In the mid-17th century Long Island was a land of opportunity for many New England settlers. The areas on the east and west ends offered good farmlands therefore they were settled first. In addition, there were safe harbors for fishermen and ample supplies of fish. At that time Yankee whaling began on Long Island's South Shore. The most desirable areas however were along the Long Island Sound on the North Shore. Stony Brook is believed to have been established in 1660 after Setauket in 1655. Its Indian name was Wopowog, meaning "land at the narrows", referring to the the narrow inlet from the Sound into Stony Brook Harbor. The first approved road, which is now Main Street, was mentioned in town records on May 25, 1685.

The first grist mill was built in 1699 by Adam Smith, only to be destroyed in a storm in 1751. It was rebuilt shortly after, still standing to this day and listed on the National Register of Historic Places. A general store, blacksmith shop and a bakery were developed, with the mill as a focal point. In the mid-18th century the center of activity started to shift to the north towards the harbor. The Three Village Inn, then known as the Old Homestead, was established in 1751. A large house with a liberty pole in front - the Old Brick House, was built close to the harbor on Main Street. Stony Brook's first school, which no longer exists, was built around 1750 on what is now Christian Avenue. On April 23, 1790 President George Washington passed by the school's yard after a visit to Setauket. The first post office at Stony Brook (spelled Stoney Brook until 1884) was established on March 24, 1807.

The strategic position of the island made it a center of activity during the American Revolution (Stony Brook didn't play any special role in it); the War of 1812 and the opening of the Erie Canal in 1825 however made Long Island a backwater in the westward expansion of the U.S. The valley that was going to become the village of Stony Brook was getting more and more alienated and isolated. Settlers around the Stony Brook Creek concentrated around the mill area, with Harbor Road as the main connection to the west. With no churches, amenities and local authority Stony Brook "slowly drifted backward" according to historian William Pelletreau. Because of its small population and difficult harbor entrance steamers didn't stop there, making it difficult for settlers to travel.

William Sidney Mount recognized the problem and had an idea to create a new inlet at the west end of the harbor through Long Beach. The potential for economic profit was clear - since the local community survived mainly through barter, tourists and trade would provide good cash income for the community. Local investors agreed and the inlet was dredged two years in a row but filled in both times. The project was abandoned and Stony Brook remained a farmers' hamlet whose primary source of income was selling cordwood at the village dock. As cordwood was vital to the local economy, the Brookhaven town trustees retained ownership of the "cordwood landings", which are today's town beaches. The only other profitable industry in Stony Brook was shipbuilding and outfitting, through which the village had peaked economically by 1855.

After 1865, when the Civil War was over, Long Island became the center of sporting activity and a focus of summer resort. Country estates and clubs were built, first along the South Shore and then the Sound. Families from New York City often visited Long Island's hotels and boarding houses. In the 1870s the north branch of Long Island Rail Road was constructed and the clean air and quiet atmosphere of Stony Brook made it a point of interest for many travelers. At the turn of the century it had six hotels, and the popularity of Stony Brook as a resort town lasted until World War I.

Shoe manufacturers Jenny and Frank Melville Jr. developed a plan for the improvement of the Stony Brook village soon after they arrived. Their son Ward and his wife Dorothy followed in their footsteps and by the late 1930s they envisioned a new Colonial village to replace the untidy collection of buildings from the previous century. Over a dinner for around 80 guests at the Three village Inn on Jan. 19, 1940, Ward Melville, founder of the Thom McAn shoe chain, offered to rebuild the town's business section, which was visibly declining during the Great Depression. He lived in Manhattan and Old Field, but he had fallen in love with Stony Brook during his summer vacations there. He quickly persuaded about 35 merchants to sell him their properties and become renters by offering them low rents, the attraction of customers from outside the area and the prospect of national acclaim which they later got from architectural groups.

Town clearances were obtained easily and Melville spent $500,000 in 1940-41 for the rebuilding of Main Street into a crescent-shaped business district of attached stores, gathered around the Federal-Greek Revival-style post office on a two-acre village green. Melville gave permanent control of the project to the nonprofit Stony Brook Community Fund, which was recently renamed the Ward Melville Heritage Organization.

Many structures were moved, modified or demolished. Ward Melville's houses were restored by the removal of Victorian modifications and the addition of Colonial architectural features. In 1946 Suffolk County made a proposal to shift Main Street to the west and widen the new village green, thus easing the traffic into Christian Avenue. Again, buildings were moved and repositioned. In the end, the village that now stands in downtown Stony Brook was complete.

Between 1956 and 1968, in further pursuit of his goal, Melville donated 600 acres of land and funds to the state of New York for the creation of the State University of New York at Stony Brook, as well as for the local school district. He thought the building would look like the Virginia colonial college "William and Mary", and it would complement what he dubbed his "living Williamsburg.'' On this land the state opened the university in 1963, bringing many assets and opportunities to the area, along with thousands more homes, increased traffic and commercial buildup on surrounding major roads. As the university center grew, additional lands were used to build the university hospital and medical school. But Melville, who died in 1977 at the age of 90, lived to resent the brick-and-block look of what is now the 1,100-acre campus. Nevertheless, of all the events that took place in Stony Brook, none influenced the development of the area as much as Ward Melville's invitation to NY State to build a university there. It positively affected the development of all the surrounding villages, including Setauket, Port Jefferson and Smithtown.

A few prominent families stand out in the history of Stony Brook; they controlled the area in the 18th and 19th century. These families include the descendant of Arthur Smith - John Smith, whose home became the Stony Brook Hotel; Jonas Smith, whose home became the Three Village Inn; David Bayles, who built the sawmill and lumberyard complex; Reverend Dr. John Carson, founder of the Stony Brook Assembly; Edward Kane, who purchased the Stony Brook mill and millpond; Jennie and Frank Melville; and Dorothy and Ward Melville, who created the present village center and funded the Museums at Stony Brook, saved historic homes and provided valuable community assets, leaving a legacy that has clearly altered the community's rather unremarkable background.

Today Stony Brook is known for the abundance of 18th and 19th century buildings still in use. Much of the old hamlet is preserved in the streets leading up to and out of its center; few 20th century structures interrupt the progression of homes which once belonged to farmers, craftsmen, and seamen.