The book Madison: Three Hundred Years By the Sea (1976), p. 39, dates the Abraham Cruttenden (or Crittenden) House to 1639. Abraham Cruttenden was one of the original settlers of what would become Madison and he arrived in New Haven with other settlers of Guilford in 1639. A more recent dating of the house, which is a Colonial cape at 123 Boston Post Road in Madison, is 1735. In 1967, Yale architect Albert Riese erected a mid-century modern box at the rear of the house as a wing for his elderly mother. Riese’s daughter and her husband extensively renovated the house in 2012.
The Shelley House, at 248 Boston Post Road in Madison, dates to the late seventeenth/early eighteenth Century, with specific dates variously given that include 1709/1710 and 1730. This exceptionally well-preserved structure is a rare surviving example of a house that was clearly built in several stages, following a pattern believed to have been common at the time: starting with a one-room, two-story dwelling with a stone wall at one end (the east half), a second section added later (the west half) and finally a lean-to at the rear. Traditionally known as the William Shelley House and also known as the Stone-Shelley House, it underwent a controversial restoration c. 2008.
The large Italianate house with a cupola at 17-19 Wall Street in Madison was built in 1854. It was the home of Alva Orrin Wilcox (1799-1887). His entry in the 1893 book The Descendants of William Wilcoxson, Vincent Meigs, and Richard Webb, complied by Reynold Webb Wilcox, reads as follows:
Alva Orrin Wilcox, of Madison, Ct, son of Return of Madison, Ct., kept a hotel and ran a stage line from New Haven to New London, carrying the mail for thirty years, m. Sept. 27, 1826, Rachel, dau. of Billy Dowd, d. Aug. 21, 1889.
The saltbox house at 566 Boston Post Road in Madison was long thought to have been built by John Dudley in 1675, making it the oldest house in town. The nomination for the Madison Green Historic District instead attributes it to Gilbert Dudley with a date of c. 1740. A plaque by the Madison Historical Society gives a date of c. 1720. On April 11, 1776, while on his way from Cambridge to New York, George Washington stopped to dine at the house, which was then a tavern run by Captain Gilbert Dudley.