The first meetinghouse in Madison was erected in 1705, on the southeast section of the town green. It had neither a bell nor a steeple and galleries were only added in 1715. A new meetinghouse was dedicated in May 1743, to which a steeple was added in 1799. The present First Congregational Church was built in the Federal style on the north part of the green in 1837-1838. As described in A Modern History of New Haven and Eastern New Haven County, Volume 1 (1918), by Everett G. Hill:
When the people built for the third time in 1838, they had the common struggle to break away from the green. There was a strong party that favored building on Deacon Hart’s lot north of the green, but so resolute was the minority that forty-seven members actually withdrew from the church in 1841, because of the change. The commanding site north of the green was chosen, and it and the building placed thereon have ever since been the pride of the people of Madison, the delight of all who visit the town. It is a building of notable architecture, acknowledged by all good judges to be one of the finest country churches of its type in New England. A handsome modern chapel was added to its equipment, on a plot just east of the green, in 1881.