The earliest part of the house was the main block with center-chimney, built around 1761-1767 for merchant Shem Burbank. In 1788, the house was purchased by the merchant, and extensive land owner, Oliver Phelps, who altered the roof to a gambrel style and added other features of the fashionable Georgian style. In 1794, he further altered the house by adding a new wing in the Federal style. The main architect of the addition was Thomas Hayden of Windsor. A young Asher Benjamin, later to become one of the most important architects of the Federal period, was one of the workers on the new wing and carved the Ionic capitals of the wing’s entryway. The interior of the Federal wing is notable for its surviving original French-made wallpaper. When Phelps died, the house was owned by the Hatheway family for a century and is currently open as a house museum, the Phelps-Hatheway House & Garden, administered by the Antiquarian and Landmarks Society.