In Connecticut Historical Collections (1836), John Warner Barber features an image of
the Mansion House of M. S. Mitchell, Esq. recently erected, and designed as a house of public entertainment. It is about three quarters of a mile north of the Congregational church. For beauty of situation and superior accommodations, it is not exceeded by any establishment of the kind in any country village in the State. This edifice stands on the spot where the house of the first minister of the place, Mr. Graham, formerly stood.
The Mansion House, on Mansion House Road in Southbury, was built in 1827-1829. One of the builders was James English, later a governor of Connecticut. The house was later owned by the famous furniture-maker Duncan Phyfe, who left the mansion in 1853 to his daughter, Mary, who had married Captain Sidney B. Whitlock, but was a widow by that time. There exists a table, made by Phyfe around 1840, that is known as the “Wedding Cake” table because it held the wedding cake of Phyfe’s grandson, Duncan Phyfe Whitlock, when he married Margaret Donaldson in Southbury.
