Captain Benjamin Williams built an impressive brick house in Middletown in 1791. As described in New England Families, Genealogical and Memorial, Vol III (1913), compiled by William Richard Cutter,
Benjamin Williams came to America from the Island of Bermuda when a young man, and settled in Middletown, Connecticut, where he died June 15. 1812, at the age of forty-five years. He built and lived in the house on East Washington street subsequently known as the De Koven place, and at present as the Wadsworth House, He became a large ship owner and had many vessels plying between the East and West Indies and the port of Middletown, the towns on the Connecticut river having extensive shipping interests in those days. Then came the war of 1812, and French privateers captured the greater number of his ships. He expected that the government would reimburse him for this loss, and died in the hope that his widow would receive what was her due, but this was never done.
The house was later owned by Henry L. deKoven, who was also involved in merchant shipping and was the first president of the Middlesex County Bank in 1830. In 1900. the house passed to Clarence Seymour Wadsworth, who used it as a business office after he built the Mansion on his Long Hill Estate. In 1941, he bequeathed the house to the Rockfall Corporation, which he had founded in 1935 and has been dedicated to environmental education, conservation projects and planning initiatives in Middlesex County. Restored in 1942, the house opened as a Community Center for non-profit organizations in Middlesex County.