The house at 591 Main Street in Cromwell was built on the site of an earlier house, purchased by Amos and William Savage from the estate of Joseph Ranney in 1756. Samuel Spencer (1744-1818) purchased half of the house and land in 1771 and the other half six years later. He may have incorporated the earlier residence into the new house he built c. 1777. After Spencer‘s death, the house passed to his daughter, who was married to Dr. Titus Morgan. Another daughter, Sarah “Sally” Spencer, married Joseph Morgan, Dr. Morgan‘s cousin, who was the grandfather of J. Pierpont Morgan. In 1873, the house became the first Cromwell residence of Russell Frisbie, who later bought the house on Main Street that is now home to the Cromwell Historical Society. It was probably Frisbie who added the Italianate decorative elents to the facade of the Spencer House, which originally had a gambrel roof.