The house at 16 Main Street in Bethel is thought to have been built circa 1790 by Daniel Barnum, a joiner and cabinetmaker who was the cousin of P. T. Barnum‘s father, Philo. In 1825, Daniel Barnum sold the house to his daughter, Anna, and her husband, John Benedict III. The couple sold the house in 1844 to Dr. Hanford N. Bennett (1818-1868), who sold it in 1853 to another physician, Dr. Ransom Perry Lyon (1826-1863). During the Civil War, Dr. Lyon was a surgeon in the 28th Regiment Connecticut Volunteer Infantry. He died during the siege of Port Hudson, Louisiana. His widow, Sophia, sold the house in 1871 to the Danbury and Norwalk Railroad. The house lost its original large center chimney when the entire building was moved 150 feet west in 1872 to make way for a new rail line along Main Street that connected the Danbury and Norwalk to the Shepaug Railroad. The building was then used as a boardinghouse for railroad workers until 1881, when George M. Cole purchased the house. He built the addition on the east side of the house, as well as the front porch.