Nathaniel Harrison II (1692-1760) built the house at 124 Main Street in Branford in 1724. The house was once thought to have been built around 1680, at which point the land was owned by Daniel Swain, so it is listed as the Swain-Harrison House in the National Register of Historic Places. The house passed to Nathaniel Harrison III and then to his daughter Martha, who married Nicodemus Baldwin. Martha sold the house to Joseph and Lorany (Bradley) Linsley in 1800, so it is also known as the Harrison-Linsley House. The Linsleys’ daughter, Lorany Linsley Smith, lived in the house until her death in 1915 at the age of 100. The Smith family owned it until 1938, when it was acquired by the architectural historian and preservationist J. Frederick Kelly, who restored the house. Upon his death in 1947 Kelly bequeathed the house to the Society for the Preservation of New England Antiquities, now Historic New England. Under a long-term lease, the house is maintained by the Branford Historical Society as its museum and headquarters.