Waterbury industrialist Elisha Leavenworth built a Greek Revival house facing the Green in 1845. He moved in with his new wife, Cynthia Fuller Leavenworth, who died in 1854 with her infant child. According to The Town and City of Waterbury, Vol. II (1896):
[Leavenworth] entered into partnership with his father in the drug business, under the firm name of F. Leavenworth & Son. In 1850 he took Nathan Dikeman, Jr., of Northampton, as a partner, and the firm became Leavenworth & Dikeman, and so remained until its dissolution in 1890. […] Soon after the partnership with Mr. Dikeman was formed Mr. Leavenworth ceased to take an active part in the business, and devoted himself to his other interests. On his father’s death, in 1840, he succeeded him as postmaster, and held the office until 1849. He held the same position again, from 1853 to 1861. He represented the town in the legislatures of 1863, 1864, 1867, 1868. In 1875 he was elected judge of probate, and again in 1877 and 1878. He was for many years the acknowledged manager of the Democratic party in the town. He was the largest contributor to the Industrial School building, having given $10,000 for this purpose. Leavenworth hall was named by the managers in recognition of the gift. He was the first president of the Dime Savings bank.
Elisha Leavenworth, who never remarried, left his house, upon his death in 1911, to the Waterbury Girl’s Club. That year, a Masonic Temple, now part of the Mattatuck Museum, was built on the site of the house, which was moved nearby to 35 Park Place. The Girls Club is now known as Girls Inc.
