Sherman is the least populous town of Fairfield County, Connecticut, United States. The population was 3,581 at the 2010 census.