Carversville is a small village that has retained its rural character, and looks much like it did during an earlier era. It has so far been spared the development and commercialism that has altered other areas of the county. The village was originally settled by the Lenni Lenape Native Americans who called the town Paunakussing, meaning "where powder was given to us," but the exact meaning of this reference is unknown. A creek by the same name flows through the village.

European settelers first referred to the area as "Indian Village", and by the early 1800s it was referred to as Milton, which probably was a corruption of "Milltown." The first post office was founded in 1833 with Thomas Carver as the postmaster, after whose family the village was renamed. Several dams once existed along the creek to power the local mills, but these were largely destroyed in a flood in 1885.