According to the United States Census Bureau, the town has a total area of 66.5 square miles (172.2 km²), of which, 66.4 square miles (172.0 km²) of it is land and 0.1 square miles (0.2 km²) of it (0.09%) is water.

The east town line is the county line of Madison County.

US 20 is an east-west highway through the town. NY 91, a north-south highway, intersects US-20 in Pompey village. NY 91 cuts across the northeast corner of Pompey.