Roads connect people. Towns are groups of people. Bigger roads connect larger groups of people. Bigger roads are the reasonable choice to convert to highways. People like/find utility in highways and roads, so they live near them.
It's an unfortunate extension of this logical process that results in 18-wheeler jake brakes thundering to a stop at the one red light in downtown Podunk, Nowhere at 3AM, on their way between Megatropolis and Port Industry, which leads to complaining at the town hall that they need "no engine braking" local ordinances and proposals that the 5-lane road speed limit should be reduced from 35 to 25. Nevermind that Podunk was established as a town because it was a convenient distance to stop at in horse-drawn buggies passing through 200 years ago between Megatropolis and Port Industry, and that every business in the town survives only because travelers stop there and inject money into the local economy...
Maybe at some point in the past, the major road should have been diverted around downtown by a half mile. But the local optimum the town currently sits at is far below the peak desirable state not because people love noise and pollution but because it's a straightfoward hill climb to the present state and it's hard to avoid that.