You are right. I am not contesting the Highway system, which actually still seems to be a good case for Federal public spending, both because it benefits the military and because it is inter-state.
My point is entirely regarding the roads that were built out from the dense urban cores to connect with sub-urbs, and the roads built to connect sub-urbs with one another, and those inside of suburbs too. If these were paid for by those who used them (and not inner-city dwellers) then perhaps urban sprawl would not have happened, and we would have saved ourselves from a whole host of externalities we are only discovering now.