France and Sweden are both far more centralized and less federalized. There's a reason more federal countries such as Switzerland or Germany have more decentralized policing. I think it would be beyond unacceptable to most Americans, myself included, for the US to have a single police department with its bureaucracy in Washington. If anything, I think far too power in the US is already centralized in DC which contributes to bad governance due to the vastly increased distance (in many senses) between politicians and their constituents.
I would be fine with 50+6 state/district/territorial police departments, which is sort of the Germany or Canada (with Ontario and Quebec) approach.
And France and Germany are each twice as populous as the most populous US state.
So you're not really correct.
There's absolutely no reason why having a single bureaucracy wouldn't work well.
It actually seems like it would be much easier to have a single bureaucracy.
The US often seems to have more layers of administration and law-making than some other places. Making individual officers work through which set of powers and laws they're dealing with in a given moment seems like it might at times be a daunting prospect. You could in theory organize them into a single bureau, but I suspect you'd inevitably wind up with specialized sections to deal with the particular laws in given cities.