I fully agree with you.
My point on spectrum wasn't relating to free speech but rather free speech regulation.
Open discourse is universally recognized as a generally good thing for society.
So, we can try to embrace Free Speech ideals even as we grant our government right to censor some speech and draw (a somewhat arbitrary) line of what's allowed.
We've seen Russia, for example, abuse that power even though free speech is written into their constitution, they use that power to censor political speech and what they refer to as blasphemy, so we know this power can and has been abused but it's also possible, so, perhaps, you are right.