For example, owning property is a negative freedom, because the state must not act against your property to ensure this freedom.
Being free of violence is a positive freedom, as the state will have to act to ensure this freedom (by sending a police officer when you dial 911 for example). In this note, not being a victim of theft would be another positive freedom, as the state will act to prevent it or redress you as a victim.
Negative rights don't have to be removed, as you fear, they're simply rights that the state can passively guarantee without having to interfere. Generally, such negative rights/freedoms don't require protection from the state, (hence they're negative).
On the flipside, the positive rights and freedoms do need to be protected. You have a freedom of speech/opinion, which is positive in that the state should act that you can enjoy them. If it were a negative freedom, then they could be lost at the toss of a hat the next time a truth dictator comes to power, as they aren't required to be protected.
So by the very nature you say you fight the removal of these rights, you're already reinforcing that freedom of speech is a positive right. (I don't even get were truth dictator comes in?)
To point out, there is gov departments in a few countries that already do what I describe or very close to. While generally not perfect, they have been useful in digging out neonazi groups and fighting for your freedoms more than the armies of those countries.