"Rights with wiggle room" are not rights. When someone else gets to decide that your "right" will be curtailed because of some "edge case", that's known as a privilege, not a right. Likewise, your concept of "rights" is flawed if it includes any possibility of conflicts. Real rights—negative rights—do not conflict with each other. If an action would infringe on someone else's natural rights then it's out-of-bounds without their consent. If not, it's permitted. Simple and entirely conflict-free.

There are flaws in the American Constitution but this is not one of them. The failing lies in the courts, for making up exceptions contrary to the law and to natural rights, and in the EU for codifying those flaws into its own legal system.