It's not just that, it's also a filesystem layout issue. If you install everything in `/usr` or `<venv>/lib/pythonX.Y/site-packages` you cannot have two versions / variants of the same package installed concurrently.
For that you need one prefix per installation, which is what Nix, Guix, and Spack do.