Because it will become unpractical. It’s like saying your SO shouldn’t have access to your bedroom, or the maid should only have access to a single room. Instead what you do is having trusted people and put everything important in a safe.
In my case, I either use apt (pipx for yt-dlp), or use a VM.
