There's no rule anywhere saying we have to treat computer programs as if they're humans and extend them the same rights. Why would we do that? Says who?

Also, we don't do that, not with AI. We're not talking about paying AI for their labor, or giving them digital housing or something. We only care about the human rights we can extract profits from, other than that these are digital slaves.

Which, I'm fine with digital slaves. They're bits on a computer. But then we need to do that, and we can't be doing this whole "welllll they're basically people" bit we're doing with learning.

For the same reason that I think it is (1) a bad idea to use an LLM to make your own OS, (2) when the AI is good enough to do that, it won't matter which OSes are or are not open-sourced, nor which OS's source code was used to train a model, as by that point the AI can make any OS for peanuts anyway — given those two points, I think the copyright holders fighting against their stuff teaching AI are missing the forest for the trees.