Only if you ask it to. At which point the person asking is at the very least culpable as well of violating someone's IP.
It is also illegal for me to pay someone to write Micky Mouse fan fiction (though if I don't publish it, this gets more murky).
> The argument you're suggesting will hold is essentially "yes we're using copyrighted works, but we're doing it at scale and blending it, so that's ok".
I want to flip this on its head: the argument you are suggesting is essentially "LLMs should be illegal because they can be asked to break copyright at scale!" It isn't illegal to be an author for hire, even though someone could potentially ask you to write fan fiction for their personal collection in the style of Tolkien, but because an LLM can do it at scale, it is illegal?
