Considering that I'm not allowed to copy any of the physical books, films or songs that I own as it breaks copyright then copying it and putting it into a generative AI should also be illegal. Just because you change it from words into weights, it's just another form of compression. I'd rather not live in the world where AI models have better rights than people.
Can you point to any substantial reproductions of copyrighted novels that are output by LLMs? If it’s just compression, you should be able to pull a substantial reproduction of the works out of them.