You are correct that the "model" may be a separate piece of work with it's own copyright but that does not mean that the model does not produce copyrighted works.

I’m talking about outputs not the model itself, the GitHub copilot example has lots of shared bytes