You can say the same about compiled executable code though.
There is an argument also that LLMs are derivative works of the training data, which I'm somewhat sympathetic to, though clearly there's a difference and lots of ambiguity about which contributions to which weights correspond to any particular source work.
Again IANAL, and this is my opinion based on reading the law & precedents. Consult a real copyright attorney for real advice.