If a human reads GPL code and outputs a recreation of that code (derivative work) using what they learned - that is illegal.
If an AI reads GPL code and outputs a recreation of that code using what it "learned" - it's not illegal?
If that is the case, then copyright holds no weight any more. I should be allowed to train an LLM on decompiled firmware (say, Playstation, Switch, iPhone) in countries where decompilation is legal - then have the LLM produce equivalent firmware that I later use to build an emulator (or competing open source firmware).