Yea, there are tons of public data to train on. Copilot's under fire right now: https://githubcopilotlitigation.com/, but we make sure we only train on permissively licenses because some companies are sensitive to the IP issues here.
Juat reading the code shouldn't be illegal, and producing a short snippet based on the code but not identical shouldn't count as a "substantial portion", but IANAL...