I don't believe they're even close to developing their own thoughts. I'm an ardent user. And every model had a mess up. It's just marketting paid for. Excuse my ignorance but what is here already is solid. I don't need AGI.
I'm really not sure how I can define that further.
If your argument is that they're not "their" thoughts, I would agree with that. But I'd also say that no one develops their own thoughts. We're all just developing thoughts from the knowledge bank we have from our experiences in the same way that AI is drawing from its weights.