It's not about having the artificial notion of self. It's about having a real self. Memories of who you are, what is your history, what is your place in the world, who are your friends and your relationship with them, I can keep going but it's obvious ChatGPT has none of it and has no simple way to acquire it because it is not even designed to learn anything from its own interactions except a small 3000 tokens context window.
Every human learned everything except few innate instincts from their own experiences. They didn't get it fed through some backdoor in their head specifically designed to learn. That's how I see "training" AI. None of what they learned was from an interactive experience of the world. All their knowledge came from some backdoor interface directly hacking their neurons to fit.
You know what you know because of experiences, not because your creator was hacking with your neurons. It's also much more demanding intellectually to understand the world through such a limited interface. Children understand language from orders of magnitude less data.