A bit among the lines of "if my grandmother had wheels, she would be a bicycle". Emulating neurons doesn't make intelligence.
Sentience isn't even well-defined and I'm not sure we can even point to any unique quality of any individual human as indicative of sentience. We certainly can't agree on what point an individual human becomes sentient, as evidenced by the abortion debate. At best we seem to have some statistical evidence based on what we've achieved as a species being superior to what oak trees have achieved, but at an individual level, I don't know how to prove to anyone that I'm not just a very advanced LLM.
There is another sense where sentience is just being used to mean "that which is human", and by definition, nothing aside from humans will ever qualify as that.
What about animals makes them sentient? Until we can answer that, people are just going to be talking past each other. Even if you forget about AI, whether animals are sentient, and, if so, which animals are sentient, is a big argument in biology/ethics/law that's been going on for at least half a century. The UK recently passed a law that declares animals to be sentient, but not all invertebrates are considered sentient in that law. Their justification is that they don't contain a central nervous system, but what special property about a CNS confers sentience?
There aren't any similarities in the way animal species and LLMs develop.
There have been rumours that the current ChatGPT loses money even at $20/month, and that to economise on running costs they've changed to a less capable model.
And if the modern LLMs have already mined everything there is to get out of pirated ebooks and the common crawl dataset - who knows how long it'll take for them to make the next big step forward?
ChatGPT discussion here has been totally dominated by this problem from the beginning, but usually it's people raising the bar on what you have to do to get good results from it.
But synthetic training data has its problems. Oh, it's great if you want a limitless supply of templated high school math problems. In other fields, though? If GPT-whatever is a bit unclear on whether Magnus Hirschfeld was a Nazi or a victim of the Nazis, and you use it to generate synthetic training data - you can't expect the student model to know better than the teacher model does.
Why not? People outgrow their teachers all the time, even in fields where there is no new external data, like maths. Often improved understanding comes just from reflecting on an issue from new perspectives, which synthetic data can provide. Perhaps a teacher/student model helps AI develop those new insights, just as it does for us.
I don't believe anybody is suggesting that we exclusively use synthetic data, but rather that synthetic data can augment other types of training. The other thing to consider is that less sophisticated models can be prone to hallucinating nonsense, but the hallucinations are usually inconsistent, whereas truthful responses tend towards consistency in various directions: between each response, internal consistency, and consistency with reality.
It's conceivable that a more sophisticated model would be able to learn a sense of certainty based on the consistency of its training data, much as we do. If you consider your education, I'm guessing you probably had lots of people tell you incompatible nonsense over the years. In my case, I've had teachers give me a ton of inconsistent explanations about how electrons "know" which path to take in a circuit - probably one of my biggest questions since a child. Only one explanation turned out to be internally consistent and demonstrable with experiments I've seen on YouTube. The result is that I now have, I think, a pretty decent understanding of how electric current forms a path within a circuit, or at least one that can be used to make valid predictions, despite being told a vast amount of inconsistent and wrong information over my life.