Their job is 100 percent to predict the next word, but saying that hides what you have to do in order to predict the next word
In a factual database? You have to repeat facts.
In a logical database? You have to learn logic.
In a database containing only good chess moves? You have to learn to make good chess moves.
Train any AI model in a strong enough way and it will pick up donation knowledge along the way. Yes, it's predicting words, but it's using that knowledge to do it.
It's by no means perfect, but do not underestimate the potential here, and remember that reductive definitions (it's just x) can make most great things seem bad or mundane.