I agree that we don't know what is possible in hypothetical future models, but claims about understanding in the current models are overblown (at least by people like the article author).

For example, I gave GPT-4 some chess games and asked it to evaluate the final move of each game as good or bad and explain its evaluation. It clearly had some understanding of the board state and what makes a move good or bad, but the answers were mostly plausible-sounding nonsense, far worse than what a novice chess player would come up with. I have to assume that its level of understanding of other topics isn't any different; it's just easier to expose the flaws with something that has objectively true and false answers.