As the ideology behind this bait-and-switch leaks into the wider culture, it slowly corrodes our own self-understanding. If you try to point out, in a large lecture or online forum on AI, that ChatGPT does not experience and cannot think about the things that correspond to the words and sentences it produces — that it is only a mathematical generator of expected language patterns — chances are that someone will respond, in a completely serious manner: “But so are we.”
And a very logical next step after this neat and tidy dehumanization is, as history has shown, the gas chambers for the obviously malfunctioning “machines”. Because whatever, it’s not like they’re actually feeling anything, right?
(edit)
When you reduce persons to the status of things, you are going to get people treated like things.
And every single time, that’s a recipe for disaster.