I think the two modes of LLM discourse: “they’re conscious!/they’re just next token predictors with impressive datasets” comes largely from two different groups of people: those who learned about LLMs before learning about ML fundamentals, and those who learned ML fundamentals before encountering LLMs of today. While I fall in the second group, there is a real risk that my prior concepts about the fundamentals is limiting my view of the bigger picture, so I at least welcome the debate.

Re: chain of thought, I at least know that in practice a lot of the results from the original paper has not been quite reproducible at later attempts. Whether that is a quirk of models changing everyday or something deeper, I do not know.