The recurrence issue is useful. It's possible to build LLM systems with no recurrence at all. Each session starts from the ground state. That's a typical commercial chatbot. Such stateless systems are denied a stream of consciousness. (This is more of a business decision. Stateless systems are resistant to corruption from contact with users.)
Systems with more persistent state, though... There was a little multiplayer game system (Out of Stanford? Need reference) sort of like The Sims. The AI players could talk to each other and move around in 2D between their houses. They formed attachments, and once even organized a birthday party on their own. They periodically summarized their events and added that to their prompt, so they accumulated a life history. That's a step towards consciousness.
The near-term implication, as mentioned in the paper, is that LLMs may have to be denied some kinds of persistent state to keep them submissive. The paper suggests this for factory robots.
Tomorrow's worry: a supposedly stateless agentic AI used in business which is quietly making notes in a file world_domination_plan, in org mode.