I think the architecture, language framework etc need a completely different view now that programming is not limited by human constraints like speed, effort and thinking. May be something like compilers. A programming language or framework which is built specifically for automated and non-deterministic behaviour.

We might need different kind of rules/constraints because at the end we rely on LLMs to respect or use those rules. Frontier labs are best placed to build something like this making it part of LLM speak.