'i am sure it will plateau'

'only thing that will stop/slow down progress is computation power'

Seems a bit contradictory? When has 'computation power' ever 'plateaued'?

Yes but since LLMs are a very specific application that are heavily heavily dependent on memory and there is massive investment pressure, there will be multiple newish paradigms for memory-centric computing and or other radical new approaches such as analog computing that will be pushed from research into products in the next several years.

You will see stepwise orders of magnitude improvements in efficiency and speed as innovations come to fruition.