Around 1980 my family got their first computer. I've followed this business ever since, and I was amazed that a CRAY could do MFLOPS. My MS basic interpreter could do hundreds or even thousands of FLOPs on its 8080A. I watched as the high end went to hundreds of MFLOPS, then GIGAFLOPS which seemed insane. There were national efforts to reach TFLOPS, and reading about the challenges (IIRC at the time interconnect was a huge deal) made it seem like the end was near. Moore's law was always in danger. Then came PETAFLOPS consuming megawatts of power.
And now I play VR on a battery powered gizmo doing about 1 TFLOP strapped to my head, and EXAFLOPS are basically here. This is all with at least TSMC 5nm, 3nm, 2nm, and multi-layer left on the table. After watching this relentless advance for 4 decades I'm pretty sure it will go beyond even that, but we just don't know what it will look like yet.
It's become everyday tech to me, but if I look back the progression is absolutely astounding.