> I am of the opinion that Nvidia's hit the wall with their current architecture

Google presented TPUs in 2015. NVIDIA introduced Tensor Cores in 2018. Both utilize systolic arrays.

And last month NVIDIA pseudo-acquired Groq including the founder and original TPU guy. Their LPUs are way more efficient for inference. Also of note Groq is fully made in USA and has a very diverse supply chain using older nodes.

NVIDIA architecture is more than fine. They have deep pockets and very technical leadership. Their weakness lies more with their customers, lack of energy, and their dependency on TSMC and the memory cartel.