AI Chips: Past, Present and Future

There are something like three dozen startups working on different chips that can speed up this whole process. The big question for all of these companies and their architectures is how flexible they will be for anything beyond a very specific implementation of an algorithm. This is particularly important in the AI/ML world because the algorithms are changing so rapidly. The challenge here goes well beyond the chip architecture. So if you think about the benchmarks for CPUs today, those can be blazing fast, but when you try them out on real applications they run a lot slower. Those differences may be magnified with parallel approaches. Different applications utilize different levels of parallelism with uneven performance improvements. It's not just how the processing is parsed, but also how effectively the results can be recombined after processing.

Disappointing article. The history and future of neural network chips is much much larger than GPUs and TPUs. There was a post a few months ago from IEEE about neural chip architectures, which was pretty good for the scope, but there is actually a lot more once you add in the materials research.