He points to Intel’s Sapphire Rapids as an example of hype-driven engineering versus the kind that could pay real dividends for broader HPC. “Maybe in their second generation they’ll put high-bandwidth memory in the CPU, but why did that take so long? It’s been available for years. We think this is attributed to LINPACK, for needing higher rankings. And now they’re just thinking about dense linear algebra and not about memory or throughput or how the memory and cache can allow these vector engines to work effectively. That’s just bad engineering. If we continue like this, we’re just going to keep wasting resources.”
To be fair, NVIDIA is also guilty of this. Their marketing is highlighting their tensor core units, claiming order-of-magnitude speedups on deep learning workloads, when in practice, workloads that involve convolutions are actually memory bandwidth bound and will not see a significant benefit.