A lot of the latest revolutions (good or bad, that's up to the reader); crunching huge data, ML, every more realistic simulations etc comes from every faster machines. If that growth stops, the article suggests we do something that was (and still is in some circles) normal in the 70-80s with homecomputers and consoles; because you could not upgrade the hardware and almost nothing was compatible (which is the most common reason the IBM PC won) to the next generation, you optimised the software to get everything from that existing hardware you had on your desk. And people are still doing that.
One of my personal miracle examples; my first love was the MSX computer, which is a Z80 3.58mhz computer with (in my case) 128kb in it. This machine could do nice games for the time and some business applications. Many years later, that same physical hardware (I still have my first one) can do this [0]. Obviously the hardware was always capable of doing this, but it needed many years (decades) for programmer(s) to figure out how to get every ounce of performance and memory utilization out of these things and push it beyond what anyone thought possible.
If the improvements in performance stagnate, there is a lot of room for getting most out of the existing hardware. I would think though that in the case of modern hardware, the geniuses that get to that point will get some language to compile to this optimised optimum instead of having to handcode and optimise applications like the Symbos guy did.
Case in point: a matrix library I used to use needed to a full row/column pass each time. We put a layer in between it and our code. Reduced lookups required by 30%. We were processing the same amount of data and getting the same results but requiring far less time. That layer also reduced memory requirements. Now we could process larger datasets faster with the same hardware. Thats just one example.
Your choice of CPU and other hardware isn't always the limiting factor. Even the language choice has an impact. Some languages/solutions require more data processing overhead than others to get the same final result.
Even the way your program's Makefile or module composition can have an effect on compiling performance. I remember the use of a code generator we included that meant it had to regenerate a massive amount of code each run due to its input files being changed. We improved it by a ridiculous amount simply by hashing its inputs and comparing the hashes prior to running the code generator. Simply not running that code generator each time meant we sped up the build significantly. 30 minute build times reduced by 5-10 minutes. Same hardware. And that was easily triggered by a trivial file change.
still, even twenty minutes is a long time to wait and see if your latest change actually works. in the foreseeable future, there will be complex projects that take a long time to build. you will eventually have to touch things that everything else depends on and recompile most of the code. people that work on these projects can always benefit from faster desktop-class hardware.
Having much better hardware in these cases helps you be actually productive and not twiddle your thumbs waiting for the compiler.
My philosophy at the moment is to use HPC only when I've exhausted other possibilities. I think many people jump to HPC prematurely. The simpler approaches are so much cheaper that I think it's usually worthwhile. I'm skeptical of the argument that it's cheaper to use HPC than it is to use more efficient methods in this case, because the more efficient methods are often something like a few days spent reading to find the right equation or existing experimental data vs. at least that much setting up a simulation and longer to run it.
Edit: Bill Rider has a bunch of blog posts that make similar points:
https://wjrider.wordpress.com/2016/06/27/we-have-already-los...
https://wjrider.wordpress.com/2015/12/25/the-unfortunate-myt...
https://wjrider.wordpress.com/2016/05/04/hpc-is-just-a-tool-...
https://wjrider.wordpress.com/2016/11/17/a-single-massive-ca...
https://wjrider.wordpress.com/2014/02/28/why-algorithms-and-...
None of this is extraordinary now but the result was we reduced multiple times the budget requirements and processing times just using code improvements. A lot of times, the head-on solution just needed optimisation. Sideways improvements also helped such as small optimisations also helped. The more exotic equipment is still useful but its an accelerator.
One last thing: we received a LOT of grief and criticism for our approach. There was peer pressure to use particular solution types even though they were wildly inappropriate. We had funding pulled or threatened to be pulled by some and other backers. One lessons we learnt includes: Don't underestimate how vested certain interests are for the use of various toolkits. "Use this or else!" is the not-so-subtle threat.
I'm so glad to not rely on only academic work now.
I should have spoken more generally to OP's point though. What I was really hoping to get at is that there are applications other than games that require non-trivial amounts of compute, and speeding them up would make meaningful differences in their users' lives.
And by the way I believe the software code is not the only place which could be made more efficient. What if we removed all the legacy stuff from the x86 architecture - wouldn't it become more efficient? What if we designed a new CPU with a particular modern programming language and advanced high-level computer science concepts in mind - wouldn't it make writing efficient code easier?
Also, what are the actual tasks we need so much number-crunching power for, besides things of questionable value like face recognition, deep-fake, passwords cracking and algorithmic trading?
Rendering is also an insanly parallelizable task. In the worst case we can always slap 2 of the same gpu on one card and get them to render half the screen or for vr, one gpu per eye.