The only thing I can think of is simple image operations like a 5x speedup for a Gaussian blur filter... but then I also wonder if you'd even bother writing code to do it on the GPU in the first place.
Sure, there are valid scenarios where the slower speed fallback is acceptable. Note that I agree with the GP comment; fallback should be explicit not automatic.
- Developer adoption. Not everyone has a GPU, but with a fallback path everyone can try it out and write code that will work on the fast path.
- Application development. Along the same lines as developer adoption, not every customer has a GPU either, so if I write an app I may want to let the user try a new feature (with a warning that it’s slow) rather than tell them they can’t use the feature at all until they upgrade their hardware.
- Heterogeneous compute farms. I’m thinking in particular of render farms for VFX and CG movies, as an example, how a render farm is large and may not be full of GPU nodes due to cost. A fallback path allows time/perf to be balanced against whatever the farm’s budget for GPU nodes is.
- Test machines. Running automated end-to-end integration tests to check for correctness doesn’t depend on speed, and if you’re renting your test machines, it might be cheaper to fallback to CPU than to rent GPU nodes. Think Selenium tests running on a bunch of AWS nodes, for example.
In fact, in practically ALL applications it's better to work than not work at all. The only exception I can think of is directly-interactive software like video games and VR imaging, where minimum fps really does matter.
Many situations for various customers aren't massive datasets but modest sets, so you might be on the left of the J-curve. Requiring 2 seconds, instead of 0.2 seconds, for a result is simply not important in most cases.
I think it's fair to warn the user that "GPU acceleration not working" if the performance is expected to be much better with a GPU.
Look at how ISPC had to change the C language to make it parallelizable for actual problems. I've attempted to write compute frustum culling in GPU.js and it came out slower than my CPU approach. Hand-written shader performed much better than either.
A 5x speed-up is very significant in many applications! I've spent a bunch of energy on several projects just aiming for a 2x speed-up on GPUs - battling against SIMD and other optimisations on the CPU side.
As for real-world examples: The tensforflow.js CPU back-end runs something like 10x slower than the WebGL back-end, and I've got some great mileage out of the CPU back-end in simple node.js apps that don't have a GPU attached to their server.
And current CPUs are faster than old GPUs for some things, especially if vector instructions are used.
And GPUs are sometimes slow due to technical issues such as bus traffic, even if the GPU itself is fast.
So it's not as simple as "never use the CPU".
Some of your customers have large datasets that are only manageable with hardware acceleration, and others don't? Maybe you don't want to maintain two separate codebases?