The transition from HDD to SSD technology is a counter example to your claim. It was a drop-in replacement (same SATA interface) and the tech significantly improved performance without any other software modification needed.
Octane was an incremental improvement in performance when compared to SSD, and in cost when compared to RAM. An awkward spot to be in.
Optane PM was byte-addressable and had latency of ~300 ns. It rendered the entire block storage abstraction obsolete. It was so radical that using it effectively would require throwing out the assumptions all OSes have made about IO for the past fifty years.
The fact that it offered an incremental improvement even when its unique capabilities were completely ignored shows how phenomenally capable the technology was.
For a superior but new tech to stick, it has to find a viable market to be first self-sustaining, only then can it attack larger and more lucrative markets (Back in those days, PC was probably more lucrative than mobile). Only when SSDs matured enough, did they start eating up the entire storage market.
They came in 2.5 inch form factor from the very beginning, the laptop shape... which fits pretty much anything. Desktops and servers use it.
The packaging was trivial. They were just cost prohibitive for anything more than an operating system drive.
This worked out to our benefit, the OS is where random access performance shines
It might be fair to say flash used in SSDs started in other applications... but it's important to remember that's also different flash.
It wasn't until the flash changed significantly that we got SSDs (durability), changing everything we once knew. Price, applications, etc.
The difference between Optane and an SSD was never so striking.