I've been thinking about a hypothetical CPU that has explicit cache and RAM access. It's pretty close to OP's alternative universe.
What would happen is people would write wrappers for such a system so that you could think about your RAM in a linear way. And you're now doing what a Haswell CPU does.
Of course, this approach has advantages for specific tasks Let's say, we want to implement constant-timing crypto primitive and put all of the required data in cache. We can now do it.
This would also help with implementing an efficient VM, OS primitives, or many other interesting things. But it would not change the way people write C/Rust/JS/Python.
Of course, it's a useful model for your performance, and you can now think about cache misses as "internet download", and synchronization primitives as "LAN share", etc, etc.
And, of course: