I've spent a lot of time archiving and scraping my own content and optimizing my home network (and ingress points, routing, etc.) to access all of these things with minimal (< 3 ms) retrieval latency and the effects are joyful. The UX is bad as it's something I've written for low latency use and my UX skills aren't very good, but it makes me feel like I have "instant" access to content, which feels like a superpower. I'd love to explore topics like this for a broader audience for more than just nerdy media I consume.
Somehow games can run at 240fps on a modern PC, but desktop user interfaces (not to mention web apps) are still sluggish.
Still, it could be worse, at least desktop environments aren't as bad as the mobile world where it seems like everything has to have a transition and animation.
This is not a model for applications in general.
That doesn't mean that the sluggish desktop apps should not be fixed, but "be more like games" is not really the right advice.
I've spent a decent amount of time in both spaces in my career, it is totally possible to build performant, fluid applications but it requires a degree of care and investment. Usually "good enough" suffices and that's where things land but there are also developers who prioritize that and short of very restrictive environments you can build responsive applications.
Besides, at the end of the day all UI is doing is rendering, with a direct path to the GPU just like games do :).
That makes me wonder... How much of an OS could be offloaded to a GPU? Specially for CPUs built into the processors that share the same memory bus, the data wouldn't need to go through PCIe to get to the GPU and back to main memory.
Does any modern OS offer a standard and architecture-neutral way to run workloads on GPUs?
> That doesn't mean that the sluggish desktop apps should not be fixed, but "be more like games" is not really the right advice.
If we can offload as much of the GUI rendering to the GPU, that's still a win because it frees the CPU for other things.
...really? My PC is orders of magnitude more powerful than the 166Mhz Pentium PC my family used in the 90s, yet the interface is no faster. Was interacting with the OS somehow orders of magnitude faster back then? I can run Win98 in a virtual machine with all the associated overhead... hell, I can do that on top of f'ing javascript in a web browser, and it is still more responsive than modern OSs.
I'm glad you've recognised the delights of a low-latency interface. The ideal interface is invisible, unnoticed by the user - my own thoughts are that an unpredictable response time is jarring, causing the interface to be noticed in a way that it shouldn't be - I've heard it called "jank" in the context of cellphone apps. So I believe people do recognise it, but probably underestimate how much it detracts from a pleasant user experience.
Of course, the underlying problem is that it's not always easy to know where the user's attention is (or if they have walked away, as you mention), but it was a simple "somewhat right" trick.
I run an RT kernel all the time. Plenty of things are still janky. Improving UX latency has more to do with UI toolkit and application design than the kernel.
We've spent the last decade-plus focused mostly on slick brochures you can poke at with your finger. Where the point is not to get stuff done, but to "engage" and manipulate the user.
But not all is lost. There are some high-performance niches in tools for professionals, and in some facets of video games.
That said, VR is the unsung hero of finally improving the awful jitter that most gaming hardware and software was developing 10 years ago. I don't care if your new graphics card can maintain 200fps on average if it has 100ms latency spikes every second. Smooth is important.
I hadn't considered the VR aspect, but you're absolutely right - the jarring nature of UI "jank" would be literally nauseating in VR.
Like you say, modern devs really need to remember to test against a high latency unreliable network. Or even just against a lack of network at all... it constantly astounds me that companies like Microsoft can let critical, central components like the start menu cease functioning without an internet connection. I recently had the start menu refuse to find any programs on my computer because it was trying to get to Bing and I was on a remote site with only local Ethernet plugged in.
I've seen Sun's Crossbow stuff in use once, but it made quite an impression - it allowed testing many failure scenarios with relatively little effort.
Another example: using the payment thing at a 7-11. You have to say if you want cashback or not by tapping on the screen (can’t use pin pad). Tapping a selection results in a minuscule change in contrast on the button that’s almost impossible to see in part because the buttons are smaller than a finger. And then it still doesn’t move on to the next screen for a few seconds, leaving you to wonder if you actually made a selection of if it’s still waiting because it didn’t register your tap.
So a new user hops on the treadmill, leans on the speed-up button until it feels about right, lets go then hammers the speed down button in a panic as it keeps getting faster! Fun to watch, but a poster-child for how not to do UX!
I think the digital version of the volume knob is a serious downgrade from the old analog version. It's probably cheaper though. The manufacturer saves $10 and I spend the next 5 years swearing at my car right after starting it.
And on top of that, either completely eliminate anything that bogs down the interactivity or at least give me a range of options between Win 98 and Modern lush.
Hard real time problems are ones where people die, stuff is destroyed etc. when deadlines are missed.
User interaction is never (or almost never) like this.
It does not imply that people die or that stuff is destroyed. I've used hard real time, soft real time and regular OSs for GUI based interaction and the only ones that really feel like they are satisfactory are the hard real time ones. The soft real time ones are obviously better than the general purpose ones but they still tend to give regular moments of unresponsiveness.
Yesterday's topic about the memory footprint of desktop environments touched on this - the fact that if a memory-hungry process triggers a swapstorm it can take minutes to regain control of a Linux desktop.
I call that sense of immediacy the feeling of "having the computer's full attention" - and ensuring that nothing required for interactivity gets paged out is the only robust way I can see of creating that feeling - but instead we use the somewhat crude, brute-force approach of throwing more RAM at the computer until the swap-outs stop happening!
It sounds like programming computers back then would have been a joy.
Casey Muratori's talks[0] on performance and optimization sparked an interest in me to start putting thought into what the CPU actually does (which wasn't taught when I learned web development). Since discovering his talks, I've been on the hunt for more learning materials.
[0] https://www.youtube.com/watch?v=Ge3aKEmZcqY&list=PLEMXAbCVnm...