Is the notable delay between keyboard sounds and video rendering due to the SSH/software?
The main culprit was network delay as I am transmitting raw pixels (one u8 per pixel) compressed with zlib. That's a hit of ~140ms for half a screen.
Next in line is the screen refresh (unmeasured, perceived).
Then the optional post processing (~20ms for half a screen), and housekeeping, like keeping track of dirty regions (about as long).
Lastly writing to the framebuffer (less than 20ms, I don't remember exactly how long).
I took great care to optimise the process, and my next step was to transmit multiple pixels as a single u8 int, the physical display cannot render 255 distinct shades of gray.
By the way I suspect compressing multiple pixels into one is unnecessary - just quantise them and let the compression deal with it.
Also zlib is not designed for image compression. I'm sure there is something more suitable, e.g. QOI.
In fact, given that you're mostly compressing mono text I wouldn't be surprised if some kind of dynamic sprite atlas kind of system was better, like in JBIG2.
Anyway if it is network latency that seems like good news because you should be able to get it to near 0. What is the ping to the reader?
If you wish you could copy your comment verbatim as an issue over here, so we can discuss it further: https://github.com/everydayanchovies/eink-vnc/issues
I will have a chance to test the ping tonight, possibly.
The explanation is that I take end-to-end network measurements (from request of update to a full buffer of pixel bytes). That delay might be due to the slow processor on device, or an inefficiency in the networking code in my application.
Using an Onyx BOOX with Termux, terminal lag is not perceptible. The lagged updates look like packet jitter to me.