Eink refresh is slow-ish, but only because if say scrolling a bunch of pixels, you need to invalidate and redraw the whole screen.
The advantage of terminals is that you can easily keep a matrix of squares, and by looking at the flux of information, you can pick the opportunistic times for refreshes.
With characters, there should be more recouping. (a B and a P overlap except in the bottom right)
With line editing, the problem should be limited to the line
All these small gains should add up to provide an excellent terminal experience.