Why not just wrap/abstract the native platform APIs (like what wxWidgets does)?
Count of pixels in displays was improving way faster than RAM bandwidth or CPU performance. Currently, this does not matter much on desktops, matters somewhat on laptops, and critically important for many mobile and embedded platforms.
When you want to render anything at all at native display resolution and refresh rate, GPU stuff is the only option that works well enough.