As I mentioned in the text, I think that's the difference between WSDISPLAYIO_MODE_MAPPED and WSDISPLAYIO_MODE_DUMBFB. The former allows access to the hardware registers whereas the latter does not. Obviously, if you choose to use the former, then you are tied to a specific graphics driver. (But I'm not sure of this.)
And yes, agree, what I described is not great for performant results, but there is a lot you gain from the simplicity of this approach if all you want is toy around. I feel we lost a lot of this simplicity over the years, but it's "still there" if you don't care about optimal performance.