Check my reply earlier in the thread. I was working under the assumption that SWI is akin to x86's OUT opcode. Maybe you can decode the quoted stuff and set my brain straight.
If you want to do straight to the hardware bare metal then you need to do something like this (https://www.rpi4os.com/part5-framebuffer/) but you’re going to have to do all the other system init work that a modern system needs. Old computers were simpler!