Edit: Yes, the "ATX12VO" spec.
I have a desktop HP that has a standard-sized PSU but has a weird connector that only plugs into the motherboard. The drives are powered from a separate connector that also plugs into the motherboard. And the PSU - MB connector is much smaller than standard ATX.
There are also boards like this that have the RAM mounted flat, a lower profile set of ports on the back, and take in 19v DC: https://www.asrockind.com/en-gb/IMB-193
You'd have to make something custom, but I think it would be much simpler than typical heat sinks. With air cooling, the vertical placement would be ideal, since convection will tend to suck cool air from the bottom and release it on the top. (I think the iMac does/did this). Even for water cooling, it would be simpler with far fewer turns in the pipe. Plus, I just like the idea of laying everything out flat instead of the weird 3-D origami that is modern PC design.