Memory safety is orthogonal to side-channels, and hardware-enforced isolation (e.g. IOMMU) is more powerful than compiler-enforced isolation (but both are good!)
Oh no now I have to worry about shaders row hammering my OS ram /s
You really do have to worry about that!
Isn't this how the Xbox 360 got hacked? Not necessarily rowhammer but other methods.. IIRC some shader code in King Kong was able to affect CPU execution or something like that.