Smaller stuff that still reflects the "add-in-card" intention of the original design is fine, you can still have a bunch of PCIe slots for network cards and USB controllers if you want. But the GPU is not really an "add-in-card" anymore and it's problematic in a design sense to keep acting like it is, let alone to have every single product on the market doing something completely different dimensionally.
That's the "pre-USB-micro" era problem that USB faced.
There's no need for water though, that's a whole can of worms.
I am aware that modern gaming consoles are basically this but they make lot's of tradeoffs that a Desktop PC doesn't necessarily have to. Single Die GPU-CPU Combination for example. On the PC world you can reasonably pay twice as much and get twice the silicon for twice the performance, a tradeoff Consoles can't make.
and as for the server this could make essentially large 1U GPU racks possible by just having one big board per U with all the gpus in it.
In any case, I've always used the standard water cooling kits and they never failed me (as opposed to constantly breaking fans and huge temperatures). Maybe it's my environment.