How would you power and cool such a beast? What kind of workloads would benefit from that much RAM while running happily on a CPU that fits on a typical SBC layout?

If you want a tiny workstation-class system, there are entirely decent mini ITX board you can stuff a recent AMD or Intel chip and tons of RAM into. You'll need a desktop-class cooling system and power supply, though, and it'll still have the footprint of 6-8 of these board, and probably about 15x the total volume and power consumption.