The trouble with all this is that people are forced to buy these insanely expensive systems merely for the benefit of fitting all that stuff in the vram even if they don't end up using the compute cores on these machines (which, let's be honest, aren't really all that better than the gaming gpus).

The needs are more in line with consumer server hardware with user-choice on cpu/ram etc. Sounds to me like there's a market for disruption. Pity that the deep-learning community is under the choke-hold of nvidia's software.