I don't work with ML, so maybe this is stupid, but wouldn't a "regular consumer PC" have orders of magnitude less GPU power than a "supercomputer"? I'm assuming the latter has some form of GPU farm, whereas I don't know if you can fit more that 3-4 fat GPUs in the former.

So if you'd be able to split the workload such that it would be able to run on consumer PCs in a reasonable time, wouldn't that also split the storage requirements the same way?

Or, if instead you're OK with waiting for ages for the regular GPU to do its thing, is transfer speed that much of an issue?