The practical question of interest is: will this make it possible to run GPT-3 size models on normal desktops with GPU? Like Stable Diffusion.
At some point the practical question would be how do you get all the data onto the desktop.
People are already downloading 100GB games. And data rates are growing much faster than RAM capacities. The logistics of downloading a model smaller than GPU RAM are unlikely to ever get complicated.
Using bittorrent, of course :)
Once AI models become usable on desktops without esoteric hardware, their datasets will go the way of all large media which people have been distributing for many years.
Storage isn’t much of an issue these days. The bottleneck is computing values from RAM, fp8 can improve this drastically
I feel like the fastest tier of ram can only get so big before speed of light delays become relevant.