While this is a MINST classifier (and can be run in the browser) you can get an idea of the math behind it. https://www.3blue1brown.com/lessons/gradient-descent and https://www.3blue1brown.com/lessons/neural-network-analysis
When dealing with a LLM, it's being run again and again - token by token. Pick the best next token, append it to the input, run it again.
If you want to generate 100 tokens (rather small amount of data when compared to much of the GPT-4 conversations), that means running it 100 times.
The network traffic makes the entire system much slower than running it all in one spot.
Consider also the "are you the input to everyone?" and the privacy implications of that.
I think the problem is that it's hard to parallelize this effectively across a network with Internet-scale latency - individual matrix multiplications parallelize very well, but there would need to be coordination of each result, which would be much slower than just doing the computation locally.
In the case of Folding@home, evaluating possible folds could be done completely in parallel and only need to be coordinated on discovery of a plausible fold (which is rare), so distributing over a high-latency network is still beneficial.
I also remember something more directly analogous to @home, but I'm having a hard time finding it.
Edit: Petals! Thank you sibling commenter!