So this is an ensemble of many LLMs?

I wonder how well a bunch of LLMs trained on personal computers, so fairly small, could perform together?

Train a LLM on your emails, train an LLM on a text book, download a bunch of arbitrary LLMs from the net you find interesting, throw them all together into a big pile, and use a moderator LLM that knows how to format their output into an assistant format.

So, the email LLM would try to autocomplete sentences from your emails, and the text book LLM would try to autocomplete sentences from the text book. People could offer LLMs to download, almost as a way of compressing information, download the LLM of your favorite programming language, and TV series, etc. The important part would be having a moderator algorithm that can shape these LLMs from dumb sentence autocompleters (barely more than a fancy Markov chain) into a coherent assistant format. For example, the text book LLM would just endlessly spew semi-random sentences from the text, but a good moderator algorithm could see that it has sufficiently answered the question and cut it off.

In short, it's interesting that separate LLMs can integrate with each other and strengthen each other and it makes me wonder if we could build modular LLMs.