MoE is a type of neural network architecture, not an approach to lifelong learning. As they say " We propose a new Shared Knowledge Lifelong Learning (SKILL) challenge, which deploys a decentralized population of LL agents that each sequentially learn different tasks, with all agents operating independently and in parallel." What they propose is effectively an MoE with separately learned routing as far as I can tell, with the heads of the LLM being task specific and there being a task mapper to assign training labels. The baselines based on Parameter-Isolation methods are sort of similar MoEs, in that additional weights are added and trained on each tasks (somewhat as experts would).