Speed, assuming you have the RAM to have it all loaded, it's faster than a fully connected network of the same size by 4x
It's better if you're hosting inference, worse if you are using it for a dedicated purpose. Presumably in the future it might make sense to share one local MoE among the different programs that use it, especially for a demand-heavy application like programming.
The model is quicker to evaluate. So quicker responses and more throughput.
It's faster to run inference and training. Less memory bandwidth needed.