FreeToken: Datacenter-class intelligence on the hardware you own

Source: https://github.com/FlashML-org/FreeToken

> FreeToken is an edge-native Mixture-of-Experts (MoE) serving engine designed for running frontier-scale open-weight models on personal and consumer hardware. It treats heterogeneous edge resources—GPUs, CPUs, host memory, and interconnects—as a unified, elastic inference platform.

Nothing new, the models have to fit in RAM+VRAM contrary to the expectation you might have from the title.

Sounds almost too good to be true. Has anyone tried it?