Show HN: Capy-flash – a 125B MoE model on two 16 GB consumer GPUs
github.com
github.com
The first one I've seen (although I'm sure there are and will be more as more time progresses...)
It seems to require a bit more system memory (70GiB free RAM as opposed to 32GB) and two newer nVidia RTX 50 series graphics cards with 16GB (as opposed to a larger choice of multi-vendor older single graphics cards with a minimum of 12GB), but on the flip side, it does seem to make up for that with faster token generation speed...
Anyway, it seems to be worth watching...
Strata may have a worthy competitor here!