AlpinDale··on Lossless compression of LLMs to 12-bitsA lossless encoding of BF16 weights to NF12, with speedups roughly proportional to amount of memory saved.
AlpinDale··on Over 1k tok/s on an RTX 5090 with Qwen3 0.6BGitHub seems to be down right now, but the kernel is here: https://github.com/AlpinDale/qwen_megakernel (1,008 tok/s now, a bit improved after I made the blogpost)