I have a demo that runs llama3-{1,3,8}B in browser on cpu. It can be integrated with this thing in the future to be fully local

https://galqiwi.github.io/aqlm-rs