There are some crazy 1/1.5 bit quants now. If you're curious I'll try to dig up the papers I was reading.
1.5bit can be done to existing models. The 1 bit (and less than 1 bit iirc) requires training a model from scratch.
Still, the idea that we can have giant models running in tiny amounts of RAM is not completely far fetched at this point.