om8

Not if you use a language that bans implicit type conversions (like go)

> What i say to people for at least 2 years now, is that "Remember when governments were not just some cryptographic algorithms?" Yeah, that's gonna change. Cryptography is here to stay, it is not as dead as people think and it's gonna make a huge blast.

The thing about cryptography and government is that it's easy to imagine for a great technology to be adapted on the governmental level because of its greatness. But it is another thing to actually implement it. We live in a bubble, where almost anyone knows about cryptographic hashes and RSA, but for most of the people it is not the case.

Another thing is that political actors are tending to try to concentrate power in their own hands. No way they will delegate a decision making to any form of algorithm — being cryptographic or not.

Don't you think that LLM inference is already very democratic? Not saying that there is no room for improvements there -- there is still a lot to do in the space of speculative decoding, quantization and other stuff. I'm saying that every 16 year old with a decent enough personal computer can run fully locally latest open weights model like Llama3-8b that beats almost everything we had a year ago.

The part of this ecosystem that is as non-democratized as it can be is training. It's currently impossible to train decent enough model with resources that are available to one person.

P.S. They acknowledge this: “... our preliminary experiments show that such method might not be helpful, and even harmful, to open-ended text generation tasks. During generation, ∞-gram can make odd mistakes (e.g., predicting totally irrelevant tokens) which makes the model to digress. Thus this combined model is not ready to replace neural LMs. Additional investigation is required to make inf-gram best contribute to text generation.”

Perplexity results are impressive. I wonder, how does combined models perform on MMLU and other problem-solving benchmarks. It can be that this infini-gram method is exactly the thing that “hacks” perplexity without adding any “understanding” to the model.

CLIP is a very interesting technology.

On my previous job ML department created internal tool, where you could search through city panoramas (like google street view) using text.

It could find you in a second all road pits, overfilled dumpsters and other ugly (and beautiful) things you wanted.