Hacker News Reader

I think these models would get interesting at extreme scale. Generate a novel in 40 iterations on a rack of GPUs.

At some point in the future, you will be able to autogen a 10M line codebase in a few seconds on a giant GPU cluster.

Diffusion LLMs also follow scaling laws - https://proceedings.neurips.cc/paper_files/paper/2023/file/3...

Is it possible that combining multiple AIs will be able to somewhat bypass scaling laws, in a similar way that multicore CPUs can somewhat bypass the limitations of a single CPU core?

I’m sure there are ways of bypassing scaling laws, but I think we need more research to discover and validate them

Those aren't the modern type with discrete masking based diffusion though.

Of course, these too will have scaling laws.

I read a wikipedia article of a person who was very intelligent but also suffered from a mental illness. He told people around him that his next novel will be of exactly N number of words and it will end with the sentence P.

I don't remember article. I read it a decade ago. It's like he was doing diffusion in his mind, subconsciously perhaps

Seems pretty easy to achieve if you have text editing software that tells you the number of words written so far?

Reply on news.ycombinator.com