neurallambda

"AI heats the planet"... really? You mean marginally?

I'll assume you're asking in good faith. Using NNs allows this project to stand on the shoulders of giants: philosophically, mathematically, programmatically, but also I expect this to plug in to OSS LLMs, and leverage their knowledge, similarly to how a human child learns in a Pavlovian/intuitive response, and only later starts to learn to reason.

Wrt inefficiency, training will be inefficient, but the programs can be extracted to CPU instructions / CUDA kernels during inference. Also, I'm interested in using straight through estimators in the forward pass of training, to do this conversion in training too.

Cyc looks cool, but from my cursory glance, is it capable of learning, or is its knowledge graph largely hand coded? Neurallambda is at least as scalable as an RNN, both in data and compute utilization.

It can get this simple example right if it does chain of thought. If you ask it to just output the answer without answering other bullet points, it will very likely not get it right. Chain of thought is duct tape to actual reasoning, and the errors/hallucinations compound exponentially. Try to get chatgpt to reason about concurrent state issues in programming. If it's not a well worn issue that it already memorized, it'll be useless. It's also perfect, for instance at Advent of Codes that it's memorized, and near 0% accuracy on new Advents.

I'm trying to tackle this problem more head-on, by outfitting LLMs with lambda calculus, stacks, queues, etc. directly in their internals, operating over their latent space. [1]

I'll read your paper, but, LLMs famously fail horribly at "multi jump" reasoning, which to me means they can't reason at all. They can merely output a reflection of the human reasoning that was baked into the training data, and they can also recombine it combinatorially. Eager to see if you've solved this!

[1] https://github.com/neurallambda/neurallambda