Neural Nets vs. Cellular Automata | Hacker News Reader

Neural Nets vs. Cellular Automata

nets-vs-automata.net

83 pointstodsacerdoti12 comments

I know it's not the same idea, but I think it's worth mentioning the adjacent concept of 'neural CA':

https://www.neuralca.org/

https://google-research.github.io/self-organising-systems/di...

https://google-research.github.io/self-organising-systems/is...

I can see why Mordvintsev et al are up to what they are doing, but to be honest I'm struggling with understanding the point of using a neural-net to 'emulate' CAs like OP seems to be doing (and as far as I can gather, only totalistic ones too?).

It sounds a bit like swatting a fly using an H-bomb tbh, but maybe someone who knows more about the project can share some of the underlying rationale?

I'm not involved in this project, but I partially replicated the results from Mordvintsev et al. a few years ago because I found the idea interesting. The key idea for me was learning the possibly unknown rules of a CA from training examples. This sounded to me like something that could be useful in science, to learn about spatially distributed processes. Or in ML as a new idea for image classification or segmentation. The hope was always that a CA could be learned which would have a simple discrete representation which could then be used in inference with much lower computational needs than a full neural net. But unfortunately we never managed to succeed here, and I have the impression that this area is not as active anymore as it was some years ago.

I suppose the idea of this project is the same: show the correspondence between both in order to understand them better.

Anyway, some interesting papers from back then:

Cellular automata as convolutional neural networks: http://arxiv.org/abs/1809.02942

Image segmentation via Cellular Automata: http://arxiv.org/abs/2008.04965

It's Hard for Neural Networks To Learn the Game of Life: http://arxiv.org/abs/2009.01398

I think the biggest advantage NNs have over CA is the fact that most CA only provide localized computation. It can take a large number of fixed iterations before information propagates to the appropriate location in the 1d/2d/3d/etc. space. Contrast this with arbitrary NN topology where instant global connectivity is possible between any elements.

CNNs are CA if you don't insert fully connected layers, actually.

Yep, see my comment above and especially http://arxiv.org/abs/1809.02942

You also need to make the CNN recurrent, allow it to unfold over many steps, ensure input and output grid are same size and avoid non-local stuff like global pooling, certain norms, etc.

Either way, parent comment is correct. An arbit NN is better than a CA at learning non-local rules unless the global rule can be easily described as a composition of local rules. (They still can learn any global rule though, its just harder and you run into vanishing gradient problems for very distant rules)

They are pretty cool with emergent behaviors and sometimes they generalise very well

I don't get it, does the prediction go backwards or forward along CA generations?

it has the signature style of an app generated from the claude web ui. There isn’t necessarily an it to get.

Exploring the site, the about page and the related links made me quite confident this isn't just vibe coded with claude.

It seems like a passion project and a niche interest by the author.

Nice website, but the "vs" is a bit misleading here. Unless I'm missing the point?

I clicked in the hope that it would tell me something about how CAs can be 'trained' and 'used' to make useful predictions somehow.

Instead, I got a neural network which is trained to predict the t+3 step of a CA based on an initial state.

Am I missing something?

Any non-trivial property on CA is undecidable in any dimensions, IIRC you resort to Medvedev reducibility, oracles, etc... pretty quick with them.

IMHO this is semi interesting because having ANNs predict the outcome of deterministic dynamical systems may help with some planning tasks.

intersting idea to do it in a distributed way with people help.

Reply on news.ycombinator.com