Neural networks were basically negatively critiqued for decades. I think there is something to cellular automata and there's definitely a relationship between CAs and DNNs that is unexplored.
There might be a relationship between CAs and DNNs, but if you find it you can be sure that Wolfram will take the credit for it.