Neurosymbolic Programming (2021) [pdf]

One of the benefits mentioned is better accessibility for existing analysis tools. If I understand the concept correctly, could this also make it easier to implement the approximated functions in environments that lack efficient numerical algorithms and data structures? These two are usually only found in technical computing languages (e.g. FORTRAN, Julia, etc.), or as libraries for a handful of general-purpose languages (e.g. Numpy, Numba, etc.)

In my very old McGraw-Hill book (“C++ Power Paradigms”, McGraw-Hill 1994. Genetic Algorithms, Neural Networks, and Constraint Programming), I had an example using a genetic algorithm for synthesizing a recurrent network. I had lunch with John Koza, he said that it was an interesting idea but that it wouldn’t scale - and he was correct about that.

> The holy grail for research on neurosym- bolic programming would be a general learning algorithm that could efficiently generate arbitrary combinations of neural components and symbolic code.

This is one area where the authors may be missing the point of the parallel distributed processing approach. The authors treat ANN’s as blind guesses from massive training data, but they forget that in most cases the choice of training data is the a priori specification.

Additionally, an ANN can be comprised of multitudes of types of artificial neurons. While some learning models are designed to discover relationships between complex data, other learning models are designed for simple data. The simpler the training data, the easier to assign a meaning to the output.

Simple back-propagation was followed by the development of deep learning and complex network architectures. The explosion in interest in deep learning is, in part, due to the realization that complex neural architectures can be comprised of a variety of neural components in a variety of ways and that the resulting neural architectures would almost certainly avail us with solutions that span the entirety of possible problems to solve.

That is to say, deterministic behavior (“symbolic program synthesis”) could be implemented by an ANN, if the ANN had the required minimum complexity.

Since the minimum complexity of such an ANN is currently incomprehensibly high, due to deep learning being in its infancy, there has been a heavy focus on striking results with low complexity.

In my view, instead of trying to bring back the zombie of symbolic programming for AI, the authors’ time would be better spent on contributing novel, highly complex neural architectures.

For example, some biological neurons in the visual cortex have been shown to be responsive to single individuals faces. If there is one neuron per face, then there could be thousands or more of face neurons in the visual cortex.

Where normal deep learning would focus on parameterized feature recognition for face detection, a highly complex neural architecture would focus on multitudes of small models that are trained with minute data sets of individuals per model.

A standard deep learning “face detection” model could then feed the training of the “face neurons”

Layering other types of neural architecture around and within that multitude of models, along with some other neural architecture that generates new “face neurons” to be included in the multitude, would provide both the robustness of deep learning and the determinism of symbolic programming. Each “face neuron” would have an a priori specification of which particular face data was used in its training. One could even imagine further granularity such as “smile neurons”, “chin neurons”, “pimple neurons”, and so on.

That is neuro-glial programming, a subset of Parallel Distributed Processing, not neuro-symbolic programming.

https://www.igi-global.com/chapter/artificial-neuroglial-net...