If you look at state of the art pre-trained transformers, they gobble up unfathomable quantities of dense textual data that far exceed what even the most learned and smart humans could traverse in centuries. Yet, their results are barely intelligent.
The human neural network, on the other hand, stores and is able to recall an estimated average of a few bits per waking second for a decade or two, barely a few GB of useful storage [1]. A great deal of effort and substance of the brain is dedicated to compressing the sensory input into an internal representation useful for rational though - a problem much simplified for a text-fed neural network. The ability of blind and deaf people (or even blind-deaf) for rational reasoning show these ancillary tasks are not intrinsic to human reason.
So things seem to indicate that what we lack is not hardware, but the right architectures and algorithms to unlock human level reason, the exceptionally tight self training loop of humans that can make sense of the world and even push humanity's understanding further using only a few GB of compressed training data.
[1] LANDAUER TK. How much do people remember ‐ some estimates of the quantity of learned information in long‐term‐memory.
Actually this is untrue. Yann Lecun himself said it. A kid at the age of 6 has had much much more data go through his brain (based on the visual data) than what any transformer has ingested. Moreover, words are already compressed data with a lot of lost information, nowhere near the raw experience about the world that our brains have. On top of that there's the evolutionary stuff.
That's exactly the point, 99.9% of the information processed by the 6 year old has went into training his visual cortex, a subsystem not necessary for reasoning. This will eventually be used 8 hours a day for transforming visual representation of text on a computer screen into a compressed textual form which the inner rational loop can easily manipulate. The holy grail of AI, for anything but artistic creation, is just this inner loop.
But are they necessary for reasoning? Records of deaf-blinds with IQs a few standard deviations above average seem to suggest even highly restricted sensory inputs can allow for rational thought. It stretches credulity that you could convey more than a few bits of meaning per second to a child purely by hand gestures and touch, let alone the exabytes claimed.
Of course someone like Yann Lecun will say scale is the only solution, he leads one of the juggernauts that uses scale as a competitive moat. We shouldn't expect AI researchers in positions of power to be any less vane, power hungry or human than any other humans.
For you and me, a verbal description of a pattern test such as "You are given seven squares. Square A contains a filled cross in the upper left position, an empty circle in the middle... etc." would send us grasping for pen and paper. But with training many people can play mental chess, there are famous blind theoretical physicists and so on. This all seems to suggest that natural language could be called "Turing complete", to bastardize the term, for describing the real world to a rational entity and formulating solutions to real world problems.
Evolution definitely still has us beat on optimization-work put into thinking.
Some implemented hacks:
- emotions as shortcuts for reasoning
- learning through imitation
- self-regulation through relation
Active dendrites the that can do operations like xor before anything reaches the soma, or use spike timing, once again before the soma are a couple of examples.
SNNs, or spikey artificial NNs have been hitting limits of the computable numbers.
Riddled basins as an example, which are sets with no open subsets. Basically any circle you can draw will always contain at least one point on a boundary set.
https://arxiv.org/abs/1711.02160
This is a much stronger constraint than even classic chaos.
While I am sure we will continue to try to make comparisons, wetwear and hardware are cheese and chalk.
Both have things they are better at and neither is a direct replacement for the other.
It's possible that I'm not aware of some breakthroughs on this topic, though.
Living beings are more "Rube Goldberg" than "Swiss Watch" - by virtue that they came to being guided not by intention, but circumstance.
On many occasions to get ambitious things to work despite a lack of good support, special subsystems were created with whatever wacky solution could be made to work, wrapped in sane surface API’s so the system as a whole could remain organized and accomplish the task well.
Five or so years after any of these were created, they could have easily been replaced with more reasonable code. But they worked reliably and efficiently. Their API’s were sensible and stable. So they accumulated.
If you reviewed all the code as a whole for the first time it was like finding a secret circus.
https://lozierinstitute.org/dive-deeper/the-newborn-senses-s...
The genes randomly mutate, and their result is subject to fitness testing.
The fascinating thing about evolution is that it's not actually necessary. The "unevolved" creatures persist just fine. The continued survival of cockroaches and whatnot proves that evolving to human levels is superfluous.
The Sperm carries the requisite DNA to activate the egg, which then just executes the body's Replicate() method. The sperm being required is so people don't spontaneously get pregnant.
The DNA coding for proteins is just to build the replicate() method, which then starts copying attributes from the host's (read: mother's) body. Much like how there are differences in compression/expansion algorithms (zip, tar, gz, etc..), the sperm that initiates the replication/unpacking adds non-neutral variance, hence the genetic traits of both parents present in the child.
Put a different way, why reinvent the wheel if I could just grab the memoized visual processing from the parent? Then, if a trait isn't present on the host, say, mid-digital hair, I (as the replication algorithm), "wouldn't know what I wouldn't know" which could perhaps be seeing as synonymous with "evolution."
In this way, DNA does control the blueprint for the brain, down to the initial generic wiring scheme of the cortex - six layers of differentiated neuron types, with a specific pattern of inter-connectivity. As a rough analogy, you could think of this DNA-controlled initial wiring scheme as something like an untrained transformer... the architecture is there, but won't do anything useful until it has been exposed to a lot of data, which will complete the wiring scheme (synapses = brain's model weights).
The brain is going to start to learn as soon as it has sufficiently developed, which is certainly before birth, and newborn babies have been shown to already respond to things like their mothers voice which they were exposed to in the womb.
Intelligence - ability to learn - is different from knowledge. A newborn baby has very little knowledge, but it does have intelligence since that comes from the brain architecture, hence ability to learn, that our DNA "encodes".
But I have no idea what is known nowadays.
Computers are already great peripherals for brains. Maybe brains will become peripherals for computers or become completely replaced.