I'm interested why he's so pessimistic about the simulating a brain approach. Yes it's the boring and obvious approach but it also seems the most direct.
Also found this quote interesting
> Might have to make it illegal to evolve AI strains or an upper bound of computation per person and closely track all computational resources on earth.
First, the human brain is built on a computational substrate that is completely and utterly unlike silicon. It is extremely inefficient to effect computation by simulating a computing model on silicon that is almost pathological for silicon to express. The abstract computational model of the human brain necessarily has an equivalent direct expression in computing hardware we actually have thanks to Turing equivalence. It just may look nothing like a human brain once you build it with algorithms optimized for silicon.
Second, and related, the abstract mathematical nature of intelligence is well-understood and the human brain must be an expression of that. However, there is currently a huge gap between that abstract theory and reduction to practice i.e. our computer science for building intelligence from first principles is severely lacking. There are many things that are easy to express in mathematics that go for decades before some reduces it to practical computable algorithms and data structures. Given the fundamental limitations and inherent complexity of simulating (poorly) the human brain, many people feel that applying a similar amount of effort to this direct approach is much more likely to produce a viable result.
And in any case, a simulated human brain would be completely eclipsed eventually by a more pure design by virtue of being several orders of magnitude more efficient computationally. Simulating a human brain is not a particularly productive detour on the long-term path given this.
Analog CMOS circuits can approach the energy efficiency of real neurons.
Do you have any other resources about serious attempts at using subthreshold FETs to simulate neurons?
Analog neuromorphic approaches to not attempt to simulate neurons, they emulate them in silicon. Partly because of a belief that research in this area is required to produce ultra low power computational devices and partly to explore the real time dynamics of spiking neural networks.
There are very few research groups working on this, but you can look up Kwabena Boahen's group at Stanford. They do large scale real time emulation and are currently building a sub threshold neuron accelerator for the neuro engineering framework from Chris Eliasmith at Waterloo which is famously used to create SPAUN. There is also the Karlheinz Meier group at Heidelberg university which does wafer-scale networks of neurons in accelerated time. Giacomo Indiveri at ETH Zurich has silicon neurons with on-chip learning circuits that the others are missing.
That's nice, but whenever I researched, I only got vague explanations involving recursion and unicorns.
http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.464...
A stacked version of that in a 3D mesh could probably handle quite a bit of intelligence.
>[OpenWorm] project coordinator Stephen Larson estimates it as "only 20 to 30 percent of the way towards where we need to get. (wikipedia)
That said, figuring what we can about how the human brain works and trying to make a computer equivalent seems quite promising. For example modern artificial vision seems to process data in a way similar to human vision even if it doesn't simulate human neurons.
The goal of current HTMs is to capture as much of the functions of neurons and the network (as they are currently understood) within the capability of typical computers and in areas that can be made readily useful such as image processing. For example, feedback from higher levels and motor control are not attempted because it is not yet understood how to incorporate them and binary instead of variable synapses are used because they were determined to be sufficient in the current HTM capabilities.
It doesn't seem like they are even close to a simulation that could accurately model something like C. elegans.
[1] https://en.wikipedia.org/wiki/Hierarchical_temporal_memory
That and the fact that the brain simulation guys don't have anything to show for. There are no human-level tasks that could be replicated by this approach yet.
Forgetting isn't a bug, it's a feature. Forgetting is basically like dimensionality reduction on input data - we extract the principle components/exemplars, remember a weighting, and trash the redundancy. Training a ML model is a lot faster on smaller data, and the same is true for us.
Don't compare ANNs with the brain strictly on a time/time basis. Time isn't the only factor, power consumption and heat production are also factors, and if you include them the brain comes out way ahead.
People with an engineering background almost universally underestimate how freaking awesome biology is. Our brains are self-constructing, self-replicating, self repairing (mostly) hyper-efficient pattern recognition systems. The more we learn about them the more awesome we realize they are. Don't be so arrogant as to assume a few hundred years of engineering will universally eclipse hundreds of millions of years of evolution.
Engineering ability however appears to be growing exponentially. It took millions of years for the brain to reach the place it is now but it only took a few thousand years to get to the moon and create the Internet. I wouldn't count out the power of engineering because biology is complex. Particularly given ever more powerful tools of computation and communication that have only recently (historically speaking, not in lifespans of javascript frameworks) come on line.
- The first part of a sigmoidal curve looks exponential.
- Evolution is massively parallel.
It is also limited by the efficiency that it must attain in order to operate under the energy conditions of our environment.
The energy efficiency of biological systems might be a hint that we should more-directly employ them in the "artificial" minds we build. You're right that the human brain is limited because of its context, but we'll only get superhuman hard artificial intelligence when we're able to build/grow a big one in a vat.
> A neural net like ResNet-150 is capable of doing essentially what 1/3 of the brain is doing (vision).
Is it? There is probably a lot more going on in that 1/3 of our brain than mapping images to words.
Oh come on.
I'd say ResNet-150 is far from solving human vision, in the edge cases.
Can it distinguish between a baseball texture and an actual baseball?
How about between an actual cat versus a toy cat? Sometimes you want the two to be in the same category, other times you don't.
The human can put the same two images in the same or in different categories, based on the higher task at hand. Neural networks are far from that, because they don't (yet) have a model of the world.
My take is that it's a lot of R&D work, and it's not clear which approach will get to a human level first nor which is safer. The emulation approach seems lower variance to me than an intelligence-from-scratch approach, even though there's tons of variance there too. We're just looking at a very uncertain future.
While theoretically possible, I think going with that approach would be admitting we don't understand the origin of general intelligence, so let's just copy the wetvare.
I wonder if nebulous fears about AI soon will be added to the ranks of famous justifications for horrendously overbearing laws, like stopping terrorism, the war on drugs, or thinking of the children.