Hacker News Reader

Comment by Causality1 | Hacker News Reader

Personally I think we're going to need a revolution in the fundamental physics of computation. The example I like to use is that a dragonfly brain uses just sixteen neurons to take input from thousands of ommatidia and track prey in 3D space, plot intercept vectors, and send that data to the motor centers of the brain. Calculate how many transistors and watts of power you'd need to replicate that functionality. Now multiply that number by how many neurons you think it takes the human brain to generate sapience.

It doesn't really matter what your guesses are, none of the results are good news.

View on HN

I wonder if there isn't some fundamental misunderstanding here. What if it's not "just the neurons". If you found a Regency TR-1 radio you could wonder "how can this 4-transistor device produce a continuous stream of music, much like Spotify, which requires billions of transistor to run?". Of course, the radio also has an antenna, which is a completely different device than a transistor.

The device running Spotify may also have an antenna, but I hope you get the analogy. My analogy is not meant to be taken faithfully, so that we need to start looking for antennas now instead of neurons. I am just saying that maybe the neuron-counting game is not the only thing. Maybe there is something else -- not magical, not divine, but physical and as-of-yet unknown. Humanity didn't always know everything, and maybe still doesn't.

Exactly my point. If all you want to do is replicate the TR-1 with four transistors, that's easy, just like making a human mind by creating a baby is easy. But making AGI with silicon, while demanding functionality completely alien to a human brain, is like making a TR-1 that can save your playlists and pause/resume the audio while still only using four transistors.

> The example I like to use is that a dragonfly brain uses just sixteen neurons to take input from thousands of ommatidia and track prey in 3D space, plot intercept vectors, and send that data to the motor centers of the brain.

Human optic nerve can't send more than ~10Mbit/s. Yet, somehow, 60fps at 640x480 screen isn't best possible movie watching setup for one-eyed people, even though it delivers uncompressed 9Mbit/s.

Lots of calculations (like aggregating data to lower-quality image; eg. input of human rod cells is aggregated through interneurons) happen around of body. 16 neurons that you are referring to are likely fed with carefully processed input, not raw input.

I tend to think in similar terms. There's so much going on under the surface with even the simplest creatures in the natural world that the physics and computational fundamentals seem really intimidating here. That's not to say that we could never get there -- certainly, many hold out hope for our abilities continuing to compound over time. But it's kind of a bummer to think about the glimmer of true AGI only materializing much further along an exponential growth curve that, to me, doesn't seem guaranteed to continue indefinitely.

We don't need the first AGI to be human efficient. Nobody would mind if it would require 10 data centers and a nuclear power plant to run.

ENIAC also started big and slow. Now it fits in a microSD card.

> Calculate how many transistors and watts of power you'd need to replicate that functionality.

I'm curious as to your answer. Because if one's building a purpose-built analog computer for the task, my estimate is a few hundred transistors, a few thousand passives, and ... an absolutely trivial amount of power on modern process.

I'm curious how we're even going to manage 420,000 pixels' worth (60,000 ommatidia, approximately 7 pixels each) of input with only a few hundred transistors, let alone do vector analysis on it.

But let's say we can. Let's say we need 320 transistors, which would be 20 transistors per pixel. That's pretending 99.7% of the seven thousand synapses each neuron has are useless for our purpose, but we'll do it. A chimp brain runs all the autonomous physical processes of a humanoid body while only having 22 billion neurons. We'll also pretend, wrongly, that chimps have no mind or emotions at all and that we only need the extra human neurons to make a sapient mind.

Humans have 86 billion neurons. Subtracting 22 gives us 64 billion, times 20 transistors per neuron gives us 1.28 trillion transistors.

1.28 trillion transistors, even with a bunch of handwaving to make it easier, and even pretending we exactly understood how sapience worked in the first place.

> I'm curious how we're even going to manage 420,000 pixels' worth (60,000 ommatidia, approximately 7 pixels each) of input with only a few hundred transistors, let alone do vector analysis on it.

If you define the problem as importing 420,000 pixels, and target recognitions, and vector analysis, then you need a whole lot more computation than the organism uses. But presumably you're going to also get better results. We both know that's not exactly what's happening, I think.

That is, we know we can solve similar tracking problems with a whole lot less state.

> That's pretending 99.7% of the seven thousand synapses each neuron has are useless for our purpose

Not really... I think we can imagine a whole lot of passives / linear operations involved, along with the big nonlinear processes we need transistors for.

We're also assuming there's no net benefit to cognition that can happen using transistors, I'll note-- e.g. they have a ton of bandwidth compared to neurons, can be multiplexed more readily, etc....

> Humans have 86 billion neurons. Subtracting 22 gives us 64 billion, times 20 transistors per neuron gives us 1.28 trillion transistors.

So about half the number packed onto Cerebras WSE-2 today.

> even pretending we exactly understood how sapience worked in the first place.

This is the big problem.

> 1.28 trillion transistors

So, basically, 45 x RTX 3080?

Feels like we're handicapping ourselves, at least in this specific domain, with digital computing.

Reply on news.ycombinator.com