Comment by reasonattlm | Hacker News Reader

Comment by reasonattlm | Hacker News Reader

MIRI is one of the success stories to emerge from the transhumanist community of the 80s to 00s. Others include the Methuselah Foundation, SENS Research Foundation, Future of Humanity Institute, and so on.

When it comes to prognosticating on the future of strong AI MIRI falls on the other side of the futurist community from where I stand.

I see the future as being one in which strong AI emerges from whole brain emulation, starting around 2030 as processing power becomes cheap enough for an entire industry to be competing in emulating brains the brute force way, building on the academic efforts of the 2020s. Then there will follow twenty years of exceedingly unethical development and abuse of sentient life to get to the point of robust first generation artificial entities based all too closely on human minds, and only after that will you start to see progress towards greater than human intelligence and meaningful variations on the theme of human intelligence.

I think it unlikely that entirely alien, non-human strong artificial intelligences will be constructed from first principals on a faster timescale or more cost-effectively than this. That line of research will be swamped by the output of whole brain emulation once that gets going in earnest.

Many views of future priorities for development in the futurist hinge on how soon we can build greater than human intelligences to power our research and development process. In particular should one support AI development or rejuvenation research. Unfortunately I think we're going to have to dig our own way out of the aging hole; we're not going to see better than human strong AI before we could develop rejuvenation treatments based on the SENS programs the old-fashioned way.

I agree with you in terms of approach that AI will emerge first from brain emulation. I disagree with you on the timeline. I know you say 'starting around 2030', but I think that's a little ambitious.

While I'm an AI/machine learning practitioner now, my recent Ph.D. work was on computational modeling of the nervous system; namely the cerebellum. The reason I say 2030 is ambitious, is because there are still a lot of unknowns to perform whole brain simulation. To start, we need whole brain connectivity or wiring diagrams at an extremely detailed level. There are some efforts that are part of the BRAIN initiative that are taking a stab at this, but I don't think they're be ready by 2030. Second, you have to understand the physiology of these neurons in order to simulate them. This is incredibly complex and poorly understand. While we understand neuronal physiology in general, there are a great many details that vary by cell type. Additionally, you have to capture neuron morphology, synaptic plasticity, the effect of neuromodulators, ... the list goes on. By capture, I mean understand them well enough to describe them mathematically so that they can be simulated computationally.

Until then, traditional machine learning and artificial neural networks will be increasingly useful and interesting.

Seems like it would be quicker to obtain full knowledge of how DNA and cell replication work. Then the simulation could grow a brain without having to fully understand it.

We could grow neurons on silicon chips, or use small tubes that attract axons and dendrites to grow through them (seen a paper about it once) and use them as an I/O interface to a lab-grown brain. We can already grow 5mm size mini brains with human neural cells. It might be more energy efficient and we could take brains to a whole new level.

Going down to modeling at the level of proteins instead of neurons adds a LOT of quantitative complexity - it could be quicker to obtain enough knowledge to start that, but it could easily add 20-30 extra years of waiting for the available computing power to arrive after the already many years we still need to wait for computing power needed for a full brain simulation at neuronal level.

Note that MIRI's current position no longer suggests the development of actual artificial consciousness, just the development of human-equivalent optimization processes. In other words, they argue that you can develop a process capable of solving human-level and harder problems without giving it self-awareness. And that seems like a feature: if you avoid building self-aware machine intelligences, you don't have to worry about what they want; you can build them to only care about what existing sapient beings want.

Keep in mind that this does not sidestep the biggest practical concern with AIs, namely misalignment of values. You don't need a self-aware, conscious being to have a system with wants and values. In context of AIs, it's good to understand intelligence (including that of ourselves) as a very strong, multi-domain optimization process.

I absolutely agree that the problem remains hard. However, it's not so much that you can't avoid building a system with wants and values of its own; it's that you have to implement a system for how exactly to value what we value, especially when there are a lot of us and we don't all share identical values.

Your statement about needing rejuvenation treatments before we can develop strong AIs makes sense. The average age of Nobel Prize winners is increasing over time[1], we could imagine a future where this average age is greater than the average human lifespan. At that point only those scientists who have exceedingly long careers and lifespans will be able to further their respective fields.

[1] http://priceonomics.com/why-nobel-winning-scientists-are-get...

This is the same as saying "we must become immortal before we can become God-like".

"Nobel-winning scientist age" is not a good proxy for "productive scientist age", for a variety of reasons. They mention specifically lag time between discovery & recognition, but you also have issues where the "name" behind the discovery is the guy in charge of the lab, but the actual discovery (and sometimes the idea) is generated by the 30-year-old postdoc / assistant prof / etc.

There is also a sampling bias issue where scientists in academia as a whole are getting older because the boomers still have a death grip on institutional positions, and academia as a whole is shrinking.

"[...] starting around 2030 as processing power becomes cheap enough [...]"

Surely you are not relying on an (arguably naïve) extrapolation of Moore Law-type progress? Recall we are fast reaching the end of it.

    I see the future as being one in which strong AI
    emerges from whole brain emulation, starting around
    2030 as processing power becomes cheap enough for an
    entire industry to be competing in emulating brains
    the brute force way, building on the academic efforts
    of the 2020s.

Computational power is not sufficient for whole brain emulation. You also need to know how the brain works in a huge amount of detail.

For example, we've had fast enough computers to emulate nematode brains for 20+ years but we are still not able to emulate one and have it learn.

>whole brain emulation

we dont need whole thing

http://rifters.com/real/articles/Oliveira-et-al-2012-Revisit...

Reply on news.ycombinator.com