Hacker News Reader

Comment by sinenomine | Hacker News Reader

Can't help but think about the obvious: maybe we should just skip all the drudgery of the natural convulsive progress and ruthless competition, push "accelerate" and get to the mature AGI civilization regime as soon as possible?

And there we should just ask our new creative oracle machines the obvious: "devise an understandable minimally invasive method of uplifting our own cognition and extending our biological lifespan". Surely at least some gene/cell/whatever therapy which does what we need is theoretically possible - the human beings aren't special enough for this not to be the case.

At least that's the zeitgeist I perceive among others who understand the significance of the unfolding progress.

View on HN

Pushing "accelerate" costs a great deal. Investors will want a return on their capital: either in money pressed from unequal access to the resulting paradise or in direct power over this new god.

With billions upon billions spent on Big Science, asking the government to spend just 1 billion on an AI training run with guaranteed spectacular results doesn't sound too outlandish.

Why should we pay taxes, if the government won't even train AI for us, given such opportunity?

AGI is an existential threat to human-led governments. Why would they build something to displace them?

We all know that analogies from Hollywood classic scifi movies aren't productive or even at all useful in our mundane reality. The general-purpose AI is going to be a very useful, obedient, capable tool, and the great powers are going to seek ownership over it, like they do with various military and space technologies.

Current safety techniques are far from sufficient for accelerating AGI progress to be a good thing.

AI safety discourse is mostly still in 2000s, with outdated intuitions informing loads of rigid concepts specific to this time and community.

Large language models trained on cross-entropy loss for next token prediction aren't going to be dangerous per se, even in the limit - simply due to the nature of the objective and the distribution they are set to approximate.

We should make sure these powerful tools don't fall into a small exclusive set of power-hungry interests, though. Transformative tools such as these should benefit all of humanity and let it prosper.

If large language models trained on cross entropy loss for the next token prediction in the limit aren’t dangerous, then they aren’t enough by themselves for AGI.

An AGI, by virtue of being at least as general as a human, would be able to effectively form a plan for how to act in the world to achieve fairly general goals, in fairly general conditions.

Even if it isn’t agentic, if a program can formulate these plans in a way people can use, then the combined program+user still implements the optimization process.

That's why we have to make sure the user (committee) is safe and aligned. The usual silicon valley tech execs, or east coast financial bros aren't going to cut it this time.

The democratic solution here looks better than the expected rule of the (very) few.

That isn’t enough if the plans which are enacted have unintended undesirable consequences.

Yeah, give me a second, singularity exists.

Reply on news.ycombinator.com