Hacker News Reader

> No, programming and math is basically what AI research is.

Most importantly, in these fields results can be easily verified which means that if you have enough compute, that you can generate correct training data. OpenAI had an early story - which could be a lie of course - about how they tried to train a robot but the actuators would be toast before the robot would have learned. Math and programming doesn't have such problems.

There is a hierarchy. Do robots need software or does software need robots?

There are lots of fields with easily verifiable outputs but that doesn’t make them valuable targets.

Eventually you’re going to need robotics to kick off the singularity, sure, but I would start with programming and math (and compute) which is exactly what they are doing.

Let’s say you start with robots, like we did, but we lack compute, we lack dev tools, we lack the ability to generate vast training environments, we cannot automate mathematical research of any kind in any way shape or form not even trivially, in general we are bottlenecked by math, code and compute. “Doing robotics” first without tackling the others is taking an astronomical detour which we as humanity are fond of.

I find the notion we automate cognition itself first and then tackle the secondary problems quite rational.

> RSI is the play and it’s already happening.

Sure, if you redefine what that means.

Before this year, "recursive self-improvement" in the context of AI research meant the holy grail: an artificial intelligence that was capable of improving itself, unattended, in ways that humans can't even imagine, so that within a relatively short period of time, it reaches superintelligence and ignites the Singularity.

What's happening now is emphatically not that. And if you think it is, you are buying straight-up propaganda. "I gave an LLM a tool that lets it make largely-random decisions about how to train its next iteration when I tell it to do so" bears little to no resemblance to RSI. Because it is, beyond a rounding error, completely missing out on the "improvement" part.

> It’s completely obvious what the play is here and it has nothing to do with pay. We pay peanuts for tokens so automating a developer for 200-500 a month worth of tokens ain’t getting you to a trillion a year.

Could be hallucinating by I just asked Gemini to estimate global average compensation of developers and multiply by estimate of total number of developers in the world, and it came up with $1 trillion to $2 trillion.

So the total addressable market is the right order of magnitude, let alone all the other automatable high wage occupations..

That would be interesting except nobody will pay 50-100k for tokens not even if they would supplant the humans completely.

Market forces drive it down hard. The USA is not the only player. Open source models are not vastly behind. My guess it that the 100-200 range will remain the baseline no matter the capabilities.

Exactly. AGI is explicitly the goal, where AGI is defined as an AI that can improve itself autonomously. Training an AI to make improvements to itself is training an AI to program.

> RSI is the play and it’s already happening. Our work is easier and faster and anyone that claims otherwise I have a very hard time taking serious.

No it isn't. Doing work more efficiently through the use of LLMs is not RSI. RSI requires a closed loop automated LLM iteration process which we are not at all close to right now.

Correct, the first thing that needs to become obsolete for RSI are the AI labs themselves.

A closed loop is the final destination, it’s not the starting point. We’re working towards it.