If you’re just about to build a superintelligent machine god and usher in the singularity then what use is an IPO? Why would you want to give away control? You would know that your shares would be getting given away at an enormous discount.
Yeah great point.
The entire AI sector been on the cusp of building “superintelligence” any day now since 2023. Yeah, we’ve seen dramatic improvements since not even those three years ago, but the “I” in AI is a misnomer; they are no more intelligent than any other probability engine. I’m even sure they can obtain primitive forms of RSI, given what we’ve seen of Claude improving smaller models via weight-space programming. The problem is that I don’t see how an LLM could improve itself if it doesn’t already know the thing it wants to know, and so an “RSI-capable superintelligence” would be restricted by the procurement of new training data to solve what it can’t currently solve, which naturally brings in issues of how it can independently create that data.
But does it need to know ahead of time or is it enough to be able to test 10k different directions at once then analyze the results? I have thought the same way in the past, that LLMs can't innovate due to how they are trained, but many of humanity's most important discoveries (fire, antibiotics, microwaves, etc) were discovered accidentally in the pursuit of something else. Even if the AI can't truly invent, all it needs to do is generate and test hypotheses then analyze to see if there is and there there for a human to check out.
"it's really dangerous, you better believe me!" screams a frantic Dario, pointing wildly with a gun "I'll do it, I swear I will. Just make me a trillionaire and no one gets hurt".
> Anthropic warns that it's close to RSI capability