Hacker News Reader

Comment by Animats | Hacker News Reader

It's amusing to read people in the past writing about the prospect of superhuman intelligence. The real problems have turned out to be different. Sycophancy and hallucinations, which are part of being confidently wrong, remains a big problem. Needing square miles of data centers was an issue in 1950s science fiction, and disappeared by the 1980s. Yet now they're being built, with private funding and the prospect of profit. The need for way too much training data indicates something is still wrong with the current approach.

None of that was predicted.

View on HN

I predicted on this site in 2016 the massive social and economic impacts AGI would have and specifically when RL data loops are not available to anyone but major players:

https://news.ycombinator.com/item?id=12168228

I even wrote up a whole article that specifically called RL loop based development as the future:

https://medium.com/@andrewkemendo/the-ai-revolution-will-be-...

> Reinforcement Learning tasks rely on ridiculous amounts of data. Whereas with traditional software architecture, where you accomplish tasks through explicit task instruction, RL trains for tasks based on millions of tests through a reward system. Most importantly once you have trained it to some minimum level, if you deploy it correctly, then it should continue improving — so long as you bake feedback into the UX. Imagine that instead of telling excel what to do, you and every other user will have a conversation with excel, improving the system incrementally.

Agreed, and this is exactly what we see happening. Your posts back then were prescient ... there's literally now 'Copilot for Excel' and 'Claude for Excel' etc. But what do you propose the people/commons can still do at this stage to redistribute the inherent power found in RL data loops to a more stable equilibria of sharing participants?

Great question and there’s two steps in my opinion:

First is to become as free as possible from lock in and own your own data. The best way to do this is the self host your own technology.

This is really not possible for the majority of people though.

So practically I always suggest that you have multiple providers for services, don’t pool your data any one place (other than your own place) and own your backups. This is basic stuff that we’ve been teaching since the 90s and still very applicable today.

The harder and more impactful thing is to then create community owned technology that is outside of the commerce model.

So for example imagine that instead of FAANG running the world, the largest tech and data orgs would look more like wikimedia foundation, Annas archive, scihub, Graphene, Linux etc…. and more generally that technology and governance are open and not bound to commerce/taxation/coercion based organizations.

Ultimately we need to create a democratic-technology movement such that capitalists don’t monopolize technology, which is currently the trend. This is not some kind of simple thing by the way, this is revolutionary economics is what I’m talking about.

My suggestion is to read Post-Scarcity Anarchism by Murray Bookchin

> The pressing ethical questions in machine learning are not about machines becoming self-aware and taking over the world, but about how people can exploit other people, or through carelessness introduce immoral behavior into automated systems.

I think the main point still stands. (And there have been some pretty prescient depictions, e.g. Marvin the paranoid android was a pretty spot-on prediction of Bing. Or perhaps the fact that Marvin was in the training set was what led to Bing?)

We don't really have superhuman intelligence yet, as in way ahead. The current stuff is ahead at some things and bad at others.

Some sci-fi content differentiates between Artificial Intelligence, and Virtual Intelligence, where AI is a true "human-like" general intelligence that (often) has a sense of self and is capable of deriving+learning new things by itself.

VI is close to what we have now, software that has some fixed intelligence, it can only really imitate what it has been taught and is not very adaptable. Useful for kiosks, drones, essentially just a tool rather than something we would see as a separate being.

Reply on news.ycombinator.com