Hacker News Reader

The only things currently consuming resources are humans and at the rate we are consuming resources we would need 2 Earths to support that rate of consumption. Worrying about self-reproducing AI is putting the cart before the horse. No one has yet figured out how to make existing humans aligned with biospheric conditions which would be sustainable. I personally welcome any form of intelligence which can make a sustainable civilization.

View on HN

I am sympathetic to the point. I was somewhat concerned about nuclear war with Russia's Ukraine invasion, and I've become more concerned about climate change over time. I'm hoping either solar continues to expand - woot! - or we get over our fear of nuclear, or fusion finally becomes viable.

My point still stands - an AI that is reproducing (and/or mutating) by any method is going to be subject to selection pressure, just as any biological process is. All ML systems are effectively under selection pressure from the training process, and so they learn unexpected and mostly undesirable optimizations like "use a game-engine glitch to gain infinite flight and go to the goal without solving the maze" or "read text off the scanned X-ray sheet to get additional clues as to the correct diagnosis".

Reply on news.ycombinator.com