Hacker News Reader

Comment by teej | Hacker News Reader

You’ve just invented Universal Paperclips.

Meh, as long as they're not designed to improve themselves, and there are strict protocols for ensuring all active units are controllable, it's not necessarily a recipe for solar system paperclipization.

Given enough units any protocol will be broken by pure chance sooner or later.

With appropriate organization and accounting, there's no reason malfunctioning units couldn't be repaired or reprocessed. It seems like that would be a sensible regulation for companies building self replicating machines.

Ah, but if they’re in space who’s doing the regulation?

> there's no reason malfunctioning units couldn't be repaired or reprocessed

If you have enough of them, and a long enough time scale, they might mutate in such a way to avoid being detected.

It's an important concern, but I think mitigatable - especially since we're not neccessarily talking about actual self contained replicating machines, and instead more likely automated factories producing drones - a multi step process where everything has to go right for replication, and into which safeguards can be engineered/studied. Basically, the process needs to be designed to fail due to entropy without external intervention/maintenance.

If we become advanced enough to consider building such things, we can engage in a serious investigation of the risks - for now it's just speculation.

Reply on news.ycombinator.com