I don't think widespread polyamory, group homes, radically different social norms around language use and conversation, or reorganizing your entire life around mitigating AI risk are "completely normal."
I don't think widespread polyamory, group homes, radically different social norms around language use and conversation, or reorganizing your entire life around mitigating AI risk are "completely normal."
But I don't know, clearly smart and educated people can believe extremely dumb things. The smarter they are the weirder the setups they get themselves into. One almost comes to appreciate the existence of some religions with a history of many centuries to "chill out" and see what works, instead of any of the new age cults.
Utilitarian consequentialism is the most evil of all philosophies. If someone is merely sadistic their sadism will be sated after some finite amount of abuse. If someone is greedy then their harm will at least be limited to what they can profit from and there is no profit in ruling over a cinder. But the harm possible by someone convinced that their actions are good in some abstract sense is without any bound.
The irony in the lesswrongers fixation on consequentialist morality is that their great fear of machine superintendence is derived from an expectation that its evil will arise from the same sorts of reasoning they engage in themselves. They simply fail to pause and ask "Are we the baddies?"
I usually leave it out of my complaints because they're so ineffectual that it's not a real threat. But the LW solution to "unaligned" machine intelligence is to create an AI god in their own image first, so that it can enslave humanity and all other lifeforms within its lightcone for their own best interest, and suppresses the creation of any competing "unaligned" God. So they're literally out to create the very thing they fear, but with the hubris to imagine that if it was theirs it would be "good". The ultimate fantasy of both the utilitarian consequentialist and most authoritarian mass murderers. If their doomsday fears come true my bet is that it will be at their own hand or that of their followers.
It's also the ultimate horseshoe for militant internet atheists-- "there is no God; and this is OK" becomes "there is no God; and we're gonna create one in our own image".
I'll give the rationalists that: they were out talking about AI ethics and risk way before AI risk was cool. Total hipsters.