When I checked back a few years later, the community seemed to be all about surviving the Rapture, I mean “Singularity”, summoning God, I mean the “Friendly AI”, and maximizing the happiness of the expected 10 to the trillion lives in Heaven, I mean the “Simulation”. I found all of that a lot less convincing.
There are several social reasons why this shift happened, but the root problem, to me, is utilitarianism taken to a dangerous extreme. Effective Altruism was arguably always rooted in some form of utilitarianism as a guiding principle. The problem is, the concept seems to have attracted some very logical, abstract thinkers who somehow didn’t realize when they started to lose contact with reality.
Note that all of this is just my opinion, but here’s my point. I find it always problematic to assign a numerical goodness value (“utilons”) to a human life, though I concede that it can work as a heuristic in small, non-extreme cases. What definitely doesn’t work is to take that number, multiply it by population sizes and claim to have solved ethics. No matter how much you wish it was, ethics is simply not a subbranch a mathematics. To put it very bluntly: That road has been traveled before, and it leads to genocide.
To be clear, this isn’t hyperbole. I won’t link any sources to limit exposure, but you can find articles by self-proclaimed altruists who argue that it’s better to lose a billion lives to climate change than to delay AI research by even one year. The suggested reason being that delaying the Ascent into Heaven — I mean the “Upload into the Simulation” — costs humanity a collective 10 to the gazillion utilons, compared to the mere billions of utilons potentially lost to the climate catastrophe. Just think about the implications.