Going a bit off topic here but your remarks on religion makes me curious. I'm from a firmly -- albeit lazily -- protestant country and don't describe myself as religious in any meaningful sense, but still an interesting subject.

My reference to religious "singularities" was to the apocalypse, or however you want to name it. The End of Days. The Second Coming. Stuff like that. I thought these ideas played a similar role in all branches of Christianity, although ignored by most people for most of the time. Is Catholicism and Protestantism so different in that regard?

More on topic, I see the evolution of (better) AI as a logical extension to the computer revolution, and I see the computer revolution as equivalent in scope to the industrial revolution. So I'm not saying it's a small deal. I just don't believe a whole lot in self-improving AI as a concept all that different from what has been going on in, say, processor design, for decades, and I certainly don't believe in the idea of a runaway, self-improving AI.

To become rapidly and exponentially more intelligent, it would have to either work smarter or work harder (to use an overused phrase). Working smarter would imply that if you arranged your CPU instructions in increasingly clever ways, you could extract exponentially rising performance from your programs. That's not how I remember the assembler programming that I did as a kid; you could be clever but there were some pretty hard limits. The strategy of working harder, on the other hand, would imply that someone would be willing to supply exponentially rising amounts of energy. Again, even if some Zuckerberg or some shady government agency would be willing to play this game, you'd quickly run into hard limits.

So yeah, AI is not the first game changing phenomenon in the history of humankind but it will change stuff. Like did bronze, railroads, or nukes.