From your position, catastrophes will be the fault of whoever was dumb enough to put a probabilistic system in charge of critical infrastructure. Totally different fear than runaway intelligence and takeover.
From your position, catastrophes will be the fault of whoever was dumb enough to put a probabilistic system in charge of critical infrastructure. Totally different fear than runaway intelligence and takeover.
The AI labs use reinforcement learning to train LLMs, and have spent a while now giving each new generation of LLM an increasing role in training the next one. If not quite an "intelligence explosion" as yet, it isn't unreasonable to already describe it as an "intelligence conflagration".
"AI psychosis" is also essentially a reinforcement loop, on how much LLM output makes users feel warm fuzzies about how competent the AI is, which was a proxy for (and yet actually different to) how competent the AI really is.
(Aside: "a probabilistic system in charge of critical infrastructure" was never not true, this is one of those phrases people use rhetorically, but it's kinda a thought-terminating cliché given how often human thought randomly fails us. The difference is which things fail how often, not that it happens at all).