We need to start talking about AI safety and nationalizing AI research like yesterday. ChatGPT is already having destabilizing effects and it is frightening how little we are discussing (this hugely powerful tool)'s potential as a weapon.
EDIT: also manipulation of wikipedia articles using proxies and hard to verify sources.
The downside of course is even generally accepted credible websites still display tremendous bias, which leads to tremendous bias in Wikipedia in general as editors pick and choose sources and silence dissenters.
As to why that is dangerous, there are many reasons.
1. It devalues human life for those in power. Technology has strictly increased wealth inequality over the last century and this takes it to the nth degree. 2. Even in its current form, it is having society destabilizing effects, go on reddit and see posts from high school students asking what they should even study when its clear ChatGPT will be able to do (research, programming, math) better than a degree will prepare them to. 3. Google the paperclip problem. 4. The amount of computing resources it takes to run ChatGPT is shockingly / absurdly low, we are far far from the hardware scaling limits of AI so it is obvious that it will continue to improve, even without further algorithmic breakthroughs.
Fundamentally disagree with you there. This is a natural language model, it is certainly not an AGI. That's why it gets things wrong so often. When humans converse with each other, there is a pattern to it, and this AI is simply very good at mimicking that pattern.
To our ape brains who have only ever known how to judge sentience by how well something communicates, it presents as very life-like. And there are phenomena happening in that network that might even be considered "thought". But it's not an AGI, just a building block toward one.
I believe an order of magnitude stronger ChatGPT is an unacceptable risk to us all, it will let those who own and control it wield power which our government shows no sign of being able to regulate. We don't allow private research and ownership of nukes...