It’s not necessarily bad. The problem with the singularity is that that we can’t tell if it’s bad or not.
As humans we have a tendancy to make linear future predictions based on past observations. Over the timescales that matter significant effects occur from previously unseen kinds of events that become important via interactions with other events.
The important measurement would be what the length of the event horizon - the length of time before our predictions rapidly decrease to zero certainty. In a singularity we would expect that length of time to decrease close to zero. What has it been historically? I would claim that 5-10 years is a difficult period to make meaningful predictions about. I think that 20 years has proven to be very difficult but possible in the past. I am unaware of any 100+ year predictions that have landed with better than random chance.
Interestingly I think we are entering a period where 5 years will be the upper bound, and even predictions over shorter 2-3 year timespans are going to become difficult.
However I feel that these LLMs are not like the internet or the release of the iPhone. We’ve gone in a very short time from LLMs in the lab to ChatGPT to passing the bar exam.
Considering the rate of research that’s being published and the fact that what we see today is generally at least many months behind the state of the art, I don’t feel that we can even have any certainty about the next 6 months.
Yes, I think that progress has become rapid enough that we can't predict six months out. I suspect that we are only 1-2 inventions away from something quite large and transformative. Obviously LLMs have already created a lot of excitement and opened up new areas of content generation already, but I think something larger is coming.
When people today talk about the AI singularity though it’s a slightly different definition from the more general technological singularity which is that the AI itself is delivering that improvement with no input from humans.
Prediction for the overwhelming majority of things is still reasonably easy, or we wouldn't be able to survive.
We're very, very far from a singularity if it really means that "our predictions about our future becomes close to zero".
If you want to keep surviving you have to be 100% correct in your ability to predict the correct choice. Now the thing is the vast majority of predictions you have to make about anything are not necessarily life or death and/or rapidly changing. LIke you said, cars are still cars, and food is still food.
The problem is the change can be massively abrupt. Watching the use of drones in Ukraine is seemingly a decent example of this. Small cheap drones are being used in mass as surveillance platforms, for artillery targeting, and even for direct enemy bombing. They cost of defending against them seems to be far higher than the cost of using them to attack.
Now this isn't exactly unpredictable in itself. Where is gets more questionable about predictions, is what is it going to look like when AI is given direct control of swarms of these things rather than individuals with an xbox controller? Suddenly decades of battle field planning is out the window and there is a massive shakeup in fighting capabilities.