It can be fixed, and "Google" referenced as "a whole lot of intelligent people" could fix it. Easily.
They just don't have any massive advantage to do so.
Their goal has ways always been capture. Keep us on their portal. Well they're succeeding, because they fill pages of response with their own data, and the alternative is to click poor results.
They have a vested interest to keep you on their page.
It doesn't matter that their buffonerific answers are often gibberish, and so many times very wrong. Or worse subtly wrong.
Those answers keep 99.9% of the people on Google only.
And just take another look, too. How many of the responses aren't just their horrid AI junk, but many of the rest are youtube.
Why spend time fixing main search, when it being broken pushes people to stay on google.com, or go to youtube?
They can fix it, they just don't want to.