But isn't something like this true everywhere?
And blaming the individual is futile. I like Scott Alexander's Moloch [0] concept more useful. If you apply a high standard against your own research regarding thoroughness, not overclaiming etc. never looking at the test set etc., you won't beat the benchmark, which contains entries you know are too good to be true. But if you don't beat them, you'll get rejected. The pace is extreme, lots of groups work on very similar things and it's a race to publish very similar stuff first. As a PhD student you need publications. You are in your late twenties, early thirties, you can't afford to lose several years.
The big lie is that there is so much discovery happening as suggested by the thousand upon thousands of papers at big conferences, most of which is never really engaged later on. We have to pretend that all grad students in every group can contributing some actual valuable, bulletproof novel thing to science every few months and pretend that you can have multiple papers at conferences while remaining painfully critical towards your own work. I mean how could it be that people happen to always find something that works out in the end? How can people sit into a project at the start of the PhD and immediately crank out a valuable novel contribution? It isn't realistic, but we have to pretend because this is the background assumption behind a PhD. But the current state of affairs is inflating away the value and prestige of a PhD. It means less and less because everyone knows that it's not just a measure of scientific research skill and knowledge.
However, what people are suggesting, namely making it less based on peer review and more on some social media like system is also misguided. It's like the instagrammization of the field: the sexiness, the dopamine hit, the celebrities.
But this isn't specific to AI or CS or academia. This is how humans are. It's how status hierarchies are always gamed. Think office politics, actual politics, dating.
Atvthe same time actual AI methods are really getting better, it's not a lie and not just hype. It's actually quite paradoxical how well the system works despite the flaws. Vision algorithms work really damn well, its very obvious to anyone who remembers the pre-2010 times. Perhaps there is somewhat of a stagnation in the last few years but also a lot is happening in smaller niches, as the big deep leaning ideas are adapted and applied specifically to particular tasks. The methods work, but the literature is distorted. You need to be an insider to know what to actually believe.
[0] https://slatestarcodex.com/2014/07/30/meditations-on-moloch/