If the universities wanted the problem fixed it would be fixed. I truly don't believe research universities are focused on producing value (other than to themselves).
Why did the culture deteriorate so much and become so myopically focused on weakly informative metrics like publication count?
1) due to population effects, academic positions are much more competitive now than they were in say 1970; if you figure that the top 50 research universities are not generally expanding the number of professors, and that new professors generally also come from those top 50 research universities, then on average a top-50-research-university professor will generate one new such professor in a career, despite having 10-100x as many graduate students (this was different in the 70s when the university system was rapidly expanding).
2) the increasing desire for fairness in hiring and promotion (by itself, a good thing) means that you need to be able to resolve hiring and promotion disputes with something both objective and external to the university (in the same way some undergraduate institutions put more admissions weight on external and objective metrics like standardized tests compared to more easily game-able internal metrics like high school class grades)
However, now that many research fields are so massive that it is impossible to personally know the majority of individuals, institutions need other ways of judging individuals. The number of papers published is a weak signal, but it’s better than nothing. Now that it’s being so heavily gamed by so many individuals, that signal strength is decreasing even more.
There’s also a second strong corrupting factor that many, if not most, of these individuals do not want to become professors, they want to get a high paying job in industry, which means their short term output is far more important to them than their long term reputation in the field.
I honestly don’t know what can be done to fix this that wouldn’t have negative side effects. But perhaps the side effects would be better than the situation we are in now.