In an ideal world, traffic numbers would be correlated with some combination of product quality mixed with a sort of load-balancing effect so that each quality company could match their available supply to the demand. I still believe, perhaps naively, that search engines could have profitably built infrastructure to serve the needs of consumers in this way -- focus group-like services for commercial search that would require companies to pay for the privilege of submitting to objective evaluations of their products in exchange for more traffic. In a structure like this, the incentives of search engines and end users could be aligned.
Instead, Eric Schmidt and the rest of the market share MBAs came into the picture and we got precisely the opposite -- instead of holding businesses beholden to the needs of end users, now we have this insidious and dystopian perpetual advertising machine where not only are businesses now beholden to the needs of a faceless SEO algorithm, funneling money away from R&D and into this annoying waterfall of trite crap noted by this article, but IMO even worse is that end users are essentially beholden to the needs of the corporations -- either by getting creepily data-mined and harvested for the benefit of the highest AdWords bidder, or by getting inundated by constant ads, as anyone who has tried watching YouTube without AdBlocker on can attest to. I don't think I'm being hyperbolic when I say that nobody wins except the search engine stockholders.
I was totally heartbroken when Google sold out and created AdWords. I naively had thought it would never happen -- Brin and Page were literally on record saying, “Advertising funded search engines will be inherently biased towards the advertisers and away from the needs of consumers” (Anatomy of a Large-Scale Hypertextual Web Search Engine, 1998)[0].
So I've had embarrassingly heated discussions with Googler family members where I raise this issue, and their defense has been essentially that AdWords is a necessary evil that funds all their infrastructure expenses, as well as their cool R&D and "20% time" and all their ML advancements that they get to do, like AlphaGo, Waymo etc.
But I really think that when your company depends on perverse incentives in this way, the end result will ultimately drift away from their original job of serving consumers, and eventually exist only to inflate their own share price and run what amounts to a protectionist racket. The only difference is that the threat of withholding SEO traffic is much cheaper and efficient option than mafia shakedowns.