From a business level google's job is to sell ads, it sell ads by providing valuable search results 'for free', and one part of the intrinsic quality of the search results is that they are 'good' in the sense that you describe.
Aside from that google really doesnt owe anyone anything, if tomorrow google found that showing small businesses search results negatively affected their bottom line, your mom and pop ecommerce site and all like it would disappear (until searchers moved on to another service). Google's 'fairness' only exists and should be counted on as long as game theortic/min max optimization of bottom line happens to align googles interests with your own...
Google is the epitome of a consumer web business model. They sell users. So yes, they have to have buyers, but their most important job is to have users to sell.
> If Google doesn't have users, its ad space isn't worth anything, and its business model ceases to exist.
Yes but that is a second order effect. Basically the majority of "products" have to be convinced that Google is not good enough _and_ there is another alternative (I like DuckDuckGo for example), then as you say, people will stop using Google Search. Then if Gmail goes South, people stop using that. Then Android goes South, they start using iPhones. Eventually Google will provide a lower quality "product" to their customers and Google's stock will take a hit. However, don't be fooled into thinking you are Google's primary customer because you use them for search.
It's not ironic, and it's not true. Being a developer gives more freedom and more pay than the vast majority of jobs. The fact that there are some developer jobs that are gasp dependent on large companies in the field does nothing to change this.
And you're right that a developer/entrepreneur doesn't have only Google and app stores as choices, but the options don't go much further than that. Facebook and Twitter, perhaps? In the end, the company that controls your customer acquisition channel controls you.
I mean we wouldn't tolerate an energy supplier disconnected arbitrary supplies because they didn't feel like selling there any more.
If you have a gas-powered merchant plant, and the spark spread isn't good, you turn it off until energy prices rise again. http://en.wikipedia.org/wiki/Spark_spread
Having said that, it's important to remember that without a site like Google, many of these small businesses wouldn't exist in the first place. For all the criticism, having a search engine that does a good if imperfect job is almost certainly a big net win for society. As long as that is the case, and as long as Google aren't actively/unreasonably harming anyone, I think we're a long way from saying they are dominant and dangerous enough to require statutory regulation, which would be the normal next step if an actual monopoly provider of an actually essential service was misbehaving.
I think this gets to the heart of the conflict of interest which Google experiences and those who depend on them are afraid of. On the one hand, they have a reputation for valuable and high-quality search results which are not vulnerable to gaming. On the other hand, they have an established monopoly on search, which will not disappear overnight, and a business selling ads which compete against their own search results for clicks. Their interests do not currently align with those who appear in their organic search results, except tangentially and in the long-term (as a way of keeping eyeballs on those results because everyone views them as trustworthy).
So there is a tension there as good search results mean less ad clicks short-term, and less ad clicks means less money for Google. In an ideal world for Google as a corporation, their first page would simply be search results, tailored to a user's request, with no organic search results whatsoever, and indeed, their space on the results page does have a growing advertising component (paid ads, G+, ads for google places or other services like shopping). Obviously at present Google has managed to walk that tightrope and provide great search results, downgrade spammy results and ignore bad links, while making money from advertising displayed with those results, but it's a delicate balance, and there are commercial pressures to improve advertising clicks, but no short-term commercial pressures to improve search result clicks.
If those results can be poisoned or gamed easily, it doesn't really affect Google in the short term, only in the long term, and only if users start to notice and desert their service (over a period of years probably) - that's not necessarily the case if users don't have many comparison points because of Google's scale and dominance. It might even be good for Google in the short term, as businesses start to buy more adwords visits when their organic results decline. I'm not trying to suggest there is some conspiracy at Google, just pointing out the intrinsic tension at the heart of their business between good results and good ad revenue.
Those 1 out of 1000 good sites that were penalized do mean that "Google isn't doing its job" in that its job is never complete, but that doesn't mean that it wasn't the right choice to make for its users (while it then tries to refine new signals to get a better set of results).
The crap filtered out can vastly outweigh the good results (temporarily?) lost, depending on the good sites that were filtered out. Yes, it sucks, but false positives always come with this type of filtering.
Also, remember that "best" is itself just a confluence of signals, and that the guy next to you may very well disagree with you for any particular "best" choice.
No.
Google is not there to help you, me or anyone find better cat pictures. It is not a charity. Google's job is to make money. It makes money primarily by selling ads. It can sell ads better if it can target people better so they provide free email and then also parse all the email contents.
Ads show up in search and by providing good search results they ensure people go to google.com and not facebook.com or bing.com so _they_ can show you ads as opposed to someone else showing you ads.
"If my website is the best but it doesn't appear at the top because a competitor has attacked it" try Bing, or other search engines. You can try to shame Google into providing better searches by going public and hoping their PR team will notice, that's valid, but I don't see at all how this their "job".
You are ignoring Google's customers, those of us paying for advertising. If Google's content (it's search results) goes south, then the value in advertising goes down.
This is how advertising works. People don't go to Google to see advertising. They go to find things. I pay to appear when the best the web has to offer isn't good enough. It also works wonders when someone is searching for something because they trust google's content, and is actually looking to buy, so they specifically look at the advertisements knowing these are places they can pay for a service.
And yes, I'm giving money to Google, essentially hiring them to provide that service. And that service is not merely to show advertisements, but rather to show advertisements beside excellent search results. The same way the rest of advertisement works in every other medium. I pay for advertising on radio at certain times because of the reach, and this reach is based not on advertising, but on the quality. So I have a direct monetary incentive to ensure the quality of the content my advertising surrounds.
Of course, that also means Google has a directly responsibility to ensure the quality of the search as well, because that's what they are selling to me.