Unlike you though, spam (at least recently) has not been a very strong issue of mine. If I had to name one, I'd say that "personalization" of the results was the worst offender to the quality my search results, and the positive thing is that we can turn that one off.
As crazy as it sounds, I wish another good search engine comes, as Google feels more and more like the "least bad" option, rather than the "good" one to me, and I'm just talking about search results, voluntarily excluding privacy & other topics from the debate.