A non-exhaustive list of things that annoy me:
Random closure of services that had many happy users, for example code search
Rigging the search page like a mine-field with distracting and inconsistent pop-ups. (Hello! the absence of such distractions were the reason I switched to Google)
Removing two column format for news (Indian Edition).
No immediate way to paginate image search.
[EDIT: Paginations help in visually scanning a full page of new images and also anchors it to a particular link that I can go back to.The visual scan seems parallel and hence faster. The non-paginated version is closer to a more serial access, with no precise control over going back to where I want to. My other pet peeve is that I have to hover over every image for the meta data to pop up.]
Dropping links to cache: text_only: and similar: pages.
Dropping scholar from the drop-down menu.
[EDIT: It used to be a flick of a mouse to try the same search on search/scholar.]
Instant, though thankfully I can turn it off.
Closing down labs. (Much of the fun in the brand is gone with it).
Almost blindly copying Bing regardless of whether the changes are perceived as useful or not.
An oft repeated excuse I hear is that they are now apparently focusing on search. Speaking strictly from my own experience, that focus is actually hurting the quality I perceive.