Search is definitely a case of "the devil in the details", each person got their own very "simple" test or search improvement but to make search for everyone "generally" valuable is a very hard tuning problem.
In my past job as head of search for a large price-comparison service, what we build and found work best is to handle search-tuning as the same as test-driven-development.
We will actually write test in the form of "When someone types apple, there should be at least xyz products from category cellphones and their pricing should be between $XYZ--$ABC", whenever we had a manager or a person just walk in with oh search should work like this, we will put it in a test and thus as we tune, weights and vectors we can always see how it impacted previous cases of "Managers walking in telling us, his fav search query is bogus"