Hacker News Reader

Comment by encomiast | Hacker News Reader

I had to give up when the first example of a "bug" that the author could see, which others are blind to, was search results.

"In some cases, people sent me their actual search results. In every such case, the search results did not contain a good result that I could see"

Software that does not meet expectations, especially in a field like search, which is basically a long running war between SEO and search engines, is not a 'bug'. That's like saying there is 'bug blindness' in the publishing industry because when I pick up a random book, it sucks, even though others think it's fine. Sure, maybe something is wrong, could even be improved, but lets save the word 'bug' for something more specific.

View on HN

> the first example of a "bug" that the author could see, which others are blind to, was search results

The author explicitly says he is using less specific and compelling examples for good reasons:

> I don't want to give any specific examples where it was my job to see how well the thing worked because, even if the internal examples are meant in a constructive, blameless, way, they may not always read that way when re-posted externally, so I'll give a few less interesting and less well supported "random" examples.

https://danluu.com/bug-blind/

I also think it's reasonable to use the term "bug" in a broad or colloquial sense. If a web server is overloaded, that isn't a bug in the strictest sense, but it sure feels like a bug to the person trying to use the website. A search engine is software, and if it fails to surface the most relevant document from its index then I'd say that counts as a bug. Also the author acknowledges this SEO spam issue, e.g. in this quote for the linked post:

> Here's a fun experiment to try. Take an open source project such as yt-dlp and try to find it from a very generic term like "youtube downloader". You won't be able to find it because of all of the content farms that try to rank at the top for that term

https://danluu.com/seo-spam/

At what point does search quality degrade to the point where is becomes a bug?

A bug is when actual behavior does not match stated or intended behavior. If the search bears little connection to what was searched for, then it is a bug, as far as the user is concerned.

Or maybe you mean to say we should reserve the word bug for deviations that are unintentional?

The trouble is, there are three different "versions" of "the search didn't work," only one of which I would say is definitely a bug:

- The thing the user searched for doesn't exist, so it wasn't found. This is not a bug.

- The user did a good job of searching, using relevant keywords etc, but the relevant thing (which exists) didn't come up. This is a bug.

- "Search didn't read my mind" -- if search is a core competency this is a bug, but otherwise I would consider it a deficiency or a missing feature.

For me the idea that poor quality search results are a bug muddies the idea of what a bug is. It's like saying the colour of the paint in my living room is buggy because I don't like it. It might be an ugly colour that I don't like, but it is what it is. A bug would be if the colour doesn't match what was shown on the tin.

> If the search bears little connection to what was searched for, then it is a bug, as far as the user is concerned.

A bad product perhaps, but not a bug.

So if I search for A, and I get results for B instead (where B has only the slightest connection to A), it is not a bug?

What about if B has no connection to A whatsoever? Still not a bug? If it is not, then we may have discovered a software domain (the first for me I should say) where bugs are not possible. Should a be a nice market to launch products for, then.

> So if I search for A, and I get results for B instead (where B has only the slightest connection to A), it is not a bug?

It depends. There could be a bug in the system like query = query.replace(A,B), or it could be that there are no good results for A. Returning nonsense results is a bad design, but not necessarily a bug.

I guess my underlying point is that bugs are about software not conforming to some specified behaviour, and trying to specify behaviour like "i should type in a search and get great results" is too loose of a spec to be meaningful so we should avoid saying things which don't meet that criteria are bugs. To use a concrete example, perhaps the set of search results returned for a query look useless to you but are actually useful for someone else.

Compare that with something like a calculator app where part of the spec is "Must handle integer addition with for inputs i,j where -10,000 <= i <= 10,000 and -10,000 <= j <= 10,000". Then if in your calculator you do 1+1 and get 3, then that is clearly a bug per the spec.

Bugs vs. bad design is a useful distinction to make imo.

That's because as computer people we're very invested in the idea of bugs, while your users are not.

In products your users are stuck with it doesn't matter much, they'll have to suffer with what they think are bugs.

In products that users can switch easily, you can get dropped for a product that you would consider crappier if the user believes it's less 'buggy'.

> At what point does search quality degrade to the point where is becomes a bug?

It depends entirely on the commitment made by the service, so e.g. my desktop file search program results has bugs whereas Google's has none.

Later in the article he uses "quality blindness" instead, which is probably a better description of most of the issues he talks about.

Dan isn't necessarily Bertrand Russell so ill give him a pass for not using the perfect word in every place.

But this article is in my opinion one of the best attempts at explaining a fundamental problem in software product work that has never (in my experience) been laid out this well. I will implore you to finish it.

> Software that does not meet expectations, especially in a field like search, which is basically a long running war between SEO and search engines, is not a 'bug'.

While bug has various definitions, not meeting reasonable expectations is common to all.

> That's like saying there is 'bug blindness' in the publishing industry because when I pick up a random book, it sucks

No, because there's no reasonable expectation that a random book won"t suck.

I dunno... people keep telling me books are good but that isn't my experience

Just like when a do a search and the results aren't relevant

Obviously this means both books and web searches are useless

Later on he gives an example of Blackboard software that was very disliked. So I don't see what was unique in his perspective either. To me this article is not very focused and feels mostly like rambling. The core idea that we get accustomed to bugs/quality in software we use is sound. Just the article needs streamlining or better a complete rewrite.

Reply on news.ycombinator.com