Hacker News Reader

Comment by jared314 | Hacker News Reader

> We saw this with Heartbleed too: given sure confidence that there is a vulnerability in a particular diff, skilled security researchers can find it very quickly.

That reminds me of the old QA technique / tactic of only telling the developers where a bug was found. Sometimes you don't even have to find an issue, you can just pick a complicated module.

View on HN

There was a study done decades ago where code was seeded with a number of bugs. Developers were told that there were a number of bugs. They found that many, but the overlap was not total. That is to say, they found bugs that the study authors did not seed.

If you can dig up the citation, I'd love to read this. That's fascinating.

I was not successful, but a sibling comment to yours https://news.ycombinator.com/item?id=8458030 provides a very useful description of the phenomenon.

Phew, I sure am glad that we instituted a company-wide policy of not putting any bugs in the code in the first place. Just makes life easier all around.

Well, I remember at the time reading this that if one picked an arbitrary number, and told the developers that there were that many, they would find that many even if none were inserted.

You laugh, but this is closer to most companies "quality policy" than otherwise.

True quality comes from good process and systems, nothing more, nothing less.

You can use overlap to estimate total number of bugs:

http://en.wikipedia.org/wiki/Lincoln_index

http://allendowney.blogspot.com/2014/07/last-year-my-occasio...

Reply on news.ycombinator.com