What does $1265 of bugs look like?
daemonology.net
daemonology.net
Unlike some other bounty schemes, there are small bounties available. One can look for ages without finding a $50 bounty, so long that boredom would probably set in and the hunt would be abandoned. But coming across the odd $1 or $5 bounty keeps interest going at small cost to Colin and still improves the overall source quality. It's almost like a bit of a game, a treasure hunt, where small pickings on route keep one's hopes up of rich pickings down the road. A game that can be set down and picked up later as time allows.
Secondly, Colin was very fair in judging the bounty to be awarded. It soon became apparent that I could trust his judgement; whenever I thought a bug was undervalued another would come along that might be overvalued. On mentioning this to him, it turns out it was sometimes deliberate; he'd be wavering and thought it fair to even out which side of the boundary the bug fell.
As for the OCD, I boggle a bit at some of those style bugs and don't think they were all me! :-) I do recall sending in some trivial thing suggesting it wasn't bountiful but perhaps should be fixed anyway; Colin generously still gave a dollar. I thought it worth reporting even without expecting a bounty because I think anything that interrupts the flow of the reader, causes them to pause, irked at a possessive "its" misspelt as "it's", detracts from the odds of them spotting something more serious.
Overall, a well crafted and executed bounty scheme. In some ways I'm surprised other companies don't do something similar, even if it was with closed source under an NDA. If the feedback is prompt then the bounty hunter can decide to stop if the bounties aren't awarded fairly enough in his opinion, or the company can cancel, perhaps having shelled out their budget or unhappy with the quality of the bugs, without the hunter having wasted much time since the last bounty.
I found it amusing (and unsurprising) that the distribution of reported bugs matches what you would find in code reviews within most companies: a small handful of genuine issues buried in a giant pile of superficial nitpicks.
Before anyone interjects, no, I'm not saying misspellings and stylistic inconsistencies don't matter. They matter to me a great deal--I am crazily OCD about that shit. Seeing 'shear' misspelled as 'sheer' throughout most of Unreal Engine 2 and 3 without being able to fix it (there were too many expected breaking changes to licensee code for such a cosmetic change to make sense) annoyed me like you wouldn't believe. But when I force myself to take a deep breath and consider the matter more rationally, it's clear that such issues are far down the list in importance. While I'm reviewing someone else's code and my OCD-ish code aesthetics sense is stroked, I always try to remind myself of that.
But I wouldn't call the harmless bugs "superficial nitpicks". Most of them could become problems later -- if other code changes result in the functions they are in being called with different parameters; or if an error is no longer treated as fatal but instead Tarsnap continues and retries the failed operation; or with more aggressive compiler optimizations.
Better to fix the bugs before they cause problems than to wait for them to cause problems.
The fact that it satisfied my OCD is just a side benefit. ;-)
edit: Look, I know that Humphreys is the agile bogeyman, but why the downvote? Engage, don't merely click an arrow.
The misspelling referer originated in the original proposal by computer scientist Phillip Hallam-Baker to incorporate the field into the HTTP specification.[1] The misspelling was set in stone by the time of its incorporation into the standards document Request for Comments (RFC) 1945; document co-author Roy Fielding has remarked that neither "referrer" nor the misspelling "referer" were recognized by the standard Unix spell checker of the period.
I have a piece of C code that I license to companies (it does multi-label classification of documents---mostly email) and I have very good test coverage and have been obsessive about its design and operation. Since it does a lot of low level C tricks for speed I wanted to be absolutely sure that it wouldn't crash, and it has to work on Windows (32 and 64), Solaris, HP/UX, AIX, Linux, ... The bottom line is that being obsessed about the stability of the code has meant that in the five years it's been shipping not a single client has reported a bug and I have never heard of the code crashing.
This actually turned out to be a problem for me, because I had a hard time getting some people to renew maintenance on the code because it never failed.
Maintenance would consist of lowering the multiplier of "crash" occurance. So your Program is always improving. I see little difference between this and the way hardware makers cripple lower end hardware
You can sell maintenance then. Slimy, i know. But business is business.
This last is then followed by
* You have to verify the compiler,
* You have to verify the hardware,
* You have to verify that there wasn't an error in loading the code,
* You have to use radiation-hardened hardware
* ...
and so on.
They miss the point!
Bugs in your code are way more likely than any of the other problems, and they matter.
I've generally given up, but at least it's nice to see the attitude here not being instantly shouted down.
As might have been predicted the bug that Raymond was talking about wasn't caused by the optimizer at all (http://esr.ibiblio.org/?p=1705#comment-248328).
It's depressing to think that when your code works, it's almost always your fault.
Obviously, this false, but until you are smart enough you'll never know when the compiler is wrong.
And they were right - my code had bugs in it.
So I recrafted my code and effectively, informally proved it to be correct. Then I sent it in again and got a formal acknowlegement that I had, indeed, found a bug.
Over the next 2 years I found 5 bugs in three different compilers, but in the first few cases I spent ages crafting my examples to prove the problem wasn't my code. Then it dawned on me, why not right the code to be correct in the first place, then I wouldn't have to re-write it when I found bugs that might be mine, and might be someone elses.
My productivity improved dramatically, and I was hooked.
I'm not disagreeing with your premise, but I find my attitude can be different depending on the scope of the project. For example, if I'm working on a smaller project, I find I can focus on the overall quality and strive for "no bugs." But on a larger project (often behind schedule and over-budget, most often not because of the dev team, directly), there are lots of edge cases that get ignored, at least during the first release cycle.
With Tarsnap, my incentives are perfectly aligned: Writing good code results in happy Tarsnap users, which results in more Tarsnap users.
I'm definitely glad you put out these bounties - you set a very good example for us all.
To resolve this you could only conduct the bounties periodically so there’s always enough of a backlog of bugs to be found.