Hacker News Reader

Comment by wccrawford | Hacker News Reader

I mostly agree.

That MMO that is getting away with using customers as QA, and only fixing bugs that NEED fixing? I bet they're making a nice profit. I'm sure the customers are annoyed, but in my experience, gamers won't quit playing over bugs. Bad customer service, price hikes, etc... Sure. But not bugs. Even if they stick around for years.

However, business applications are a different story. Businesses can't afford to be stopped from doing what they need to. And a QA person makes all the difference in the world there.

For those that don't understand why, a programmer can't find their own bugs because they've already done their utmost to find bugs in the code. They've already stretched themselves to the limit before they sent the code to QA. You can train, coddle, beat, or otherwise influence them and only get a few more bugs found by programmers. It's much better for everyone's sanity and wallets if you just pay for proper QA.

View on HN

"However, business applications are a different story."

You've sure got that right. Our application conducts real-time purchasing events (reverse auctions) where bidders are ranked based on a couple of different configurable algorithms. An error in calculation means incorrect ranking, which means the entire outcome is invalidated. Considering that it can take weeks to put together a purchasing event of this type means that there is a lot riding on that single series of calculations.

Simply releasing and finding bugs in production would be financial suicide. QA isn't even a question in our business. Everything is tested thoroughly when it goes out the door.

I write software for clinical trial data collection, so I'm in the same boat. Not only is every product tested, but QA must thoroughly document their tests. At the end of the day, yeah, it slow the process down, but the software is significantly more robust when it heads into field.

Having worked at many companies that didn't believe in QA at all, I think having a team of testers is beneficial not only for software quality, but also filtering potential bug reports from the field. When a client reports a potential, QA does the initial leg work in duplicating the issue, then passes it off to engineering.

Lately my interest has been piqued by analytical code verification techniques and/or fuzzing (going beyond simple static verification or "checking the code style").

I think a lot of bugs made in day-to-day coding could be found given such methods. It would be best to integrate something like that into the compiler (and IDE) as it has the necessary information and structures. I believe the clang team is working on this (http://clang-analyzer.llvm.org/).

I guess it'd require defining "higher level" pre- and post conditions and assertions and such in the code...

Not a substitute for "proper QA", but I think sometimes that "software engineering" is simply dabbling in the dirt as a engineering discipline, a computer could probably run the function you are editing right now 1000000x per second and/or verify the hell out of it while you are thinking about the next letter to type :p

Hell yes you are spot on right there.

I've got into a habit of defining pre and post-call checks on just about everything. It's knocked down my regression and defect rate quite considerably. Assumptions are dangerous - so always check them.

Microsoft are actually leading this if you ask me with PEX:

Try it here: http://www.pexforfun.com/

Reply on news.ycombinator.com