Hacker News Reader

Comment by edw519 | Hacker News Reader

For a lark I'm going to take an old project that has been running for years without a hitch and I'm going to retro-actively add tests to the code. I'm really curious how many bugs and unexpected behaviors will turn up.

Last week I got an enhancement request from a customer that basically said, "Add B capability and make it work exactly like A capability."

"Cool," I thought, "This should be easy."

So I examined all the "A" stuff, which had been in production since 2008. Then I cut, pasted, modified, and added a whole bunch of stuff. (I know, I know, every once in a while, a programmer's just gotta take the lazy way out.)

When I started unit testing my B stuff, I broke everything on almost every try. This was before I even assembled a test plan, it was just a hacker beating up his own work.

How could this be? So I went back and beat up the A stuff and broke it in all the same places. Stuff thousands of people have used thousands of times. Sigh.

View on HN

How is the A stuff wrong when thousands of people have used it successfully(presumably for its intended purpose) thousands of times? I understand where you're coming from. But the longer I do this, the less dogmatic about testing I get. If the code works for its intended purpose then it's probably all right. Now, adding features and having confidence in refactoring is another story with untested code.

I recently discovered something interesting. The requirements gathering we do kinda sucks. How often do we bring in end users and watch them use the system as they do? Why do we not trust user input with our code but do with our ears? I recently got the opportunity to do this for a project that's being rewritten. The information you gather from this is gold.

The reason thousands of people have used it successfully is because most people are not developers, aren't lazy and willingly accept having to do something inefficiently over and over again. If there is some flaw in the way the system works then people will adapt and go around the mountain. They won't even think about this and, if it's so automatic that it's buried in their unconscious, they may never even think to tell you.

The other reason people have used it successfully is because they assumed that the broken output was correct. If it's not tested, how do you know that your report generator is spitting out the correct figures?

Also, kind of related, just as one of the benefits of unit testing is that you write well structured and testable code, one of the benefits of functional testing is that you think about the interface. I'd consider any inefficient interface to be a bug.

Because the people using it for the intended purpose are not fuzzing it for potential security flaws.

Do you write lots of security fuzz unit tests? Cause I don't. I write tests that hit the edge cases I can think of, but invariably, I can't think of everything. I really don't think the security argument has much to do with unit testing.

Reply on news.ycombinator.com