Hacker News Reader

A couple of years ago, I committed a test case called TestHowDatabasesWork because of a team argument about how Oracle handled concurrency and locking. It got a good laugh, but it also documented a few behaviors specific to the problem at hand.

There is a good possibility that in documenting what you saw working you failed to notice some important details of its optimistic locking policy.

If you're ever again faced with such questions about Oracle I highly recommend picking up an appropriate book by Tom Kyte. For a developer in particular I would recommend http://www.amazon.com/exec/obidos/tg/detail/-/0072230657.

Agree. TDD in the middle tiers of the app is easy, but that's not where the difficulties lie.

Testing the UI layer is quite difficult (at least in the absence of something like MVC, which too many developers can't handle and only want VB-like or WebForms-like coding). And in any paradigm I've ever heard of, real testing of the bottom layers of DB interaction is darned near impossible.

As the OP asserts, imposing a well-defined testing regimen is where most of the bang for the buck derives.

> And in any paradigm I've ever heard of, real testing of the bottom layers of DB interaction is darned near impossible.

Is it that because the code is written in an untestable way? A pretty standard mocking approach should be able to test this.

In my experience, problems "at the bottom" usually occur because underlying third-party components like the database (or the filesystem or the network stack or the remote server) don't behave the way you thought they did. They're buggy or undocumented, or you just misunderstood their specs. When you mock them away for testing, the mock objects act like you expect and don't have the same quirks as their real-life counterparts. If you don't mock them away, then you have hard-to-run or hard-to-reproduce tests, or tests that give different results with different instances of the external dependencies.

Ah. That makes sense. I guess I've had the luxury of only working with MySQL, Postres, and sqlite, and have never run into bugs with them.

I think the issue is that a mock of a DB doesn't imitate all the nuanced behaviors of the real thing. Transactionality, concurrency, isolation, error generation, etc. So, while you can prove functional correctness with a DBUnit/etc., you can't prove that the difficult stuff works.

Isn't that the job of the DB package, and not of you, the user?

It's the job of the DB package to perform as specified. The problem is that one can write code that works correctly under the wrong assumptions of DB behavior. Here's an example I've experienced:

We were building (to be purposefully vague) an auction system where an auction could end at any point in time. Bids flowed in via a queue. It didn't really matter within reason which order the bids were processed, but we had to make sure that a bid which caused an auction to end was processed in isolation from other bids that depended on the auction still being open. Our code relied upon the DB's locking behaviors (SELECT FOR UPDATE) to ensure that the code processing a particular bid laid claim to the state (auction status, etc) used for its work. All fine and good except that we didn't quite understand how Oracle implemented locking at various isolation levels. So, some code which would have worked against, say, DBUnit, failed under Oracle. This wasn't a failure of the DB, but we wouldn't have caught this problem unless we ran an integration test.

Ah. Interesting.

Databases have always been one of my weaker points, and lately, ORMs have allowed me to be pretty lazy on that front. Thanks for walking me through that.

No, and I'll give you an example. Take a simple relational table with some kind of constraint. Begin a transaction and insert some data into this table that violates the constraint. What happens? Guess what, it depends on the db. Some databases/drivers will throw an exception immediately upon insert. Some will wait until the commit has occurred to throw the exception.

Subtle differences like these exist all across the various relational db world. Not to mention that they all do locking and concurrency differently, which is probably the biggest problem.

Interesting - in the embedded software world we have the exact same problem with hardware. Mocking the hardware can confirm the internal consistency of your ideas about how it works, but most problems stem from your ideas being inconsistent with how it actually works.

Until now I always assumed that the environment from which TDD emerged (presumably database-driven "Enterprise" apps) was different in a way that made it seem like a better idea than it does to me, but your comment is causing me to seriously rethink that assumption.

Suppose your code needs to interact with stored procedures in the database. How does a standard mocking approach in your code help you test the stored procedures?

Testing all of the ins and outs of possible DB behavior is out of scope for most software projects. You need to assume your driver works until proven otherwise.

As for testing interactions with the database, one approach that I've seen work is to have data in your test cases with which you build up database objects that you then interact with. If your classes all have appropriate helpers and appropriate ways of inserting reasonable defaults on fields not explicitly set in the testing code, this makes testing the expected database interactions fairly straightforward.

See, I really enjoy freezing the API early. The API should be very, very simple. I tend to code by designing top down, then coding from the bottom up.

APIs should _not_ be from the perspective of "this is how I solve this problem" and should be from the perspective of "this solves my problem."

For example,

    TempCalc tc = new TempCalc();
    tc.f = 50;
    System.out.println("50 degrees farenheit is " + tc.convertToCelcius() + " degress celcius");

is bad design. Yes, you may need to get a farenheit value, and change it to celcius, but I don't care about setting internal state, then calling functions to change it. I just want an "fToC(int f)" function.

This is an admittedly contrived example, but I see this principle in API design all the time. Everything should be from the perspective of the user, not the coder.

"I'd love to use this api if it looked like this."

Anyway... as far as that integration stuff goes, have you used Selenium? I'm not super up on Java testing stuff, as I'm a Rubyist, but I use it, and I know it has Java bindings of some kind...

Selenium 2 (the Google "Webdriver" stuff) works quite well and isn't specific to Java in any significant way. If you can bring up your UI with a bunch of mocks/stubs under it, Selenium is a great way to do automated regression testing of your web UI.

On your API design comments:

My issue with TDD is that it's implicitly big-design-up-front. I used to think I understood a problem before I stated coding, but I don't anymore. So, anything I design in the beginning ends up being modified (if not scrapped entirely). There's no better way for me to understand a problem than by trying to solve it.

Also, for important enough APIs (public stuff, widely used in a codebase, or whatever), the design of the API is dependent on the problem being solved. Let's say you are building an API for a yet-to-be-developed photo storage SaaS. For any solution of significant sophistication, can you really think through all the issues that have API design implications before attempting to create the service itself? Is it better to stream back the bits of a photo found via a tag search or should a "ticket" in the form of a GUID be provided instead? I don't know, but I'll have a better idea once I spend some time solving the problem.

Thanks for confirming on the Selenium thing.

I guess it just comes down to personal taste, then. I feel that I can't understand how the coding should be laid out until after I've designed how it will be used.