Comment by simias | Hacker News Reader

Comment by simias | Hacker News Reader

Parent Full thread

I actually implemented a basic unit test framework in C yesterday and started with basic asserts like you point out, however once I got it working I switched to return values not unlike TFA. The obvious problem with assertions in that they kill your test program immediately at the first failure instead of doing additional tests and potentially give a clearer picture of what's broken exactly. They're also not very flexible if you want to customize the error message since they only stringify the assert parameter. "Assertion `ret == 0' failed." is not super helpful. As sibling comments mention there are ways to work around that by using literal strings and && but it doesn't help if you want to wrap around tests using helper functions etc...

I'm also tempted to say that if you need to use GDB to figure out where your tests have failed exactly your framework is not particularly user friendly. Normally when a test fails it should be pretty clear where it failed and why. The root cause of the issue might be further away of course but then again assertions won't help with that either.

Assertions don't actually have to kill the program. They send an abort signal, which means you can catch them with a signal handler.

I similarly wrote a simple C testing framework for a little project I was working on, based on assertions. The framework uses signal handlers for aborts and segfaults, and then setjmp/longjmp to handle resuming the test suite at the next test case. This has the particularly nice effect of turning segfaults into (marked as such) test failures, instead of just terminating everything. It probably wouldn't be too hard to fit custom messages in too, but I hadn't felt the need yet.

It would be safer to fork before the segfault, to preclude other less obvious memory errors.

I concede that being able to handle segfaults as test failure is a very nice feature, however dealing with signal handlers is a bit more involved that simply using return values and I believe that it might cause portability issues (does it work on Windows for instance?).

I agree that for a decent test framework it might be the best approach but I think at this point we're no longer talking about "a minimal unit testing framework" which was the point of TFA.

I mostly agree with you, but I would like to make something clear.

>I'm also tempted to say that if you need to use GDB to figure out where your tests have failed exactly your framework is not particularly user friendly.

Am I missing something?

  #include <assert.h>
  
  int main(void)
  {
  	assert(0);
  	return 0;
  }

When using clang I get a clear message that includes the filename, the line, and the function:

  assert.bin: assert.c:5: int main(void): Assertion `0' failed.

My point was that depending on how you structure your tests knowing the location of the assert might not help you, for instance you could have a helper function that would validate something and generate a failure if something goes wrong, in this case knowing that this specific function failed doesn't help much understanding what test specifically failed. My point was that if you need GDB to figure out what exactly failed then you don't have a very user friendly test framework.

Oh, I see now. Yeah, trying to make assert output a dynamically constructed message does seem like a PITA.

Reply on news.ycombinator.com