I'm toying with the newish JUnit version 4 for setting up test cases for my current Java project. I had tried the 3.x series, and didn't find the experience quite as satisfying. Setting up parameterized suites with the TestSuite class, and having to rely on method naming conventions seemed… well… kinda ugly.
JUnit 4 is a radically different API, and it depends on lots of newish features in the Java language, including annotations on classes and methods. I really knew nothing about this feature until I saw it used in JUnit, and I still know very little. I'm a language nerd, but not a Java nerd, I guess. But it may be worth exploring further.
Anyway, I need to be able to pull test cases from files within a directory, so the Parameterized ‘runner’ is just the thing. You designate one method to produce all the test cases as a Collection, and then JUnit instantiates your class with arguments taken from that collection.
Unfortunately, when a test fails, the output leaves a lot to be desired. It prints the name of the class, the name of the method, but as far as which test case failed… they're just numbered serially, like this:
There were 4 failures: 1) tryFib[1](FibTest) java.lang.AssertionError: expected:<1> but was:<0> 2) tryFib[2](FibTest) java.lang.AssertionError: expected:<1> but was:<0> 3) tryFib[3](FibTest) java.lang.AssertionError: expected:<2> but was:<0> 4) tryFib[4](FibTest) java.lang.AssertionError: expected:<3> but was:<0>How are you supposed to know which test actually failed? As someone else pointed out, there really needs to be a way to annotate a name or description for each case.
Until then, here's my silly work-around: just wrap your whole test method in a try/catch block, and rethrow the exception with a new message containing a description of the test case. I use the toString() method to produce that description. Complete example follows.
Here is the failure output now:
There were 4 failures: 1) tryFib[1](FibTest) java.lang.Error: fib(1): expected:<1> but was:<0> 2) tryFib[2](FibTest) java.lang.Error: fib(2): expected:<1> but was:<0> 3) tryFib[3](FibTest) java.lang.Error: fib(3): expected:<2> but was:<0> 4) tryFib[4](FibTest) java.lang.Error: fib(4): expected:<3> but was:<0>The differences are highlighted. I guess maybe this isn't such a convincing example, since the test description – fib(2) – is not sufficiently different from the serial number (2). In my application, test cases will be described by the input file name.
