contrapunctus, by Christopher League
 

Parameterized tests in JUnit 4

I’m toying with the newish JUnit version 4 for setting up test cases for my current Java project. I had tried the 3.x series, and didn’t find the experience quite as satisfying. Setting up parameterized suites with the TestSuite class, and having to rely on method naming conventions seemed… well… kinda ugly.

JUnit 4 is a radically different API, and it depends on lots of newish features in the Java language, including annotations on classes and methods. I really knew nothing about this feature until I saw it used in JUnit, and I still know very little. I’m a language nerd, but not a Java nerd, I guess. But it may be worth exploring further.

Anyway, I need to be able to pull test cases from files within a directory, so the Parameterized ‘runner’ is just the thing. You designate one method to produce all the test cases as a Collection, and then JUnit instantiates your class with arguments taken from that collection.

Unfortunately, when a test fails, the output leaves a lot to be desired. It prints the name of the class, the name of the method, but as far as which test case failed… they’re just numbered serially, like this:

There were 4 failures:
  1) tryFib[1](FibTest)
  java.lang.AssertionError: expected:<1> but was:<0>
  2) tryFib[2](FibTest)
  java.lang.AssertionError: expected:<1> but was:<0>
  3) tryFib[3](FibTest)
  java.lang.AssertionError: expected:<2> but was:<0>
  4) tryFib[4](FibTest)
  java.lang.AssertionError: expected:<3> but was:<0>

How are you supposed to know which test actually failed? As someone else pointed out, there really needs to be a way to annotate a name or description for each case.

Until then, here’s my silly work-around: just wrap your whole test method in a try/catch block, and rethrow the exception with a new message containing a description of the test case. I use the toString() method to produce that description. Complete example follows.

Here is the failure output now:

There were 4 failures:
  1) tryFib[1](FibTest)
  java.lang.Error: fib(1): expected:<1> but was:<0>
  2) tryFib[2](FibTest)
  java.lang.Error: fib(2): expected:<1> but was:<0>
  3) tryFib[3](FibTest)
  java.lang.Error: fib(3): expected:<2> but was:<0>
  4) tryFib[4](FibTest)
  java.lang.Error: fib(4): expected:<3> but was:<0>

The differences are highlighted. I guess maybe this isn’t such a convincing example, since the test description—fib(2)—is not sufficiently different from the serial number (2). In my application, test cases will be described by the input file name.

One response

You may leave a response below, or a trackback from your own site. You can follow responses to this post using this RSS feed.

1
Rumble
1 May 2008 @22:56

I managed to create my runner based upon the Parameterized runner. The first parameter is used as the description instead of the index of the array.

import static org.junit.Assert.assertEquals;

import java.lang.annotation.Annotation;
import java.lang.reflect.Constructor;
import java.lang.reflect.InvocationTargetException;
import java.lang.reflect.Method;
import java.lang.reflect.Modifier;
import java.util.ArrayList;
import java.util.Collection;
import java.util.List;

import org.junit.internal.runners.CompositeRunner;
import org.junit.internal.runners.MethodValidator;
import org.junit.internal.runners.TestClassMethodsRunner;
import org.junit.internal.runners.TestClassRunner;
import org.junit.runners.Parameterized.Parameters;

/**
 * Class DescParameterized - this class is a direct copy of
 * org.junit.runners.Parameterized, with the getName and getTestName
 * methods altered.
 */
public class DescParameterized extends TestClassRunner {

    public static Collection eachOne(Object... params) {
        List results= new ArrayList();
        for (Object param : params)
            results.add(new Object[] { param });
        return results;
    }

    // TODO: single-class this extension
    private static class TestClassRunnerForParameters
        extends TestClassMethodsRunner {

        private final Object[] fParameters;
        private final int fParameterSetNumber;
        private final Constructor fConstructor;

        private TestClassRunnerForParameters
            (Class klass, Object[] parameters, int i) {
            super(klass);
            fParameters= parameters;
            fParameterSetNumber= i;
            fConstructor= getOnlyConstructor();
        }

        @Override
            protected Object createTest() throws Exception {
            return fConstructor.newInstance(fParameters);
        }

        @Override
            protected String getName() {
            return (String) fParameters[0];
        }

        @Override
            protected String testName(final Method method) {
            return String.format("%s[%s]", method.getName(), fParameters[0]);
        }

        private Constructor getOnlyConstructor() {
            Constructor[] constructors= getTestClass().getConstructors();
            assertEquals(1, constructors.length);
            return constructors[0];
        }
    }

    // TODO: I think this now eagerly reads parameters, which was
    // never the point.
    public static class RunAllParameterMethods extends CompositeRunner {
        private final Class fKlass;

        public RunAllParameterMethods(Class klass) throws Exception {
            super(klass.getName());
            fKlass= klass;
            int i= 0;
            for (final Object each : getParametersList()) {
                if (each instanceof Object[])
                    super.add(new TestClassRunnerForParameters
                              (klass, (Object[])each, i++));
                else
                    throw new Exception
                        (String.format
                         ("%s.%s() must return a Collection of arrays.",
                          fKlass.getName(), getParametersMethod().getName()));
            }
        }

        private Collection getParametersList()
            throws IllegalAccessException, InvocationTargetException,
                   Exception {
            return (Collection) getParametersMethod().invoke(null);
        }

        private Method getParametersMethod() throws Exception {
            for (Method each : fKlass.getMethods()) {
                if (Modifier.isStatic(each.getModifiers())) {
                    Annotation[] annotations= each.getAnnotations();
                    for (Annotation annotation : annotations) {
                        if (annotation.annotationType() == Parameters.class)
                            return each;
                    }
                }
            }
            throw new Exception("No public static parameters method on class "
                                + getName());
        }
    }

    public DescParameterized(final Class klass) throws Exception {
        super(klass, new RunAllParameterMethods(klass));
    }

    @Override
        protected void validate(MethodValidator methodValidator) {
        methodValidator.validateStaticMethods();
        methodValidator.validateInstanceMethods();
    }
}
Download this code: 2008/DescParameterized.java

Your response