Testing Patterns for Java Unit Tests

http://parasoft.developpez.com/tutoriels/java/testing/patterns/

 

I. Identify common patterns in test cases, and use them to create new tests

In 1995 the famous Gang of Four-Erich Gamma, Richard Helm, Ralph Johnson, and John Vlissides-introduced the concept of design patterns for software development. Design patterns increase development efficiency by helping developers identify common problems and apply standard solutions to these problems. Most developers use design patterns for writing application code, but few realize that design patterns are also helpful for writing test cases. Let's explore how to identify common patterns in test cases and use these patterns to create new tests.

II. Patterns for JUnit Tests

Today, writing unit tests is an obligatory part of the software development process for most developers. For Java developers, unit testing means producing a JUnit test class for every class. If you are not fortunate enough to have access to sophisticated test-generation software, writing unit tests is probably a normal part of your day.
If you think that writing unit tests is often tedious, unrewarding, and even boring, you are not alone. Most developers would rather write code "that does something." Designing meaningful unit tests after you have written a class can take considerable time (unless you are using Test-Driven Design [TDD]). Moreover, writing unit tests is often a routine process that lends itself to mechanization. If you watch yourself closely while writing unit tests, you will notice that you very often apply a recurring pattern to create the unit test for a particular class. For example, one of the simplest and most familiar testing patterns consists of an invocation of a tested method on some test input, followed by an assertion of the corresponding expected output:

import junit.framework.*; public class StringBufferTest extends TestCase { public void testAppend() throws Throwable { StringBuffer testObject = new StringBuffer("a"); testObject.append("b"); assertEquals("ab", testObject.toString()); } }

Note that the StringBuffer class is used only as an example; in real life you probably would not write tests for library classes.

Tests that follow this basic structure can already be considered applications of a simple testing pattern. This basic assertion pattern performs only an isolated test of a single method. It is a useful pattern for verifying data processing algorithms with known input and output, but it does not provide much help for testing the soundness of a whole class.

To verify the overall correctness of a class, you usually need a testing pattern that involves multiple methods. An example of such a multiple-method testing pattern is the combined testing of getter and setter methods. In this pattern, an attribute of the tested object is set to a certain value; when that attribute is queried afterward, that same value should be returned. This seemingly trivial testing pattern can help you uncover a number of common mistakes, such as the aliasing of field and parameter names or wrong fields being set or returned:

import junit.framework.*; import java.util.*; class DateTest extends TestCase { public void testSetTime() throws Throwable { Date testObject = new Date(); testObject.setTime(100000L); assertEquals(100000L, testObject.getTime()); } }

For simple set/get methods, there is usually only one possible execution path. Nonetheless, you should still write a number of tests that use different values or objects. A single test that uses zero or a null reference as a test value may give you a false sense of security. There are a number of other common method pairs that you can use as the basis for similar testing patterns. Collection classes and classes that support event listeners usually have methods for adding and removing items. A common testing pattern is to add an item, then remove the item again and assert that the test object be in the same state that it was in before the two methods were invoked (if that is indeed the expected semantics of the tested class).

You can extend these basic testing patterns to involve more than just one or two methods. To test more complicated classes, it is often necessary to create an object, invoke a number of methods to manipulate that object, and then compare the final object state with the expected outcome.

III. Identifying Patterns

It is not easy to establish general patterns for testing arbitrary classes, but the code itself can provide hints on identifying useful testing patterns. For the example of the get/set testing pattern, the hint simply lies in the names of the methods. Method pairs whose names start with set and get (where the type of the single parameter of the set method is the same as the return type of the get method) are usually accessor methods for a particular field of the object. Your code is likely to contain standard get/set or add/remove pairs, but you should also look for additional domain-specific method names that point to multiple methods that could be tested with the same pattern.
If you are developing based on a Design-by-Contract (DbC) methodology and routinely document your methods' preconditions and postconditions, then your DbC annotations are another good source of hints (see Listing 1). Based on the contract information, you can form simple test patterns that invoke a method and assert the postcondition:

import junit.framework.*; public class SimpleSetTest extends TestCase { public void testConstructor() throws Throwable { SimpleSet testObject = new SimpleSet(); assertTrue( testObject.isEmpty()); } }

However, if you are indeed using DbC, you will probably have a DbC checker or run-time DbC instrumentation that already performs these simple checks. The real advantage of knowing method preconditions and postconditions is that you can create test chains that involve longer sequences of tested methods, which you can do by matching the postconditions of a tested method with the preconditions of the next tested method in the sequence. Methods that have no preconditions can be used at any point in the chain:

import junit.framework.*; public class SimpleSetTest2 extends TestCase { public void testSequence() throws Throwable { Object testItem = "test item"; SimpleSet testObject = new SimpleSet(); assertTrue( testObject.isEmpty()); testObject.add(testItem); assertTrue( testObject.contains (testItem)); testObject.remove(testItem); assertTrue( !testObject.contains (testItem)); } }

The quality of your DbC annotations has a big influence on how easily these annotations can be used for constructing test sequences. Just asserting that arguments and return values cannot be null is not very helpful for creating a meaningful test sequence.

In addition to drawing hints from naming conventions and DbC annotations, you might also want to look at a tested class's superclasses and implemented interfaces. Very often, you can use the same testing pattern to test all classes that implement a certain interface or extend a certain superclass. For example, a simple set of testing patterns can be used to provide basic test coverage for any class that implements the java.util.Iterator interface. After you have determined a good testing pattern for the Iterator interface, you can apply that pattern to all classes that implement the Iterator interface.

To start, consider the contract of the Iterator interface. This contract is easy when you already have DbC annotations. However, certain semantic aspects of classes cannot be expressed easily as DbC conditions. In such cases, you could start with this set of rules that each Iterator should satisfy:

1. If hasNext() returned true, then a subsequent invocation of next() must not throw a NoSuchElementException.
2. If hasNext() returned false, then a subsequent invocation of next() must throw a NoSuchElementException.
3. Multiple invocations of hasNext() must return the same value if next() was not called in between.
4. As soon as hasNext() has returned false, it must never return true thereafter.
5. As soon as next() has thrown a NoSuchElementException, subsequent invocations must also throw an exception.

This set of conditions already covers a lot of common problems with the implementation of the Iterator interface. You can translate these conditions into testing patterns. The only remaining challenge is the generation of Iterator objects in different states that allow you to exercise all of the code's execution paths (see Listing 2).
You can freely choose the value of the constant REPEAT_COUNT. Depending on the program logic behind your iterator, it might be possible that-because of some obscure b-the Iterator reverts to returning more elements after hasNext() has already returned false. A repeat count of 2 may be sufficient to detect flopping between two states. If you suspect that a wrong state change could occur after a large number of repeats, choose a larger repeat count value. The same pattern, with different repeat values, may also be used for other classes.

IV. Extract Test Patterns

If you designed your classes based on use cases-either documented in Unified Modeling Language (UML) diagrams or in some other form-you have another good resource for testing patterns: the use cases themselves.

Use cases describe how classes interact with other classes when a certain scenario is executed. Often, you can directly extract a sequence of method invocations from a use case description. These sequences can be used as testing patterns. Testing patterns that you extract from use cases might not be specific to a particular class. Instead, a whole set of interacting classes might be tested, and the sequence of tested methods verifies the overall interaction rather than individual classes.
You can identify testing patterns based on naming conventions, DbC annotations, superclasses/interfaces, and use cases. As you have seen, once a suitable pattern is identified, it can be applied to tested classes in a rather mechanical fashion. Sometimes, even the process of identifying a testing pattern is a rather mechanical one. Indeed, the techniques explored in this article ideally lend themselves to automation.
Currently available automated test generation tools can automatically identify suitable testing patterns and create test cases based on these patterns. Such tools can provide you with basic test coverage for your code and free up more of your time for writing the application code and maybe a few tricky test cases that cannot be generated automatically.

However, all of this information assumes that there is already code to be tested, and test cases are created based on that existing code. If you are following a methodology like TDD, testing patterns are problematic because you will write tests before the tested code exists.

V. Listings

V-A. Listing 1

Your DbC annotations can be a good source of hints if you are developing based on a Design-by-Contract methodology and you document preconditions and postconditions for methods.

public class SimpleSet { /** @post this.isEmpty() */ public SimpleSet() { //... } /** @post this.contains(item) * @post !this.isEmpty() */ public void add(Object item) { //... } /** @pre !this.isEmpty() * @pre this.contains(item) * @post !this.contains(item) */ public void remove(Object item) { //... } public boolean contains(Object item) { //... return false; } public boolean isEmpty() { return false; } }

V-B. Listing 2

In meeting the challenge of generating Iterator objects in different states that let you exercise all of the code's execution paths, you can freely choose the value of the REPEAT_COUNT constant.

import java.util.*; import junit.framework.*; public class SimpleIteratorTest extends TestCase { private final static int REPEAT_COUNT = 10; public void testSimpleIterator() throws Throwable { Iterator testObject = new SimpleIterator(); while (testObject.hasNext()) { // Assert condition #3 for hasNext() == true: // for (int repeat = 0; repeat < REPEAT_COUNT; repeat++) { assertTrue(testObject.hasNext()); } // A violation of condition #1 will be // automatically reported by JUnit: // testObject.next(); } // Assert condition #2: // String caught = null; try { testObject.next(); } catch (Throwable throwable) { caught = throwable.getClass().getName(); } assertEquals( NoSuchElementException.class.getName(), caught); // Assert condition #4: // for (int repeat = 0; repeat < REPEAT_COUNT; repeat++) { assertFalse(testObject.hasNext()); } // Assert condition #5: // for (int repeat = 0; repeat < REPEAT_COUNT; repeat++) { caught = null; try { testObject.next(); } catch (Throwable throwable) { caught = throwable.getClass().getName(); } assertEquals( NoSuchElementException.class.getName(), caught); } } }