Pattern Sequence test cases using test scripts

Problem:

There is a need to specify sets of test cases that together achieve some specific testing requirement. At times these test cases must be executed in a specific sequence in order to achieve that goal.

Context:

An individual test case presents a single stimulus to the Object Under Test (OUT). A particular test case may operate properly but leave the OUT in a state in which further stimuli will not be handled properly. An object can only be considered correct when it has been tested over its complete life cycle. That is, after an object has received a set of messages that has thoroughly exercised all of its intended actions, often denoted as all paths through the state model of the object.

Forces:

• automation of test cases- Application of test cases can be partially automated by grouping in a single test method several test case messages.
• interaction of messages- Method interaction can be tested by execution of multiple test cases on a single OUT rather than beginning each new test with a fresh object. These actions could be programmed as a single method rather than calling the other, existing, methods.

Solution:

Create test methods that serve as scripts for invoking multiple test cases. The public interface of each test class contains only test scripts that the tester may invoke. The test case methods are protected from direct access. Each test script creates an OUT and so any test cases executed within the test script are stimuli to that single OUT.

Some standard scripts that are created are the baseline test script, Create a baseline test script, and functional, structural and interaction test scripts that ensure that the OUT is tested to specified coverage criteria. For projects with specific performance criteria, there would be a performance suite that concentrates on defining specific types of loads on the system.

Structure:

A test script is simply a method, but one that has responsibility for creating the OUT, applying the test cases and then deleting the object under test.

void SimpTest::test_script9()
{
	OUT = new CUT;
	*ostr << "Start an interaction of test cases 4 and 6" << endl;
	if (test_case4() && test_case6())
		*ostr << "Interaction test case completed successfully" << endl;
	else
		*ostr << "Interaction test case completed unsuccessfully" << endl;
	delete OUT;
}

Participants:

The CUT and the associated test class are the primary participants. Methods in the test class either present a single stimulus to the OUT or multiple stimuli.

Applicability:

Use the sequence test cases by using test scripts pattern for tests that can be composed from other, simpler, tests or when a test is investigating the effects of multiple messages on a single OUT.

Benefits:

• Test scripts provide a means of using multiple test cases to build and test more complex interactions.
• Various types of standard test coverages such as functional and structural, required for the HIT analysis, can be achieved by aggregating test cases by use of test scripts.

Risks:

Incorrect sequencing of test cases in scripts can result in misinterpreted test results.

Resulting Context:

This compositional organization of test cases results in an increased ease of automation of executing tests.

Patterns Referenced:

Pattern Test a method in the Context of a class