Many software testers find that testing a software application is an uphill battle.  The test group is usually understaffed and over worked.   There never seems to be enough emphasis put on Quality Assurance and testing in the project life cycle. Testers often find themselves with a large set of test cases but not enough time to run them.  A major source of the problem is the sheer number of test combinations in today’s complex software applications.  Attempts to exhaustively test all combinations is not always possible.  This leaves us with an important question:  Which test combinations should be tested and how does one know those test combinations provide  the best test coverage?  Testers need to be able to minimize the test effort but maximize test coverage and quality.  By singling out and testing all combinatorial pairs each state transition is tested once.  Pairwise testing is a proven technique that provides this.

The following example illustrates pairwise testing:

Let’s pretend that you work for the power company and want to test the transmission of power through newly installed power substations and lines across Texas, New Mexico, Arizona and California.  All states have 3 Sub stations that are connected to each sub station in the adjoining state.  The grid looks like this:

From this we know that our test data is as follows:

We can determine the total number of test combinations by multiplying the number of values in each column with each other resulting in 3x3x3x3=81 total test combinations.  The tests can be constructed as follows:

Test 1: CA Station 1 to AZ Station 1 to NM Station 1 to TX Station 1.

Test 2: CA Station 1 to AZ Station 1 to NM Station 1 to TX Station 2.

Test 3: CA Station 1 to AZ Station 1 to NM Station 1 to TX Station 3.

Test 4: CA Station 1 to AZ Station 1 to NM Station 2 to TX Station 1.

Test 5: CA Station 1 to AZ Station 1 to NM Station 2 to TX Station 2.

Test 6: CA Station 1 to AZ Station 1 to NM Station 2 to TX Station 3.

Test 7: CA Station 1 to AZ Station 1 to NM Station 3 to TX Station 1.

Test 8: CA Station 1 to AZ Station 1 to NM Station 3 to TX Station 2.

Test 9: CA Station 1 to AZ Station 1 to NM Station 3 to TX Station 3.

…and so forth…up to 81 tests.

So the first test would be to transmit power from CA Station 1 to AZ Station 1 to NM Station 1 and end at TX Station 1.  Let’s say it takes 5 minutes to perform each test.  That means it will take us 6 hours and 45 minutes to complete all 81 tests.  However, is it really necessary to run all 81 tests?  If not, which of the 81 tests do you run?

Take a look at Tests 1 – 9 in the above table.  Let’s say the power line between CA Station 1 and AZ Station 1 is broken.  In this scenario, each of tests 1 – 9 would fail because power cannot be transmitted all the way to the Texas Power Station.  Each of the first 9 test cases requires this section of the line to work.  So if test 1 fails we know tests 2 through 9 will also fail and hence they do not need to be run.  It is only necessary to test power transmission between CA Station 1 to AZ Station 1 once.  This is just one pair.

By letting SmartTest determine all the pairs for this example, we end up with 9 unique pairwise test combinations.

The above 9 tests ensure that all of the pairs for our test data are tested at least once, giving us the same coverage as if we ran all 81 exhaustive tests.  For this example, running 9 tests would take us only 45 minutes – saving us 6 hours of testing time but still providing the same results.  Generating pairwise test cases minimizes the test effort but maximizes the test coverage.