Test adequacy measurement using a combination of criteria

The traditional method of assessing software quality is through software testing. In the past two decades, numerous test criteria have been proposed. Although the generation and evaluation of a test set is important, the ultimate goal is to ensure the quality of the software under test. It is risky to validate software by using test sets with respect to an arbitrarily selected test adequacy criterion as it can lead to incorrect conclusions on the quality of the software. In this paper we examine how different test criteria can be combined into one measurement to assess the adequacy of test sets. Based on the subsumption relation between these criteria, a multi-criterion decision making method, Analytic Hierarchy Process (AHP), is used to determine the weight of each test adequacy criterion. A case study reported here suggests that the combined criterion so generated provides a more objective and precise measurement of the fault detection capability of test sets than does a single-member test criterion.