Practical Measurements For Software Testing

Every software development company focuses on developing quality software. The only way to track the software quality is

evaluating it at every stage of its development. It requires some kind of metrics, which is obtained through effective

testing methods. Each stage of software testing is effectively monitored for the software QA.

1. Software measurements are used for:

2. Deriving basis for estimates

3. Tracking project progress

4. Determining (relative) complexity

5. Understanding the stage of desired quality

6. Analyzing defects

7. Validating best practices experimentally

Here, some software testing metrics are proposed for black box testing that has real world applications. It discusses:

Importance of software testing measurement

Different techniques/processes for measuring software testing

Metrics for analyzing testing

Methods for measuring/computing the metrics

Advantages of implementing these metrics

These metrics helps in understanding the inadequacies in different software QA stages and finding better correcting


What is measurement and why it is required?

The process of assigning numbers or symbols to attributes of real world entities for describing them according to defined

rules is called measurement.

For developing quality software, several characteristics like requirements, time and effort, infrastructural cost,

requirement testability, system faults, and improvements for more productive resources should be measured.

Measuring software testing is required:

1. If the available test cases cover all the systems aspects

2. For tracking problems

3. For quantifying testing

Choose the suitable metrics

Several metrics can measure software-testing process.

Here, the following types of metrics are identified:

Base metrics:

These raw data are collected in a testing effort and applied in formulae used to derive Calculated Metrics.

The Test Metrics comprise of the Number of

Test Cases Passed, Failed, Under Investigation, Blocked, Re-executed and Test Execution Time.

Calculated metrics:

They convert the Base Metrics data into useful information. Every test efforts must implement the following Calculated


% Complete

% Defects Corrected

% Test Coverage

% Rework

% Test Cases Passed & Blocked

% Test Effectiveness & Efficiency

% 1st Run Failures

% Failures

Defect Discovery Rate

Defect Removal Cost

Measurements for Software Testing

The corresponding software testing process in software development measures each step for ensuring quality product delivery.

1. Software Size:

The amount of functionality of an application determines this and is calculated by

Function Point Analysis

Task Complexity Estimation Methodology

2. Requirements review:

Before software development, the Software requirement specifications (SRS) from the client are obtained. It must be:











The Review Efficiency is a metric that offers insight on the review quality and testing.

Review efficiency=100*Total number of defects found by reviews/Total number of project defects

3. Effectiveness of testing requirements:

It is measured by maintaining Requirement Trace-ability matrix and specification of requirements, which should have:

SRS Objective, purpose


Functional Capabilities

Performance Levels

Data Structures/Elements Safety




Constraints & limitations

Next comes the updating of the crucial requirement trace-ability matrix or RTM, which determines the number and types of


While measuring the mapping of test cases, the number and priority of requirement it tests, its execution effort and

requirement coverage must be determined.

The Requirement compliance factor (RCF) measures the coverage provided by the test cases to one or set of requirement(s).




j is a set of requirements and (j=1-m);

Xi=2, if the test case (say Tj) tests requirements Ri completely,

=1, if it tests partially,

=0, if otherwise.
Effectiveness=RCFj/Ej where Ej=Time required for executing a test case

4. Evaluating estimation accuracy

Relative error=(A-E)/A where E is estimate of a value and A is actual value.

For a collection of estimates, the mean RE for n projects is

__ n

For a set of n projects, the mean magnitude of RE (MRE) is

___ n

Of a set of n projects, an acceptable level for MRE is less than 0.25.

If K is the number of projects whose mean magnitude of relative error is less than or equal to q,
then the prediction quality pred(q)=K/n

5. Measurement of Efficiency in testing process

In software testing, we must keep tabs on what we had planned and what we have actually achieved for measuring efficiency.

Here, the following attributes play major roles: -

Cost: The Cost Variance (CV) factor measures the risk associated with cost.

CV=100*(AC - PC)/PC, AC=Actual Cost, PC=Planned/Budgeted Cost.

Effort: Effort Variance (EV) measures effort.

EV=100*(AE - PE)/PE
(AE=Actual Effort, PE=Planned Effort)

Schedule: Schedule Variance (SV) is important for project scheduling.

SV=100*(AD-PD)/PD where AD=Actual duration and PD=Planned duration.

Cost of quality: It indicates the total effort expended on prevention, appraisal and rework/failure activities versus all

project activities.

Prevention Effort=Effort expended on planning, training and defect prevention.
Appraisal Effort=Effort expended on quality control activities.

Failure effort=Effort expended on rework, idle time etc.

COQ=100*(PE + AE + FE)/Total project effort.

Product -

Size variance: It is the degree of variation between estimated and actual sizes.
Size Variance=100*(Actual Software Size-Initial Estimated Software Size)/Initial Estimated Software Size

Defect density: It is the total number of defects in software with respect to its size.

Defect density=Total number of defects detected/software size

Mean Time Between Failures: MTBF is the mean time between two critical system failures or breakdowns.

MTBF=Total time of software system operation/Number of critical software system failures.

Defects: Defects are measured through:

Defect distribution: It indicates the distribution of total project defects.
Defect Distribution=100*Total number of defects attributed to the specific phase/Total number of defects.

Defect removal effectiveness: Adding the number of defects removed during the phase to the number of defects found later

approximates this.

Benefits of implementing metrics in software testing:

Improves project planning.

Understanding the desired quality achieved.

Helps in improving the processes followed.

Analyzing the associated risks.

Improving defect removal efficiency.

Share this post