UNIT-2

Software quality metrics are a subset of software metrics that focus on the quality
aspects of the product, process, and project. These are more closely associated with
process and product metrics than with project metrics.
Software quality metrics can be further divided into three categories −
 Product quality metrics
 In-process quality metrics
 Maintenance quality metrics

Product Quality Metrics

This metrics include the following −

 Mean Time to Failure
 Defect Density
 Customer Problems
 Customer Satisfaction
Mean Time to Failure
It is the time between failures. This metric is mostly used with safety critical
systems such as the airline traffic control systems, avionics, and weapons.
Defect Density
It measures the defects relative to the software size expressed as lines of code or
function point, etc. i.e., it measures code quality per unit. This metric is used in
many commercial software systems.
Customer Problems
It measures the problems that customers encounter when using the product. It
contains the customer’s perspective towards the problem space of the software,
which includes the non-defect-oriented problems together with the defect
problems.
The problems metric is usually expressed in terms of Problems per User-Month
(PUM).
PUM = Total Problems that customers reported (true defects and non-defect
oriented Problems) for a time period + Total number of license months of the
software during the period
Where, Number of license-months of the software = Number of installed license of
the software × Number of months in the calculation period
PUM is usually calculated for each month after the software is released to the
market, and also for monthly averages by year.
Customer Satisfaction
Customer satisfaction is often measured by customer survey data through the five-
point scale −
 Very satisfied
 Satisfied
 Neutral
 Dissatisfied
 Very dissatisfied
Satisfaction with the overall quality of the product and its specific dimensions is
usually obtained through various methods of customer surveys. Based on the five-
point-scale data, several metrics with slight variations can be constructed and used,
depending on the purpose of analysis. For example −
 Percent of completely satisfied customers
 Percent of satisfied customers
 Percent of dis-satisfied customers
 Percent of non-satisfied customers
Usually, this percent satisfaction is used.

In-process Quality Metrics

In-process quality metrics deals with the tracking of defect arrival during formal
machine testing for some organizations. This metric includes −
 Defect density during machine testing
 Defect arrival pattern during machine testing
 Phase-based defect removal pattern
 Defect removal effectiveness

Defect density during machine testing

Defect rate during formal machine testing (testing after code is integrated into the
system library) is correlated with the defect rate in the field. Higher defect rates
found during testing is an indicator that the software has experienced higher error
injection during its development process, unless the higher testing defect rate is
due to an extraordinary testing effort.
This simple metric of defects per KLOC or function point is a good indicator of
quality, while the software is still being tested. It is especially useful to monitor
subsequent releases of a product in the same development organization.
Defect arrival pattern during machine testing
The overall defect density during testing will provide only the summary of the
defects. The pattern of defect arrivals gives more information about different
quality levels in the field. It includes the following −
 The defect arrivals or defects reported during the testing phase by time
interval (e.g., week). Here all of which will not be valid defects.
 The pattern of valid defect arrivals when problem determination is done on
the reported problems. This is the true defect pattern.
 The pattern of defect backlog overtime. This metric is needed because
development organizations cannot investigate and fix all the reported
problems immediately. This is a workload statement as well as a quality
statement. If the defect backlog is large at the end of the development cycle
and a lot of fixes have yet to be integrated into the system, the stability of the
system (hence its quality) will be affected. Retesting (regression test) is
needed to ensure that targeted product quality levels are reached.
Phase-based defect removal pattern
This is an extension of the defect density metric during testing. In addition to
testing, it tracks the defects at all phases of the development cycle, including the
design reviews, code inspections, and formal verifications before testing.
Because a large percentage of programming defects is related to design problems,
conducting formal reviews, or functional verifications to enhance the defect
removal capability of the process at the front-end reduces error in the software.
The pattern of phase-based defect removal reflects the overall defect removal
ability of the development process.
With regard to the metrics for the design and coding phases, in addition to defect
rates, many development organizations use metrics such as inspection coverage
and inspection effort for in-process quality management.
Defect removal effectiveness
It can be defined as follows −
DRE = Defect removed during a development phase /Defects latent in the
product * 100%
This metric can be calculated for the entire development process, for the front-end
before code integration and for each phase. It is called early defect removal when
used for the front-end and phase effectiveness for specific phases. The higher the
value of the metric, the more effective the development process and the fewer the
defects passed to the next phase or to the field. This metric is a key concept of the
defect removal model for software development.

Maintenance Quality Metrics

Although much cannot be done to alter the quality of the product during this phase,
following are the fixes that can be carried out to eliminate the defects as soon as
possible with excellent fix quality.
 Fix backlog and backlog management index
 Fix response time and fix responsiveness
 Percent delinquent fixes
 Fix quality
Fix backlog and backlog management index
Fix backlog is related to the rate of defect arrivals and the rate at which fixes for
reported problems become available. It is a simple count of reported problems that
remain at the end of each month or each week. Using it in the format of a trend
chart, this metric can provide meaningful information for managing the
maintenance process.
Backlog Management Index (BMI) is used to manage the backlog of open and
unresolved problems.
BMI = Number of problems closed during the month /Number of problems arrived
during the month* 100%
If BMI is larger than 100, it means the backlog is reduced. If BMI is less than 100,
then the backlog increases.
Fix response time and fix responsiveness
The fix response time metric is usually calculated as the mean time of all problems
from open to close. Short fix response time leads to customer satisfaction.
The important elements of fixing responsiveness are customer expectations, the
agreed-upon fix time, and the ability to meet one's commitment to the customer.
Percent delinquent fixes
It is calculated as follows −
Percent Delinquent Fixes =Number of fixes that exceeded the response time
criteria by severity level/Number of fixes delivered in a specified time * 100%
Fix Quality
Fix quality or the number of defective fixes is another important quality metric for
the maintenance phase. A fix is defective if it did not fix the reported problem, or if
it fixed the original problem but injected a new defect. For mission-critical
software, defective fixes are detrimental to customer satisfaction. The metric of
percent defective fixes is the percentage of all fixes in a time interval that is
defective.
A defective fix can be recorded in two ways: Record it in the month it was
discovered or record it in the month the fix was delivered. The first is a customer
measure; the second is a process measure. The difference between the two dates is
the latent period of the defective fix.
Usually the longer the latency, the more will be the customers that get affected. If
the number of defects is large, then the small value of the percentage metric will
show an optimistic picture. The quality goal for the maintenance process, of
course, is zero defective fixes without delinquency.
Software quality indicator

An indicator usually compares a metric with a baseline or expected result.

Software quality indicators act as a set of tools to improve the management
capabilities of personnel responsible for monitoring software development
projects.

The following quality indicators are used during the software testing &
development life cycle.
1) Progress: Measures the amount of work accomplished by the developer in each
phase. This measure flows through the development life cycle with a number of
requirements defined and baselined, then the amount of preliminary and detailed
design completed, then the amount of code completed, and various levels of tests
completed.
2) Stability: Assesses whether the products of each phase are sufficiently stable to
allow the next phase to proceed. This measures the number of changes to
requirements, design, and implementation.
3) Process compliance: Measures the developer’s compliance with the
development procedures approved at the beginning of the project. Captures the
number of procedures identified for use on the project versus those complied with
on the project.
4) Quality evaluation effort: Measures the percentage of the developer’s effort
that is being spent on internal quality evaluation activities. Percent of time
developers are required to deal with quality evaluations and related corrective
actions.
5) Test coverage: Measures the amount of the software system covered by the
developer’s testing process. For module testing, this counts the number of basis
paths executed/covered, & for system testing it measures the percentage of
functions tested.
6) Defect detection efficiency: Measures how many of the defects detectable in a
phase were actually discovered during that phase. Starts at 100% and is reduced as
defects are uncovered at a later development phase.
7) Defect removal rate: Measures the number of defects detected and resolved
over a period of time. Number of opened and closed system problem reports (SPR)
reported through the development phases.
8) Defect age profile: Measures the number of defects that have remained
unresolved for a long period of time. Monthly reporting of SPRs remaining open
for more than a month’s time.
9) Defect density: Detects defect-prone components of the system. Provides a
measure of SPRs / Computer Software Component (CSC) to determine which CSC
is the most defect-prone.
10) Complexity: Measures the complexity of the code. Collects basis path counts
(cyclomatic complexity) of code modules to determine how complex each module
is.