Software Quality

Unit 4. Costs, Defects Characterization, & Ethical

Considerations

Grado en Ingeniería del Software

E.T.S. de Ingeniería en Sistemas Informáticos
Universidad Politécnica de Madrid
Curso 2024-25
Software Quality

Unit 4: Costs, Defects

Characterization
& Ethical Considerations
4.1 Costs

Software Quality 2
Cost of Quality
• Quality costs in an organization are those required
to manage quality plus those due to errors.
• Total Quality can be defined as the set of
approaches and methods which allow for
producing products at the lowest possible cost,
satisfying customers needs, and considering
employees self-realization.

Software Quality DIAPOSITIVA 3

Cost of Quality
• In the 80’s Juran pointed out that in the U.S.
business sector the third part of the work was used
to repair what was done wrong in the other two
thirds.
• Crosby notes that in a company without quality
culture costs of non-conformance are around the
20% of the business. Even more, if the company
provides services, these costs could be around the
40%.

Software Quality 4
Cost of Quality
“Costs of Quality are those which
don’t exist, if all the needed activities
were well done at first”

DIRFT: Do it Right for the First Time

Philip B. Crosby
Software Quality 5
 Costs Of Quality?

• Prevention Costs: costs incurred to prevent poor quality

(keep failure and appraisal cost to a minimum)
• Appraisal Costs: costs incurred to determine the degree
of conformance to quality requirements (measuring,
evaluating or auditing)
• Internal & External Failure Costs: costs incurred by poor
quality, costs associated with defects found before or
after the customer receives the product or service.

Software Quality 6
Are There Good Costs?
• In business, a cost can be seen to be good when it
is an investment. Thus it should be profitable.
• Hence, only costs for achieving quality can be
considered as good costs. They are costs for
prevention. Increasing prevention, revisions and
failures will go down.

Software Quality 7
Controllable Costs
• They are costs dependent on decisions taken by the
organization regarding quality.
• They are prevention and appraisal costs.
• These costs could be unlimited. The question is to
define the level in which they are kept profitable.

Software Quality 8
Prevention Costs
• They are costs incurred to keep failure and
appraisal costs to minimum

• Quality planning.
• Quality consultancy.
• Quality training.
• To implement Quality management systems.
• To get UNE-EN ISO certifications.

Software Quality 9
Prevention Costs
• EFQM self-evaluation systems.
• Use of Poka-Yoke systems.
• Application of QFD during design and development
phases, and FMEA to failure analysis.
• Preventive maintenance.
• Projects for Improvement.
• Quality arranged with suppliers.

Software Quality 10
Prevention Costs

• Studies related to customers needs and expectations.

• Quality prevention manuals.
• Study of internal suggestions for improvement.
• Value analysis and feasibility studies.
• Costs related to quality circles.

Software Quality 11
Appraisal Costs
They are costs incurred to determine the degree of
conformance to quality requirements:

• Verification at materials reception.

• Inspections, homologations, and quality revisions.
• Test equipment.
• Rejected products in tests.

Software Quality 12
Appraisal Costs
• Claims analysis.
• Analysis of non-conformities.
• Customer satisfaction studies.
• Employee satisfaction polls.
• Quality audits.
• Training of tester staff.

Software Quality 13
Cost Of Non-Conformance
They represent costs incurred by failures:
• Internal failures: costs associated with defects
found before the customer receives the product or
service.
• External failures: costs associated with defects
found after the customer receives the product or
service
Custormer delivery
Design Production Sale

Redesign, Scraps, Customer service, Legal ramifications,

Delays Accidents, Reprocess... Lost Custom/reputation, Site repair, ...
Internal failures External failures
Software Quality 14
Cost Of Non-Conformance
It can be classified into:
•Tangible: it can be calculated in an objective way. It
is usually accompanied by a payout.
•Intangible: it must be calculated in a subjective way.
It is consequence of loss customers or reputation.

Costs of a failure have to be always estimated in a

marginal manner. Moreover, it must include the
profit that an organization hasn't earned.

Software Quality 15
Internal Failures Costs
These costs are usually tangibles:
• Corrective actions.
• Reworks.
• Machinery break-down, (unexpected) software
faults, software crashes.
• “Expired” products (deprecated, new libraries not
updated)
• Redesigns.
• “Waste” of any kind, scraps, craps.
Software Quality 16
Internal Failure Costs
• Reprocessing works.
• Software, equipment and machinery maintenance.
• Downtime (lack of availability).
• Retesting.
• Excessive Stock (hardware) or software units
”ready” but not used in practice.
• Loss of sales by lack of foresight.

Software Quality 17
External Failures Costs
TANGIBLES
• “Withdrawal” of defective products.
• Customers service.
• Complaints.
• Failures analysis time and its consequences.
• “Rectification” of returned products.
• Penalties for delays.

Software Quality 18
Calidad del Software
External Failure Costs
TANGIBLES
• Order cancellations.
• Warranty claims.
• Payment of compensations.
• Expenses for legal processes.
• Advertising campaigns to reduce the negative
effects of external failures.

Software Quality 19
External Failures Costs
INTANGIBLES
• Loss of income due to loss of image and reputation that
represent failures.
• Costs derived from lack of motivation of employees for
non-quality, including absenteeism and staff rotation.

Software Quality 20
Total Cost Of Quality
• TCOQ: total amount of costs associated with
prevention and appraisal (controllable) plus those
associated with product failure(poor quality).

• Logically, when increasing controllable costs, the

cost of non quality should go down.

• The optimum cost of quality can/should be found.

Software Quality 21
Total Cost Of Quality
• Failure costs have to be foreseen at any level of
controllable costs.

• There is always a level in which failure costs are so

reduced that it is not worth keep on increasing
controllable costs. This level is close to the situation
of “zero defects”.

Software Quality 22
Total Cost Of Quality
The optimum level is achieved when zero defects and
only controllable costs are got:

Total Cost Of Quality

Unitary Quality Cost

Failure costs
Optimum

Prevention and appraisal costs

Comformance

Software Quality 23
Total Cost Of Quality
Costs of Quality are an excellent tools for decision
making.
To know how much it is saved in quality costs has an
important effect in organizational management.
Mainly because it permits to obtain resources for
quality improvement and maintenance; and it
suggests the total amount of funding needed for
these activities.

Software Quality 24
Theoretical Cost Models
• There are several models for assisting in quality
management. However, none of them is better
than other. They must be considered as
complementary.
• Organizations must select the model according with
their own stage in the implementation of the
quality management system.

Software Quality 25
Theoretical Cost Models
• Prevention, Appraisal and Failure Model (PAF).
• Cost-Benefit Model.
• Taguchi’s Loss Function Model.
• Quality Process-Cost – BS 6142-1:1992
• ABC Model (Activity-Based-Costing)
• Financial and Nonfinancial Measures.
• Total Quality Management Model.

Software Quality 26
Prevention, Appraisal & Failure
Model (PAF-Model)
Classical Juran’s Model (1951)
Total Cost of Quality

Costs per Prevention +

product Appraisal costs
unit Failure
costs

Quality Level

Improvement Zone Indifference Zone

Failure costs > 70% Perfection Zone
Failure costs ≈ 50%
Prevention costs< 10% Failure costs <10%
Prevention costs ≈ 10%
Prevention costs > 70%

Software Quality 27
Prevention, Appraisal & Failure
Model (PAF-Model)
Revised approach by Juran & Gryna 1988
• It is admitted that it is possible to get perfection with
finite costs.

Total Cost Of Quality

Quality Cost

Prevention & appraisal costs

Internal & external failure cost

Quality of Conformance
Software Quality 28
Cost-Benefit Model

• This model aims to analyze the influence that the cost of

quality has on the optimum number of sales to achieve
a particular advantage in the financial year.
• The purpose of the cost - benefit model is to help
companies to decide how, when and where to invest in
prevention or equipment.

Software Quality 29
Total Quality Management Model
• One of the pillars of Total Quality Management is that, before you
can solve a problem, first it must be measured; because if the
problem cannot be measured, we cannot determine if the
solution is correct and the desired improvement has been
produced (Stanleigh, 1993).
• This model doesn’t accept the optimal point between prevention
and failures. The model focus the attention on long term defect
prevention, because the consequence will be total costs
decreasing.

Software Quality 30
Total Quality Management Model
• Traditional approach of quality cost program used to justify
quality control activities, is not enough to get total quality
management targets (Pippit, 1969).
• It is not considered necessary to separate prevention and
appraisal cost as it is done in traditional costs system. As the
model is focused on continuous improvement, to keep prevention
and appraisal cost low is not priority (Daniel y Reitsperger, 1991).
• Some authors propose to eliminate prevention costs from the
model, because they consider those costs as investment.

Software Quality 31
4.2 Defects
Characterization

Software Quality 32
Defects
(1) imperfection or deficiency in a work product where that
work product does not meet its requirements or
specifications, and needs to be either repaired or replaced
(2) an imperfection or deficiency in a project component
where that component does not meet its requirements or
specifications and needs to be either repaired or replaced
(3) generic term that can refer to either a fault (cause) or
a failure (effect)
(4) imperfection or deficiency in a work product or
characteristic that does not meet its requirements or
specifications

1. ISO/IEC 23531:2020, Systems and software engineering

2. A Guide to the Project Management Body of Knowledge (PMBOK(R) Guide) --
Sixth Edition
3. IEEE 982.1-2005 IEEE Standard Dictionary of Measures of the Software Aspects
of Dependability
4. IEEE 2675-2021, IEEE Standard for DevOps: Building Reliable and Secure
Systems Including Application Build, Package, and Deployment

Software Quality 33
Defects Detection

Find out defects in an early phase of the process

allows to correct them promptly, and to keep the rest
of phases within the initial foresight. This is very
important to satisfy business and clients interests.

Software Quality 34
 Defects detection (1)
Usually, defects are searched during test stages, not during
initial stages of development. This means that it is necessary
to make a bigger effort in quality management.

80%

60%

40%

20%

0%
Requirement Development Acceptance Production
and Design and Unitary Tests
Tests

Software Quality 35
Defects detection (2)
Quality control allows to save the maximal amount of
money when it is done at the beginning of the
software development process.
Errors detected at the beginning of the software
development process are solved easily and they are
less expensive than those detected later on.
A defect detected in system requirement definition
needs one hour to be solved. The same defect
detected in production phase will need at least 100
hours. (Barry Boehm, Software Economics).

Software Quality 36
Defects classification
Where defects occur Why defects occur

• Specification • Missing
• Design • Unclear
• Code • Wrong
• Environment • Changed
• Documentation • Better way

Software Quality 37
Specification defects
Incorrectness in the definition of customers needs
for a component or for the system.
• Requirements don’t properly describe customer’s
needs (Specification).
• Product characteristics incompatible or wrong
(Functionality).
• Wrong specifications about product interaction
with its environment (Interface).
• Wrong description about what software does
(Functional description).
Software Quality 38
Design defects (1)
Errors in system or components design.
Such errors can occur in algorithms, logic of control, data
structures, access to data bases, input/output formats,
interfaces descriptions.
Defects can be produced by a poor requirement
specification, and it may lead to an incorrect design
which will will also produce incorrect code.

Software Quality 39
Design defects (2)
• Problems with incorrect design about how product
interacts with users and environment. (Interface)
• Design doesn’t catch module/product functionality.
They are defects found out during design
inspection or during implementation (Functional
description).
• Processes interfaces or processes communications
which don’t work properly (Processes
communications).

Software Quality 40
Design defects (3)
• Incorrect module/product data structure design
(Data definition).
• Problems with control and execution flow between
processes (Module design).
• Logic is badly described during design (Logic
description).
• Design doesn’t fit to standards (Standards).

Software Quality 41
Code defects (1)
Bugs or errors during program implementation. They
can be in the product, in test code, files, etc.
Defects can be produced by a poor design
comprehension, by a bad selection of data structures
and algorithms, or by logical errors or syntactical
errors.

Software Quality 42
Code defects (2)
• Loss of precision, incorrect equations, etc.
(Computational problems).
• Incorrect initialization, wrong data access or data
storage, wrong scales or unit, bad data sizing, etc.
(Data manipulation problems).
• Forgotten steps, duplicated logic, unnecessary
functions, etc. (Logic).
• Problems related to calls, parameters, sub-
processes halts, etc. (Implementation).
• Code doesn’t follow standards (Standards)
Software Quality 43
Environment defects
They are resulting from development and test
environments such as: configuration errors, errors in
tools integration, etc.
•Test software problems.
•Test hardware problems.
•Inappropriate behavior of development tools.
•Integration software problems.

Software Quality 44
Defects in Documentation
• Errors in manuals, installation instructions,
demonstrations, all of them delivered to
customers

Software Quality 45
Defect Classification
• Missing: information doesn’t appeared in the
intermediate product.
• Unclear: information is misleading, ambiguous, or
difficult to be understood.
• Wrong: information is clearly incorrect.
• Changed: changes in the intermediate product will
cause changes in other products.
• Better way: there is a better way to perform
intermediate product, motivated by: efficiency,
performance, legibility, maintainability, etc.
Software Quality
Defect Classification by severity
• Critical severity: the whole system is stopped
• High severity: the whole system is not but essential
parts can be operated
• Medium severity: the system can be operated but
main functionalities are affected, though work-
arounds can be found
• Low severity: the operation of the system (main
functionalities) is not disrupted

Software Quality
Defects and failures
• A defect can cause a failure
• Failure severity (Unit 2)
• Defect removal priority will depend, at least, on the
failure severity

Software Quality 48
 Failure Severity (ECSS-Q-ST-30-02C, 6 March 2009 )
Severity Severity Dependability Safety effects
category level effects (as specified in ECSS‐Q‐ST‐40)
(as specified in
ECSS‐Q‐ST‐30)

Catastrophic 1 Failure propagation Loss of life, life‐threatening or permanently disabling injury

or occupational illness.
Loss of an interfacing manned flight system. Severe
detrimental environmental effects. Loss of launch site
facilities.
Loss of system.
Critical 2 Loss of mission Temporarily disabling but not life‐threatening injury, or
temporary occupational illness.
Major detrimental environmental effects. Major damage to
public or private properties. Major damage to interfacing
flight systems. Major damage to ground facilities.
Major 3 Major mission degradation

Minor or 4 Minor mission degradation

negligible or
Any other effect
Software Quality 49
Static Analysis and defects (1)
Static analysis is as important as formal reviews.
• Static analysis searches for defects without
code execution. It is carried out once the code
is written.
• Its target is to find out defects in source code,
and in SW models to improve their quality.
• Static analysis can find out errors that are very
difficult to be found during tests execution.

Software Quality 50
Static Analysis and defects (2)
ADVANTAGES
• Early defects detection, before tests execution.
• Early warning about suspicious aspects of code or
even design.
• To detect failure to follow development standards.
• To improve code maintainability and design.

Software Quality 51
Static Analysis and defects (3)
• Defect classification:

• Variables which never are used.

• Code which never shall be executed (dead code).
• Security vulnerability.
• Use of variables without value.
• Infringement of programming standards.

Software Quality 52
Quality and Defects
• 50% SW cost can be attributed to errors corrections
(USA Defense Department).
• Where are software problems coming from?
• Requirement misunderstanding: 50%
• Design doesn’t fit requirements: 30%
• Bad coding: 20%

Software Quality 53
4.3 Ethical
Considerations

Software Quality 54
 IEEE-ACM code of ethics
See The Software Engineering Code of Ethics and Professional Practice

Code of ethics focuses on the SW engineer as the main

ethical subject.
Autonomous systems or AI-based systems postulate the
system/product as another ethical agent interacting with
society and thus new principles could be added to the
code:

• Transparency and Explainability: for example,

GDPR alludes to the right of an individual to receive
an explanation about decisions made by an
automated system.)
• Legal compliance: the system shall adhere to the
legal requirements at all times and be accountable for
that.

Transparency and Explainability Legal compliance

Figure adapted from Weyns, Danny. "Towards a code of ethics for autonomous and self-adaptive systems." Proceedings of the
IEEE/ACM 15th International Symposium on Software Engineering for Adaptive and Self-Managing Systems. 2020.

Software Quality 55