Test Principles and Concepts

Domain 1
 Key Components
• Why do we test software
• Understanding the testing challenge
• Structural versus functional techniques
• Quality principles
• Life cycle testing
• The people challenges of software testing
 Why Do We Test Software?
• What is a defect?
– What is quality software?
• IT’s view - means meeting requirements
• User’s of software view - means fit for use
• The two software quality gaps
– User gap
– IT gap
• What is a defect to a software tester?
Why Does a Development Process
Produce Defects?
• Variability is the “enemy” of quality – the
concepts behind maturing a software
development process is to reduce
variability.
• What does it mean for a process to be in
or out of control?
• Special causes of variation
• Common causes of variation
Reducing the Frequency of Defects
in Software Development
• Five levels of maturity
– Ad hoc
– Control
– Core competency
– Predictable
– Innovative
• Testers need to understand process maturity
An Effective Development Process
that Minimizes Defects
Incorporates
• The concept of quality
• Reducing variability
• Reducing or eliminating defects
 PDCA View of a Process
• P - Devise a plan
• D - Execute (do) the plan
• C - Check the results
• A - Take the necessary action
 Workbench View of a Process
• Each workbench is built on the following
two components
– Objective – why the process exists
– People skills – roles, responsibilities and
associated skill sets needed to execute a
process
 Workbench Components
• Inputs
• Procedures
• Deliverables
• Standards
• Tools
 Why do we test software – The
Answer
• We test software because the processes
to build software are defect-prone
– Traditional way of software testing identifies
where the specification has been incorrectly
implemented, incompletely implemented or is
missing
– Software testing compensates for the fact that
the software development process does not
identify the true needs of the user
• Software testing can add significant value
to the IT organization.
– Finding defects early costs considerably less
– Remove defects before going into production
– Identify weaknesses in development process
so that those processes can be improved
– Mature processes produce software more
effectively and efficiently
 Understanding the Testing
Challenge
• Test Objectives
– A testing Goal
– Enable test manager and project manager to
gauge testing progress and success
– Enhance communication within and without
project team by defining scope of test effort
– Statement of the objective & expected results
– Created from system requirements document
– Use a checklist to ensure process is followed
 Strategic Risks
• A risk is a condition that can result in a
loss
• We cannot eliminate risks, but we can
reduce their occurrence and/or impact of
the loss
• One of the most effective methods to
reduce computer system strategic risk is
testing
 Economics of Testing
• Most problems associated with testing
occur from one of the following causes:
– Failing to define testing objectives
– Testing at the wrong phase of the life cycle
– Using ineffective test techniques
 Testing Policy

• Definition of testing
• Testing system
• Evaluation
• Standards
 What Are You Testing For?
• Why are defects hard to find?
– Not looking
– Looking, but not seeing
• Corrected condition causes another defect
• Life cycle testing
 Verification and Validation
• Verification ensures that the system (software,
hardware, documentation and personnel)
complies with the organization’s standards and
processes
– Answers question “Did we build the right system?”
• Validation physically ensures that the system
operates according to plan by executing the
system functions through a series of tests that
can be observed and evaluated
– Answers question “Did we build the system right?”
 Functional and Structural Testing
• Functional testing
– Advantages
• Simulates actual system usage
• Makes no system structure assumptions
– Disadvantages
• Potential of missing logical errors in software
• Possibility of redundant testing
• Structural testing
– Advantages
• You can test the software’s structure logic
• You test code that you wouldn’t use if you preformed only functional testing
– Disadvantages
• Does not ensure that you’ve met user requirements
• It’s tests may not mimic real-world situations
 Testing Techniques
• White-box testing
– Statement coverage
– Decision coverage
– Condition coverage
– Decision/condition coverage
– Multiple condition coverage
• Black-box testing
– Equivalence partitioning
– Boundary analysis
– Error guessing
• Incremental testing
– Top-down
– Bottom-up
• Thread testing
• Independent testing
• Regression testing
 Reviews and Inspections

• Three types of reviews

– In-process
– Decision-point or phase-end
– Post implementation
• Three classes of reviews
– Informal or peer
– Semiformal or walkthroughs
– Formal or inspections
 Structural System Testing
Techniques
• Stress
• Execution
• Recovery
• Operations
• Compliance (to process)
• Security
 Functional System Testing
Techniques
• Requirements
• Regression
• Error handling
• Manual support
• Intersystem
• Control
• Parallel
 Quality Principles
• What is Quality?
– Five perspectives of quality
• Transcendent – I know it when I see it
• Product-based – Possesses desired features
• User-based – Fitness for use
• Development- and manufacturing-based –
Conforms to requirements
• Value-based – At an acceptable cost
 Definitions of Quality
• Frequently defined as meeting the customer’s
requirements the first time and every time
• Conformance to a set of customer requirements
that, if met, result in a product that is fit for its
intended use
• Much more than the absence of defects
• Requires controlled process improvement
• Can only be seen through the eyes of the
customers
 Why Quality

• Cost of poor quality

• Quality philosophies
– Deming philosophy
– Michael Tveite’s regrouping of Demings 14 points
– Juran’s ten steps to quality improvement
– Cost of quality
• Prevention costs
• Appraisal costs
• Failure costs
 Quality Assurance versus Quality
Control
• Definitions
– Quality assurance is a planned and systematic set of
activities necessary to provide adequate confidence
that products and services will conform to specified
requirements and meet user needs. Staff function.
– Quality control is the process by which product quality
is compared with applicable standards, and the action
taken when nonconformance is detected. Line
function
 Life Cycle Testing

• Requirements
• Design
• Program (build/construction)
• Test process
• Installation
• Maintenance
 Quality Factors
• Correctness
• Authorization
• File Integrity
• Audit Trail
• Continuity of Processing
• Service Levels
 Developing a Test Plan
• Select and rank quality factors
• Identify the system development phases
• Identify the business risks associated with
the system under development
• Place risks in the matrix
Type of Development Methodology
Impacts the Test Plan
• Traditional system development (and most
major changes to existing software
systems
• Iterative development/prototyping/CASE
• System maintenance
• Purchased/contracted software
 People Challenges of
SoftwareTesting
• Negative view of testing and testers
– Testers hold up implementation
• Top ten people challenges
– Training in testing
– Relationship building with developers
– Using tools
– Getting managers to understand testing
– Communicating with users about testing
– Making the necessary time for testing
– Testing “over the wall” software
– Trying to hit a moving target
– Fighting a lose-lose situation
– Having to say “no”
 Raise Management Awareness of
the Importance of Testing
• Calculate costs of testing and ensure the costs are
understood by management
• Show management ways that the testing costs can be
reduced by using more effective techniques
• Relay other benefits of testing
• Show that test training is built into your own personal
goals and objectives, and discuss your training needs
with your immediate supervisor
• Collect and distribute articles about testing
• Inquire about the budget for testing
• Use creative means to enlighten management
 Model for Test Process
Improvement
• Examine the organization’s needs and business
goals
• Conduct assessment
• Initiate process improvement
• Analyze assessment output and derive action
plan
• Implement improvements
• Confirm improvements
• Sustain improvement gains
• Monitor performance