II.

TESTING PROCESS

Testing Processes: Processes in General, Test Planning and Control, Test

Analysis and Design, Test Implementation and Execution, Evaluating Exit
Criteria and Reporting, Test Closure
 Introduction
 Processes in general
 Test planning and control
 Test analysis and design
 Test implementation and execution
 Evaluating exit criteria and reporting
 Test closure

We need a process to do any task. either in real life or in s/w product building. the
development of a s/w product also need a process, testing of a s/w product also need a process.
Testing process is having several steps like
 Process in general
 Test planning and control
 Test analysis and Design
 Test implementation and execution
 Evaluating exit criteria and reporting
 Test closure
Process in General:
A process is a series of activities performed to fulfill a specific purpose. Based on an input to
the process and following activities a tangible output is produced.
 Concept of process
 Monitoring of process
 Dependency of process
 Overall test process
Concept of process:-
A process description must always include.
• A definition of the input .
• A list of activities- the procedure.
• A description of output.
For a more comprehensive and more useful process description the following information
could also be included
 Purpose—A description of what must be achieved ?
 Role—Who is going to perform the activities?
 Methods, techniques, tools—How exactly are we going to perform the activities?
 Measurements—What metrics are we going to collect for the process?
 Templates—What should the output look like?
 Verification points—Are we on the right track?
 Exit criteria—What do we need to fulfill before we can say that we have finished?
Monitoring Process:-
It is the responsibility of management in-charge of a specific area to know how the
processes are performed. here the monitoring is held on the below things
 How test is performed
 How the test is progressing
 How the test is improved
 What are the diff. metrics…
Dependency of process:-
 how o/p of one process will become i/p to another process
 identify top level ,mid level, low level processes among all processes.
The overall generic test process:- planning and control are constant activities
consists of five activities . Those are in the sense that they are not just done once
• Test Planning in the beginning of the test assignment.
• Test Analysis and design Monitoring of the process should be done on
• Test implementation and execution an ongoing basis, and controlling and
• Evaluating exit criteria and reporting replanning activities performed when the
• Test closure activities need arises.
Sometimes test analysis and design is
performed in parallel with test
implementation and execution. A model is
not a scientific truth; when using a model,
even a very well-defined model, we should
be open for necessary tailoring to specific
situations
The generic test process is iterative—
not a simple straightforward process. It must
be foreseen that we'll have to perform the
activities more than once in an iterative way
before the exit criteria have been fulfilled.
The iterations to be foreseen in the test
The test activities need not be process are shown in the figure here.
performed in strict sequential order. Test
Test planning and control:
The planning must first be done at the overall level resulting in a master test plan. The
detailed planning for each test level is based on this master test plan. Identical planning
principles apply for the overall planning and the detailed planning
The purpose of the control part is to ensure that the planned activities are on track by
monitoring what is going on and take corrective actions as appropriate.
The inputs on which this process is based are:
• Test strategy
• Master test plan
• Information about how the testing is progressing
The activities are:
• Verify the mission and define the objectives of the testing
• Decide and document how the general test strategy and the project test plan apply to the
specific test level: what, how, where, who
• Make decisions and initiate corrective actions as appropriate as the testing progresses
The output consists of:
• Level test plan
Input to test planning and control:
The test level plan must be consistent with the master test plan. It must also be consolidated
with the overall plan for the project in which the testing is a part. This is to ensure that
schedules and resources correspond, and that other teams, which interface with the test team in
question, are identified.
The decisions to make in the test planning and control process are guided by the expected
contents of the test plan. Don't get it wrong: The decisions are not made for the purpose of
writing the plan, but for the purpose of getting agreement and commitment of all the
stakeholders in the test to be performed.
The planning and control of the test are continuous activities. The initial planning will take
place first. Information from monitoring what is going on as the testing progresses may cause
controlling actions to be taken.
Documentation of test planning and control:
Here it maintains test plan identifier and test plan approval.
Test plan identifier
1. Introduction (scope, risks, and objectives)
2. Test item(s) (test object(s))
3. Features to be tested
4. Features not to be tested
5. Approach (targets, techniques, templates)
6. Item pass/fail criteria (exit criteria including coverage criteria)
7. Suspension criteria and resumption requirements
8. Test deliverables (work products)
9. Testing tasks (analysis, design, implementation, execution, evaluation,
reporting, and closure; all broken down into more detailed activities in
an appropriate work break down structure)
10. Environmental needs
11. Responsibilities
12. Staffing and training needs
13. Schedule
14. Risks and contingencies
Test plan approvals
The level test plan produced and maintained in this process is input to all the other detailed test
processes. They all have the level test plan as their reference point for information and
decisions.
Activities In test planning
Test planning should start as early as possible.
The planning of the acceptance testing can start as soon as a draft of user requirements is
available.
Early planning has number of advantages like more time for planning, early identification of
problems,..
The steps here are..

Defining test object and test basis: The most common test basis and coverage items are..

 Defining approach: approach must at least cover:

  The test methods and test techniques to use
 The structure of the test specification to be produced and used
 The tools to be used The interface with configuration management
 Measurements to collect
 Important constraints, such as availability or "fixed" deadline

The interface with configuration management covers:

 How to identify and store the configuration items we produce in the test process
 How to get the configuration items we need (for example, design specifications,
source code, and requirements specifications)
 How to handle traceability
 How to register and handle incidents

 Defining completion criteria: The most appropriate completion criteria vary from
test level to test level. Completion criteria for the test may be specified as follows:
 Specified coverage has been achieved
 Specified number of failures found per test effort has been achieved
 No known serious faults
 The benefits of the system are bigger than known problems (The time has
run out)
 Defining work products and their relationships: Typical deliveries or work
products from a test level are….
 Level test plan
 Test specification test environment
 Test logs and journals
 Test reports
 Scoping and the test efforts.

Everything needs to be balanced. The time and cost of testing to enhance the quality must
be balanced with the cost of missing a deadline or having remaining defects in the product when
it goes on the market.

Work Breakdown Structure : One of the things on which the test planning is based, is a list of
all the tasks to be performed. This list should be in the form of a work breakdown structure of
the test process at hand. If we use the test process defined here the overall tasks are planning,
monitoring, control, analysis, design, implementation, execution, evaluation, reporting, and
closure, all broken down into more detailed activities in an appropriate work breakdown
structure.

Defing test roles: roles to handle the testing tasks may be

 Test leader (manager or responsible)
 Test analyst/designer
 Test executer
 Reviewer/inspector
 Domain expert
 Test environment responsible
 (Test) tool responsible
Test teams are formed by all these roles. We need different teams depend- ing on which test
phase we are working in, but the principles are the same:

 All relevant roles must be present and filled in the team

 A role can be filled by one person or more people, depending on the
size of the testing assignment at hand
 One person can fill one role or more roles, again depending on the size (but keep in mind
that less than 25% time for a role = 0% in real life)

Producing the schedule: A framework for the resources and the schedule must be obtained
from the overall project plan, and the result of the test scheduling must be reconciled with the
project plan.Estimations for all the tasks are input to the scheduling. Once the tasks are
estimated they can be fitted into the project time line. The schedule is also based on the actual
people performing the tasks, the people's efficiency, and their availability
Activities In test control:
As the testing progresses the control part of test management is about staying in control and
making necessary corrections to the plan when it no longer reflects the reality.Measurements
are collected in the test monitoring activities for all the detailed activities in the test processes,
and these measurements are analyzed to understand and follow the actual progress of the
planned test activities and the resulting coverage. Decisions must be made if things are
deviating significantly from the plan, and corrective actions may be necessary.
Metrics for test plan and control: Metrics to be defined for the monitoring and control of the
test planning and control activities themselves may include:
 Number of tasks commenced over time
 Task completion percentage over time
 Number of tasks completed over time
 Time spent on cach task over time.
This will of course have to be compared to the estimates and schedule of the test planning and
control activities.
Test analysis and design:
The purpose of the testanalysis and design activities is to produce test designs with test
conditions and testcases and necessary test environment based on the basis and the test goals
Inputs on which process is based are:
 Level test
 basis documentation
The activities are:
 Analysis of basis documentation
 Design of high-level test cases and test environment
Output consists of:
 Test design
 Test environment design and specification.
Input to Test Analysis and Design :The input from the level test plan that we need for this
process is:
 Test objectives
 Scheduling and staffing for the activities
 Definition of test object(s)
 Approach—especially test case design techniques to use and structure and contents of the
test specification
 Completion criteria, not least required coverage Deliverables
We of course also need the test basis—that is, the material we are going to test the test object
against.
Documentation of Test Analysis and Design :The result of the test analysis and design should
be documented in the test specification. This document or series of documents encompasses
 The test designs also called test groups
 The test cases-many test cases per test design
 Test procedures often many-to-many relationship with test cases
The overall structure of the test specification is defined in the level test plan. The detailed
structure is …
The test specification documentation is created to document the decisions made during the test
development and to facilitate the test execution.
Activities in Test Analysis and Design: The idea in structured testing is that the test is
specified before the execution. The test specification activity can already start when the basis
documentation is under preparation.
The test specification aims at designing tests that provide the largest possible coverage to meet
the coverage demands in the test plan. This is where test case design techniques are a great help.
 Defining test designs
 Identification of test conditions
 Creation of test cases
 Requirements
 Traceability: reference b/w test specification elements, ref. b/w test specification and
basis documentation.
 Metrics for analysis and design: no. of test conditions, no. of defects, time spend on each
task…..
Requirements are only mentioned here for understanding of the tester about the system and
its nature.these requirements are described in many phases…
 Requirement levels
 Business requirements
 User requirements
 System requirements
 Requirements types
 Environment requirements
 Functional requirements
 Non functional requirements
 Project requirements:cost,resources,time.
 Requirements styles
 Ststements
 Tasks
 Models
 Tables
Test implementation and execution: it is to organize the test cases in procedures and to
perform the physical test in the correct environment.
The inputs on which this process is based are:
 Level test plan
 Test conditions and test design
 Other relevant documents
 The test object
The activities are:
 Organizing test procedures :unique identification,description,references to highlevel test
cases..

Quality assurance of test specification will be needed to estimate the execution

time for manual execution of test procedure.

 Design and verify the test environment : h/w, s/w, perifherals, n/w, toolsand utilities,
data, physical environment, communication facilities…
 Execute the tests:checking execution entry crieteria,test execution,identifying failures,test
execution logging,..
 Record the testing: confirmation testing and regression testing are used to record the test
with more confidence.
Confirmation test : is the first to be performed after the defect correction.this is used to
execute again to give chance to success this time.
 Regression test: repetation of tests that have already been performed with out problemsto
ensure the defects have been introduced or uncovered as a result of the change.

 Check the test results using metrics for I&E:

 Number of created test environments over time
 Number of created test data over time
 Number of created test procedures over time
 Number of initiated test procedures over time
 Number of passed test procedures over time
 Number of failed test procedures over time
 Number of passed confirmation tests over time
 Number of test procedures run for regression testing over time
 Time spent on the various tasks

The output consists of:

 Test specification
 Test environment
 Test logs
 Incident reports
Evaluating exit criteria and reporting:
Test execution ,recording,control,retesting and regression testing must be continued until
we believe that the exit criteria have been achieved. All the way we need to follow what is
going on.The purpose of the test progress and completion reporting is to stay in control of the
testing and deliver the results of the testing activities in such ways that they are understandable
and useful for the stakeholders.
The inputs on which this process is based are:
 Test plan
 Measurements from the test development and execution processes
The activities are:
 Comparing actual measurements with estimates and planned values
 Reporting test results
The output consists of:
 Presentation of test progress
 Test report
Input to Test Progress and Completion
Reporting The input from the level test plan that we need for this process is:
Scheduling and staffing for the activities Exit criteria
Documentation of Test Progress and
Completion Reporting The documentation of the progress must be presented in various ways
according to who is receiving it. The audience may be the customer, higher management,
project management and participants, and testers. At the completion of each test level a test
summary report should be produced. The ultimate documentation of completion is the final test
summary report for the entire test assignment.
Activities in Test Progress and Completion Reporting: The activities related to the test
progress and completion reporting are discussed in the sections referenced above.
Metrics for progress and completion reporting:
 Number of tasks commenced over time
 Task completion percentage over time
 Number of task completed over time
 Time spent on each task over time
This will of course have to be compared to the estimates and schedule of the test progress and
completion activities.

Test Closure :The purpose of the test closure activities is to consolidate experience and place
test ware under proper control for future use.
The inputs on which this process is based are:
  Level test plan
 Test ware, including test environment
The overall procedure consists of the activities:
 Final check of deliveries and incident reports
 Secure storage/handover of test ware
 Retrospection
The output generated in this process is:
 Test experience report
 Configuration management documentation
Input to Test Closure The input from the test plan that we need for this process is:
 Scheduling and staffing for the activities
 Planned deliveries
Furthermore we need all the test ware, both the test plans and specification, we have produced
prior to test execution, the test environment, and the logs, incidents, and other reports we have
produced during and after test execution. We also need the experiences made by all the
participants and other stakeholders. These are often in the form of feelings and opinions of what
has been going on.
Documentation of Test Closure The documentation from this process is an experience report
or a retrospective report from the retrospective meeting.
Other documentation will exist in the form it is specified in the organization's and/or customer's
configuration management system.
Activities in Test Closure
Check Completion Again Before we definitively close the door to the testing assignment we
need to make extra sure that we have met the part of the exit criteria. This is both in terms of
test coverage and deliveries we are to produce. If this is not in order or any discrepancies not
clearly documented we'll have to make sure it is before we proceed.
Delivering and Archiving Test Ware The test ware we have produced are valuable assets for
the organization and should be handled carefully. For the sake of easy and economically sound
future testing in connection with defect correction and development of new versions of the
product we should keep the assets we have produced.
It is a waste of time and money not to keep the test ware we have produced.
If the organization has a well-working configuration management system this is what we must
use to safeguard the test ware.
If such a system does not exist, we must arrange with those who are taking over responsibility
for the product how the test ware must be secured. Those taking over could, for example, be a
maintenance group or the customer.
Retrospective Meeting The last thing we have to do is to report the experiences we have
gained during our testing. The measurements we have collected should be analyzed and any
other experiences collected and synthesized as well. This must be done in accordance with the
approach to process improvement expressed in the test policy and the test strategy
It is important that we as testers finish our testing assignment properly by producing an
experience report.
For the sake of the entire process improvement activity, and hence the entire organization, it is
important that higher management is involved and asks for and actively uses the test experience
reports. Otherwise, the retrospective meetings might not be held, because people quickly get
engrossed in new (test) projects and forget about the previous one.