UNIT II LEVELS OF TESTING

The Need for Levels of Testing- Unit Test Planning –

Designing the Unit Tests – The Test Harness – Running
the Unit tests and Recording Results – Integration Tests
– Designing Integration Tests – Integration Test
Planning – Scenario Testing – Defect Bash Elimination.
System Testing – Acceptance testing – Performance
testing – Regression Testing - Internationalization
testing - Ad-hoc testing – Alpha, Beta Tests- Testing
OO systems – Usability and Accessibility Testing –
Configuration Testing - Compatibility Testing – Testing
the documentation – Website Testing - Case Study for
Unit and Integration Testing.
 Levels of Testing
 Execution-based software testing, especially for large
systems, is usually carried out at different levels.
 In most cases there will be 3-4 levels of testing:

1. unit test,
2. integration test,
3. system test, and
4. acceptance test
At each level there are specific testing goals.
For example, at unit test a single component is tested.
 Levels of Testing (Cont.,)
A principal goal is to detect functional and
structural defects in the unit.
 At the integration level several components are
tested as a group, and the tester investigates
component interactions.
 At the system level the system as a whole is
tested.
 And a principle goal is to evaluate attributes
such as usability, reliability, and performance.
 System test begins when all of the components
have been integrated successfully.
Levels of Testing (Cont.,)
 Unit Testing
It is the smallest software component.
It can be characterized in several ways.
For example,
◦ It performs a single function;
◦ It can be compiled separately;
◦ It is a task in a work breakdown structure
(from the manager’s point of view);
◦ It contains code that can fit on a single page or
screen.
 Unit test
A unit is a function or procedure in a procedural
programming language.
In object-oriented systems both the method and
the class/object.
It may also be a small-sized COTS.
Commercial-Off-the-Shelf (COTS) Component.
Products that are ready-made and available for
sale to the general public are known as
Commercial-Off-the-shelf component.
For example Microsoft office.
 Unit Test (Cont.,)

Some components suitable for unit test.

 Unit Test
Advantages
◦ Since the software component being tested is
relatively small in size and
◦ It is easier to design,
◦ Easier to execute,
◦ It is easier to analyze and record test results.
◦ If a defect is discovered by the tests it is easier
to locate and repair.
◦ Since only the one unit is under consideration.
Unit Test – The Need for Preparation
The principal goal for unit testing is
Each individual software unit is functioning
according to its specification.
Planning includes designing tests to reveal
defects such as
◦ functional description defects,
◦ algorithmic defects,
◦ data defects, and
◦ control logic and
◦ sequence defects.
Unit Test – The Need for Preparation
1. Resources should be allocated
2. Test cases should be developed, using both
white and black box test design strategies.
 The unit should be tested by an independent
tester (someone other than the developer)
 The test results and defects found should be
recorded history (made public).
 Defects found are often not recorded by the
developer; they are private (not public).
 Each unit should also be reviewed by a team of
reviewers, preferably before the unit test .
Unit Test – The Need for Preparation
To prepare for unit test the
developer/tester must perform several
tasks. These are:
(i) plan the general approach to unit testing;
(ii) design the test cases, and test procedures
(these will be attached to the test plan);
(iii) define relationships between the tests;
(iv) prepare the auxiliary code necessary for unit
test.
 Unit Test Planning
A general unit test plan should be prepared.
It may be the master test plan or as a stand-alone
plan.
Documents that provide inputs for the unit test
plan are
◦ The project plan, as well the requirements,
specification, and design documents.
A unit test plan is developed to cover all the
units within a software project;
However, each unit will have its own associated
set of tests.
 Unit Test Planning
Phase 1: Describe Unit Test Approach and Risks
 In this phase the general approach of unit test planning
is outlined.
 The test planner:

(i) identifies test risks;

(ii) describes techniques to be used for designing the
test cases for the units;
(iii) recording of test results;
(iv) describes the requirements and other software that
interfaces with the units to be tested,
 Unit Test Planning
Phase 2: Identify Unit Features to be Tested
 The planner determines which features of each unit will be
tested,
 For example: state transitions, control structures, messages,
and data flow patterns.
 If some features will not be covered by the tests.
 They should be mentioned and the risks of not testing them
be assessed.
 Input/output of each unit should also be identified, such as
variables with an allowed ranges of values and performance
at a certain level.
 Unit Test Planning
Phase 3: Add Levels of Detail to the Plan
 In this phase the planner refines the plan.
 The planner adds new details like, resource, and
scheduling portions.
 As an example, existing test cases that can be reused for
this project can be identified in this phase.
 Unit availability and integration scheduling information
should be included in the revised version of the test
plan.
 Test-related documents that will be required for this
task, for example, test logs should be described.
Unit Test Worksheet
 Designing the Unit Tests
 The preparation work for unit test involves unit test
design.
 It is important to specify

(i) the test cases (including input data, and expected

outputs for each test case), and,
(ii) the test procedures
 Test case data should be tabularized for ease of use, and
reuse.
 As part of the unit test design process, developers /
testers should also describe the relationships between
the tests.
 Designing the Unit Tests
 Test cases may be reused from past projects.
 Suppose, If the organization has been careful to store them.
 They are easily retrievable and reusable.
 Test case design at the unit level can be based on the black and
white box test design strategies.
 Considering the relatively small size of a unit, it makes to focus on
white box test design.
 This approach gives the tester the opportunity to exercise logic
structures and/or data flow sequences, of the unit.
 Some black box–based testing is also done at unit level.
 However, the bulk of black box testing is usually done at the
integration and system levels.
 The test harness
A test harness or automated test framework is a
collection of software, test drivers, test data and other
supporting tools that are constructed to test an
application by executing it under different environments
and analyzing its results.
 Test harnesses allow for the automation of tests.
 A test harness should allow specific tests to run.
 A test harness provides stubs and drivers, which will be
used to replicate the missing items, which are small
programs that interact with the software under test.
 The typical objectives of a test harness are to:

1. Automate the testing process.

2. Execute test suites of test cases.
3. Generate associated test reports.
The test harness
 The test harness
 The auxiliary code developed to support testing of units and
components is called test harness.
 The harness consists of drivers that call the target code and
stubs that represent modules it calls.
 The development of drivers and stubs requires testing resources.
 The drivers and stubs must be tested themselves to insure they are
working properly and that they are reusable for subsequent releases
of the software.
 stubs and drivers are a computer program which acts as a
replacement for some other modules (which are not accessible for
testing).
 These computer programs will re-create the functionalities of the
other modules thus facilitating the software testing process.
The test harness
 The test harness
For example, a driver could have the
following options and combinations of
options:
(i) call the target unit;
(ii) do 1, and pass inputs parameters from a
table;
(iii)do 1, 2, and display parameters;
(iv)do 1, 2 and display results (output
parameters).
 Stubs and Drivers
A test harness provides stubs and drivers.
The Stub is a piece of code emulating a called
function.
Stubs are used in top down testing approach,
when one has the major module ready to test,
but the sub modules are still not ready yet. ...
These dummy pieces of code are the stubs.
On the other hand, Drivers are the ones, which
are the "calling" programs. 
Drivers are used in bottom up testing approach.
 Stubs and Drivers
Call verb is usedto transfer the control from
one program to another program.
The program that contains
the CALL verb is the Calling Program and
the program being called is known as
the Called Program. 
Calling program execution will halt until
the called program finishes the execution.
 Running the unit tests and
recording results
Unit tests can begin when
(i) the units becomes available from the
developers (an estimation of availability is
part of the test plan),
(ii) the test cases have been designed and
reviewed, and
(iii)the test harness, and any other
supplemental supporting tools, are
available.
 Running the unit tests and
recording results
 The status of the test efforts for a unit, and a summary of the test
results, could be recorded in a simple format
 These forms can be included in the test summary report, and are of
value at the weekly status meetings that are often used to monitor
test progress.
 It is very important for the tester at any level of testing to carefully
record, review, and check test results.
 The tester must determine from the results whether the unit has
passed or failed the test.
 If the test is failed, the nature of the problem should be recorded in
what is sometimes called a test incident report.
 Differences from expected behavior should be described in detail.
 This gives clues to the developers to help them locate any faults.
During testing the tester may determine that additional tests are
required.
 Running the unit tests and
recording results
 When a unit fails a test there may be several reasons for
the failure. The most likely reason for the failure is a
fault in the unit implementation (the code). Other
likely causes :
 a fault in the test case specification (the input or the
output was not specified correctly);
 a fault in test procedure execution (the test should be
rerun);
 a fault in the test environment (perhaps a database was
not set up properly);
 a fault in the unit design (the code correctly adheres to
the design specification, but the latter is incorrect).
 Integration tests
Integration test for procedural code has
two major goals:
(i) to detect defects that occur on the
interfaces of units;
(ii) to assemble the individual units into
working subsystems and finally a
complete system that is ready for system
test.
 Integration tests
Integration test should only be performed on
units that have been reviewed and have
successfully passed unit testing.
A tester might believe erroneously that since a
unit has already been tested during a unit test
with a driver and stubs, it does not need to be
retested in combination with other units
during integration test.
However, a unit tested in isolation may not
have been tested adequately for the situation
where it is combined with other modules.
 Integration tests
One unit at a time is integrated into a
set of previously integrated modules.
The interfaces and functionality of the
new unit in combination with the
previously integrated units is tested.
When a subsystem is built from units
integrated in this stepwise manner, then
performance, security, and stress tests
can be performed on this subsystem.
 Integration tests
 Two different strategies of integration testing are top-
down and bottom-up.
 These charts show hierarchical calling relationships
between modules.
 Each node, or rectangle in a structure chart, represents a
module or unit.
 The edges or lines between them represent calls between
the units.
 Bottom-up Integration tests
 The upper level module calls the lower module.
 Bottom-up integration of the modules begins with
testing the lowest level modules M6, M7, M8, M9,
M10, M11.
 These are modules that do not call other modules.
 Bottom-up Integration tests
 Driversare needed to test these modules.
 The next step is to integrate modules on the next upper
level of the structure chart whose subordinate modules
have already been tested.
 For example, if we have tested M6, M7, and M8, then
we can select M2 and integrate it with M6, M7, and M8.
 Bottom-up Integration tests
 Accordingly we can integrate M9 with M3 when M9 is
tested, and M4 with M10 when M10 is tested, and finally
M5 with M11 and M10 when they are both tested.
 The M4 and M5 subsystems have overlapping
dependencies on M10.
 Bottom-up Integration tests
 To complete the subsystem represented by M5, both
M10 and M11 will have to be tested and integrated.
 M4 is only dependent on M10.
 In that way the system is finally integrated as a whole.
 A rule of thumb for bottom-up integration says that all
of a module’s subordinate modules must have been
tested previously.
 Top-down integration
The rule of thumb for selecting candidates for
the integration sequence says that when
choosing a candidate module to be integrated
next, at least one of the module’s superordinate
(calling) modules must have been previously
tested.
We begin top-down integration with module
M1. We create four stubs to represent M2, M3,
M4, and M5.
M2-M1,M2-M6,M7,M8
Same as M3-M9,M4-M10,M5-M11
 Acceptance Tests
 Suppose software is being developed for a specific
client.
 Acceptance tests are carried out after system testing.
 The acceptance tests must be planned carefully with
input from the client/users.
 Acceptance test cases are based on requirements.
 Acceptance tests are a very important milestone for the
developers.
 At this time the clients will determine if the software
meets their requirements.
 Development organizations will often receive their final
payment when acceptance tests have been passed.
 Acceptance Tests
 After acceptance testing the client will point out to the
developers which requirement have not been satisfied.
 Some requirements may be deleted, modified, or added
due to changing needs.
 If the client is satisfied that the they give their approval.
 Then the next step is to install the system at the client’s
site.
 If the software has been developed for the mass market,
then testing it for individual clients/users is not
practical.
 So, this type of software undergoes two stages of
acceptance test.
 Acceptance Tests
 The first is called alpha test.
 This test takes place at the developer’s site.
 A users and members of the developer’s organization are
invited to use the software.
 Developers observe the users and note problems.
 Beta test sends the software to a users who install it and
use it under real world working conditions.
 The users send records of problems with the software to
the development organization.
 Where the defects are repaired sometimes in time for the
current release.
 In many cases the repairs are delayed until the next
release.
 Alpha Testing
Alpha and Beta Tests are types of Acceptance
Testing.
When software is being developed for a specific
client, acceptance tests are carried out after
system testing.
Retesting may have to be done to ensure that the
software works as required by the client. This is
called installation test.
Alpha, Beta and Acceptance tests allow users to
evaluate the software in terms of their
expectations.
Alpha Testing
 Alpha Testing
 Itis performed to identify all possible issues/bugs
before releasing the product to everyday users or
public.
 The aim is to carry out the tasks that a typical user
might perform.
 Alpha testing is carried out in a lab environment and
usually the testers are users and internal employees of
the organization.
 This kind of testing is called alpha only because it is
done early on, near the end of the development of the
software, and before beta testing.
 Alpha Testing
Advantages
 Ability to provide early detection of errors with
respect to design and functionality.
 Provides better view about the reliability of the
software at an early stage.

Disadvantage
In depth functionality cannot be tested as
software is still under development stage.
 BETA TESTING
 Beta Testing of a product is performed by "real users"
of the software application in a "real environment"
and can be considered as a form of external user
acceptance testing.
 Beta version of the software is released to a limited
number of end-users of the product to obtain feedback
on the product quality.
 Beta testing reduces product failure risks and
provides increased quality of the product through
customer validation.
 It is the final test before shipping a product to the
customers.
 BETA TESTING
 Traditional Beta testing: Product is distributed to the target
market, and related data is gathered in all aspects.
 Public Beta Testing: Product is publicly released to the outside
world via online channels and data can be gathered from anyone.
Based on feedback, product improvements can be done. Example –
Windows 8.
 Technical Beta Testing: Product is released to the internal
group of an organization and gather feedback/data from the
employees of the organization.
 Focused Beta: Product is released to the market for gathering
feedback on specific features of the program.
 Post release Beta: Product is released to the market and data is
gathered to make improvements for the future release of the
product.
 BETA TESTING
Advantages
Reduces product failure risk via customer
validation.
 Improves product quality via customer
feedback.
Disadvantages
Finding the right beta users and maintaining
their participation could be a challenge.
PHASES OF ALPHA & BETA
TESTS

• Pre-Alpha:- Software is a prototype. UI is complete. But not

all features are completed. At this stage software is not
published.
• Alpha: Software is near its development and is internally
tested for bugs/issues.
 PHASES OF ALPHA & BETA
TESTS
 Beta: Software is stable and is released to limited
user base. The goal is to get customer feedback on the
product and make changes in software accordingly.
 Release Candidate (RC): Based on the feedback of
Beta Test, you make changes to the software. At this
stage, there is no radical changes in Functionality but
just checking for bugs.
 Release: All works, software is released to public.
 Integration Test Planning
Integration test must be planned.
Planning can begin when high-level design is
complete so that the system architecture is
defined.
Other documents relevant to integration test
planning are the requirements document, the
user manual, and usage scenarios.
The testing objectives are to assemble
components into subsystems and to demonstrate
that the subsystem functions properly.
 Integration Test Planning
Testing resources and schedules for
integration should be included in the test
plan
The plan includes the following items:
(i) clusters this cluster is dependent on;
(ii) a natural language description of the
functionality of the cluster to be tested;
(iii) list of classes in the cluster;
(iv) a set of cluster test cases.
 Regression Testing
Regression testing is not a level of testing.
It is the retesting of software.
When changes are made in old software
◦ To ensure that the new version of the software has
retained the capabilities of the old version and
◦ No new defects have been introduced due to the
changes.
Regressiontesting can occur at any level of test.
For example, when unit tests are run the unit.
 Regression Testing
 The unit is repaired and then retested with all the old test
cases.
 To ensure that the changes have not affected its
functionality.
 Regression tests are especially important when multiple
software releases are developed.
 Test cases, test procedures, and other test-related items
from previous releases should be available.
 So that these tests can be run with the new versions of
the software.
Alpha, Beta, and Acceptance Tests
 Users have played a supporting role for various testing
activities.
 And they have been involved in requirements analysis
and reviews.
 The clients along with test planners design the actual
test cases
 And that will be run during acceptance test.
 Users/clients may also have participated in prototype
evaluation, and in the various stages of usability testing.
 After the software has passed all the system tests and
defect repairs have been made,
 the users take a more active role in the testing process.