Software Testing Foundation

Test Levels

9/21/2022 09e-BM/DT/FSOFT - ©FPT SOFTWARE – Fresher Academy - Internal Use 1

Lesson Objectives

 Be introduced test levels (UT, IT, ST, UAT)

 Have chance to discuss with experience testers

9/21/2022 09e-BM/DT/FSOFT - ©FPT SOFTWARE – Fresher Academy - Internal Use 2

AGENDA

• Test Levels

• Unit Test (UT)

• Integration Test (IT)

• System Test (ST)

• User Acceptance Test (UAT)

9/21/2022 09e-BM/DT/FSOFT - ©FPT SOFTWARE – Fresher Academy - Internal Use 3

Section 1
TEST LEVELS

9/21/2022 09e-BM/DT/FSOFT - ©FPT SOFTWARE – Fresher Academy - Internal Use 4

Test Levels – What?

 A group of test activities that are organized and managed

together.
 Each test level is specifically service for a development
stage -> V-model

9/21/2022 09e-BM/DT/FSOFT - ©FPT SOFTWARE – Fresher Academy - Internal Use 5

V-Model

9/21/2022 09e-BM/DT/FSOFT - ©FPT SOFTWARE – Fresher Academy - Internal Use 6

Test Levels

Test levels are characterized by the following attributes:

 Specific objectives

 Test basis, referenced to derive test cases

 Test object (i.e., what is being tested)

 Typical defects and failures

 Specific approaches and responsibilities

9/21/2022 09e-BM/DT/FSOFT - ©FPT SOFTWARE – Fresher Academy - Internal Use 7

Test Levels

 Unit testing (Component testing)

 Integration testing (Combination testing)

 System testing

 Acceptance testing (User Acceptance testing)

9/21/2022 09e-BM/DT/FSOFT - ©FPT SOFTWARE – Fresher Academy - Internal Use 8

Section 2
UNIT TEST

9/21/2022 09e-BM/DT/FSOFT - ©FPT SOFTWARE – Fresher Academy - Internal Use 9

Unit Test – What?

 Verify that individual UNITs of software program are working

properly follow detailed design

 A UNIT(component) is the smallest testable part of an application,

it can be:
Program statements
Class, Function, Procedure
Module
Individual program
9/21/2022 09e-BM/DT/FSOFT - ©FPT SOFTWARE – Fresher Academy - Internal Use 10
Unit Test – Why?

$14,000
 Faster Debugging
85% % Defects
 Better Design

Percentage of Bugs
Introduced in
this phase
 Reduce Future % Defects
Cost (Quality found in
in this phase
$1000
Effort & $ Cost to
Correction Cost) $25 $130 $250 repair defect
in this phase
Coding Unit Funct Field Post
Test Test Test Release

9/21/2022 09e-BM/DT/FSOFT - ©FPT SOFTWARE – Fresher Academy - Internal Use 11

Unit Test – Inputs (Test Basic)

 Detailed design
 Code
 Data model
 Component specifications

9/21/2022 09e-BM/DT/FSOFT - ©FPT SOFTWARE – Fresher Academy - Internal Use 12

Unit Test – Implementation

 Doer: developer/coder, who develop the code

 How: using one or more unit testing types in following orders:
 Top-Down Unit Testing: stubs are required
 Bottom-up Unit Testing: drivers are required
Stubs and drivers are usually used in Unit Testing to replace
missing components, software

9/21/2022 09e-BM/DT/FSOFT - ©FPT SOFTWARE – Fresher Academy - Internal Use 13

Top-Down Unit Testing
Stubs: called by software component

Stub

Stub

B
Stub
9/21/2022 09e-BM/DT/FSOFT - ©FPT SOFTWARE – Fresher Academy - Internal Use 14
Bottom-up Unit Testing
Driver: call software component
Test Test
Drivers Drivers

Level N Level N

Test
Drivers

Level N-1
9/21/2022 09e-BM/DT/FSOFT - ©FPT SOFTWARE – Fresher Academy - Internal Use 15
Unit Test – Types/Technique

Statement Testing: execute all statements at least

once
Decision/Branch Testing: execute each
decision/branch at least once
Path Testing: execute all paths in the program at least
once

9/21/2022 09e-BM/DT/FSOFT - ©FPT SOFTWARE – Fresher Academy - Internal Use 16

Statement Testing

Execute every line of code (mean: all statements in the

programs should be executed at least once)
Example:
READ A
READ B
C =A + 2*B
IF C> 50 THEN
PRINT “large C”
ENDIF
=> 100% statement coverage: A = 20, B = 25
9/21/2022 09e-BM/DT/FSOFT - ©FPT SOFTWARE – Fresher Academy - Internal Use 17
Decision (Branch) Testing
Execute every branches (all branches in the program should be executed at
least once). Example:
z = 0;
if (a > b) then
z = 12;
x = 72 / z
Statement coverage may miss bugs
One test needed for statement coverage
(a = 5, b = 4)  Passes
Select inputs to force each decision to execute both possible ways (T/F)
Now two test cases are needed for coverage
(a = 5, b = 4)  Passes
(a = 4, b = 5)  Finds bug
9/21/2022 09e-BM/DT/FSOFT - ©FPT SOFTWARE – Fresher Academy - Internal Use 18
Path Testing

Is combination between statement and P1

decision testing T F

Paths in the program executed at least

once. P2
T F
Independent path can be defined in
terms of CFG – Control Flow Graph.
Test the each independent path.
9/21/2022 09e-BM/DT/FSOFT - ©FPT SOFTWARE – Fresher Academy - Internal Use 19
Path Testing

READ A (1)
READ B (2)
C = A + 2 * B (3)
IF C> 50 THEN (4)
PRINT “large C” (5)
ENDIF (6)
Path coverage:
Path 1: 1, 2, 3, 4, 5, 6
A = 20, B = 25
Path 2: 1, 2, 3, 4, 6
A = 20, B= 10
9/21/2022 09e-BM/DT/FSOFT - ©FPT SOFTWARE – Fresher Academy - Internal Use 20
Path Testing

Read P, Q (1)
If P+Q > 100 then (2)
Print "Large“ (3)
If P > 100 then (4)
Print "P is Greater“ (5)
Endif (6)
Path coverage:
Path 1: 1, 2, 3, 4, 5, 6
P= 110, Q = 50
Path 2: 1, 2, 3, 6
P = 80, Q = 70
Path 3: 1, 2, 6
P=80, Q=10
9/21/2022 09e-BM/DT/FSOFT - ©FPT SOFTWARE – Fresher Academy - Internal Use 21
Unit Test – Typical defects and failures

Incorrect functionality (e.g., not as described in

design specifications)
Data flow problems
Incorrect code and logic

9/21/2022 09e-BM/DT/FSOFT - ©FPT SOFTWARE – Fresher Academy - Internal Use 22

Unit Test – Tools

 Java: Junit (www.junit.org)

 C/C++: cppUnit
(http://sourceforge.net/projects/cppunit/)
 Python: pyUnit (http://pyunit.sourceforge.net/)
 Perl: PerlUnit (http://perlunit.sourceforge.net/)
 Visual Basic: vbUnit (http://www.vbunit.com/)
 C#: csUnit (http://www.csunit.org/)
 .NET: Nunit (http://www.nunit.org/)
9/21/2022 09e-BM/DT/FSOFT - ©FPT SOFTWARE – Fresher Academy - Internal Use 23
Section 3
INTEGRATION TEST

9/21/2022 09e-BM/DT/FSOFT - ©FPT SOFTWARE – Fresher Academy - Internal Use 24

Integration Testing – What?

Integration testing focuses on interactions between

components or systems

Notes: Test the interface between components or

system and do not concentrate on functionalities of
each function itself.

9/21/2022 09e-BM/DT/FSOFT - ©FPT SOFTWARE – Fresher Academy - Internal Use 25

Integration Testing – Why, When?

Why:
Unit tests only test the unit in isolation
Many failures result from faults in the interaction of
subsystems
Ensure integrate functions/modules follow architecture
design or high level design (HLD)
Ensure communication between functions/modules work
properly
When:
Depends on integration level
9/21/2022 09e-BM/DT/FSOFT - ©FPT SOFTWARE – Fresher Academy - Internal Use 26
Integration Testing – Inputs (Test Basis)

 Software and system/architecture design

 Sequence diagrams
 Interface and communication protocol
specifications
 Use cases
 Architecture at component or system level
 Workflows
 External interface definitions
9/21/2022 09e-BM/DT/FSOFT - ©FPT SOFTWARE – Fresher Academy - Internal Use 27
Level of Integration

Components integration: integrate between

components.
Ex: integrate create new, cancel and update order.
Systems integration: integrate between systems,
packages, and microservices.
Ex: integrate “Test Insight” system and “Defect
management” system.

9/21/2022 09e-BM/DT/FSOFT - ©FPT SOFTWARE – Fresher Academy - Internal Use 28

Component IT – Typical defects and failures

 Incorrect data, missing data, or incorrect data encoding

 Incorrect sequencing or timing of interface calls
 Interface mismatch
 Failures in communication between components
 Unhandled or improperly handled communication
failures between components
 Incorrect assumptions about the meaning, units, or
boundaries of the data being passed between
components

9/21/2022 09e-BM/DT/FSOFT - ©FPT SOFTWARE – Fresher Academy - Internal Use 29

System IT – Typical defects and failures

 Inconsistent message structures between systems

 Incorrect data, missing data, or incorrect data encoding
 Interface mismatch
 Failures in communication between systems
 Unhandled or improperly handled communication
failures between systems
 Incorrect assumptions about the meaning, units, or
boundaries of the data being passed between systems
 Failure to comply with mandatory security regulations

9/21/2022 09e-BM/DT/FSOFT - ©FPT SOFTWARE – Fresher Academy - Internal Use 30

Types of Integration

Big-bang integration: integrate all

components, systems at one time.
Incremental integration: integrate one
by one component

9/21/2022 09e-BM/DT/FSOFT - ©FPT SOFTWARE – Fresher Academy - Internal Use 31

Big Bang Integration

Test A
A
Test B

Test C B C D
Test
Test D A, B, C, D,
E, F, G
Test E E F G

Test F
Integrate all components, systems at
Test G
one time
9/21/2022 09e-BM/DT/FSOFT - ©FPT SOFTWARE – Fresher Academy - Internal Use 32
Big Bang Integration

Advantages:
 Everything is finished before integration testing starts.
Disadvantages:
 Time consuming
 Difficult to trace cause of fault
 Only effectively when the software expected to have no
defect.

9/21/2022 09e-BM/DT/FSOFT - ©FPT SOFTWARE – Fresher Academy - Internal Use 33

Incremental Integration - Types

Top-down: from menu to function (use

stub)
Bottom-up: from detail function to menu
(use driver)
Functional incremental: each function is
integrated, tested one by one.

9/21/2022 09e-BM/DT/FSOFT - ©FPT SOFTWARE – Fresher Academy - Internal Use 34

Top-down Integration Testing

A
Layer I

B C D Layer II

E F G
Layer III

Test
Test A Test A, B, C, D A, B, C, D,
E, F, G
Layer I
Layer I + II
All Layers
9/21/2022 09e-BM/DT/FSOFT - ©FPT SOFTWARE – Fresher Academy - Internal Use 35
Top-down Integration Testing

Advantages:
No drivers needed
Test cases can be defined in terms of the
functionality of the system (functional requirements)
Disadvantages:
Large number of stubs may be required, especially
if the lowest level of the system contains many
methods.
Writing stubs is difficult: Stubs must allow all
possible conditions to be tested.
Some interfaces are not tested separately.
9/21/2022 09e-BM/DT/FSOFT - ©FPT SOFTWARE – Fresher Academy - Internal Use 36
Bottom-Up Integration Testing
A
Layer I

B C D Layer II
Test E

Test B, E, F E F G
Layer III

Test F

Test C Test
A, B, C, D,
E, F, G

Test D,G
Test G
9/21/2022 09e-BM/DT/FSOFT - ©FPT SOFTWARE – Fresher Academy - Internal Use 37
Bottom-Up Integration Testing

Advantages:
No stubs needed
Useful for integration testing of the following systems
•Object-oriented systems
•Real-time systems
•Systems with strict performance requirement
Disadvantages:
Tests the most important subsystem (user
interface) last
Drivers needed
9/21/2022 09e-BM/DT/FSOFT - ©FPT SOFTWARE – Fresher Academy - Internal Use 38
Functional Incremental

Advantages:
Test cases can be defined in terms of the
functionality of the system (functional requirements)
Disadvantages:
Both stubs and drivers are needed.
Some interfaces are not tested separately.
Difficulty for integrator
9/21/2022 09e-BM/DT/FSOFT - ©FPT SOFTWARE – Fresher Academy - Internal Use 39
Section 4
SYSTEM TEST

9/21/2022 09e-BM/DT/FSOFT - ©FPT SOFTWARE – Fresher Academy - Internal Use 40

System Testing – What?

System testing is defined as testing the behavior of a

system/software as per software requirement
specification.

Is concerned with the behavior of the whole system or

product as defined by the scope of a development
project or product, included functional requirements
and non-functional requirements.

9/21/2022 09e-BM/DT/FSOFT - ©FPT SOFTWARE – Fresher Academy - Internal Use 41

System Testing – Why?

It is done to verify, validate the functional, non

functional, business, technical requirements of the
software.

Is testing in an environment that closely resembles the

production environment where the application will be
finally deployed

It is the final test on behalf of development to verify

that the system 09e-BM/DT/FSOFT
9/21/2022
to be delivered
- ©FPT SOFTWARE – Fresher Academy - Internal Use 42
System Testing – When?

Complete software system should be developed

Unit testing must be completed
Integration testing must be completed
Specifications for the product have been completed
Test cases, test data and test schedule are ready

9/21/2022 09e-BM/DT/FSOFT - ©FPT SOFTWARE – Fresher Academy - Internal Use 43

System Testing – Techniques

System testing includes:

Functional testing: using back box test method
Non-functional testing: performance and reliability.
Re-testing and Regression testing
 Doer: Tester

9/21/2022 09e-BM/DT/FSOFT - ©FPT SOFTWARE – Fresher Academy - Internal Use 44

Black Box Test Method

 Focuses on input and output of the software without regard to the internal
code of the program
 Focuses on functional requirement of the software
 Design sets of input conditions that will fully exercise all functional
requirements for a program

9/21/2022 09e-BM/DT/FSOFT - ©FPT SOFTWARE – Fresher Academy - Internal Use 45

System Testing – Environment

Environment for system test should be the

most likely/closely to the production
environment to minimize environment-specific
failure.
Production environment is where the
software/system/application will be finally
deployed
9/21/2022 09e-BM/DT/FSOFT - ©FPT SOFTWARE – Fresher Academy - Internal Use 46
System Testing – Types

Functional Testing types

Function Testing
User Interface (UI) Testing
Data and Database Integrity Testing
Business Cycle Testing
Access Control Testing
9/21/2022 09e-BM/DT/FSOFT - ©FPT SOFTWARE – Fresher Academy - Internal Use 47
System Testing – Types

Non-functional Testing types

Performance testing
Load testing Testing related to changes
Stress testing types
Volume testing Confirmation testing
Regression testing

9/21/2022 09e-BM/DT/FSOFT - ©FPT SOFTWARE – Fresher Academy - Internal Use 48

System Testing – Typical defects and failures
 Incorrect calculations
 Incorrect or unexpected system functional or non-functional behavior
 Incorrect control and/or data flows within the system
 Failure to properly and completely carry out end-to-end functional tasks
 Failure of the system to work properly in the production environment(s)
 Failure of the system to work as described in system and user manuals

9/21/2022 09e-BM/DT/FSOFT - ©FPT SOFTWARE – Fresher Academy - Internal Use 49

Section 5
(USER ) ACCEPTANCE TEST

9/21/2022 09e-BM/DT/FSOFT - ©FPT SOFTWARE – Fresher Academy - Internal Use 50

Acceptance Testing – What?

Also known as user acceptance testing

Do by customer.
Is the last phase of the software testing process
Confirm that the system meets the agreed upon criteria
Is a user-run test that demonstrates the application’s
ability to meet user’s requirements (mapping to V-model)
Concentrates on validation types of testing to determine
whether the system is fit for use.

9/21/2022 09e-BM/DT/FSOFT - ©FPT SOFTWARE – Fresher Academy - Internal Use 51

Acceptance Testing – Why?

Accept product based on acceptance criteria

Ensure function needed and expected by customers is
present in a software product

9/21/2022 09e-BM/DT/FSOFT - ©FPT SOFTWARE – Fresher Academy - Internal Use 52

Acceptance Testing – Inputs

 Business processes
 User or business requirements
 Regulations, legal contracts and standards
 Use cases
 System requirements
 System or user documentation
 Installation procedures

 Risk analysis reports

9/21/2022 09e-BM/DT/FSOFT - ©FPT SOFTWARE – Fresher Academy - Internal Use 53
Acceptance Testing – When?

After software product is released and

System testing is implemented

(Note: time scale of AT may overlap on ST – alpha

testing)

9/21/2022 09e-BM/DT/FSOFT - ©FPT SOFTWARE – Fresher Academy - Internal Use 54

Acceptance Testing – Environment

For most aspect should represent for the production

environment.

9/21/2022 09e-BM/DT/FSOFT - ©FPT SOFTWARE – Fresher Academy - Internal Use 55

Acceptance Testing – Types

 Operational Acceptance Testing – OAT: ensures that all the

non-functional aspects of a system are tested to ensure that
the system is within the specified parameters. It may includes:
• Backup & restore
• Installing, uninstalling and upgrading
• Disaster recovery
• User management
• Maintenance tasks
• Data load and migration tasks
• Security weakness
• Performance testing

9/21/2022 09e-BM/DT/FSOFT - ©FPT SOFTWARE – Fresher Academy - Internal Use 56

Acceptance Testing – Types

 Compliance Acceptance Testing: is performed against the

contract’s acceptance criteria for producing custom-developed
software, such as government regulations.
Eg: does the accounting report adhere to the law?

 Mass market Acceptance Testing contains 2 phases:

Alpha testing: do by users in development site.
Beta testing: do by users in customer site.

9/21/2022 09e-BM/DT/FSOFT - ©FPT SOFTWARE – Fresher Academy - Internal Use 57

Alpha Testing

Do by end users/customers

Conducted at the developer’s site / development site

Software is used in a natural setting with developers

watching intently (under the supervision of the
developer)

9/21/2022 09e-BM/DT/FSOFT - ©FPT SOFTWARE – Fresher Academy - Internal Use 58

Beta Testing

Do by end-users/customers
Conducted at end-user sites
Do after alpha testing
Developer is generally not present
The end-user records all problems that are
encountered and reports these to the developers at
regular intervals

9/21/2022 09e-BM/DT/FSOFT - ©FPT SOFTWARE – Fresher Academy - Internal Use 59

AT – Typical defects and failures

System workflows do not meet business or user requirements

Business rules are not implemented correctly
System does not satisfy contractual or regulatory requirements
Non-functional failures such as security vulnerabilities,
inadequate performance efficiency under high loads, or improper
operation on a supported platform

9/21/2022 09e-BM/DT/FSOFT - ©FPT SOFTWARE – Fresher Academy - Internal Use 60

References
 https://www.istqb.org/ , Foundations of Software Testing,
Chapter 2

9/21/2022 09e-BM/DT/FSOFT - ©FPT SOFTWARE – Fresher Academy - Internal Use 61

Lesson Summary

• Session 1: Test Levels

• Session 2: Unit Test (UT)

• Session 3: Integration Test (IT)

• Session 4: System Test (ST)

• Session 5: User Acceptance Test (UAT)

9/21/2022 09e-BM/DT/FSOFT - ©FPT SOFTWARE – Fresher Academy - Internal Use 62

Thank you

9/21/2022 09e-BM/DT/FSOFT - ©FPT SOFTWARE – Fresher Academy - Internal Use 63