2023-12-28

Testing
Introduction to Software Engineering

Associate Professor
Hasan Abdulkader
COMPUTER SCIENCE AND TELECOMMUNICATIONS, PhD

Fall 2023-2024

Testing & Devops Introduction to Software Engineering Introduction to Software Engineering

1 2

Software testing Program bugs

• Software testing is a process in which you execute your program using • If the behaviour of the program does not match the behaviour that you
data that simulates user inputs. expect, then this means that there are bugs in your program that need to
be fixed.
• You observe its behaviour to see whether or not your program is doing
• There are two causes of program bugs:
what it is supposed to do.
• Programming errors You have accidentally included faults in your program
• Tests pass if the behaviour is what you expect. Tests fail if the behaviour differs
code. For example, a common programming error is an ‘off-by-1’ error where
from that expected.
you make a mistake with the upper bound of a sequence and fail to process the
last element in that sequence.
• If your program does what you expect, this shows that for the inputs used, the
program behaves correctly.
• Understanding errors You have misunderstood or have been unaware of
some of the details of what the program is supposed to do. For example, if your
• If these inputs are representative of a larger set of inputs, you can infer program processes data from a file, you may not be aware that some of this
that the program will behave correctly for all members of this larger input data is in the wrong format, so your program doesn’t include code to handle this.
set.

Testing & Devops Introduction to Software Engineering 3 Testing & Devops Introduction to Software Engineering 4

3 4
 2023-12-28

Types of testing
Functional testing
Functional testing
Test the functionality of the overall system. The goals of functional testing are to • Functional testing involves developing a large set of program tests so
discover as many bugs as possible in the implementation of the system and to provide
convincing evidence that the system is fit for its intended purpose.
that, ideally, all of a program’s code is executed at least once.

User testing • The number of tests needed obviously depends on the size and the
Test that the software product is useful to and usable by end-users. You need to show functionality of the application.
that the features of the system help users do what they want to do with the software.
You should also show that users understand how to access the software’s features • For a business-focused web application, you may have to develop thousands of
and can use these features effectively. tests to convince yourself that your product is ready for release to customers.
Performance and load testing
Test that the software works quickly and can handle the expected load placed on the • Functional testing is a staged activity in which you initially test individual
system by its users. You need to show that the response and processing time of your units of code. You integrate code units with other units to create larger
system is acceptable to end-users. You also need to demonstrate that your system units then do more testing.
can handle different loads and scales gracefully as the load on the software increases.
• The process continues until you have created a complete system ready for
Security testing release.
Test that the software maintains its integrity and can protect user information from theft
and damage.

Testing & DevOps Introduction to Software Engineering 5 Testing & Devops Introduction to Software Engineering 6

5 6

Functional testing processes Functional testing

Unit testing
The aim of unit testing is to test program units in isolation. Tests should be designed to execute
all of the code in a unit at least once. Individual code units are tested by the programmer as
they are developed.

Feature testing
Code units are integrated to create features. Feature tests should test all aspects of a feature.
All of the programmers who contribute code units to a feature should be involved in its testing.

System testing
Code units are integrated to create a working (perhaps incomplete) version of a system. The
aim of system testing is to check that there are no unexpected interactions between the
features in the system. System testing may also involve checking the responsiveness, reliability
and security of the system. In large companies, a dedicated testing team may be responsible
for system testing. In small companies, this is impractical, so product developers are also
involved in system testing.

Release testing
The system is packaged for release to customers and the release is tested to check that it
operates as expected. The software may be released as a cloud service or as a download to be
installed on a customer’s computer or mobile device. If DevOps is used, then the development
team are responsible for release testing otherwise a separate team has that responsibility.

Testing & DevOps Introduction to Software Engineering 7 Testing & DevOps Introduction to Software Engineering 8

7 8
 2023-12-28

Unit testing Equivalence partitions

• As you develop a code unit, you should also develop tests for that code.

• A code unit is anything that has a clearly defined responsibility. It is

usually a function or class method but could be a module that includes a
small number of other functions.

• Unit testing is based on a simple general principle:

• If a program unit behaves as expected for a set of inputs that have some shared
characteristics, it will behave in the same way for a larger set whose members
share these characteristics.

• To test a program efficiently, you should identify sets of inputs

(equivalence partitions) that will be treated in the same way in your code.

• The equivalence partitions that you identify should not just include those
containing inputs that produce the correct values. You should also identify
‘incorrectness partitions’ where the inputs are deliberately incorrect.

Testing & Devops Introduction to Software Engineering 9 Testing & DevOps Introduction to Software Engineering 10

9 10

Unit testing guidelines (1) Unit testing guidelines (2)

Test edge cases Overflow and underflow

If your partition has upper and lower bounds (e.g. length of strings, numbers, etc.) If your program does numeric calculations, choose test inputs that cause it to
choose inputs at the edges of the range. calculate very large or very small numbers.

Force errors Don’t forget null and zero

Choose test inputs that force the system to generate all error messages. Choose If your program uses pointers or strings, always test with null pointers and strings.
test inputs that should generate invalid outputs. If you use sequences, test with an empty sequence. For numeric inputs, always
test with zero.
Fill buffers
Choose test inputs that cause all input buffers to overflow. Keep count
When dealing with lists and list transformation, keep count of the number of
Repeat yourself elements in each list and check that these are consistent after each
Repeat the same test input or series of inputs several times. transformation.

One is different
If your program deals with sequences, always test with sequences that have a
single value.

Testing & DevOps Introduction to Software Engineering 11 Testing & DevOps Introduction to Software Engineering 12

11 12
 2023-12-28

Feature testing Types of feature test

• Features have to be tested to show that the functionality is implemented • Interaction tests
as expected and that the functionality meets the real needs of users. • These test the interactions between the units that implement the feature. The developers of the
units that are combined to make up the feature may have different understandings of what is
• For example, if your product has a feature that allows users to login using their required of that feature.
Google account, then you have to check that this registers the user correctly
and informs them of what information will be shared with Google. • The integration may also reveal bugs in program units, which were not exposed by unit testing.

• Usefulness tests
• Normally, a feature that does several things is implemented by multiple,
interacting, program units. • These test that the feature implements what users are likely to want.

• For example, the developers of a login with Google feature may have implemented an opt-out
• These units may be implemented by different developers and all of these default on registration so that users receive all emails from a company. They must expressly
developers should be involved in the feature testing process. choose what type of emails that they don’t want.

Testing & Devops Introduction to Software Engineering 13 Testing & Devops Introduction to Software Engineering 14

13 14

System testing Release testing

• System testing involves testing the system as a whole, rather than the • Release testing is a type of system testing where a system that’s intended for
individual system features. release to customers is tested.

• The fundamental differences between release testing and system testing are:
• System testing should focus on four things:
• Release testing tests the system in its real operational environment rather than in a
• Testing to discover if there are unexpected and unwanted interactions between test environment. Problems commonly arise with real user data, which is sometimes
the features in a system. more complex and less reliable than test data.

• Testing to discover if the system features work together effectively to support • The aim of release testing is to decide if the system is good enough to release, not to
what users really want to do with the system. detect bugs in the system. Therefore, some tests that ‘fail’ may be ignored if these
have minimal consequences for most users.
• Testing the system to make sure it operates in the expected way in the different
environments where it will be used. • Preparing a system for release involves packaging that system for
deployment (e.g. in a container if it is a cloud service) and installing software
• Testing the responsiveness, throughput, security and other quality attributes of and libraries that are used by your product. You must define configuration
the system. parameters such as the name of a root directory, the database size limit per
user and so on.

Testing & Devops Introduction to Software Engineering 15 Testing & Devops Introduction to Software Engineering 16

15 16
 2023-12-28

Test automation Figure 9.4 Automated testing

• Automated testing is based on the idea that tests should be executable.

• An executable test includes the input data to the unit that is being tested,
the expected result and a check that the unit returns the expected result.

• You run the test and the test passes if the unit returns the expected
result.

• Normally, you should develop hundreds or thousands of executable tests

for a software product.

Testing & Devops Introduction to Software Engineering 17 Testing & DevOps Introduction to Software Engineering 18

17 18

# TestInterestCalculator inherits attributes and methods from the class

# TestCase in the testing framework unittest
Automated tests
class TestInterestCalculator (unittest.TestCase):
# Define a set of unit tests where each test tests one thing only
# Tests should start with test_ and the name should explain what is being tested • It is good practice to structure automated tests into three parts:
def test_zeroprincipal (self):
#Arrange - set up the test parameters • Arrange You set up the system to run the test. This involves defining the test
p = 0; r = 3; n = 31 parameters and, if necessary, mock objects that emulate the functionality of
result_should_be = 0 code that has not yet been developed.
#Action - Call the method to be tested
interest = interest_calculator (p, r, n) • Action You call the unit that is being tested with the test parameters.
#Assert - test what should be true
self.assertEqual (result_should_be, interest) • Assert You make an assertion about what should hold if the unit being tested
has executed successfully. In Program 9.2, I use AssertEquals, which checks if
def test_yearly_interest (self): its parameters are equal.
#Arrange - set up the test parameters
p = 17000; r = 3; n = 365
• If you use equivalence partitions to identify test inputs, you should have
#Action - Call the method to be tested
result_should_be = 270.36
several automated tests based on correct and incorrect inputs from each
interest = interest_calculator (p, r, n) partition.
#Assert - test what should be true
self.assertEqual (result_should_be, interest)
Program 9.2 Test
methods for an
interest calculator

Testing & DevOps Introduction to Software Engineering 19 Testing & Devops Introduction to Software Engineering 20

19 20
 2023-12-28

System testing Figure 9.5 The test pyramid

• System testing, which should follow feature testing, involves testing the
system as a surrogate user.

• As a system tester, you go through a process of selecting items from

menus, making screen selections, inputting information from the
keyboard and so on.

• You are looking for interactions between features that cause problems,
sequences of actions that lead to system crashes and so on.

• Manual system testing, when testers have to repeat sequences of

actions, is boring and error-prone. In some cases, the timing of actions is
important and is practically impossible to repeat consistently.
Unit tests are the easiest to automate, so the majority of your tests should be unit tests.
• To avoid these problems, testing tools have been developed that can record a Mike Cohn, who first proposed the test pyramid, suggests that 70% of automated tests
series of actions and automatically replay these when a system is retested should be unit tests, 20% feature tests (he called these service tests), and 10% system tests
(UI tests).

Testing & Devops Introduction to Software Engineering 21 Testing & DevOps Introduction to Software Engineering 22

21 22

Test-driven development Stages of test-driven development (1)

Identify partial implementation

• Test-driven development (TDD) is an approach to program development Break down the implementation of the functionality required into smaller mini-
that is based around the general idea that you should write an units. Choose one of these mini-units for implementation.
executable test or tests for code that you are writing before you write the
Write mini-unit tests
code. Write one or more automated tests for the mini-unit that you have chosen for
implementation. The mini-unit should pass these tests if it is properly
• It was introduced by early users of the Extreme Programming agile implemented.
method, but it can be used with any incremental development approach.
Write a code stub that will fail test
• Test-driven development works best for the development of individual Write incomplete code that will be called to implement the mini-unit. You know
program units and it is more difficult to apply to system testing. this will fail.

Run all existing automated tests

• Even the strongest advocates of TDD accept that it is challenging to use
All previous tests should pass. The test for the incomplete code should fail.
this approach when you are developing and testing systems with
graphical user interfaces.

Testing & Devops Introduction to Software Engineering 23 Testing & DevOps Introduction to Software Engineering 24

23 24
 2023-12-28

Stages of test-driven development (2) Figure 9.8 Test-driven development

Implement code that should cause the failing test to pass

Write code to implement the mini-unit, which should cause it to operate correctly

Rerun all automated tests

If any tests fail, your code is probably incorrect. Keep working on it until all tests
pass.

Refactor code if necessary

If all tests pass, you can move on to implementing the next mini-unit. If you see
ways of improving your code, you should do this before the next stage of
implementation.

Testing & DevOps Introduction to Software Engineering 25 Testing & DevOps Introduction to Software Engineering 26

25 26

Benefits of test-driven development Disadvantages of TDD

• It is a systematic approach to testing in which tests are clearly linked to • TDD discourages radical program change
sections of the program code. I found that I was reluctant to make refactoring decisions that I knew would cause many tests to
fail. I tended to avoid radical program change for this reason.
• This means you can be confident that your tests cover all of the code that has
• I focused on the tests rather than the problem I was trying to solve
been developed and that there are no untested code sections in the delivered
A basic principle of TDD is that your design should be driven by the tests you have written. I
code. In my view, this is the most significant benefit of TDD. found that I was unconsciously redefining the problem I was trying to solve to make it easier to
write tests. This meant that I sometimes didn’t implement important checks, because it was
• The tests act as a written specification for the program code. In principle difficult to write tests in advance of their implementation.
at least, it should be possible to understand what the program does by
reading the tests. • I spent too much time thinking about implementation details rather than the programming
problem
Sometimes when programming, it is best to step back and look at the program as a whole rather
• Debugging is simplified because, when a program failure is observed, you than focusing on implementation details. TDD encourages a focus on details that might cause
can immediately link this to the last increment of code that you added to tests to pass or fail and discourages large-scale program revisions.
the system.
• It is hard to write ‘bad data’ tests
• It is argued that TDD leads to simpler code as programmers only write Many problems involving dealing with messy and incomplete data. It is practically impossible to
code that’s necessary to pass tests. They don’t over-engineer their code anticipate all of the data problems that might arise and write tests for these in advance. You
might argue that you should simply reject bad data but this is sometimes impractical.
with complex features that aren’t needed.

Testing & Devops Introduction to Software Engineering 27 Testing & Devops Introduction to Software Engineering 28

27 28
 2023-12-28

Security testing Risk-based security testing

• Security testing aims to find vulnerabilities that may be exploited by an • A risk-based approach to security testing involves identifying common
attacker and to provide convincing evidence that the system is risks and developing tests to demonstrate that the system protects itself
sufficiently secure. from these risks.

• The tests should demonstrate that the system can resist attacks on its • You may also use automated tools that scan your system to check for
availability, attacks that try to inject malware and attacks that try to known vulnerabilities, such as unused HTTP ports being left open.
corrupt or steal users’ data and identity.
• Based on the risks that have been identified, you then design tests and
• Comprehensive security testing requires specialist knowledge of checks to see if the system is vulnerable.
software vulnerabilities and approaches to testing that can find these
vulnerabilities. • It may be possible to construct automated tests for some of these
checks, but others inevitably involve manual checking of the system’s
behaviour and its files.

• Once you have identified security risks, you then analyze them to assess
how they might arise.

Testing & Devops Introduction to Software Engineering 29 Testing & Devops Introduction to Software Engineering 30

29 30

Code reviews Figure 9.9 Code reviews

• Code reviews involve one or more people examining the code to check
for errors and anomalies and discussing issues with the developer.

• If problems are identified, it is the developer’s responsibility to change

the code to fix the problems.

• Code reviews complement testing. They are effective in finding bugs that
arise through misunderstandings and bugs that may only arise when
unusual sequences of code are executed.

• Many software companies insist that all code has to go through a

process of code review before it is integrated into the product codebase.

Testing & Devops Introduction to Software Engineering 31 Testing & DevOps Introduction to Software Engineering 32

31 32
 2023-12-28

Code review activities (1) Code review activities (2)

Setup review Discussion

The programmer contacts a reviewer and arranges a review date. The reviewer and programmer discuss the issues and agree on the actions to
resolve these.
Prepare code
The programmer collects the code and tests for review and annotates them with Make to-do list
information for the reviewer about the intended purpose of the code and tests. The programmer documents the outcome of the review as a to-do list and shares
this with the reviewer.
Distribute code/tests
The programmer sends code and tests to the reviewer. Make code changes
The programmer modifies their code and tests to address the issues raised in the
Check code review.
The reviewer systematically checks the code and tests against their
understanding of what they are supposed to do.

Write review report

The reviewer annotates the code and tests with a report of the issues to be
discussed at the review meeting.

Testing & DevOps Introduction to Software Engineering 33 Testing & DevOps Introduction to Software Engineering 34

33 34

Software support

• Traditionally, separate teams were responsible for software

development, software release and software support.

• The development team passed over a ‘final’ version of the software to a

release team. This team then built a release version, tested this and
DevOps and Code Management prepared release documentation before releasing the software to
customers.

• A third team was responsible for providing customer support.

• The original development team were sometimes also responsible for

implementing software changes.

• Alternatively, the software may have been maintained by a separate

‘maintenance team’.

Introduction to Software Engineering Testing & Devops Introduction to Software Engineering 36

35 36
 2023-12-28

Figure 10.1 Development, release and support

DevOps

• There are inevitable delays and overheads in the traditional support model.

• To speed up the release and support processes, an alternative approach

called DevOps (Development+Operations) has been developed.

• Three factors led to the development and widespread adoption of DevOps:

• Agile software engineering reduced the development time for software, but the
traditional release process introduced a bottleneck between development and
deployment.

• Amazon re-engineered their software around services and introduced an approach

in which a service was developed and supported by the same team. Amazon’s
claim that this led to significant improvements in reliability was widely publicized.

• It became possible to release software as a service, running on a public or private

cloud. Software products did not have to be released to users on physical media or
downloads.

Testing & DevOps Introduction to Software Engineering 37 Testing & Devops Introduction to Software Engineering 38

37 38

DevOps principles Benefits of DevOps

Everyone is responsible for everything Faster deployment

All team members have joint responsibility for developing, delivering and Software can be deployed to production more quickly because communication
supporting the software. delays between the people involved in the process are dramatically reduced.

Everything that can be automated should be automated Reduced risk

All activities involved in testing, deployment and support should be automated if it The increment of functionality in each release is small so there is less chance of
is possible to do so. There should be mimimal manual involvement in deploying feature interactions and other changes causing system failures and outages.
software.
Faster repair
Measure first, change later DevOps teams work together to get the software up and running again as
DevOps should be driven by a measurement program where you collect data soon as possible. There is no need to discover which team were
about the system and its operation. You then use the collected data to inform responsible for the problem and to wait for them to fix it.
decisions about changing DevOps processes and tools.
More productive teams
DevOps teams are happier and more productive than the teams involved in the
separate activities. Because team members are happier, they are less likely to
leave to find jobs elsewhere.

Testing & DevOps Introduction to Software Engineering 39 Testing & DevOps Introduction to Software Engineering 40

39 40
 2023-12-28

Code management Code management and DevOps

• During the development of a software product, the development team • Source code management, combined with automated system building, is
will probably create tens of thousands of lines of code and automated essential for professional software engineering.
tests.
• In companies that use DevOps, a modern code management system is a
• These will be organized into hundreds of files. Dozens of libraries may fundamental requirement for ‘automating everything’.
be used, and several, different programs may be involved in creating and
running the code. • Not only does it store the project code that is ultimately deployed, it also
stores all other information that is used in DevOps processes.
• Code management is a set of software-supported practices that is used
to manage an evolving codebase. • DevOps automation and measurement tools all interact with the code
management system
• You need code management to ensure that changes made by different
developers do not interfere with each other, and to create different
product versions.

• Code management tools make it easy to create an executable product

from its source code files and to run automated tests on that product.

Testing & Devops Introduction to Software Engineering 41 Testing & Devops Introduction to Software Engineering 42

41 42

Figure 10.3 Code management and Devops

Code management fundamentals

• Code management systems provide a set of features that support four

general areas:

• Code transfer Developers take code into their personal file store to work on it
then return it to the shared code management system.

• Version storage and retrieval Files may be stored in several different versions
and specific versions of these files can be retrieved.

• Merging and branching Parallel development branches may be created for

concurrent working. Changes made by developers in different branches may be
merged.

• Version information Information about the different versions maintained in the

system may be stored and retrieved

Testing & DevOps Introduction to Software Engineering 43 Testing & Devops Introduction to Software Engineering 44

43 44
 2023-12-28

Code repository Features of code management systems

Version and release identification

• All source code management systems have the general form with a Managed versions of a code file are uniquely identified when they are submitted to the
system and can be retrieved using their identifier and other file attributes.
shared repository and a set of features to manage the files in that
repository: Change history recording
The reasons why changes to a code file have been made are recorded and
• All source code files and file versions are stored in the repository, as are other maintained.
artefacts such as configuration files, build scripts, shared libraries and versions
of tools used. Independent development
Several developers can work on the same code file at the same time. When this is
• The repository includes a database of information about the stored files such as submitted to the code management system, a new version is created so that files are
version information, information about who has changed the files, what changes never overwritten by later changes.
were made at what times, and so on.
Project support
All of the files associated with a project may be checked out at the same time.There is
• Files can be transferred to and from the repository and information about no need to check out files one at a time.
the different versions of files and their relationships may be updated.
Storage management
• Specific versions of files and information about these versions can always be The code management system includes efficient storage mechanisms so that it doesn’t
retrieved from the repository. keep multiple copies of files that have only small differences.

Testing & Devops Introduction to Software Engineering 45 Testing & DevOps Introduction to Software Engineering 46

45 46

Benefits of distributed code management Git repositories

• Resilience

• Everyone working on a project has their own copy of the repository. If the shared
repository is damaged or subjected to a cyberattack, work can continue, and the
clones can be used to restore the shared repository. People can work offline if they
don’t have a network connection.

• Speed

• Committing changes to the repository is a fast, local operation and does not need data
to be transferred over the network.

• Flexibility

• Local experimentation is much simpler. Developers can safely experiment and try
different approaches without exposing these to other project members. With a
centralized system, this may only be possible by working outside the code
management system.

Testing & Devops Introduction to Software Engineering 47 Testing & DevOps Introduction to Software Engineering 48

47 48
 2023-12-28

Branching and merging Branching and merging

• Branching and merging are fundamental ideas that are supported by all
code management systems.

• A branch is an independent, stand-alone version that is created when a

developer wishes to change a file.

• The changes made by developers in their own branches may be merged to

create a new shared branch.

• The repository ensures that branch files that have been changed cannot
overwrite repository files without a merge operation.

• If Alice or Bob make mistakes on the branch they are working on, they can easily
revert to the master file.

• If they commit changes, while working, they can revert to earlier versions of the
work they have done. When they have finished and tested their code, they can then
replace the master file by merging the work they have done with the master branch

Testing & Devops Introduction to Software Engineering 49 Testing & DevOps Introduction to Software Engineering 50

49 50

DevOps automation Aspects of DevOps automation

Continuous integration
• By using DevOps with automated support, you can dramatically reduce Each time a developer commits a change to the project’s master branch, an
the time and costs for integration, deployment and delivery. executable version of the system is built and tested.

Continuous delivery
• Everything that can be, should be automated is a fundamental principle
A simulation of the product’s operating environment is created and the
of DevOps. executable software version is tested.

• As well as reducing the costs and time required for integration, Continuous deployment
deployment and delivery, process automation also makes these A new release of the system is made available to users every time a change is
processes more reliable and reproducible. made to the master branch of the software.

• Automation information is encoded in scripts and system models that Infrastructure as code
Machine-readable models of the infrastructure (network, servers, routers, etc.) on
can be checked, reviewed, versioned and stored in the project
which the product executes are used by configuration management tools to build
repository. the software’s execution platform. The software to be installed, such as compilers
and libraries and a DBMS, are included in the infrastructure model.

Testing & Devops Introduction to Software Engineering 51 Testing & DevOps Introduction to Software Engineering 52

51 52
 2023-12-28

System integration Continuous integration

• System integration (system building) is the process of gathering all of the • Continuous integration simply means that an integrated version of the
elements required in a working system, moving them into the right directories, system is created and tested every time a change is pushed to the
and putting them together to create an operational system. system’s shared repository.
• Typical activities that are part of the system integration process include: • On completion of the push operation, the repository sends a message to
• Installing database software and setting up the database with the appropriate schema.
an integration server to build a new version of the product

• Loading test data into the database. • The advantage of continuous integration compared to less frequent
integration is that it is faster to find and fix bugs in the system.
• Compiling the files that make up the product.
• If you make a small change and some system tests then fail, the problem
• Linking the compiled code with the libraries and other components used.
almost certainly lies in the new code that you have pushed to the project
• Checking that external services used are operational. repo.

• Deleting old configuration files and moving configuration files to the correct locations. • You can focus on this code to find the bug that’s causing the problem.
• Running a set of system tests to check that the integration has been successful.

Testing & Devops Introduction to Software Engineering 53 Testing & Devops Introduction to Software Engineering 54

53 54

System building Continuous delivery and deployment

• Continuous integration is only effective if the integration process is fast • Continuous Integration • Continuous deployment
and developers do not have to wait for the results of their tests of the
integrated system. • Continuous integration means • However, the real environment in which
creating an executable version of software runs will inevitably be different
• However, some activities in the build process, such as populating a a software system whenever a from your development system. When
database or compiling hundreds of system files, are inherently slow. change is made to the your software runs in its real, operational
repository. The CI tool builds the environment bugs may be revealed that
• It is therefore essential to have an automated build process that system and runs tests on your did not show up in the test environment.
minimizes the time spent on these activities. development computer or project
integration server.
• Fast system building is achieved using a process of incremental building,
where only those parts of the system that have been changed are rebuilt • Continuous delivery means that, • This means that you have to test it in a
after making changes to a production environment to make sure
system, you ensure that the that environmental factors do not cause
changed system is ready for system failures or slow down its
delivery to customers. performance.

Testing & Devops Introduction to Software Engineering 55 Testing & Devops Introduction to Software Engineering 56

55 56
 2023-12-28

Continuous delivery and deployment

Key points 1

• The aim of program testing is to find bugs and to show that a program does what
its developers expect it to do.

• Four types of testing that are relevant to software products are functional testing,
user testing, load and performance testing and security testing.

• Unit testing involves testing program units such as functions or class methods that
have a single responsibility. Feature testing focuses on testing individual system
features. System testing tests the system as a whole to check for unwanted
interactions between features and between the system and its environment.

• Identifying equivalence partitions, in which all inputs have the same characteristics,
and choosing test inputs at the boundaries of these partitions, is an effective way
of finding bugs in a program.

• User stories may be used as a basis for deriving feature tests.

• Test automation is based on the idea that tests should be executable. You develop
a set of executable tests and run these each time you make a change to a system.

Testing & DevOps Introduction to Software Engineering 57 Testing & Devops Introduction to Software Engineering 58

57 58

Key points 2 Key points 3

• The structure of an automated unit test should be arrange-action-assert. • DevOps is the integration of software development and the management of
You set up the test parameters, call the function or method being tested, that software once it has been deployed for use. The same team is
and make an assertion of what should be true after the action has been responsible for development, deployment and software support.
completed. • The benefits of DevOps are faster deployment, reduced risk, faster repair of
buggy code and more productive teams.
• Test-driven development is an approach to development where executable
tests are written before the code. Code is then developed to pass the tests. • Source code management is essential to avoid changes made by different
developers interfering with each other.
• A disadvantage of test-driven development is that programmers focus on
the detail of passing tests rather than considering the broader structure of • All code management systems are based around a shared code repository
their code and algorithms used. with a set of features that support code transfer, version storage and
retrieval, branching and merging and maintaining version information.
• Security testing may be risk driven where a list of security risks is used to
• Git is a distributed code management system that is the most widely used
identify tests that may identify system vulnerabilities. system for software product development. Each developer works with their
own copy of the repository which may be merged with the shared project
• Code reviews are an effective supplement to testing. They involve people repository.
checking the code to comment on the code quality and to look for bugs.

Testing & Devops Introduction to Software Engineering 59 Testing & Devops Introduction to Software Engineering 60

59 60
 2023-12-28

Key points 4

• Continuous integration means that as soon as a change is committed to a project

repository, it is integrated with existing code and a new version of the system is
created for testing.

• Automated system building tools reduce the time needed to compile and integrate the
system by only recompiling those components and their dependents that have
changed.

• Continuous deployment means that as soon as a change is made, the deployed

version of the system is automatically updated. This is only possible when the
software product is delivered as a cloud-based service.

• Measurement is a fundamental principle of DevOps. You may make both process and
product measurements. Important process metrics are deployment frequency,
percentage of failed deployments, and mean time to recovery from failure.

Testing & Devops Introduction to Software Engineering 61

61