SOFTWARE ENGINEERING

Lecture No. 8

SOFTWARE TESTING - II (DYNAMIC)

BLACK BOX TESTING

By Engr. Muhammad Haseeb Nasir

Topics to Cover
 Software Testing
 Testing Process
 Testing Stages
 Principles of Testing
 Test Cases
 Testing Types
 Black Box testing
 BLACK BOX TESTING TECHNIQUES

 Equivalence Class Partitioning Testing

 Boundary Value Testing
 Fuzzy Testing
 Omission Testing
 Integration Testing
 Sandwich Testing
Security Testing
 Compatibility Testing
 Null Case Testing
 Volume Testing
 Load Testing
 Stress Testing
 Resource Testing
 Requirements/Specification Testing
 Configuration Testing
 Documentation Testing
 Smoke Testing
 Usability Testing
 Exploratory Testing
 Button Press Testing
 State Transition Testing
 Installation Testing
 Performance Testing
 Load Testing
 Stress Testing
 Volume testing
 Configuration testing
 Compatibility test
 Security testing
 Recovery testing
 Human factors testing/usability
Load Testing
 Load testing is the process of putting demand on a system or device and measuring its
response. Load testing is performed to determine a system’s behavior under both
normal and anticipated peak load conditions.
 It helps to identify the maximum operating capacity of an application as well as any
bottlenecks and determine which element is causing degradation.

 Example: Using automation software to simulate 500 users logging into a web site and performing end-
user activities at the same time.
 Example: Typing at 120 words per minute for 3 hours into a word processor.
 Stress Testing
 Stress testing is a form of testing that is used to
determine the stability of a given system or entity.

 It involves testing beyond normal operational

capacity, often to a breaking point, in order to
observe the results.

 In stress testing you continually put excessive load on

the system until the system crashes

 The system is repaired and the stress test is

repeated until a level of stress is reached that is
higher than expected to be present at a customer
site.
 Volume Testing
 Volume testing refers to testing a software application with a certain amount of data. This amount
can, in generic terms, be the database size or it could also be the size of an interface file that is
the subject of volume testing.
 For example, if you want to volume test your application with a specific database size, you will
expand your database to that size and then test the application's performance on it. Another
example could be when there is a requirement for your application to interact with an interface
file; this interaction could be reading and/or writing on to/from the file.
 Compatibility Testing [1]
 Exposes defects related to using files from output one version of the software in another
version of the software.
 Most Landmark applications are designed to be “forwards” compatible, meaning files
created in a previous release of the software can be used in the version currently under
test.
 They are not designed to be “backwards” compatible, meaning a file output in the
version under test will not work in a current released version.
 Configuration Testing [1]
 During this testing tester validate how well our current project is able to
supports on different types of hardware technologies like as different types
of printers, n/w interface cord(NIC),topology etc. This testing is also called
as hardware testing or portable testing
 Recovery Testing [9]
 In software testing, recovery testing is the activity of testing how well an application is able to
recover from crashes, hardware failures and other similar problems.
 Recovery testing is the forced failure of the software in a variety of ways to verify that recovery
is properly performed.
 Recovery testing is basically done in order to check how fast and better the application can
recover against any type of crash or hardware failure etc. Type or extent of recovery is
specified in the requirement specifications.
 Examples of recovery testing:

 While an application is running, suddenly restart the computer, and afterwards check the validness of the
application's data integrity.
 While an application is receiving data from a network, unplug the connecting cable. After some time, plug
the cable back in and analyze the application's ability to continue receiving data from the point at which
the network connection disappeared.
 Restart the system while a browser has a definite number of sessions. Afterwards, check that the browser is
able to recover all of them.
 Usability Testing
 Usability testing is a technique used to evaluate a product by testing it on users. This
can be seen as an irreplaceable usability practice, since it gives direct input on how
real users use the system. Usability testing focuses on measuring a human-made
product's capacity to meet its intended purpose.
 Usability testing measures the usability, or ease of use, of a specific object or set of
objects.
 User interviews, surveys, video recording of user sessions, and other techniques can be
used
Resource Testing
 In resource testing you have to check whether an AUT(Application under
test) utilizes more resources (e.g memory) than it should be utilizing.
 Requirements/Specification Testing
 Exposes defects in the program design/implementation by comparing the
program to every word in the Requirement

 Document or the Function Specification Document. Important these documents

kept up‐to‐date.
 Documentation Testing
 Exposes defects in the content and access of on-line
user manuals (Help files) and content of training
manuals.

 The Testing Group tests that all Help files appear on

the screen when selected.

 On-Line documentation is a Landmark requirement for

product release

 Documentation testing can be approached in two

phases:

 1st Phase is Review and Inspection, examines the

documents for editorial clarity and errors

 2nd phase is Live Test, which uses the documentation in

conjunction with the use of the actual program.
 Exploratory Testing
 Exploratory testing is an approach to software testing that is concisely described as
simultaneous learning, test design and test execution. Exploratory software testing is a
powerful and fun approach to testing.
 The essence of exploratory testing is that you learn while you test, and you design
your tests based on what you are learning
 Exploratory testing is a method of manual testing.
 The testing is dependent on the tester's skill of inventing test cases and finding
defects. The more the tester knows about the product and different test methods, the
better the testing will be.
 Button Press Testing
 Button Press Testing: (Landmark testing term, not industry standard)
 Exposes functionality defects by methodically pressing every widget (pull
down menu, pop ups, drop down lists, buttons, icons, etc.) in the program.
 Installation Testing
 Implementation testing installation testing is a kind of quality assurance
work in the software industry that focuses on what customers will need to do to
install and set up the new software successfully. The testing process may involve
full, partial or upgrades install/uninstall processes.

 This testing is typically done by the software testing engineer in conjunction

with the configuration manager.
 Regression Testing
 Exposes defects in code that should have not changed.

 Re-executes some or all existing test cases to exercise code that was tested in a previous release
or previous test cycle.

 Performed when previously tested code has been re-linked such as when:

 Ported to a new operating system

 A fix has been made to a specific part of the code.

 Studies shows that:

 The probability of changing the program correctly on the first try is only 50% if the change
involves 10 or fewer lines of code.
 The probability of changing the program correctly on the first try is only 20% if the change
involves around 50 lines of code.
 Progressive VS Regressive Testing
 When testing new code, you are performing “progressive testing.”

 When testing a program to determine if a change has introduced errors in the unchanged code,
you are performing “regression testing.”

 All black box test design methods apply to both progressive and regressive testing. Eventually, all
your “progressive” tests should become “regression” tests.

 The Testing Group performs a lot of Regression Testing because most Landmark development
projects are adding enhancements (new functionality) to existing programs. Therefore, the existing
code (code that did not change) must be regression tested.
 Regression Testing VS Retesting

 Re- test - Retesting means testing only the certain part of an application again and not
considering how it will effect in the other part or in the whole application.

 Regression Testing - Testing the application after a change in a module or part of the
application for testing that is the code change will affect rest of the application.
Validation Testing
 Acceptance Testing

 Alpha Testing

 Beta Testing
Acceptance Testing
 It is virtually impossible for a software developer to foresee how the customer will really use
a program

 When custom software is built for one customer, a series of acceptance tests are conducted to
enable the customer to validate all requirements

 Conducted by the end user rather than software engineers

 An acceptance test can range from an informal test drive to a planned and systematically
executed series of tests

 In the case of software, acceptance testing performed by the customer is known as user
acceptance testing (UAT), end-user testing, site (acceptance) testing, or field (acceptance)
testing
Alpha Testing
 In this type of testing, the users are invited at the development center where they use the
application and the developers note every particular input or action carried out by the user.
Any type of abnormal behavior of the system is noted.

 Alpha tests are conducted in a controlled environment

Beta Testing
 The beta test is conducted at end user sites. Unlike
alpha testing , the developer is generally not present.

 Therefore the beta test is a live application of the

software in an environment that cannot be controlled by
the developer

 In this type of testing, the software is handed over to

the user in order to find out if the software meets the
user expectations and works as it is expected to.

 The end user records all problems that are encountered

during beta testing and reports these to the developer
at regular intervals

 As a result of problems reported during beta tests,

software engineers make modifications and then
prepare for release of the software product
References
[1] LandMark Resource Software House testing content
[2] http://jamesmccaffrey.wordpress.com/2008/08/29/the-difference-between-unit-testing-and-module-testing
[3] http://www.faqs.org/faqs/software-eng/testing-faq/section-14.html
[4] http://en.wikipedia.org/wiki/Sanity_testing
[5] http://www.softwaretestingstuff.com/2009/12/difference-between-smoke-sanity-testing.html
[6] http://en.wikipedia.org/wiki/White-box_testing
[7] http://agile.csc.ncsu.edu/SEMaterials/WhiteBox.pdf
[8] http://en.wikipedia.org/wiki/Compatibility_testing
[9] http://en.wikipedia.org/wiki/Recovery_testing
[10] http://en.wikipedia.org/wiki/Security_testing
[11] http://searchsecurity.techtarget.com/definition/nonrepudiation
[12] http://www.enotes.com/topic/Operational_Acceptance_Testing
[13] http://www.buzzle.com/editorials/7-14-2006-102344.asp
[14] http://en.wikipedia.org/wiki/Acceptance_testing
[15] http://www.scan-associates.net/services_spa.htm
[16] http://searchsecurity.techtarget.com/definition/ethical-hacker
For any query Feel Free to ask