Scribd
Upload
101 views37 pages

Object-Oriented and Classical Software Engineering: Stephen R. Schach

Execution-based testing and nonexecution-based testing are two types of testing. Software Quality Assurance (sqa) Goes far beyond "v and v" Managerial independence >> development group >> SQA group. Sqa is responsible for ensuring that software satisfies its specifications.

Uploaded by

hsiang.liu4806
Copyright
© Attribution Non-Commercial (BY-NC)
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPT, PDF, TXT or read online on Scribd
101 views37 pages

Object-Oriented and Classical Software Engineering: Stephen R. Schach

Execution-based testing and nonexecution-based testing are two types of testing. Software Quality Assurance (sqa) Goes far beyond "v and v" Managerial independence >> development group >> SQA group. Sqa is responsible for ensuring that software satisfies its specifications.

Uploaded by

hsiang.liu4806
Copyright
© Attribution Non-Commercial (BY-NC)
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPT, PDF, TXT or read online on Scribd
You are on page 1/ 37

Slide 6.

Object-Oriented and
Classical Software
Engineering
Fifth Edition, WCB/McGraw-Hill, 2002

Stephen R. Schach
srs@vuse.vanderbilt.edu

© The McGraw-Hill Companies, 2002


CHAPTER 6 Slide 6.2

TESTING

© The McGraw-Hill Companies, 2002


Overview Slide 6.3

● Quality issues
● Nonexecution-based testing
● Execution-based testing
● What should be tested?
● Testing versus correctness proofs
● Who should perform execution-based testing?
● When testing stops

© The McGraw-Hill Companies, 2002


Testing Slide 6.4

● Two types of testing


– Execution-based testing
– Nonexecution-based testing

© The McGraw-Hill Companies, 2002


Testing (contd) Slide 6.5

● “V & V”
– Verification
» Determine if the phase was completed correctly
– Validation
» Determine if the product as a whole satisfies its requirements

© The McGraw-Hill Companies, 2002


Testing (contd) Slide 6.6

● Warning
– “Verify” also used for all nonexecution-based testing

© The McGraw-Hill Companies, 2002


Software Quality Slide 6.7

● Not “excellence”
● Extent to which software satisfies its specifications
● Software Quality Assurance (SQA)
– Goes far beyond V & V
– Managerial independence
» development group
» SQA group

© The McGraw-Hill Companies, 2002


Nonexecution-Based Testing Slide 6.8

● Underlying principles
– We should not review our own work
– Group synergy

© The McGraw-Hill Companies, 2002


Walkthroughs Slide 6.9

● 4–6 members, chaired by SQA


● Preparation—lists of items
● Inspection
– Up to 2 hours
– Detect, don’t correct
– Document-driven, not participant-driven
– Verbalization leads to fault finding
– Performance appraisal

© The McGraw-Hill Companies, 2002


Inspections Slide 6.10

● Five-stage process
– Overview
– Preparation, aided by statistics of fault types
– Inspection
– Rework
– Follow-up

© The McGraw-Hill Companies, 2002


Fault statistics Slide 6.11

● Recorded by severity and fault type


● Compare with previous products
● What if there are a disproportionate number
of faults in a module?
● Carry forward fault statistics to the next phase

© The McGraw-Hill Companies, 2002


Statistics on Inspections Slide 6.12

● 82% of all detected faults (IBM, 1976)


● 70% of all detected faults (IBM, 1978)
● 93% of all detected faults (IBM, 1986)
● 90% decrease in cost of detecting fault
(Switching system, 1986)
● 4 major faults, 14 minor faults per 2 hours
(JPL, 1990). Savings of $25,000 per
inspection
● Number of faults decreased exponentially
by phase (JPL, 1992)
● Warning
– Fault statistics and performance appraisal
© The McGraw-Hill Companies, 2002
Metrics for Inspections Slide 6.13

● Fault density (e.g., faults per KLOC)


● Fault detection rate (e.g., faults detected per hour)
● By severity (major/minor), by phase
● What does a 50% increase in the fault detection
rate mean?

© The McGraw-Hill Companies, 2002


Execution-Based Testing Slide 6.14

● Definitions
– Failure (incorrect behavior)
– Fault (NOT “bug”)
– Error (mistake made by programmer)
● Nonsensical statement
– “Testing is demonstration that faults are not present”

© The McGraw-Hill Companies, 2002


What is execution-based tested? Slide 6.15

● “The process of inferring certain behavioral


properties of product based, in part, on results
of executing product in known environment with
selected inputs.”
– Inference
– Known environment
– Selected inputs
● But what should be tested?

© The McGraw-Hill Companies, 2002


Utility Slide 6.16

● Does it meet user’s needs?


– Ease of use
– Useful functions
– Cost-effectiveness

© The McGraw-Hill Companies, 2002


Reliability Slide 6.17

● Frequency and criticality of failure


– Mean time between failures
– Mean time to repair
– Mean time, cost to repair results of failure

© The McGraw-Hill Companies, 2002


Robustness Slide 6.18

● Range of operating conditions


● Possibility of unacceptable results with valid input
● Effect of invalid input

© The McGraw-Hill Companies, 2002


Performance Slide 6.19

● Extent to which space and time constraints are met


● Real-time software

© The McGraw-Hill Companies, 2002


Correctness of specifications Slide 6.20

● Incorrect specification for a sort

● Function trickSort which satisfies this specification:


© The McGraw-Hill Companies, 2002
Correctness of specifications (coptd) Slide 6.21

● Incorrect specification for a sort:

● Corrected specification for the sort:


© The McGraw-Hill Companies, 2002
Correctness Slide 6.22

● NOT necessary
● NOT sufficient

© The McGraw-Hill Companies, 2002


Correctness Proofs Slide 6.23

● Alternative to execution-based testing

© The McGraw-Hill Companies, 2002


Example of Correctness Proof Slide 6.24

● Code segment to be proven correct

© The McGraw-Hill Companies, 2002


Example of Correctness Proof (contd) Slide 6.25

● Flowchart of
code segment

© The McGraw-Hill Companies, 2002


Example of Correctness Proof Slide 6.26

© The McGraw-Hill Companies, 2002


Correctness Proof Case Study Slide 6.27

● Never prove a program correct without testing it


as well

© The McGraw-Hill Companies, 2002


Episode 1 Slide 6.28

● 1969 — Naur Paper


● “Naur text-processing problem”
Given a text consisting of words separated by blank or by 
(new line) characters, convert it to line-by-line form in
accordance with following rules:
(1) line breaks must be made only where given text
has blank or nl ;
(2) each line is filled as far as possible, as long as
(3) no line will contain more than maxpos characters
● Naur constructed a procedure (25 lines of Algol 60),
and informally proved its correctness

© The McGraw-Hill Companies, 2002


Episode 2 Slide 6.29

● 1970 — Reviewer in Computing Reviews


– The first word of the first line is preceded by blank unless
the first word is exactly maxpos characters long

© The McGraw-Hill Companies, 2002


Episode 3 Slide 6.30

● 1971 — London finds 3 more faults


● Including:
– The procedure does not terminate unless a word
longer than maxpos characters is encountered

© The McGraw-Hill Companies, 2002


Episode 4 Slide 6.31

● 1975 — Goodenough and Gerhart


find three further faults
● Including:
– The last word will not be output unless it
is followed by blank or nl

© The McGraw-Hill Companies, 2002


Proofs and Software Engineering Slide 6.32

● Lesson:
● Even if product is proved correct,
it must STILL be tested.

© The McGraw-Hill Companies, 2002


Three Myths Slide 6.33

● Software engineers do not have enough


math for proofs
● Proving is too expensive to be practical
● Proving is too hard

© The McGraw-Hill Companies, 2002


Proofs and Software Engineering (contd) Slide 6.34

● Can we trust a theorem prover ?


● How to find input–output specifications,
loop invariants
● What if the specifications are wrong?
● Can never be sure that specifications
or a verification system are correct

© The McGraw-Hill Companies, 2002


Proofs and Software Engineering (contd) Slide 6.35

● Correctness proofs are a vital software engineering


tool, WHERE APPROPRIATE.
● If
– Human lives are at stake
– Indicated by cost/benefit analysis
– Risk of not proving is too great
● Also, informal proofs can improve the quality of
the product
– Assert statement

© The McGraw-Hill Companies, 2002


Who Performs Execution-Based Testing? Slide 6.36

● Testing is destructive
– A successful test finds a fault
● Solution
– 1. The programmer does informal testing
– 2. SQA does systematic testing
– 3. The programmer debugs the module
● All test cases must be
– Planned beforehand, including expected output
– Retained afterwards

© The McGraw-Hill Companies, 2002


When Testing Can Stop Slide 6.37

● Only when the product has been irrevocably retired

© The McGraw-Hill Companies, 2002

You might also like

pFad - Phonifier reborn

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy