CSE526 STQA From Lec 1 to Lec 16

The uniqueness of software quality assurance

DO you think that there is a bug-free software? Can software developers warrant their software applications and their documentation from any bugs or defects ? What are the essential elemental differences between software and other industrial products such as automobiles, washing machines etc?

The essential differences between software and other industrial products can be categorized as follows :
1.

2. 3.

Product complexity : # of operational modes the product permit. Product visibility : SW products are invisible. Product development and production process.

What is Software ?

IEEE Definition:

Software Is : Set of Computer programs, procedures, and possibly associated documentation and data pertaining to the operation of a computer system.

IEEE Definition is almost identical to the ISO def. ( ISO/IEC 9000-3 )

Computer programs (Code) Procedures Documentation Data necessary for operation the SW system.

So we can say
Software quality assurance always includes : Code quality The quality of the documentation And the quality of the necessary SW data

SW errors, faults and failures

An error : can be a grammatical error in one or more of the code lines, or a logical error in carrying out one or more of the clients requirements. Faults: Most of time Caused by error SW failures that disrupt our use of the software.

The relationship between SW faults & SW failures:

Do all SW faults end with SW failures?

The answer is not necessarily The SW fault becomes a SW failure only when it is activated.

Classification of the causes of SW errors

SW errors are the cause of poor SW quality SW errors can be

Code error Documentation error SW data error

The cause of all these errors are human

The nine causes of software errors

1.

2.
3. 4. 5.

6.
7. 8. 9.

Faulty requirement definition Client-developer communication failures Deliberate deviation from SW requirements Logical design errors Coding errors Non-compliance with documentation and coding instructions Shortcomings of the testing process Procedure errors Documentation errors

10

Faulty requirement definition

1.

2.
3. 4.

Erroneous definition of requirements Absence of vital requirements Incomplete definition of requirements Inclusion of unnecessary requirements

11

Client-developer communication failures

Misunderstandings resulting from defective client-developer communications. Misunderstanding of the clients requirements changes presented to the developer

In written forms Orally Responses to the design problems others

12

Deliberate deviation from SW requirements

The developer reuse SW modules taken from the earlier project Due to the time budget pressure Due to the unapproved improvements

13

Logical design errors

This is come from systems architects, system analysts, SW engineers such as:

Erroneous algorithms Erroneous definition of boundary conditions

14

Coding errors

Misunderstanding the design documentation errors in the prog. Lang. Errors in the application of CASE and other dev. Tools.

15

Non-compliance with documentation and coding

Team members who need to coordinate their own codes with code modules developed by non-complying team members Individuals replacing the non-complying team member will find it difficult to fully understand his work.

16

Shortcomings of the testing process

Incomplete testing plans Failures to document and report errors and faults Incomplete correction of detected errors.

17

QUALITY

Quality refers to any measurable characteristics such as correctness, maintainability, portability, testability, usability, reliability, efficiency, integrity, reusability and interoperability.

Software quality - Definition IEEE

1.

2.

The degree to which a system, component, or process meets specified requirements. The degree to which a system, component, or process meets customer or user needs or expectations.

19

Software Quality Pressmans def.

Conformance to explicitly stated functional and performance requirements, explicitly documented standards, and implicit characteristics that are expected of all professionally developed software.

20

Software Quality Assurance The IEEE Definition

1.

SQA is : A planned and systematic pattern of all actions necessary to provide adequate confidence that an item or product conforms to established technical requirements.

21

SQA: What is SQA?

SQA is the process of assuring people that
every effort has been made to ensure that software products have the desired level of reliability, maintainability, usability.

SQA: What is Software (System) Quality?

Absence of defects
Fitness for use Meets specifications Adherence to CMM or ISO quality standards Reliable, Maintainable, Useable.

SQA: An SQA Program

Minimizing number of defects in delivered s/w

Creating mechanisms for controlling software development and maintenance so that costs and schedules can be met Making certain that the delivered product can

be used in its intended marketplace

Improving the quality of future products

Software Quality Assurance Vs. Software Quality Control

Quality Control : a set of activities designed to evaluate the quality of a developed or manufactured product. It take place before the product is shipped to the client. Quality Assurance : the main objective is to minimize the cost of guaranteeing quality by a variety of activities performed throughout the causes of errors, and detect and correct them early in the development Process.

25

Quality Concepts

Quality of Design refers to the characteristics that designers specify for an item.

Quality of Conformance is the degree to which the design specifications are followed during manufacturing.

Quality Control is the series of inspections, reviews and tests used throughout the development cycle to ensure that each work product meets the requirements placed upon it.
6

(cont'd)...

Quality policy refers to the basic aims and objectives of an organization regarding quality as stipulated by the management.

Quality assurance consists of the auditing and reporting functions of management. Cost of Quality includes all costs incurred in the pursuit of quality or in performing quality related activities such as appraisal costs, failure costs and external failure costs.
7

(cont'd)...

Quality planning is the process of assessing the requirements of the procedure and of the product and the context in which these must be observed. Quality testing is assessment of the extent to which a test object meets given requirements Quality assurance plan is the central aid for planning and checking the quality assurance. Quality assurance system is the organizational structure, responsibilities, procedures, processes and resources for implementing quality management.
8

SQ. Factors

The requirements document is one of the most important elements for achieving SQ. What is a Good SQ requirements document ?

29

Software quality factors

Product operation factors

Product revision factors

Product transition factors

Classification of SW requirements into SW quality factors.

McCalls Factor Model This model classifies all SW requirements into 11 SW quality factors, grouped into 3 categories: Product operation: Correctness, Reliability, Efficiency, Integrity, Usability Product revision : Maintainability, Flexibility, Testability Product transition : Portability, Reusability, Interoperability.

31

McCalls Quality Factors and Criteria

Product operation factors

Correctness: extent to which a program fulfills its specification. Reliability: ability not to fail. Efficiency: use of resources execution and storage. Integrity: protection of the program from unauthorized access. Usability: ease of use of the software.
33

Product revision factors

Maintainability: effort required to locate and fix a fault in a program. Flexibility: ease of making changes required by changes in operating environment. Testability: ease of testing the program to ensure that it is error-free and meets its specification.
34

Product transition factors

Portability: Effort required to transfer a program from one environment to another system. Reusability: ease of re-using software in a different context. Interoperability: effort required to couple a system to another system.
35

Product operation SW quality factors

Correctness: Output specifications are usually multidimensional ; some common include:

The output mission The required accuracy The completeness The up-to-dateness of the info. The availability of the info.( the reaction time ) The standards for coding and documenting the SW system

36

Product operation SW quality factors

Reliability: Deals with failures to provide service. They determine the maximum allowed SW system failure rate, and can refer to the entire system or to one or more of its separate functions.

37

Product operation SW quality factors

Efficiency: Deals with the HW resources needed to perform all the functions of the SW system in conformance to all other requirements. Integrity: Deals with the SW system security, that is requirements to prevent access to unauthorized persons.
38

Product operation SW quality factors

Usability: Deals with the scope of staff resources needed to train a new employee and to operate the SW system.

39

Product revision SW quality factors

Maintainability : Maintainability requirements determine the efforts that will be needed by users and maintenance personnel to identify the reasons for SW failures, to correct the failure, and to verify the success of the corrections. Example : Typical maintainability requirements: 1. The size of a SW module will not exceed 30 statements 2. The programming will adhere to the company coding standards and guidelines.

40

Product revision SW quality factors

Flexibility : The capabilities and efforts required to support adaptive maintenance activities are covered by flexibility requirements. This factors requirements also support perfective maintenance activities, such as changes and additions to the SW in order to improve its service and adapt it to changes in the firms technical or commercial environment.

41

Product revision SW quality factors

-

Testability : Deal with the testing as well as with its operation. Providing predefined intermediate results and log files. Automatic diagnostics performed by the SW system prior starting the system, to find out whether all components of SW system are in working order. Obtain a report about detected faults.

42

Product transition SW quality factors

-

Portability : Tend to the adaptation of a SW system to other environments consisting :

Different HW Different OS Example : SW designed to work under windows 2000 env. Is required to allow low-cost transfer to Linux.
-

43

Product transition SW quality factors

-

Reusability : Deals with the use of SW modules originally designed for one project in a new SW project currently begin developed. The reuse of SW is expected to save resources., shorten the project period, and provide higher quality modules. These benefits of higher quality are based on the assumption that most SW faults have already been detected by SQA activities performed previously on it.

44

Product transition SW quality factors

-

Interoperability : Focus on creating interfaces with other SW systems or with other equipment firmware. Example:
The firmware of medical lab. equipment is required to process its results according to a standard data structure that can be then serve as input for a number of standard laboratory.

45

Alternative Models Of SW Quality Factors

Two other models for SQ factors:

Evans and Marciniak 1987 ( 12 factors ) Deutsch and Willis 1988. ( 15 factors )

Five new factors were suggested

Verifiability Expandability Safety Manageability Survivability

46

Five new factors were suggested

Verifiability: define design and programming features that enable efficient verification of the design and programming ( modularity, simplicity, adherence to documentation and prog guidelines. ) Expandability: refer to future efforts that will be needed to serve larger populations, improve services, or add new applications in order to improve usability. Safety: meant to eliminate conditions hazardous to equipment as a result of errors in process control SW. Manageability: refer to the admin. tools that support SW modification during the SW development and maintenance periods. Survivability: refer to the continuity of service. These define the minimum time allowed between failures of the system, and the maximum time permitted for recovery of service.
47

Who is interested in the definition of quality requirements ?

The client is not the only party interested in defining the requirements that assure the quality of the SW product. The developer is often interested also specially :

Reusability Verifiability Portability

Any SW project will be carried out according to 2 requirements document :

The clients requirements document The developers additional requirements document.

48

The SQA system- an SQA architecture

SQA system components can be classified into 6 classes :

Pre-project components Components of project life cycle activities assessment Components of infrastructure error prevention and improvement. Components of SQ management Components of standardization, certification, and SQA system assessment Organizing for SQA- the human components
49

Pre-project Components :

To assure that : 1. The project commitments have been adequately defined considering the resources required, the schedule and budget. 2. The development and quality plans have been correctly determined.

50

Components of project life cycle activities assessment:

1.

The project life cycle composed of two stages: The development Life cycle stage:

Detect design and programming errors Its components divided into:

Reviews Expert opinions Software testing Assurance of the quality of the subcontractors work and customer-supplied parts.
Include specialize maintenance components as well as development life cycle components, which are applied mainly for functionality improving maintenance tasks.

2.

The operation-maintenance stage

51

Components of infrastructure error prevention and improvement :

Main objectives of these components, which are applied throughout the entire organization, are : To eliminate or at least reduce the rate of errors, based on the organizations accumulated SQA experience.

52

Components of software quality management :

This class of components is geared toward several goal: The major ones being the control of development and maintenance activities and introduction of early managerial support actions that mainly prevent or minimize schedule and budget failures and their outcomes.
53

Components of standardization, certification, and SQA system assessment

The main objective of this class are: 1. Utilization of international professional knowledge 2. Improvement of coordination of the organizational quality system with other organizations 3. Assessment of the achievements of quality systems according to a common scale. The various standards classified into 2 groupes:

Quality management standards Project process standards.

54

Organizing for SQA- the human components

The SQA organizational base includes : Managers Testing personnel The SQA unit and practitioners interested in SQ. The main objectives are to initiate and support the implementation of SQA components Detect deviation from SQA procedures and methodology Suggest improvements
55

The Contract review process and its stages

Several situations can lead a SW company to sign a contract with a customer such as :

Participation in a tender Submission of a proposal. Receipt of an order from a companys customer Receipt of an internal request or order from another department in the organization

56

The Contract review process and its stages

Contract review :

is the SQA component devised to guide review drafts of proposal and contract documents. If applicable, provides oversight ( supervision ) of the contracts carried out with potential project partners and subcontractors.

57

Proposal draft review

+
Contract draft review --------------------------Contract review

The Contract review process itself is conducted in two stages :

Stage 1 Review of the proposal draft prior to submission to the potential customer ( proposal draft review ): Reviews the final proposal draft and proposals foundations:

Customers requirement documents Customers additional details and explanations of the requirements Cost and resources estimates Existing contracts or contract drafts of the supplier with partners and subcontractors.

59

The Contract review process itself is conducted in two stages :

Stage 2 Review of the proposal draft prior to signing ( Contract draft review ): Reviews the contract draft on the basis of the proposal and the understandings ( include changes ) reached during the contract negotiations sessions. The individuals who perform the review thoroughly examine the draft while referring to a comprehensive range of review subjects ( a Check-list ) is very helpful for assuring the full coverage of relevant subjects. See appendix 5A, 5B
60

Contract Review objectives:

Proposal draft review objectives( assure the following )

Customer requirements have been clarified and documented Alternative approaches for carrying out the project have been examined Formal aspects of the relationship between the customer and SW firm have been specified. Identification of development risks Adequate estimation of project resources and timetable have been prepared. Examination of the customers capacity to fulfill his commitments Definition of partners and subcontractors participation conditions Definition and projection proprietary rights.
61

Contract Review objectives:

Contract draft review objectives( assure the following )

No un-clarified issues remain in the contract draft All the understandings reached between the customer and the firm are to be fully and correctly documented. No changes, additions, or omissions that have not been discussed and agreed upon should be introduced into contract draft.
62

Who performs a contract review:

The leader or another member of the proposal team The members of the proposal team An outside professional or a company staff member who is not a member of the proposal team. A team of outside experts.

63

Implementation of a contract review of a major proposal

Recommended approaches for implementing major contract reviews :

The contract review should be scheduled. A team should carry out the contract review A contract team leader should be appointed The activities of the team leader include :

Recruitment of the team members Distribution of review tasks Coordination between members Coordination between the review team and the proposal team Follow-up of activities, especially compliance with the schedule Summarization of the findings and their delivery to the proposal team.
64

Development and quality plans

Development plans and quality plans are the major elements needed for project compliance with ISO 9000.3 standards and ISO/IEC 2001 and with IEEE 730. It is also an important element in the Capability Maturity Model ( CMM ) for assessment of SW development organization maturity. The projects needs development and quality plans that :

Are based on proposal materials that have been re-examined and thoroughly updated Are more comprehensive than the approved proposal, especially with respect to schedules, resources, estimates, and development risk evaluations Include additional subjects, absent from the approved proposal others
65

Development plan and quality plan objectives

1.

2.

3.
4. 5.

Scheduling development activities that will lead to successful and timely completion of the project, and estimating the required manpower resources and budget. Recruiting team members and allocating development resources. Resolving development risks. Implementing required SQA activities Providing mgt. with data needed for project control.

66

Elements of the development plan

1. 2. 3. 4. 5. 6.

7.
8. 9.

10.
11.

Project products Project interfaces Project methodology and development tools SW development standards and procedures The mapping of the development process.( proj. mgt. Gant ) Project milestones ( documents , code , report ) Project staff organization ( org. stru., prof. req., no of team mem., names of team leaders ) Development facilities ( SW, HW tools, space, period req. for each use ) Development risks ( see next slide ) Control methods Project cost estimation

67

Elements of Quality Plan

1. 2.

3.

Quality goals ( quantitative measures example page 102 ) Planned review activities The scope of review activity The type The schedule ( priorities ) The specific procedure to be applied Who responsible for carrying out the rev. act. Planned SW tests ( a complete list of planned SW tests should be provided ) each test The unit, integration or the complete system to be tested The type of testing activities to be carried out The planned test schedule The specific procedure Who responsible
68

Elements of Quality Plan

4.

Planned acceptance tests for externally developed SW Configuration management configuration mgt tools and procedures, including those change-control procedures meant to be applied throughout the project

5.

69

Maintenance service components

Corrective maintenance

support services and software corrections. adapts the software package to differences in new customer

Adaptive maintenance

requirements, Functionality improvement maintenance perfective maintenance of new functions added to the software so as to enhance performance, preventive maintenance activities that improve reliability and system infrastructure for easier and more efficient future maintainability

70

Maintenance software quality assurance tools

SQA tools for corrective maintenance SQA tools for functionality improvement maintenance SQA infrastructure tools for software maintenance SQA tools for managerial control of software maintenance.

71

Maintenance software quality assurance tools

SQA tools for corrective maintenance

Entail (User support services and Software corrections bug repairs) Most bug repair tasks require the use of mini-testing tool Required to handle repair patch (small-scale) tasks (small number of coding line to be corrected rapidly)

72

Maintenance software quality assurance tools

SQA tools for functionality improvement maintenance The same project life cycle tools are applied (reviews and testing etc..) Are implemented also for large-scale adaptive maintenance tasks

73

Maintenance software quality assurance tools

SQA infrastructure component for S.W maintenance

We need SQA infrastructure tools for:

Maintenance procedures and work instructions Supporting quality devices Preventive and corrective actions Configuration management Documentation and quality record control Training and certification of maintenance teams

74

Maintenance software quality assurance tools

Maintenance procedures and work instructions

Remote handling of request for service On-site handling User support service Quality assurance control Customer satisfaction surveys

75

Maintenance software quality assurance tools

Supporting quality devices

Checklists for location of causes for a failure Templates for reporting how software failure were solved Checklists for preparing a mini testing procedure document

76

Maintenance software quality assurance tools

Preventive and corrective actions

Directed and controlled the CAB Corrective Action Board Changes in content and frequency of customer requests for user support services Increased average time invested in complying with customers user support requests Increased average time invested in repairing customers software failures Increased percentage of software correction failures.

77

Configuration management Failure repair

Software system version installed at the customers site A copy of the current code and its documentation Decision making about the advisability of performing a group replacement Planning the group replacement, allocating resources and determining the timetable.

Group replacement

Maintenance documentation and quality records

78

The objectives of training and certification

To develop the knowledge and skills new staff need to perform software development and maintenance tasks at an adequate level of efficiency and effectiveness. Such training facilitates integration of new team members. To assure conformity to the organizations standards for software products (documents and code) by transmitting style and structure procedures together with work instructions.

To update the knowledge and skills of staff in response to developments in the organization, and to assure efficient and effective performance of tasks as well as conformity to the organizations style and structure procedures and work instructions. To transmit knowledge of SQA procedures. To assure that candidates for key software development and maintenance positions are adequately qualified.

The training and certification process

The operation of a successful training and certification system demands that the following activities be regularly performed: Determine the professional knowledge requirements for each position Determine the professional training and updating needs Plan the professional training program Plan the professional updating program Define positions requiring certification Plan certification processes Deliver training, updating and certification programs Perform follow-up of trained and certified staff.

All these activities converge into an integrated process in which feedback from past activities and information about professional developments stimulate a cycle of continuous training, certification and adaptation to changing quality requirements. Training and certification activities are meant to fill the needs of staff and new employees. Comprehensive follow-up of the outcomes of current programs as well as keeping track of developments in the profession are required to make sure that programs are adequately up-to-date.

Corrective and preventive actions (CAPA) definition:

Corrective actions:

A regularly applied feedback process that includes collection of information on quality nonconformities, identification and analysis of sources of irregularities as well as development and assimilation of improved practices and procedures, together with control of their implementation and measurement of their outcomes.

83

Corrective and preventive actions (CAPA) definition:

Preventive actions:

A regularly applied feedback process that includes collection of information on potential quality problems, identification and analysis of departures from quality standards, development and assimilation of improved practices and procedures, together with control of their implementation and measurement of their outcomes.

84

The corrective and preventive actions process

Information collection Analysis of information Development of solutions and improved methods Implementation of improved methods Follow-up.

85

Development of solutions and their implementation

1. 2.

3.

4. 5.

Updating relevant procedures Changes in practices, including updating of relevant work instructions Shifting to a development tool that is more effective and less prone to the detected faults. Improvement of reporting methods Initiatives for training, retraining or updating staff
86

Follow-up of activities
1.

2. 3.

Follow-up of the flow of development and maintenance CAPA records Follow-up of implementation Follow-up of outcomes

87

Organizing for preventive and corrective actions

CAPA activities depends on

the existence of a permanent core organizational ad hoc team participants

members of the SQA unit top-level professionals, development and maintenance department managers

Can be

88

Software configuration item (SCI) or configuration item (CI)

An approved unit of software code, a document or piece of hardware that is designed for configuration management and treated as a distinct entity in the software configuration management process.

SCI version The approved state of an SCI at any given point of time during the development or maintenance process. Software configuration version An approved selected set of documented SCI versions that constitute a software system or document at a given point of time, where the activities to be performed are controlled by software configuration management procedures. The software configuration versions are released according to the cited procedures.

A unit of software code, a document or piece of hardware is defined as an SCI if it is assumed that it may be needed for further development of the software system and/or its maintenance. A software configuration is composed of as many SCIs as the developers assume will be needed in the future, with each SCI approved, identified and registered.

The SCIs are generally placed into four classes,as follows: Design documents Software code Data files, including files of test cases and test scripts Software development tools.

software configuration management

software configuration management (SCM) is the task of tracking and controlling changes in the software. SCM are the practices and procedures for administering source code, producing software development builds, controlling change, and managing software configurations.

Specifically, SCM ensures the integrity, reliability and reproducibility of developing software products from conception to release. SQA processes provide assurance that the software products and processes in the project life cycle conform to their specified requirements by planning and performing a set of activities to provide adequate confidence that quality is being built into the software.

SCM is the development and use of software technical standards and processes for managing evolving software systems. SCM is closely related to the software quality assurance (SQA) activity.

The tasks of software configuration management

The tasks of software configuration management may be classified into four groups: Control software change Release of SCI and software configuration versions Provision of SCM information services Verification of compliance to SCM procedures

Software change control

Grant approval to carry out changes Control the changes and assure the quality of approved changes Document the approved changes Apply mechanisms that coordinate the changes made to the SCI by preventing more than one team from simultaneously introducing changes into the same SCI.

3 basic SQA management tools

Project Progress Control, Software Quality Metrics Software Quality Costs

Project Progress Control, enables management to oversee each project and initiate, when required. And how changes and improvements in a project is performed.

Project Progress Control

Delay in completing project mostly caused by: optimistic scheduling and budgeting unprofessional software risk management belated identification of schedule and budget difficulties underestimation

Components of Project Progress Control

1. Control of Risk Management activities, begin with the preparation of periodic assessments about the state of the SW risk items and the expected outcomes of the risk management activities performed in their wake. 2. Project schedule control, deals with the projects compliance with its approved and contracted timetables

3. Project resource control , focuses on professional human resources, internal composition or allocation 4. Project Budget Control, based on the comparison of actual with scheduled expenditure, more accurate picture of budget deviations requires.

Project Progress Control

Examples of Metrics from Everyday Life

Working and living

Cost of utilities for the month Cost of groceries for the month Amount of monthly rent per month Time spent at work each Saturday for the past month Time spent mowing the lawn for the past two times Grades received in class last semester Number of classes taken each semester Amount of time spent in class this week Amount of time spent on studying and homework this week Number of hours of sleep last night Time to drive from home to the airport Amount of miles traveled today Cost of meals and lodging for yesterday

College experience

Travel

104

Software quality metrics

Metrics: measurable ways to design and assess the software product Process and project metrics apply to the project as a whole Project metrics focus on specific attributes of the project Collected as technical tasks are being conducted

Quality metrics

a function whose inputs are software data and whose output is a single numerical value that can be interpreted as the degree to which the SW possesses a given quality attribute.

106

Quality metrics

You cant control what you cant measure

Tom DeMarco (1982)

IEEE definition A quantitative measure of the degree to which an item possesses a given quality attribute

107

Main objectives of software quality metrics

To facilitate management control, planning and execution of the appropriate managerial interventions To identify situations that require development or maintenance process improvement in the form of preventive or corrective actions

108

Software quality metrics

Depend on system size measures by (KLOC, Function Point)

KLOC: Measures the size of software by

thousands of code lines Classic metric

Function Point

It measures project size by functionality, indicated in the customers or tender requirement specification

109

Software quality metrics

Classification

Process metrics

Related to the software development process

Product metrics

Related to software maintenance

110

Process metric A metric used to measure characteristics of the methods, techniques, and tools employed in developing, implementing, and maintaining a software system Product metric A metric used to measure that characteristic of any product of the software development process

Process metrics
1.

Software process quality metrics

Error density metrics Error severity metrics.

2. 3. 4.

Software process timetable metrics Error removal effectiveness metrics Software process productivity metrics

112

Product metric

Refer to the software systems operational phase (Customer services) Customer services have two types

Help desk services (HD)

Method of application of the software and solution of customer implementation problems

correction of software failures identified by users or detected by the customer service team

Corrective maintenance services

113

Product metric

Can be classified into

1.
2. 3. 4.

HD quality metrics HD productivity and effectiveness metrics Corrective maintenance quality metrics Corrective maintenance productivity and effectiveness metrics

114

Limitations of software metrics

The main problem in software quality metrics are rooted in the attributes measured Programming style: Volume of documentation comments Software complexity Percentage of reused code Professionalism of design review and software testing teams Reporting style of the review and testing results
115

Objectives of cost quality metrics

Control budgeted expenditures (for SQA prevention and appraisal activities) Previous years failure costs Previous projects quality costs (control costs and failure costs) Other departments quality costs (control costs and failure costs).

The classic model of cost of software quality

It provides a methodology for classifying the costs associated with product quality assurance from an economic point of view. It is Developed to suit the quality situations found in manufacturing organizations, the model has since been widely implemented.

Prevention costs Costs of Control costs Cost of software quality Costs of Failure of control costs Appraisal costs

Internal failure costs

External failure costs

The model classifies costs related to product quality into two general classes: Costs of control include costs that are spent to prevent and detect software errors in order to reduce them to an accepted level. Costs of failure of control include costs of failures that occurred because of failure to prevent and detect software errors. The model further subdivides these into subclasses.

Costs of control are assigned to either the prevention or the appraisal costs subclass:

Prevention costs include investments in quality infrastructure and quality activities that are not directed to a specific project or system, being general to the organization. Appraisal costs include the costs of activities performed for a specific project or software system for the purpose of detecting software errors.

Failures of control costs are further classified into internal failure costs and external failure costs:

Internal failure costs include costs of correcting errors that have been detected by design reviews, software tests and acceptance tests (carried out by the customer) and completed before the software is installed at customer sites. External failure costs include all costs of correcting failures detected by customers or the maintenance team after the software system has been installed.

Prevention costs Costs of Control costs Cost of software quality Appraisal costs Managerial preparations and control costs Internal failure costs Costs of Failure of control costs External failure costs Managerial failure costs

* Costs of carrying out contract reviews * Costs of preparing project plans, including quality plans * Costs of periodic updating of project and quality plans * Costs of performing regular progress control of external participants contributions to projects

Managerial failure costs

* Unplanned costs for professional resources, resulting from underestimation of the resources . * Damages paid to customers as compensation for late project completion, a result of the unrealistic schedule * Damages paid to customers as compensation for late completion of the project, a result of managements failure to recruit team members.

Standards, certification and assessment: ISO 9000-1and ISO 9000-3

The benefits of use of standards

The ability to apply software development and maintenance methodologies and procedures of the highest professional level Better mutual understanding and coordination among development teams but especially between development and maintenance teams Greater cooperation between the software developer and external participants in the project Better understanding and cooperation between suppliers and customers, based on the adoption of known development and maintenance standards as part of the contract

The organizations involved in standards development

Development of SQA standards has been undertaken by several national and international standards institutes, professional and industry-oriented organizations that invest remarkable amounts of resources in these projects.

The following institutes and organizations, among the most prominent developers of SQA and software engineering standards, have gained international reputation and standing in this area: IEEE (Institute of Electrical and Electronics Engineers) Computer Society ISO (International Organization for Standardization) DOD (US Department of Defense) ANSI (American National Standards Institute) IEC (International Electrotechnical Commission) EIA (Electronic Industries Association).

Classification of SQA standards

Software quality assurance standards can be classified into two main classes: Software quality assurance management standards, including certification and assessment methodologies (quality management standards) Software project development process standards (project process standards).

Quality management standards

These focus on the organizations SQA system, infrastructure and requirements, while leaving the choice of methods and tools to the organization. ISO 9000-3 and the Capability Maturity Model (CMM) are, respectively, examples of a standard and a methodology belonging to this class.

Project process standards

These focus on the methodologies for carrying out software development and maintenance projects, that is, on how a software project is to be implemented. These standards define the steps to be taken, design documentation requirements, the contents of design documents, design reviews and review issues, software testing to be performed and testing topics, and so forth. Naturally, due to their characteristics, many SQA standards in this class can serve as software engineering standards and vice versa.

Examples of standards in use

ISO 9000-3 and IEEE/IEA 12207 are examples of comprehensive standards that cover all aspects of software quality management and the software development life cycle, respectively. Examples of specialized standards of both classes may be found in IEEE software engineering standards, such as IEEE Std 730-1998 for software quality assurance plans, IEEE Std 1012-1998 for software verification and validation, and IEEE Std 1045-1992 for software productivity metrics.

Quality management standards

Quality management standards and methodologies focus on the software quality assurance system its organization, infrastructure and requirements yet leave the choice of the methods and tools to be used in the hands of the organization. In other words these standards focus on the what of SQA and not its how. Standards belonging to this class, especially ISO 9000-3, structure the SQA certification procedures that are applied to organizations developing software.

Describe the ISO 9000-3 certification process.

To acquire ISO 9000-3 certification, organizations must: Plan the organizations activities for gaining certification Develop the organizations SQA system, including procedures Obtain approval of procedures by the certifying organization Implement the organizations SQA system Undergo certification audits of actual performance of the SQA system.

ISO 9000-3

ISO 9000-3, the Guidelines offered by the International Organization for Standardization (ISO), represent implementation of the general methodology of quality management ISO 9000 Standards to the special case of software development and maintenance. The ISO 9000-3 Standard for the software industry can be considered to provide the requirements for ISO 9000-3 certification.

ISO 9000-3 quality management system: guiding principles

Eight principles guide the new ISO 9000-3 standard; (1) Customer focus. Organizations depend on their customers and therefore should understand current and future customer needs. (2) Leadership. Leaders establish the organizations vision. They should create and maintain an internal environment in which people can become fully involved in achieving the organizations objectives via the designated route.

(3) Involvement of people. People are the essence of an organization; their full involvement, at all levels of the organization, enables their abilities to be applied for the organizations benefit. (4) Process approach. A desired result is achieved more efficiently when activities and resources are managed as a process.

(5) System approach to management. Identifying, understanding and managing processes, if viewed as a system, contributes to the organizations effectiveness and efficiency. (6) Continual improvement. overall performance should be high on the organizations agenda.

(7) Factual approach to decision making. Effective decisions are based on the analysis of information. (8) Mutually supportive supplier relationships. An organization and its suppliers are interdependent; a mutually supportive relationship enhances the ability of both to create added value.

Capability Maturity Model Integration (CMMI)

In the late 1990s a new developmental direction was taken development of integrated CMM models. Development of specialized CMM models involved development of different sets of key processes for model variants for different departments that exhibited joint processes. In practice, this created a situation where departments that applied different CMM variants in the same organization faced difficulties in cooperation and coordination.

The CMMI structure and processes areas

The CMMI model, like the original CMM models, is composed of five levels. The CMMI capability levels are the same as those of the original, apart from a minor change related to capability level 4, namely: Capability maturity level 1: Initial Capability maturity level 2: Managed Capability maturity level 3: Defined Capability maturity level 4: Quantitatively managed Capability maturity level 5: Optimizing.

Bootstrap Methodology

The Bootstrap methodology measures the maturity of an organization and its projects on the basis of 31 quality attributes grouped into three classes: process, organization and technology. A five-grade scale is applied to each of the quality attributes separately. The methodology facilitates detailed assessment of the software development process by evaluating its achievements with respect to each attribute and indicates the improvements required in the software development process and in projects.

IEEE/EIA std 12207

IEEE/EIA Std 12207 provides a framework that incorporates the entire spectrum of software life cycle processes. In this capacity, it refers the reader to other IEEE standards as sources for specialized details and prescriptive requirements.

The purposes of IEEE/EIA Std 12207, as determined by the IEEE and EIA, can be summarized thus: To establish an internationally recognized model of common software life cycle processes that can be referenced by the software industry worldwide. To promote understanding among business parties by application of commonly recognized processes, activities and tasks.

IEEE Std 1012 verification and validation

The IEEE Std 1012-1998 (IEEE, 1998) deals with the processes for determining whether a software product conforms to its requirements specifications (verification) and whether it satisfies the objectives of its intended use (validation). The standard adopts a broad range of applications, as demanded by the variety of verification and validation (V&V) methods available for use throughout the software life cycle. In response to developments in the field, the current standard has been substantially expanded from the 1986 version.

Project management responsibilities for quality assurance

Most project management responsibilities are defined in procedures and work instructions; the project manager is the person in charge of making sure that all the team members comply with the said procedures and instructions.

His tasks include professional hands-on and managerial tasks, particularly the following. Professional hands-on tasks Preparation of project and quality plans and their updates Participation in joint customersupplier committee Close follow-up of project team staffing, including attending to recruitment, training and instruction.