Terms

Quality Abstract term! Measurable? - a degree or grade of excellence or worth - the totality of characteristics of an entity that bear on its ability to satisfy stated or implied needs A characteristic property of something. A requirement that does not concern functionality. An attribute that does not concern functionality. Difference from requirement? Measurable? Same as quality requirement. All of the above...

Quality Aspects of Software Systems

A quality qualities

Mikael Svahnberg Mikael.Svahnberg@bth.se

Quality requirement Quality attribute Non-functional requirement Quality aspect

Quality attribute vs. requirement

An attribute is always present! Performance (high/low) Maintainability Security etc. A requirement puts a constraint on an attribute: The system should be able to handle at least 50 concurrent connections. A quality requirement describes not what the system will do, but how it will do it.

Why in a Software Architecture course?

Functional requirements describe what a system will do. Impact on algorithms Impact on software execution Quality requirements describe how the system will do it. Sometimes impact on implementation (e.g. performance) Impact on software structure = software architecture

Attributes of Software
More than Functionality Quality Attributes How the system works, as opposed to what it does. Affects the success as much as having the right functionality Affects What you build (e.g. Architecture) How you build (e.g. Development Process) Different attributes are relevant for different products

Example Buildings
Museum Bright Easy access Resting areas Impression of space Home Cosy Pivot around living room

Quality requirements revisited

Quality requirement = non-functional requirement (NFR) Sometimes explicit: The response time in this situation should be at most 2 seconds. Sometimes hidden: Hey, this is slow. We wanted it to be as fast as our old system! (frustrated customer) Two categories (Bosch): Development Operational

Development vs. operational requirements Development

Relevant from a SE perspective. Maintainability Reusability Flexibility Portability Extendability etc.

Operational
Relevant from a usage perspective. Availability Efciency Flexibility Integrity Security Usability etc.

Example of performance requirements

AGV system (AGV = Automatic Guided Vehicle) Sensor input should be processed within 10 ms. After that, route planning should be done within 30 ms. Finally, the nagivation system should be activated within 10 ms. Total time from sensor data to navigation: 50 ms! How do you design an architecture that can handle this? Is the specication good enough? Average values, worst-case, best-case?

Example of availability requirements

The system must be available 99,999% of the time: Downtime: less than 1 second per 24 hours! Downtime: at most 5 minutes per year! How do you support this in the architecture? Redundant components Automatic upgrades without restart Graceful degradation Automatic restart in case of unrecoverable error May need hardware support as well: Redundant hardware Hot-swap But, these require software support of course...

One or More Attributes (2 examples)

Ericsson Mobile Phones Performance only Priority Rewrote the Software from Scratch for Every new Model What happens to Time to Market? Maintainability? Flexibility? Reusability? Cost? Danaher Motion Autonomous Vehicles, running 2-3 m/s with current software Faster requires better hardware or new software More expensive, no-one is willing to pay Fast enough a balance is struck between cost and performance.

Balance of Qualities
What Danaher Motion is doing is to try and reach a balance where all important quality attributes are sufciently supported. All qualities may not be maximally achieved. Prioritise quality attributes What is most important to achieve? Does not mean that other attributes can be ignored!

Relations between Quality Attributes [McCall 1994]

Relations between Quality Attributes [Chung et al. 2000]

Negative influence Positive influence

Conflicting quality attributes

Quality attributes affect each other: Positively (e.g. maintainability and exibility) Negatively (e.g. maintainability and performance) Conict matrix for attributes A, B, C and D:
A A B C D + ++ B + C ++ D

What to do about conflicts?

You have to make trade-offs! Measure strengths and weaknesses of different architectural structures. See which architectural structure is best suited for a particular quality attribute. Quantify the quality requirements! Make them measurable. Why? Prioritize the quality requirements. Choose the appropriate architectural structure!

Available Quality Attributes

L. Chung (ed), B.A. Nixon, E. Yu, J. Mylopoulos, Non-Functional Requirements in Software Engineering, Kluwer Academic Publishers, Boston MA, 2000.

Classifications
L. Bass, P. Clements, R. Kazman, Software Architecture in Practice, Addison-Wesley Publishing Co., Reading MA, 1998. B. Boehm, H. In, Identifying Quality Requirement Conicts, in IEEE Software 13(2):25-36, 1996. J.A. McCall, Quality Factors, in Encyclopedia of Software Engineering, J.L. Marciniak (Editor), John Wiley & Sons, New York NY, pp. 958-969, 1994. ISO 9126, Software Qualities, ISO/IEC FDIS 91261:2000(E). IEEE Std-830 1993 G. Kotonya, I. Sommerville, Requirements Enineering, John Wiley & Sons, New York NY, 1998.

Accessibility, accountability, accuracy, adaptability, additivity, adjustability, affordability, agility, audiability, buffer, space performance, capability, capacity, clarity, code-space performance, cohesiveness, commonality, communication cost, communication time, compataibility, completeness, component integration time, component integration cost, composability, comprehensibility, conceptuality, conciseness, confidentiality, configurability, consistency, controllability, coordination cost, coordination time, correctness, cost, coupling, customer evaluation time, customer loyalty, customizability, data-space performance, decomposability, degradation of service, dependability, development cost, development time, deversity, distributivety, domain analysis cost, domain analysis time, efficiency, elasticity, enhanceability, evolvability, execution cost, extensibility, external cosistency, fault-tolerance, feasibility, flexibility, formality, generality, guidance, hardware cost, impact analyzability, independence, informativeness, inspection cost, inspection time, integrity, internal consistency, inter-operability, intuitiveness, learnability, main-memory performance, maintainability, maintenance cost, maintenance time, maturity, mean performance, measurability, mobility, modifiability, modularity, naturalness, nomadicity, obervability, offpeak-period performance, operability, operating cost, peak-period performance, performability, performance, planning cost, planning time, plasticity, portability, precision, predictability, process management time, productivity, project stability, project tracking cost, promptness, prototyping cost, prototyping time, reconfigurability, recoverability, recovery, reengineering cost, reliability, repeatability, replacability, replicability, response time, responsiveness, retirement cost, reusability, risk analysis cost, risk analysis time, robustness, safety, scalability, secondary-storage performance, sensitivity, security, similarity, simplicity, software cost, software production time, space boundness, space performance, specificity, stability, standardizability, subjectivity, supportability, surety, survivability, susceptibility, sustainability, testability, testing time, throughput, time performance, timeliness, tolerance, tracebility, trainability, transerability, transparancy, understandability, uniform performance, uniformity, usability, user-friendliness, validity, variability, verifiability, versatility, visibility, wrappability

Factor

Management oriented view of product quality

Classifications

Criteria

Criteria

Criteria

Software oriented attributes which provide quality

Using Classifications
Starting point to look for relevant quality attributes May nd metrics for quality attributes May nd relations between quality attributes May nd supporting design solutions

Factor Criteria Metric [McCall 1994] Observable during Runtime vs Not Observable during Runtime [Bass et al. 1998] External and Internal Quality vs Quality in Use [ISO 9126] Describes top-levels of a tree of quality attributes All attributes are not always necessary All attributes are not always equally important By and large translatable between the different classications.

Metric

Metric

Metric

Quantitative measures of those attributes

Example: Architecture Evaluations in Large Team Software Engineering Project

Drawbacks of classifications
They (often) provide no rationale: Why are certain attributes not included? (E.g., security and safety in ISO 9126-1.) Why does the hierarchy look the way it does? How to compose lower-level characteristics into higherlevel ones? How to perform measurements is not always stated. This makes it hard to judge the completeness and consistency of a classication.

Kotonya & Sommerville

Divides quality requirements into three major groups: Product Usability Reliability etc. Process Delivery Implementation Standards External Economic constraints Legal constraints Interoperability

ISO 9126-Standard
Functionality Suitability, Accuracy, Interoperability, Security, Functionality Compliance Reliability Maturity, Fault Tolerance, Recoverability, Reliability Compliance Usability Understandability, Learnability, Operability, Attractiveness, Usability Compliance Efciency Time Behaviour, Resource Utilisation, Efciency Compliance Maintainability Analysability, Changeability, Stability, Testability, Maintainability Compliance Portability Adaptability, Installability, Co-Existence, Replaceability, Portability Compliance

A Problem with ISO 9126

Only discusses those attributes directly visible on a software product Does not consider aspects such as time-to-market or cost Does not discuss relations between quality attributes

Impact of Quality Attribute

(Adapted from [Bosch 2000]) Organisation Business Organisation Process Technology Product Architecture Component System Lifecycle Development Deployment Evolution Also: Example: Security Sales pitch Staff commitment Separate Testing Dept. Staff competence Inspections, Testing Programming Lang., Std. Libraries Redundancy, Data Flow Provided Funct. (e.g. Receive le) Fit end-user into view Reuse of tested code Co-existing with other software? Conguration ??

General truths (?) about how to Address Quality Attributes in Software Architecture
Performance Streamlining Few levels of indirection Shortcuts Modiability/extendability Extra levels of indirection Modularity Maintainability No shortcuts Modularity Portability Hardware abstraction Modularity Adherence to standards Usability Separation of user interface Adherence to standards

What about methods?

Bosch Estimation of quality attributes Architecture transformation Hofmeister et al. Global analysis The Architecture Trade-off Analysis Method (ATAM) SEI The Software Architecture Analysis Method (SAAM) Bass et al. 1998 Attribute-Based Architectural Styles (ABAS) SEI

Discussion
Six foils one for each top-level quality attribute in ISO 9126 Find at least one system and one concrete situation where some aspect of the quality attribute is relevant. Describe this. Consider every quality attribute on its own, regardless of how important or unimportant it is in relation to the other attributes.

ISO 9126 - Functionality

Capability to provide functions that meet stated & implied needs Suitability Capability to provide an appropriate set of functions Accuracy Capability to provide right or agreed results or effects Interoperability Capability to interact with other systems Security Capability to protect information and data Functionality Compliance Capability to adhere to standards, conventions & regulations.

ISO 9126 Reliability

Capability to maintain specic level of performance Maturity Capability to avoid failure as a result of aws Fault Tolerance Capability to maintain specic level of performance in case of software faults Recoverability Capability to re-establish specic level of performance and recover data in the case of failure Reliability Compliance Capability to adhere to standards, conventions and regulations. Up-Time Mean Time Between Failures, Mean Time to Fix Availability Classes (e.g. Class 4=99,99% availability)

ISO 9126 - Usability

Capability of the software to be understood, learned, used and attractive to the user. Understandability Capability to enable the user to understand whether the software is suitable, and how it can be used Learnability Capability to enable the user to learn its application Operability Capability to enable the user to operate and control software. Attractiveness Capability to be attractive to the user Usability Compliance Capability to adhere to standards and conventions

ISO 9126 Efficiency

Capability to provide appropriate performance, relative to the amount of resources used Time Behaviour Response times, processing times, throughput rates Resource Utilisation Capability to use appropriate amounts and types of resources Efciency Compliance Capability to adhere to standards and conventions Peak Performance? Minimum Hardware Congurations?

ISO 9126 - Maintainability

Capability of system to be modied. May include corrections, improvements or adaptation. Analysability Capability to be diagnosed for deciencies or for parts to be modied. Changeability Capability to enable a specied modication Stability Capability to avoid unexpected side effects during modication Testability Capability to enable validation Maintainability Compliance Capability to adhere to standards and conventions

ISO 9126 - Portability

Capability to be transferred from one environment to another Adaptability Capability to be adapted to different environments without applying actions or means other than those provided for this purpose Installability Capability to install the software in a specic environment Co-Existence Capability to co-exist with other independent software Replaceability Capability to be used in place of another software for the same purpose Portability Compliance Capability to adhere to standards and conventions