Chapter-

3
A GOAL-BASED FRAMEWORK FOR SOFTWARE
MEASUREMENT

1
Software Measurement:

• Software measurement is a process of quantitatively

assessing various aspects of software development,
quality, and performance.
• It helps in making informed decisions, predicting
project outcomes, and continuously improving
processes and quality.
• Common metrics include Lines of Code (LOC), defect
density, function points, response time, and
productivity rates.
 Challenges in Software Measurement

• Selecting metrics that are relevant to specific project

goals and objectives.
• Avoiding "measurement overload" by not collecting
excessive data.
• Ensuring that metrics provide actionable insights that
support project objectives.
• The need to align software measurement with business
objectives and stakeholder expectations.
Software metrics is a term that embraces many activities ,
all of which involve some degree of software measurement:

• Cost and effort estimation • Performance Evaluation and

• Productivity measures and models
models • Structural and complexity
• Data collection metrics
• Quality models and • Capability-maturity
measurement assessment
• Reliability models • Management by
metrics
• Evaluation of methods and
tools
4
Why a Goal-Based Approach?

• A goal-based approach to software measurement

helps teams focus on specific, well-defined objectives.
• It ensures that all measurements are relevant,
actionable, and aligned with the strategic goals of the
project.
• The GQM (Goal-Question-Metric) framework,
developed by Victor Basili and colleagues, addresses
these needs by structuring the measurement process
around goals.
• The framework for software measurement is based on three
principles
 Classifying the entities to be examined
 Determining relevant measurement goals
Identifying the level of maturity that the organization
has reached

6
Classifying Software Measures

• The first obligation of any software measurement activity

is identifying the entities and attributes that we want to
measure
• Software entities can be classified as follows
 Processes: Collection of software-related
activities.
Products: Artifacts, deliverables or
documents that result from a process activity.
 Resources: Entities required by a process
activity.
• Within each class of entity, we distinguish between internal
and external attributes of a product, process, or resource:
7
Classifying Software Measures (cont.)

• Internal attribute: are those that can be measured by

examining the product, process, or resource on its own,
separate from its behavior. (e.g. Program size,
complexity, dependencies).
• External attributes: are those that can be measured only
with respect to how the product, process or resource relates
to the environment. (Ease of Navigation, Number of
failures)

8
Classifying Software Measures (cont.)

Internal External
 Size, Effort, Cost  Usability
 Code Complexity  Integrity
 Functionality  Efficiency
 Modularity  Testability
 Redundancy  Reusability
 Syntactic Correctness  Portability
 Reuse  Interoperability
7
Importance Of Internal Attributes

• Many software engineering methods proposed and

developed in the last 25 years provide rules, tools, and any
heuristics for providing software products. It is claimed
that this structure makes them easier to understand, and
tests.
• It is assumed that good internal structure leads to a
good external quality.
• This connection has rarely been established.
Processes
• We often have questions about our software-development
activities and processes that measurement can help us to
answer.
• We want to know how long it takes for a process to
complete, how much it will cost, whether it is effective or
efficient, and how it compares with other processes that we
could have chosen.
• Following are some of the internal attributes that can
be measured directly for a process
 The duration of the process or one of its activities.
 The effort associated with the process or one of its activities.
 The number of incidents of a specified type arising during the
process or one of its activities.
• E.g: AT&T developers wanted to know the effectiveness
of their software inspections. In particular, managers needed
to evacuate the cost of inspections against benefits
received. To do this, they measured the average amount of
effort expended per thousand lines of code reviewed. This
information combined with measures of the number of
faults discovered during the inspections, allowed managers
to perform a cost-benefit analysis.

1
0
Products
• Products are not restricted to the items that management is
committed to deliver to the customer. Any artifact or
document produced during the software life cycle can be
measured and assessed.
• For example developers often build prototypes for examination
only, so that they can understand requirements or evaluate
possible designs; these prototypes may be measured in some
way.
• External product attributes depend on both product
behavior and environment, each attribute measure should
take these characteristics into account.
• Internal product attributes are sometimes easy to
measure.
• We can determine the size of a product by
measuring the number of pages it fills or the number
of words it contains.
• Other internal product attributes are more
difficult to measure, because opinions differ as to
what they mean and how to measured them. For
example the complexity of codes.

1
2
Resources

• The resources that we are likely to measure include any

input for software production.
• Resources include personnel, materials, tools and
methods. Resources are measured to determine their
magnitude, cost and quality.
• Cost is often measured across all types of resources, so
that managers can see how the cost of inputs affects the
cost of the outputs.
• Resource measure combines a process measure (input)
with
a product measure (output).
Determining What To Measure

• A particular measurement is useful only if it helps you

to understand the underlying process or one of its
resultant products.
• The improvement in the process or products can be
performed only when the project has clearly defined
goals for processes and products.
• A clear understanding of goals can be used to generate
suggested metrics for a given project in the context of
a process maturity framework.

17
Goal-Question-Metric

• The GQM approach to process and metrics has proven to be

a particular effective approach to selecting and implementing
the metrics.
• To use GQM, You express the overall goals of your
organization Ask relevant questions
Measure.
• So, GQM approach provides a framework involving the
following three steps
 Listing the major goals of the development or maintenance
project
 Deriving the questions from each goal that must be answered
to
determine if the goals are being met
 Decide what must be measured in order to be able to answer 18
What is the GQM Framework?
• The GQM framework is a structured, goal-oriented approach
to software measurement.
• It provides a systematic process to define goals, formulate
questions, and identify metrics that directly relate to achieving
those goals.
• GQM operates on three levels:
• Goal Level: Define what you want to achieve.
• Question Level: Develop questions to explore how well the
goals are being met.
• Metric Level: Identify measurable data that answers these
questions.
 Benefits of GQM

• Aligns software measurement with project goals.

• Helps avoid irrelevant or redundant metrics.
• Supports decision-making and process
improvement.
• Applicable at various levels, including individual
projects, teams, or organizational goals.
Components of the GQM Framework
Goal Definition
• The goal-setting stage establishes the primary purpose of the measurement effort.
• Goals should be Specific, Measurable, Achievable, Relevant, and Time-bound
(SMART).
• Structure of a GQM Goal:
• Object: The specific item being measured (e.g., code, process).
• Purpose: The reason for measuring (e.g., to improve, assess).
• Quality Focus: The attribute being measured (e.g., reliability, performance).
• Perspective: Whose viewpoint is being considered (e.g., developer, user).
• Environment: The context in which the goal applies (e.g., development,
production).
• Example Goal:
• Improve the reliability of the software application from the end-user perspective by
reducing the number of system crashes in production.
 Components of the GQM Framework
• Question Formation
• Questions are developed to break down the goal into specific areas
of inquiry.
• Questions should cover aspects of the goal that need to be
monitored or analyzed to assess progress.
• Each goal can generate multiple questions to capture different
dimensions of the intended outcome.
• Example Questions for Reliability Goal:
• How frequently does the system crash?
• How long does it take to recover from a failure?
• What are the main causes of system crashes?
• How do users perceive the system’s reliability?
 Components of the GQM Framework
• Metric Selection
• Metrics are chosen to provide quantitative or qualitative answers to the
questions.
• Each question should have corresponding metrics that are both measurable and
meaningful.
• Types of metrics can vary:
• Product Metrics: Characteristics of the software product (e.g., defect
density).
• Process Metrics: Characteristics of the development process (e.g., time to fix
defects).
• Project Metrics: Attributes of project management (e.g., schedule
adherence).
• Example Metrics for Reliability Questions:
• Mean Time Between Failures (MTBF): Average time between system failures.
• Mean Time to Repair (MTTR): Average time to fix a failure and restore the
system.
• Defect Density: Number of defects per unit of code, such as per 1,000 lines.
• Typical goals are expressed in terms of productivity, quality,
risk, customer satisfaction, etc. Goals and questions are to
be constructed in terms of their audience.
• To help generate the goals, questions, and metrics, Basili &
Rombach provided a series of templates.
Purpose: To (characterize, evaluate, predict, motivate,
etc.) the (process, product, model, metric, etc.) in order
to understand, assess, manage, engineer, learn, improve,
etc. Example: To characterize the product in order to
learn it, maintenance in order to improve it

24
Perspective − Examine the (cost, effectiveness,
correctness, defects, changes, product measures, etc.)
from the viewpoint of the developer, manager, customer,
etc. Example: Examine the defects from the viewpoint
of the customer.
Environment − The environment consists of the
following: process factors, people factors, problem
factors, methods, tools, constraints, etc. Example:
The customers of this software are those who have no
knowledge about the tools.

25
Examples Of AT&T goals, questions
and metrics

26
e.g. driving metrics from
G&Q

27
Measurement and Process
Improvement
• Normally measurement is useful for:
 Understanding the process and products
 Establishing a baseline
 Accessing and predicting the outcome
• According to the maturity level of the process given by SEI,
the type of measurement and the measurement program will be
different. It has suggested that there are five levels of
process maturity. Namely: ad hoc, repeatable, defined,
managed and optimized.
• The SEI distinguishes one level from another in terms of key
process activity going on at each level.

28
• Level 1: Ad hoc
• At this level, the inputs are ill- defined, while the outputs are
expected. The transition from input to output is undefined and
uncontrolled. For this level of process maturity, baseline
measurements are needed to provide a starting point for
measuring.
• Level 2: Repeatable
• At this level, the inputs and outputs of the process, constraints, and
resources are identifiable. A repeatable process can be described
by the following diagram.
• The input measures can be the size and volatility of the
requirements. The output may be measured in terms of system
size, the resources in terms of staff effort, and the constraints in
terms of cost and schedule.

29
•Level 3: Defined
• At this level, intermediate activities are defined, and
their inputs and outputs are known and understood. A
simple example of the defined process is described in
the following figure.
• The input to and the output from the intermediate
activities can be examined, measured, and assessed.

30
•Level 4: Managed
• At this level, the feedback from the early project
activities can be used to set priorities for the current
activities and later for the project activities. We can
measure the effectiveness of the process activities.
The measurement reflects the characteristics of the
overall process and of the interaction among and
across major activities.

31
• Level 5: Optimizing
• At this level, the measures from activities are used to
improve the process by removing and adding process
activities and changing the process structure dynamically in
response to measurement feedback. Thus, the process change
can affect the organization and the project as well as the
process. The process will act as sensors and monitors, and
we can change the process significantly in response to
warning signs.

32
To sum-up

• At a given maturity level, we can collect the

measurements for that level and all levels below it.
I. Ad hoc: Initial, Baseline
II. Repeatable: Process dependent on individual.
III. Defined: Process defined and institutionalized.
IV. Managed: Measured process.
V. Optimizing: Improvement feedback to the
process.
• Identifying the Level of Maturity:
• Process maturity suggests to measure only what is visible.
Thus, the combination of process maturity with GQM
will provide most useful measures.
 At level 1, the project is likely to have ill-defined requirements.
At
this level, the measurement of requirement characteristics is
difficult.
 At level 2, the requirements are well-defined and the additional
information such as the type of each requirement and the number
of changes to each type can be collected.
 At level 3, intermediate activities are defined with entry and
exit criteria for each activity

34
• The goal and question analysis will be the same, but
the metric will vary with maturity.
• The more mature the process, the richer will be
the measurements.
• The GQM paradigm, in concert with the process maturity,
has been used as the basis for several tools that assist
managers in designing measurement programs.
• GQM helps to understand the need for measuring the
attribute, and process maturity suggests whether we are
capable of measuring it in a meaningful way.
• Together they provide a context for measurement.

35
Software Measurement Validation

• Even when you know which entity and attribute you want
to assess, there are many measures from which to choose.
• Sometimes, managers are confused by measurement which
is not surprising.
• One of the roots of this confusion is the lack of
software measurement validation.

36
• The validation approach depends on distinguishing
measurement from prediction
Measures or measurement systems are used to assess
an existing entity by numerically characterizing one or
more of its attribute.
Prediction systems are used to predict some attribute of
a future entity, involving a mathematical model with
associated prediction procedures.

37
Software Measurement Validation (con’t)

• Validating software Measures: is the process of ensuring

that the measure is a proper numerical characterization of the
claimed attribute by showing that the representation
condition is satisfied.
• The formal requirement for a validating measure involves
demonstrating that it characterizes the stated attribute in
the sense of measurement theory.

38
• Validating a prediction system in a given environment is
the process of establishing the accuracy of the prediction
system by empirical means; that is, by comparing model
performance with known data in the given environment.
• It involves experimentation and hypothesis testing.
• To validate the prediction system formally you must first
decide how stochastic it is, and then compare performance
of the prediction system with known data points.

39
Performing Software Measurement
Validation

• Software engineering community has always been aware of

the need for validation.
• Thus, a measure must be viewed in the context in which
it will be used.
• Validation must take into account the measurement's
purpose; measure X may be valid for some uses but not for
others.

40
Advantages of the GQM Framework

· Goal-Oriented: Directly aligns metrics with organizational or

project goals.
· Focused: Avoids irrelevant data collection, ensuring
resources are used efficiently.
· Flexible: Can be applied at different levels and for various
software attributes.
· Improves Decision-Making: Data is specifically chosen to
support decision-making and improvement.
· Systematic and Structured: Provides a clear, step-by-step
process for measurement
Limitations of the GQM Framework

· Dependent on Clear Goals: If goals are poorly

defined, the entire measurement effort may be
flawed.
· Time and Effort: Defining goals, questions, and
metrics can be time-consuming, particularly in
complex projects.
· Skill-Intensive: Requires expertise in both software
measurement and the domain being measured.
· Subjectivity in Goal Definition: Goal-setting may be
influenced by personal or organizational bias,
affecting metric relevance.
Example Application of GQM Framework in a Software Project

• Let’s consider an e-commerce platform looking to improve its transaction

speed:
• Goal: Improve the transaction processing speed of the e-commerce
application from the user's perspective to reduce checkout time.
• Questions:
• What is the average time taken to process a transaction?
• How many transactions fail due to timeout issues?
• How satisfied are users with the transaction speed?
• Metrics:
• Average Processing Time: Average time taken per transaction.
• Failure Rate: Number of failed transactions per total transactions.
• User Satisfaction Survey: Rating of transaction speed.
Conclusion

The GQM framework is a powerful tool for teams and

organizations that aim to improve processes
systematically, make data-driven decisions, and
achieve measurable improvements in software
quality and performance.