Process Assessment Issues of the ISO/IEC 29110

emerging standard
Vincent Ribaud Philippe Saliou
LISyC, Université de Brest, UEB, France LISyC, Université de Brest, UEB, France
20 avenue Le Gorgeu, C.S. 93837 20 avenue Le Gorgeu, C.S. 93837
29238 Brest Cedex 3 29238 Brest Cedex 3
+33298016971 +33298016974
ribaud@univ-brest.fr Philippe.Saliou@univ-brest.fr

ABSTRACT appropriate standards elements, referred to as Profiles [2]. For the

The emerging ISO/IEC 29110 standard “Software Engineering - VSEs developing generic software, four profiles are being
Lifecycle Profiles for Very Small Entities (VSE)” is an ISO developed: Entry, Basic, Intermediate and Advanced (ISO/IEC
initiative to provide Very Small Entities (VSE) with a suitable set FDIS 29110-4-1). The Basic Profile has been submitted to ISO
of profiles for Process Assessment and Process Improvement. The for final balloting (ISO/IEC DTR 29110-5-1-2) and is used as a
approach is conforming to ISO 15504 2-D model of process basis for the work reported in this paper. The Process Assessment
capability: a process dimension based on a Process Reference Model (PAM) of ISO/IEC 29110 standard is a two-dimensional
Model (PRM), and a capability dimension with a set of process model of process capability. In the process dimension, the
attributes grouped into capability levels. The ISO/IEC 29110 processes are defined and classified into process categories. The
standard is developing 4 profiles for VSEs developing generic capability dimension defines a set of process attributes grouped
software: Entry, Basic, Intermediate and Advanced. This paper into capability levels. Process attributes are the measurable
establishes a reduced set of Base Practices profiled from ISO characteristics of process capability. Process capability indicators
15504-5 “An exemplar Process Assessment Model (PAM)”. It are the means of achieving the capabilities addressed by the
applies recommendations of ISO/IEC 29110 DTR 29110-3 about considered process attributes. Evidence of process capability
assessment and questions the use of a separated capability indicators supports the judgment of the degree of achievement of
dimension and its usability for a VSE. the process attribute [3, Part 5, p. 79].
The process dimension of the ISO/IEC standard 15504 [3]
Categories and Subject Descriptors provides a complete view of the prescribed work to be done in a
D.2.O [Software Engineering]: General - Standards. software project. Thus, profiling this process dimension through
the ISO/IEC 29110 recommendations yields a useful set of Base
Practices (an activity that, when consistently performed,
General Terms contributes to achieving a specific process purpose [3]) for a VSE
Documentation, Standardization. intending to implement the Basic Profile. This reduced set is
presented in section 3 and may be considered as an ISO/IEC
Keywords 29110 Process Reference Model (PRM). Section 4 addresses the
ISO/IEC 29110, software engineering processes, process problem of a VSE that wishes to carry an ISO/IEC 29110
assessment model. assessment against the PRM of section 3. We make the proposal
to integrate Level 2 indicators within the PRM to provide the VSE
with a unified view of reference and assessment models.
1. INTRODUCTION
Very Small Entities (VSE) produce software components
stand-alone or integrated in large software systems. A VSE is an 2. REQUIREMENT and RELATED WORK
entity (enterprise, organization, department or project) having up 2.1 SE Standards for Very Small Entities
to 25 people [1]. Most of VSEs cannot afford the resources, nor
do they see a net benefit in establishing standardized software life 2.1.1 ISO/IEC 29110 initiative
cycle processes [1]. To rectify some difficulties, the ISO/IEC ISO started in 2005 a Working Group mandated to develop a
Working Group 24 is developing a set of documents under the set of standards and technical reports suitable to VSEs audience.
emerging ISO/IEC 29110 standard “Lifecycle profiles for Very The profiles are based on subsets of appropriate standards
Small Entities” [1]. The documents are based on subsets of elements, relevant to the VSE, for example, processes and
outcomes of ISO/IEC 12207 [4] and products of ISO/IEC 15289
[5]. ISO/IEC DTR 29110-5-1-2 provides Management and
Permission to make digital or hard copies of all or part of this work for Engineering Guide to the ISO/IEC FDIS 29110-4-1 processes.
personal or classroom use is granted without fee provided that copies are
not made or distributed for profit or commercial advantage and that 2.1.2 Process Assessment
copies bear this notice and the full citation on the first page. To copy ISO/IEC DTR 29110-3 is an Assessment Guide applicable
otherwise, or republish, to post on servers or to redistribute to lists,
requires prior specific permission and/or a fee.
to VSE profiles. It is compatible with ISO/IEC 15504-2 and
Conference’04, Month 1-2, 2004, City, State, Country. 15504-3. The assessment has two purposes: to evaluate the
Copyright 2004 ACM 1-58113-000-0/00/0004…$5.00. process capability based on a two-dimensional assessment model
(issued from the 15504 standard); to determine whether an competencies. For instance, it defines the SI.O2 objective
organization achieves the targeted VSE Profile based on the “Software requirements are defined, analyzed for correctness and
evaluated capabilities for the processes. A VSE Process testability, approved by the Customer, baselined and
Assessment Model (PAM) can be derived by selecting only the communicated. Changes to them are evaluated for cost, schedule
assessment indicators in the 15504-5 Exemplar PAM, relevant to and technical impact previously to be processed [1, Part 4, p. 8]”.
corresponding process outcomes defined in ISO/IEC 29110-4-1. ISO/IEC DTR 29110-5-2 details, for each activity of the PM and
SI processes, the tasks to be performed: role, description of the
2.2 Software Process Assessment for VSEs task, inputs and outputs products. For instance, it defines SI.2.1 to
A lot of research has been performed on Software Process SI.2.7 tasks and their associated output products: Requirements
Assessment (SPA) for small companies based either on 15504 [6, Specification, Verification Results, Change Request, Validation
7, 8] or CMMI [9, 10]. Almost all approaches aims to minimize Results, and Software User Documentation.
the assessment time and are reducing the number of assessed Table 1. Basic Profile Process Breakdown
processes. As an example, the ADEPT method [10] is selecting
12207:2008 related 15504:2006 related
six of the seven processes areas associated with CMMI maturity Activities processes or processes or Base
Level 2, and six of the 14 applicable to Level 3. Several activities Practices
approaches are using process-area interviews (or questionnaires) PM.1 Project 6.3.1 Project MAN.3 Project
as the central stage to collect evidences of process achievement. Planning Planning Management
Grünbacher reports on the experiences of an assessment and PM.2 Project Plan
improvement process in a small software company [9]. The - -
Execution
process is based on an initial self-evaluation with the SynQuest
PM.3 Project 6.3.2 Project
self-assessment tool and following structured interviews in the
Assessment and Assessment and MAN.3
company. The process areas covered in the questionnaire are Control Control
mainly related to level two and three of the CMM. We agree with
the necessity of self-assessment in a VSE but we are going further MAN.3. BP15:
PM.4 Project 6.3.2.3.4 Project
Perform project
and try to avoid (almost completely) interviews and coaching. Closure Closure
close-out review
A Brazilian approach, MARES [6], is a set of guidelines for
conducting 15504-conformant process assessments, focused on
SI.1 SW MAN.3. BP11:
small companies. MARES first component is a set of process Implementation
6.3.1.3.3 Project
Implement the project
profile patterns, identifying high-priority processes and their Activation
Initiation plan
respective capability-level goal to contribute to the organization’s
SI.2 SW 6.4.1 Stakeholder
business goals. MARES second component is a set of heuristics ENG.1 Requirements
Requirements Requirements
for adapting the patterns to a specific organization. von Definition,7.1.2 SW
elicitation,
Wangenheim and al. report that MARES results indicate the Analysis ENG.4 Software
Requirements
15504 standard’s applicability to small Brazilian companies [6]. requirements analysis
Analysis
ISO/IEC 29110 standard uses also profiles but we reached
7.1.3 SW
difficulties to implement a 2-D assessment model within VSEs. SI.3 SW
Architectural Design ENG.5 Software
Architectural and
and 7.1.4 SW design
3. Process Dimension Detailed Design
Detailed Design
3.1 Reference Models SI.4 SW 7.1.5 SW ENG.6 Software
3.1.1 Basic Profile Processes Construction Construction construction
The ISO/IEC FDIS 29110-4-1 [1] is an International Standard 7.1.6 SW ENG.7 Software
Profile (IS) identified as Basic Profile. This IS applies more SI.5 SW Integration Integration and7.1.7 integration and
specifically to a VSE that is involved in software development of and Tests SW Qualification ENG.8 Software
a single application by a single project team with no special risk Testing testing
or situational factors. 6.4.7 SW ENG.11 Software
SI.6 Product
The Basic Profile is made of 2 processes: Project Management Installation, installation,
Delivery
(PM) and Software Implementation (SI). Processes are described 6.4.8 SW SPL.3 Product
with: name; purpose; objectives; input, output, and internal Acceptance Support acceptance support
products; roles involved, activities list and activities description.
Each process objective includes a list of the chosen processes for 3.1.2 15504
the basic profile from ISO/IEC 12207 and its outcomes related to ISO 15504 [3] separates process and capability levels in two
the objective. We established the coverage of the Basic Profile for dimensions. In the process dimension, individual processes are
the 12207 and 15504 set of processes. This coverage is given in described in terms of Process Title, Process Purpose, and Process
Table 1. Some ISO/IEC 29110 activities shall be mapped to Outcomes as defined in ISO/IEC 12207 (where each life cycle
12207 Activities or 15504 Base Practices (BPs) rather than 12207 process is also divided into a set of activities; each activity is
processes (in this case, the activity name is in italics). ISO/IEC further divided into a set of tasks [4]). This decomposition is a
FDIS 29110-4-1 provides a set of cohesive tasks of each activity. Process Reference Model (PRM) aligned on the 12207 PRM. For
For instance, the starting point of the ISO/IEC 29110 use for instance, the 7.1.2 Software Requirements Analysis Process [4]
requirement is the SI.2 “Software Requirements Analysis” defines 8 outcomes, 1 activity and 3 tasks. In addition, the 15504
activity. This part establishes also VSE needs and suggested process dimension provides: a) a set of Base Practices (BP) for the
process providing a definition of the tasks and activities needed to not achieving the process purpose (or not implement the process)
accomplish the process purpose and fulfil the process outcomes; to meeting current and projected business goals [3]. Within this
b) a number of input and output Work Products (WP) related to measurement framework, the measure of capability is based upon
one or more of its outcomes; and c) characteristics associated with a set of process attributes (PA). Each attribute defines a particular
each work product [3]. As an example, ENG.4 Software aspect of process capability. Clause 6 of the 15504-5 presents the
requirements analysis Process defines 6 BPs, 4 input WPs and 6 process capability indicators related to the process attributes
output WPs [3]. The capability dimension consists of six associated with capability levels 1 to 5. Process capability
capability levels (Level 0 reflects an incomplete process) and the indicators are the means of achieving the capabilities addressed by
process capability indicators for nine Process Attributes (PA) for the considered process attributes. Most literature about Process
levels 1 to 5. Figure 1 represents the two dimensions and a Assessment for VSEs agrees that level 3 is the maximum that can
performance of process assessment. be reached by a VSE.
Table 2. Reduced 15504 Processes and Base Practices
15504
Profiled 15504 Profiled Base Practices (Total
related Objective
outcomes number of BP)
process
Supply Process Group (SUP)
PM.02
SPL.3 1) 2) BP1, BP2 (4)
SI.06
Engineering Process Group (ENG)
ENG.2 SI.02 1) BP2 (6)
SI.02
ENG.4 1) 2) 6) 7) 8) BP1, BP3, BP5, BP6 (6)
PM.03
ENG.5 SI.03 All outcomes All BPs : BP1 to BP5 (5)
SI.03
ENG.6 1) 2) 3) 4) All BPs : BP1 to BP4 (4)
SI.04
ENG.7 SI.05 3) 4) 5) 6) BP3, BP4, BP5, BP6 (6)
Figure 1. Reference Models. Performing a process assessment
ENG.8 SI.05 1) 2) 3) BP1, BP2, BP3 (4)
yields a rating for each process attribute. A rating is a judgment of
the degree of achievement (None, Partially, Largely, Fully) of the Management Process Group (MAN)
process attribute for the instance of the assessed process. PM.01 BP1, BP2, BP4, BP5, BP6,
MAN.3 PM.02 1) 3) 4) 5) 6) 7) BP7, BP8, BP9, BP10, BP11,
3.2 ISO/IEC 29110 Profile to 15504 SI.01 BP12, BP13, BP14, BP15 (15)
As specified in ISO/IEC DTR 29110-3, “a VSE specific Process MAN.5 PM.05 3) BP3 (7)
Assessment Model (PAM) can be derived by selecting only the
PM.01
assessment indicators in the 15504-5 Exemplar PAM, relevant to MAN.6
PM.02
2) 5) 6) BP3, BP5, BP6, BP7, BP8 (9)
the corresponding process outcomes defined in ISO/IEC 29110-4-
Support Process Group
1.” In ISO/IEC DTR 29110-5-1-2, each objective is associated
with a list of the processes for the basic profile from ISO/IEC SUP.1 PM.07 All outcomes All BPs : BP1 to BP5 (5)
12207 and its outcomes related to the objective. For instance, the
SUP.2 SI.07 All outcomes All BPs : BP1 to BP5 (5)
SI.02 objective profiled from the 7.1.2 Software Requirements
Analysis Process outcomes (15504 ENG.4) is reduced to 4 SUP.3 SI.07 1) 2) 3) 4) 6) BP1, BP2, BP3, BP4, BP6 (6)
selected outcomes. Applying the profiled objectives to the Base
Practices of ENG.4, we can suppress Base Practices that do not SUP.4 PM.04 1) 3) 4) 5) BP1, BP2, BP4, BP5, BP6 (6)
contribute to the selected outcomes (1, 2, 6, and 8). SUP.7 SI.06 1) 3) 5) BP1, BP4, BP5, BP6, BP7 (8)
Each PM objectives can be related in a quite simple manner to
one, two or three processes of the 15504 (mostly in the MAN SUP.8 PM.06 1) 2) 3) 4) 6) BP1, BP2, BP3, BP4, BP6 (6)
process group). Each SI objectives can also be related to one or SUP.9 PM.02 2) 5) BP2, BP3, BP9, BP10 (10)
two processes of the 15504 (mostly in the ENG process group).
Hence, profiling 15504 Base Practices for the ISO/IEC 29110 PM 4.2 Application to a VSE
and SI processes is straightforward. Grouping all profiled BP of We compiled the Process Attributes (PA) for level 1 and 2,
concerned 15504 processes gives the reduction ratio of this generic practices per PA (an activity that, when consistently
profiling operation. Grouping is presented in Table 2. performed, contributes to the achievement of a specific process
4. Capability dimension attribute [3]) and tried to figure out the implementation for each
process (see examples in Table 3).
4.1 15504 The continuous model of 15504 allows each process to be
Clause 5 of ISO/IEC 15504-2 defines a measurement framework measured independently. But we are faced with different issues
for the assessment of process capability defined on a six point regarding PM and SI processes. PM is essentially a simplification
ordinal scale: 0-Incomplete, 1-Performed, 2-Managed, 3- of MAN.3 Project Management; hence generic practices are
Established, 4-Predictable, 5-Optimizing. The scale represents tailored once for the whole PM process. But SI is a collection of 6
increasing capability of the implemented process, from Level 0: low-level processes (from Initiation to Delivery) and each generic
 practice has to be declined in 6 exemplars. It seems us affordable implement the defined activities and how to manage the execution
for the PA 2.2 Work product management attribute, but much of the activities of each Profile Processes. PA 2.2 will require
harder with the PA 2.1 Performance management attribute. highlighting how to identify work products requirements, how to
Performance management is a huge issue and it does not seem identify the activities associated with work products, how to
possible to imagine that a VSE can afford the work required to manage the configuration of work products, and how to manage
adapt PA 2.1 Performance management attribute to its different the quality of work products.
processes. Table 3 shows some generic practices related to PA 2.2 The challenge of this two-dimensional merging is to preserve
and PA 2.1. usefulness of Process Attributes while keeping processes
definition as simple as possible. However a big step will be
Table 3. Process Attributes for Levels 1 to 3 accomplished by a VSE if PM, SI and its 6 sub-processes are
Process Pro- implemented along this unified view because the VSE will
Generic practices Comment
Attribute cess probably reach a capability level 2.
Level 2 “Managed”
GP 2.1.1 Identify the objectives for the 5. CONCLUSION
performance of the process Two main ideas are presented in this paper (1) we established
PA 2.1 It may be
1. Identify the objectives for the a set of Base Practices conforming to the ISO/IEC 29110 Basic
Perfor- considered for
PM performance of the process.
mance PM, but out of Profile and reduced from ISO 15504-5 “An exemplar Process
SI 2. The scope of the process performance
mana- reach for all Assessment Model”; (2) we set the question of the suitability of
is defined.
gement sub-processes the 15504 bi-dimensional model for Process Assessment in a VSE
3. Assumptions and constraints are
attribute of SI. and we propose to develop a unified view of processes
considered when identifying the
performance objectives. incorporating Level 2 PAs. A VSE implementing Base Practices
...
of Basic Profile may be delighted to learn - as Monsieur Jourdain
GP 2.2.2 Define the requirements for in Moliere’s comedy The Middle-Class Gentleman - that the VSE
documentation and control of the work
has been “speaking” a capability level 1 without knowing it. Our
PA 2.2 products 1. 2. and 3.
Work 1. Requirements for the documentation shall be objective is to contribute to the Intermediary Profile in order to let
product PM and control of the work products are defined once, VSEs “speaking” a capability level 2 without knowing it.
mana- SI defined but are
gement 2. Dependencies between work products submitted to 6. REFERENCES
attribute are identified and understood change. [1] ISO/IEC 29110:2010. Software Engineering - Lifecycle
3. Requirements for the approval of work Profiles for Very Small Entities (VSE). ISO, Geneva.
products to be controlled are defined
... [2] Laporte, C.Y., Alexandre, S., and O’Connor, R.V. 2008. A
Software Engineering Lifecycle Standard for Very Small
4.3 Merging the two dimensions Enterprises. In Proceedings of the 15th EuroSPI Conference.
Although the MARES approach refers to the 15504, they do not CCIS Vol. 16, Springer-Verlag, Berlin Heidelberg.
use the 2-dimension model of the 15504 and concentrates on the (September 1-5, 2008), 129-141.
process dimension. The RAPID approach [9] defines a complete
set of 210 questions / indicators for each process across capability [3] ISO/IEC 15504:2004. Information technology -- Process
levels 1, 2 and 3. However most approaches such in [9], [10] are assessment. ISO, Geneva: ISO.
relating the use of the CMMI rather than the 15504 standard. [4] ISO/IEC 12207:2008. Information technology -- Software
Both models have different architecture and focus. 15504 life cycle processes. ISO, Geneva: ISO.
separates processes and capability levels in two dimensions while [5] ISO/IEC 15289:2006. Systems and software engineering --
CMMI handles them in one dimension. CMM focuses on an Content of systems and software life cycle process
organization’s capability whereas the 15504 standard focuses on a information products. ISO, Geneva.
single process capability. The latter seems more suitable to a VSE [6] von Wangenheim, C. G., Anacleto, A., and Salviano, C. F.
because improvement can be conducted process per process, 2006. Helping Small Companies Assess Software Processes.
rather than a global step as the staged CMMI. IEEE Software 23, 1 (Jan. 2006), 91-98.
However, it should be pointed out that separate process and [7] Cater-Steel, A. P. 2001. Process Improvement in Four Small
capability dimensions may discourage a VSE regarding process Software Companies. In Proceedings of the 13th Australian
assessment. Concerning capability level 1, Process Attribute (PA) Conference on Software Engineering (August 27 - 28, 2001).
indicators relate to the process dimension regarding the extent of IEEE Computer Society, Washington, DC, 262.
the process purpose and outcome achievement. As these issues
form a part of process definition in the ISO/IEC 29110 standard, a [8] Rout, T. P., Tuffley, A., Cahill, B., and Hodgen B. 2000. The
VSE implementing correctly the Basic Profile for instance, will Rapid Assessment of Software Process Capability. In
achieve fully or largely Level 1 PAs. Capability Level 2 indicators Proceedings of SPICE 2000. 47-55.
relate to define, plan, monitor and adjust performance of each [9] Grunbacher, P. 1997. A software assessment process for
process and to identify, define, document, review and adjust each small software enterprises. In Proceedings of the 23rd
work products. We make the proposal to incorporate Level 2 PAs EUROMICRO Conference (September 1-4, 1997), 123-128.
inside Intermediate or Advanced Profile Processes definitions. PA
[10] Mc Caffery, F., Taylor, P.S., and Coleman, G. 2007. Adept:
2.1 will require highlighting how to identify resource
A Unified Assessment Method for Small Software
requirements, how to plan the performance of the process, how to
Companies. IEEE Software 24, 1 (Jan-Feb 2007), 24-31.