2017 5th Intl Conf on Applied Computing and Information Technology/4th Intl Conf on Computational Science/Intelligence

and Applied Informatics/2nd Intl Conf on Big Data, Cloud Computing, Data Science & Engineering

Automated Risk Identification of CMMI Project

Planning Using Ontology

Chanapan Rojrattanakorn and Wiwat Vatanawood

Software Engineering Program
Department of Computer Engineering, Faculty of Engineering
Chulalongkorn University, Bangkok, Thailand
chanapan.r@student.chula.ac.th, wiwat@chula.ac.th

Abstract—Risk management is an important part of software development process. CMMI is one of most effective
development. Risk identification is the first major step in the risk guidelines that provides a collection of best practices for
management process that determines possible risks in a software process improvement [4]. Software development projects
project and dictates how effective the risk management process should complete activities following CMMI recommendations
will be. However, there are several approaches used to identify to develop quality products and processes. In other words, we
risk. The approaches depend on subjective decisions of risk or can use the CMMI model as a criteria to identify risk. If
domain experts which increase cost and workload. Therefore, to recommended CMMI activities cannot be performed, it may
make the risk identification more effective, a systematic and lead to risk in a software project. For example, if the estimation
automated approach is necessary. In this paper, we propose an
for effort and cost is not performed, the project may go over
alternative risk identification scheme using knowledgeable
budget or fail to meet deadlines resulting in an incomplete
ontology representing risk taxonomy and CMMI project
planning guideline. The ontology provides the relations among project.
risk factors and software project risks according to the structure Previous research works have introduced ontology for
of risk taxonomy and the relevant process areas in CMMI. In this supporting CMMI assessment based on CMMI version 1.1 [5].
paper, we focus and demonstrate the CMMI project planning However, the CMMI model has upgraded versions and
process area as an initiative to construct our ontology. The concepts in some process areas have changed so that the
resulting ontology is supportive as expected and scalable to ontology is no longer supported. Some researches proposed a
handle more on the other process areas of CMMI and the
framework for automated risk assessment using risk taxonomy
addition of the software project risks.
and CMMI based [6]. They introduced the rules and relations
Keywords—Ontology; Knowledge based; Risk Identification; between factors and best practices from CMMI that we apply
Risk taxonomy; CMMI; to our ontology. Nevertheless, this framework only supports
the basics of a work product and still requires consultants or
experts to manually review project documents to find
I. INTRODUCTION equivalent work product according to CMMI recommendation.
Risk is defined as a probability of error, damage or Thus, PAO (Project Asset Ontology) was developed to perform
unexpected situation that causes a negative impact or hinders document mappings for the gap analysis process but only
project completion [1]. Thus, risk management is an important support CMMI version 1.2 [7].
activity for software development project. The current risk
management process includes five sequential steps which are In this paper, we propose an automated risk identification
risk identification, risk analysis, risk planning, risk tracking scheme using ontology to minimize workload and subjective
and risk control [2]. Risk identification determines the possible judgement. Risk taxonomy is used as a guideline to categorize
risks in a software project, providing the key elements of the risks in combination with best practices from the CMMI model
risk management process that determine their effectiveness. as a measurement of work that should be completed to prevent
Popular risk identification techniques are decision driver identified risk.
analysis, checklist and assumption analysis [3]. However, these The remainder of this paper is structured as follows.
techniques depend on the subjective decisions of risk or Section II discusses the topic background. Section III describes
domain experts. Furthermore, the risk management process the risk identification ontology. The case study is presented in
requires additional work and increases cost. section IV and section V is the conclusion respectively.
Many software projects nowadays emphasize improving
software quality to gain trust from their customers. To improve
quality of software product, a good development process is
required. Several methodologies, standards or guidelines have
been presented to help organizations improve their

978-1-5386-3302-1/17 $31.00 © 2017 IEEE 19

DOI 10.1109/ACIT-CSII-BCD.2017.26

Authorized licensed use limited to: UNIVERSIDADE DE PERNAMBUO. Downloaded on March 15,2023 at 17:15:56 UTC from IEEE Xplore. Restrictions apply.
 II. BACKGROUND such as “schedule”, “budget”, “staff” and “facilities” that
should be considered to prevent risks.
A. Ontology
Ontology is an explicit description of concept in a domain C. CMMI
[8]. It represents a knowledge base comprised of classes and CMMI (Capability Maturity Model Integration) [11] is a
their relationships. To create an ontology, we need to define reference model for software process improvement. CMMI for
terms and their association with classes, properties, and development provides a collection of best practices that cover
instances. Classes contain information on the scope and are activities for developing products and services. CMMI version
used to explain detail of concept, properties describe relations 1.3 consists of 22 process areas that contain practices for
between classes, and instances are members and hold the project management, process management, engineering and
inherited properties of a class. Recently, web ontology support activities used in software development project. To
language (OWL) [9] is used to represent the complex implement CMMI, several documents or artifacts must be
knowledge and relation between them. OWL, developed by created and reorganized according to CMMI recommendations.
W3C Web Ontology Working Group, is used by an application These documents become objective evidence for CMMI
to process and interpret the web contents. Fig. 1 shows an certification appraisal. However, we can use the CMMI model
example of OWL in well-formed XML format. The OWL as a criteria to measure work that should be completed to
explains classes and their properties for CMMI process areas prevent risk. We use the Project Planning process area in
and work products. Typically, an ontology is alternatively CMMI as an initial model to construct our ontology. Project
represented by an OWL file. In this paper, the ontology of the planning is a part of effective project management that includes
knowledge base in terms of OWL syntax is designed to project estimation, developing a plan, getting commitment and
represent the relations of risk factors and software project risks maintaining the plan.
according to the structure of risk taxonomy and CMMI.
III. PROPOSED RISK IDENTIFICATION ONTOLOGY
B. Risk Taxonomy
In this section, we demonstrate how to consider important
Risk taxonomy [10] is a method for systematic and
terms from knowledge areas for classes and relationships and
repeatable risk identification being developed by the SEI
construct our ontology for risk identification. Overview of the
(Software Engineering Institute). This method provides a basis
approach is shown in Fig. 2. We develop risk identification
framework for eliciting and organizing the breadth of software
ontology using Protégé software and export to Fuseki server in
development risks in both technical and non-technical aspects.
OWL format. Fuseki server [12] is a java based framework that
It provides taxonomy-based questionnaires that are used as
provides the API connection from ontology bases data to web
instrument to elicit and identify risk. Taxonomy-based risk
application.
identification categorizes risk into three major classes which
are product engineering, development environment and The approach for creating a knowledge base in Fig. 3.
program constraints. These classes are divided into elements consists of three main steps. Firstly, we analyze risk structure
that describe groups of activities for each class in software with the assumption that the quality of software development
development and each element is also characterized by its process is associated with risk. Then, we use CMMI
attributes. For example, the program constraints class consists recommendations as criteria to identify risk. Finally, we
of three elements which are “resources”, “contract” and analyze and identify the relationships between existing
“program interfaces”. The resources element includes attributes software project risk, risk taxonomy, and CMMI to define
classes, relations and risk identification rules for the ontology.
The details of these steps are explained as following:

A. Analyze Risk Structure and Risk Cause

We assume that the quality of the software development
process is associated with risk. Several software process
improvement models provided the best practices that should be

Fig. 1 Example of OWL

Fig. 2 Overview of the approach

20

Authorized licensed use limited to: UNIVERSIDADE DE PERNAMBUO. Downloaded on March 15,2023 at 17:15:56 UTC from IEEE Xplore. Restrictions apply.
 Analyze risk structure and
risk cause Implement CMMI
Work Products
Existing
Identify association of Ontology
Risk Verify Equivalent Work
risk, risk taxonomy and scheme and Product
Risk CMMI rules
Taxonomy
Design and develop risk Risk Assess Process Areas
identification ontology identification List of missing
ontology Work Products Existing Risk
Identify Risk
Risk Taxonomy

Fig. 3 The approach for creating knowledge base. Return Software project
Risk
complete for the quality software development process and to
prevent software project risks. Thus, we can use best practice
in CMMI model as an indicator for risk identification. The
approach is assessment the produced work products from Fig. 4 The overview process to identify software project risks
CMMI implementation and compares with a set of work
product from best practices recommendations. Moreover, our
approach covers finding the equivalent of work product for analyze the relation between existing risks and risk factors by
assessment. The result of the comparison is a list of lacking using taxonomy-based questionnaires (TBQ) as a guideline.
work products that are used to categorize and identify risk TBQ is designed to ensure that all risk areas in software
according to risk taxonomy. In this paper, we focus on best development are systematically addressed. An example of the
practices from the CMMI project planning process area as an questions in TBQ from “program constraints” class would be
initiative to identify risks. Fig. 4 shows the overview process to “Is the staff inexperienced, lacking domain knowledge,
identify software project risks based on CMMI and risk
lacking skills, or understaffed?” [10]. Using these questions,
taxonomy.
we can map the existing risks to risk taxonomy according to
the TBQ guidelines. The sample of the mapping between
B. Identify Association of Risk, Risk Taxonomy and CMMI
existing software project risks and risk taxonomy is shown in
1) Study Existing Software Project Risks TABLE II. This table shows the relationship between the
We study existing software project risks from [13] that software project risk and risk taxonomy. For example, the
contain lists of risks and its categories. The planning and “Staff” attribute refer to an activity for the stability and
control dimension is most frequently found in software project adequacy of the staff in terms of skill levels, experience and
risk as following TABLE I [13]. Thus, the initial ontology for skills in the required application domain that should be
risk identification focuses on the project planning process area performed to assign task with relevant skills and
in CMMI. accomplishments. Thus, the risk “People’s assignments do not
2) Identify Risk Taxonomy match an individual's strength” can map to the “Staff”
This step aims to map existing software project risks and attribute in the “Resources” element.
risk taxonomy. Risk taxonomy provides a framework to 3) Identify CMMI Best Practices
categorize and organize software development risks. We We identify the association between the CMMI and
existing software project risks. According to previous step, we
TABLE I SOFTWARE RISKS FREQUENCY BY can imply the relation between risk taxonomy and the existing
DIMENSION [13]

Dimension Total frequency TABLE II MAPPING BETWEEN EXISTING RISKS AND

RESOURCES ELEMENT FROM RISK TAXONOMY
User 14 Resources
Requirement 17 Software Project Risk  Staff Schedule Budget
Complexity 4 People's assignments do not match
X
Planning and Control their strengths
27
Insufficient/inappropriate staffing X
Team 9
Wrong time estimation X
Organizational Environment 9 Budget overruns X
Total 80

21

Authorized licensed use limited to: UNIVERSIDADE DE PERNAMBUO. Downloaded on March 15,2023 at 17:15:56 UTC from IEEE Xplore. Restrictions apply.
 TABLE III THE EXAMPLE OF WORK PRODUCT COMPONENT work product component that can be used as an indicator to
Work Work product identify risks.
Specific Practice
Product component
Activities C. Design and Develop Risk Identification Ontology
Deliverables
Skill and knowledge To design and develop our ontology, we need to consider
WBS acquisition the important terms for classes and relationships found in the
Project Risk three major knowledge areas of existing software project risks,
SP1.1: Estimate the scope Work product risk taxonomy and the CMMI model. We adopted the CMMI
of the project components
ontology from [5] with additional important classes which are
Project Plan
Schedule the work product and work product component classes. For
Task risk taxonomy, we derive the term and hierarchy from their
Task Dependency
Description
Task Duration structure to identify classes. Then, we assign the relationship to
Labor Hours those classes according to association analysis from previous
software project risks to the best practices of CMMI. For steps. For example, the “consistOf” property link between
program constraints and resources class refer to resources
example, the risks that related to resources element of risk
element that is a component of the program constraints class.
taxonomy can be related to practice for estimation of efforts
The overview of risk identification ontology is shown in Fig. 5.
and costs from CMMI.
To identify risk from CMMI best practices, we use the The overview of risk identification ontology consists of the
CMMI recommendation to define the rules for checking the classes and relationships of CMMI and risk taxonomy. For
availability of work product. The best practices in CMMI CMMI model, the ontology is provided for assessment the
provides the typical work product that should be produced to work products compare with the CMMI recommendations to
satisfy the specific practice. However, our approach is find lacking work products. In this case, we populated classes
designed to support the equivalence work products in the level and relationships for project planning process area includes
of work product component which can be used as objective specific goals, specific practices, and work products. Fig. 6
shows the example of classes and relationships for project
evidence to identify risks. TABLE III shows the example of
planning process area.
CMMI

Specific Specific
producedBy achieves
Practice Goal
sastisfied

Work
isPartOf Product
ProcessArea
subclassOf
Work
subclassOf
Product
Component subclassOf
subclassOf
subclassOf

Thing
Risk Taxonomy

Facilities
subclassOf Program
dividedInto isA
Constrains consistOf Resources isA
consistOf consistOf isA Budget

Risk isA
dividedInto Contract Program
Taxonomy Interface Staff
Product
Engineering isA isA Customer
dividedInto Type of
isA Contract isA
consistOf
consistOf consistOf Schedule
Development Associate
isA isA
Require consistOf consistOf Environment Restricti Contractor
ment ons isA
consistOf
consistOf
consistOf isA Politics
...

Engineering Depende
Design consistOf
Code & Integratio Specialties ncies isA
Unit Test n & Test isA SubContra
consistOf
...

...

ctor
...

...

Developmen Developmen Work Management Management

t Process t System Environment Process Methods Prime
Contractor
...

...

...

...

...

Vendors

Corporate
Management

Fig. 5 Overview of risk identification ontology

22

Authorized licensed use limited to: UNIVERSIDADE DE PERNAMBUO. Downloaded on March 15,2023 at 17:15:56 UTC from IEEE Xplore. Restrictions apply.
 Project
properties, and instances for class. We first populate class and
Planning instance for the CMMI project planning process area as an
initial model.
sastisfied
sastisfied sastisfied
SG 3 Obtain
SG1 Establish SG 2 Develop Commitment to the IV. CASE STUDY
Estimates a Project Plan Plan
We propose the case study to demonstrate how risk
identification ontology works. To identify risks, we use the

...
...
achieves achieves
status of generated work product which is “exists” or “not
SP 1.2 Establish
exists” as an objective evidence to compare with CMMI
SP 1.1 Estimate the
Scope of the Estimates of Work recommendations. We use the list of lacking work product to
Project Product and Task identify risk according to the association of CMMI, risk
Attributes taxonomy and existing software project risks from proposed
ontology. We assume that we are conducting risk identification
...

produceBy produceBy produceBy

for the software project that implemented specific goal one
Task Work package (SG1) of the project planning process area, which is
WBS
Description description established estimation. In this case, we exemplified the project
which lacking in the areas of established project schedule, skill
Fig. 6 The example of classes and relationships for project planning and knowledge acquisition which are the part of WBS (Work
process area element breakdown structure), and software development life cycle
from specific practice 1.4. These work products may lead to
Wrong time the software project risks.
estimate
willAffect
The evaluation steps started from setting the status of
Schedule
willAffect
Schedule omits
generated work products. Afterward, we compare the work
necessary tasks product list with CMMI recommendations to identify the
missing work product. We use SPARQL to query for software
isA Budget
project risk related to the lacking work product from our
Overruns ontology. Fig. 8. shows an example of SPARQL querying for
willAffect
software project risk related to project schedule. Finally, we
Budget
isA willAffect Budget is grater compare the resulting risk identification from the query using
Resources than estimated our ontology with the assumed the software project risks. The
result of risk identification for the case study is shown in
Insufficient/
TABLE IV. This result verifies the validity of the association
isA
inappropiate staffing between software project risk, risk taxonomy and CMMI. For
willAffect example, we identify the risk that the time estimation is
Improper definition of incorrect because the project schedule is not performed
isA Staff willAffect roles and following the specific practice 1.1 (SP1.1) in CMMI.
responsibilities
willAffect According to the relationship between classes in ontology, this
People's assignments software project risk affected the “resources” element in the
do not match their
strengths
“program constrain” class and the project schedule related to
the “resources” element as well. Therefore, the ontology can
Development imply the relation between the “resources” element and the
Facilities willAffect facilities are
inadequate

Fig. 7 Sample ontology designing for identification risk in the resources

element

Fig. 7 represents the sample designing ontology for

identification risk in the resources element. There are four
attributes in the resources element which are schedule, budget,
staff and facilities. We added software project risk that relates
to resources element with the “willAffect” property. For
example, the “willAffect” property link between “schedule”
and “Wrong time estimate” means that the wrong time
estimation will affect the project schedule.
To develop our ontology, we use Protégé [14] which is an
open source editor for building ontologies. Protégé provides a Fig. 8 The example of SPARQL querying for the software project risk
graphic interface to design ontology schema and create classes, related to project schedule

23

Authorized licensed use limited to: UNIVERSIDADE DE PERNAMBUO. Downloaded on March 15,2023 at 17:15:56 UTC from IEEE Xplore. Restrictions apply.
 TABLE IV THE RESULT OF CASE STUDY FROM RISK IDENTIFICATION ONTOLOGY
Lacking Work product Related Risk taxonomy
Software project risks
list Class Element Attribute
Project schedule Wrong schedule estimate Program constrains Resources Schedule
Skill and knowledge People's assignments do not match their Program constrains Resources Staff
acquisition strengths

Software development Software development methodology is Development Development Suitability

life cycle inadequate environments process

“wrong time estimate” risk to the work product from CMMI, [3] B. W. Boehm,"Software risk management: principles and practices,"
IEEE software, vol. 8, no. 1,pp.32-41, 1991.
which is the project schedule.
[4] MK. Kulpa and KA. Johnson, "Interpreting the CMMI (R): A Process
Improvement Approach," CRC Press, 2008.
V. CONCLUSION AND FUTURE WORK [5] GH. Soydan and M. Kokar, "An OWL ontology for representing the
Risk identification is the first major step in the risk CMMI-SW model.", Workshop on Semantic Web Enabled Software
Engineering (SWESE), 2006.
management process that determines possible risks in a
[6] M. Choetkiertikul, H. K. Dam, A. Ghose, and T. T. Sunetnanta, "A
software project and dictates how effective the risk CMMI-based automated risk assessment framework," 2014 21st Asia-
management process will be. To make the risk identification Pacific Software Engineering Conference, Vol. 2, IEEE, 2014.
more effective, the use of a systematic method is necessary. [7] S. Rungratri and S. Usanavasin, "Project assets ontology (PAO) to
We propose the risk identification ontology and demonstrate support gap analysis for organization process improvement based on
how the ontology works. Our ontology is designed as a CMMI v. 1.2," International Conference on Software Process, Springer
knowledge base to identify risk using risk taxonomy and Berlin Heidelberg, 2008.
CMMI as a guideline. We use the Project Planning process [8] TR. Gruber, "A translation approach to portable ontology
specifications," Knowledge acquisition 5.2, pp199-220, 1993.
area in CMMI as an initial model to construct our ontology.
[9] P. Patel-Schneider, P. Hayes and I. Horrocks. “OWL Web ontology
For future work, we plan to complete risk identification that Language Semantics and Abstract Syntax”, 2009, Available at
covers all 22 process areas of CMMI model. Moreover, we https://www.w3.org/TR/owl-semantics/
intend to add a list of software project risks to our ontology and [10] MJ. Carr, SL. Konda, I. Monarch, FC. UIrich and CF. Walker,
"Taxonomy-based risk identification," Software Engineering Institute,
validate our approach with several software projects. Carnegie Mellon University, 1993.
[11] Software Engineering Institute (SEI), “CMMI for Development Version
REFERENCES 1.3”, Carnegie Mellon University, 2010.
[12] The Apache Software Foundation , ”What is Jena?”, Available at
https://jena.apache.org/about_jena/about.html
[13] T. Arnuphaptrairong, "Top ten lists of software project risks: Evidence
[1] A. J. Dorofee, J. A. Walker, C. J. Alberts, R. P. Higuera, and R. L from the literature survey," Proceedings of the International
Murphy, "Continuous Risk Management Guidebook," Carnegie Mellon MultiConference of Engineers and Computer Scientists, Vol. 1, 2011.
University, Pittsburgh PA, 1996.
[14] Stanford Center for Biomedical Informatics Research , “Why Protégé” ,
[2] S. Maniasi, P. Britos, and R. García-Martínez. "A Taxonomy-Based Available: http://protege.stanford.edu
Model for Identifying Risks," JIISIC, 2006.

24

Authorized licensed use limited to: UNIVERSIDADE DE PERNAMBUO. Downloaded on March 15,2023 at 17:15:56 UTC from IEEE Xplore. Restrictions apply.