See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/224243378

Requirements Engineering Tools

Article in IEEE Software · September 2011

DOI: 10.1109/MS.2011.81 · Source: IEEE Xplore

CITATIONS READS
66 9,704

6 authors, including:

Juan Manuel Carrillo de Gea Joaquín Nicolás Ros

University of Murcia University of Murcia
78 PUBLICATIONS 1,054 CITATIONS 64 PUBLICATIONS 1,057 CITATIONS

SEE PROFILE SEE PROFILE

José Luis Fernández-Alemán Ambrosio Toval

University of Murcia University of Murcia
175 PUBLICATIONS 4,971 CITATIONS 191 PUBLICATIONS 4,431 CITATIONS

SEE PROFILE SEE PROFILE

All content following this page was uploaded by Juan Manuel Carrillo de Gea on 16 October 2014.

The user has requested enhancement of the downloaded file.

Software Technology
Requirements Engineering Tools
Juan M. Carrillo de Gea, Joaquín Nicolás, José L. Fernández Alemán, Ambrosio Toval,
Christof Ebert, and Aurora Vizcaíno

Requirements engineering is the disciplined and systematic approach to

elicit, specify, analyze, commit, validate, and manage requirements while
considering user, technical, economic, and business-oriented needs and
objectives. It spans the entire lifecycle, often involving distributed teams
and supply chains. Tools facilitate consistency and efficiency in managing
requirements. Finding out which tool is suitable for given needs isn’t easy.
Our article aims to provide a brief overview on requirements engineering
tools. I look forward to hearing from both readers and prospective authors
about this column and the technologies you want to know more about. —
Christof Ebert

REQUIREMENTS ENGINEERING (RE) tools are quickly evolving. The demand for flexibility, lean
and agile development, worldwide collaboration, and advanced software and systems ecosystems is
changing how we manage requirements. For instance, agile teams are less document-centric and more
code-oriented—they expect brief requirements directly related to code changes—so their RE tool should be
lightweight. On the other hand, distributed development teams with high-quality expectations demand
strong specifications, comprehensive requirements, and traceability throughout the lifecycle. Their RE tool
should assure service for generations of software.
RE tools are adapting to these demands with changes to their design and architecture. Traditionally, RE
tools are proprietary and well maintained by their vendors; they’re often oriented toward distinct
environments and niche markets (for example, automotive, medical, and defense), development processes
(for example, agile development, product management, and prototyping), or utilization settings (for
example, local versus global software development). Enough reason to evaluate RE tools and technologies
with different use cases.

Background
To evaluate currently relevant RE tools, we surveyed their vendors and compared the results with user
experiences from typical RE use cases. Out of 94 vendors, 37 responded, representing the global
marketplace: the US (21 tools); Canada (4); the UK (3); Germany (2); and France, Israel, Japan, New
Zealand, Russia, Spain, and Switzerland (1 each).
The tools’ worldwide licensing varies: two have from 0 to 100 active licenses; nine have from 101 to
1,000; 13 have from 1,001 to 10,000; and nine have more than 10,000. Four vendors didn’t answer this
question.
The average cost per license is typically above $1,000 (17 tools). Five tools are in the $501 to $1,000
range; four are in the $100 to $500 range, and four cost less than $100. Seven vendors didn’t answer this
question.
Most tools require Windows (28 tools), although Web-based clients are quite common to facilitate
distributed and collaborative access to resources (9 tools). Other OSs, such as Linux (8), Unix (6), and the
Mac OS (4), have a more limited presence.
Vendors typically charge for the tools and keep the licenses proprietary (34 tools), but a few offer other
licensing, such as proprietary and free, open source and free, or open source and nonfree (1 each).
Tool Evaluation
We based the evaluation on typical use cases and on ISO/IEC TR 24766:2009, a framework for evaluating
RE tool capabilities. The document is a type 2 technical report (that is, there’s a future but not immediate
possibility of an agreement on an international standard). It supplements the more general ISO/IEC
14102:2008 standard, which focuses on evaluating computer-aided software engineering tools. According
to ISO/IEC TR 24766, a total of 157 RE tool capabilities fall into six major categories (see Table 1).

Table 1. The number of requirements engineering (RE) tool features.

Category of tool capability No. of features

Requirements elicitation 37
Requirements analysis 36
Requirements specification 16
Requirements verification and validation 34
Requirements management 17
Other capabilities 17
Total 157

We refined the questionnaire to comprise 126 questions organized in eight feature categories—six from
ISO/IEC TR 24766 plus two more: modeling and traceability. We also added questions to represent a
meaningful set of features (for example, traceability across a tool’s boundaries, specific reporting facilities,
and concrete modeling and specification languages). Furthermore, we added 20 questions regarding the
tools’ basic administrative information (for example, tool name, vendor name, and current version). So, the
final questionnaire had 146 questions.
Because the vendors are distributed worldwide, we designed a Web-based survey using LimeSurvey
(www.limesurvey.org), an open source survey application. Survey results are accessible online (see Table 2
for links to our survey and others on RE tools).

Table 2. Databases with RE-tool surveys.

Database source URL No. of

tools

Our tools survey (evolving) www.um.es/giisw/EN/re-tools-survey 37

Ian Alexander http://easyweb.easynet.co.uk/~iany/other/vendors.htm 67
Alarcos Research Group http://sites.google.com/site/toolsgsd/tools-1/software-requirement- 7
tools
International Council on www.incose.org/ProductsPubs/products/rmsurvey.aspx 34
Systems Engineering
Ludwig Consulting Services www.jiludwig.com/Requirements_Management_Tools.html 40
QAGuild http://qaguild.com/Toolsdirectory/RequirementManagementTools.htm 7
Volere www.volere.co.uk/tools.htm 71
@WEBO www.atwebo.com/case.htm 41
 Of the study’s 37 participants, some important vendors (for example, IBM and IRqA) didn’t address
some sections of the questionnaire. Some also showed low commitment. So, we discarded incoherent or
incomplete answers and added new questions in favor of critical and lacking aspects of ISO/IEC TR 24766
to produce high-quality data. The responses could contain bias because representatives tend to rank their
own tools positively. To counteract this effect, the questionnaire’s preface carefully explained the research
purpose and protocol. To draw valid conclusions, we used triangulation: three researchers performed and
validated the statistical work, and three other researchers revised the findings.

Ranking Summary
Table 3 shows the participants and the tools’ score in each category. The score in the Global column
represents the level of accomplishment on the basis of the entire ISO/IEC TR 24766 (that is, when
considering the distinct categories of features all together). Because some tools didn’t participate in all
categories, we performed this calculation on only those that did.

Table 3. Participants’ scores and prices.*

Tool Vendor Eli An Sp M Ve Ma Tr Othe Gl Pric

cit aly eci od rifi na ac r ob e
ati sis fic eli ca ge ea tool al rang
on ati ng tio m bil capa e
on n en ity biliti (sin
& t es gle
val seat)
id
ati
on

Acclaro DFSS Axiomatic Design + ++ + n/a ++ n/a + n/a n/a $$$$

Solutions
Aligned Elements Aligned + ++ + ++ ++ 0 + - + $$$$
Avenqo PEP Avenqo, Germany ++ ++ + -- + 0 ++ 0 + $$$
Blueprint Blueprint Software + ++ + + n/a + ++ + n/a $$$$
Systems
Bright Green Projects Bright Green ++ ++ + ++ 0 + + + + n/a; ○
Caliber RM Micro Focus ++ + + - n/a ++ + + n/a n/a
Cameo Requirements+ No Magic ++ + 0 0 + - 0 0 0 $$$
CASE Spec Goda Software n/a n/a n/a n/a ++ 0 ++ ++ n/a n/a
Cognition Cockpit Cognition ++ ++ ++ ++ ++ ++ ++ ++ ++ $$$$
Cradle 3SL ++ ++ ++ ++ ++ ++ ++ ++ ++ $$$$
GMARC Computer System ++ ++ ++ + ++ + ++ + ++ $$$$
Architects
inteGREAT eDev technologies ++ + ++ ++ ++ + ++ ++ ++ $$$$
IRqA Visure Solutions n/a n/a n/a n/a ++ ++ ++ + n/a n/a
jUCMNav jUCMNav - - - + n/a n/a n/a n/a n/a ●
Leap SE Leap Systems - - - - - 0 0 0 - $$
MacA&D / WinA&D Excel Software + n/a + + n/a - + n/a n/a $$$$
MKS Integrity MKS ++ ++ ++ 0 ++ ++ + ++ ++ $$$$
PACE ViewSet ++ ++ ++ ++ ++ ++ + ++ ++ $$$$
 Polarion Requirements Polarion Software ++ ++ + + ++ ++ ++ ++ ++ $$$
Psoda Psoda ++ ++ + + ++ ++ + ++ + $
QFDcapture International - 0 0 -- - -- - - - $$$$
TechneGroup
QPack Orcanos ++ + ++ n/a ++ ++ ++ + n/a $$$
RaQuest SparxSystems Japan + 0 + + - + + 0 + $$
Rational Doors IBM Rational + + ++ + n/a n/a n/a n/a n/a $$$$
ReqMan RequirementOne ++ ++ + + ++ + + + + ●
Reqtify & Requirement Dassault Systemes ++ ++ + ++ ++ ++ ++ ++ ++ $$$$
Central
Rational Requirements IBM Rational + + + 0 n/a 0 + + n/a $$$$
Composer
RTIME QAvantage ++ + + 0 ++ + + + + $$
Rational RequisitePro IBM Rational n/a + + - n/a 0 + n/a n/a $$$$
RMTrak Prometeo Technologies 0 - - -- + - 0 - - $$
Rommana Rommana Software + + 0 0 n/a 0 n/a n/a n/a $$$
SpiraTeam Inflectra ++ 0 0 -- + 0 0 ++ 0 $
TestTrack RM Seapine Software + ++ + - + + 0 + + n/a
TopTeam Analyst TechnoSolutions + 0 + + 0 ++ ++ + + $$$$
TraceCloud TraceCloud + + ++ + ++ ++ + ++ + $
TrackStudio TrackStudio + ++ - - + 0 0 0 0 $; ○
VisibleThread On- VisibleThread ++ 0 ++ + n/a n/a n/a n/a n/a $$$$
premise/On-demand

* For the scores, ++ = very high, + = high, 0 = medium, - = low, and -- = very low. For prices, $$$$ > $1,000, $$$ = $501$1,000, $$
= $100$500, and $ < $100, free = ●, free version available with limitations = ○. n/a = not applicable.

The RE tools’ scores in concrete use scenarios can help determining their suitability for certain tasks.
We assessed the tools’ performance in three scenarios. Next, we summarize the results and highlight
helpful features along with the percentage of tools supporting them. These features represent the less
accomplished capabilities in our scenarios—that is, capabilities that have a smaller percentage of tools
supporting them.

Scenario 1
This scenario addressed development projects in which determining user needs is critical—for instance,
those involving various types of stakeholders or geographically distributed stakeholders, or those with
insufficient time for requirements workshops. In such situations, ensuring that the final product meets
specifications and fulfills expectations is important. This scenario’s activities included requirements
elicitation and verification and validation (V&V).
The tools support elicitation by

 storing and managing elicitation templates—for example, quality function deployment or goal
question metrics (59 percent);
 storing and managing elicitation checklists (62 percent);
 storing and managing prioritization forms (59 percent); and
 providing Object Management Group Requirements Interchange Format (ReqIF) compatibility
for exchanging requirements information (27 percent)—ReqIF (formerly RIF) defines an open,
nonproprietary exchange format.

The tools support V&V by

  generating exception reports on verification plan cases with no verification procedures and on
verification procedures not linked to verification plan cases (59 percent);
 generating exception reports on validation plan cases with no validation procedures and on
validation procedures not linked to validation plan cases (62 percent);
 providing a standard format for interfacing to verification tools (59 percent);
 providing a standard format for interfacing to validation tools (59 percent); and
 implementing built-in requirements checks—that is, requirements verification capabilities that
determine whether requirements are complete, consistent, correct, modifiable, ranked,
traceable, unambiguous, understandable, or verifiable (51 percent).

The most capable tools for both requirements elicitation and requirements V&V are Cognition Cockpit,
Cradle, QPack, and Reqtify. MKS Integrity and Polarion Requirements stand out for elicitation; Aligned
Elements, Case Spec, GMARC, IRqA, PACE, ReqMan, and TraceCloud stand out for V&V.

Scenario 2
This scenario refers to organizations that want to establish a strong basis for design and implementation,
including modeling and requirements specification. The tools support modeling by providing storage and
display of

 business-process-modeling notation (46 percent),

 goal models (38 percent),
 Systems Modeling Language artifacts (32 percent), and
 data-flow diagrams (46 percent).

The tools support specification by

 inspecting the document through spell checking, grammar checking, data dictionaries, and
acronym tables (59 percent);
 generating a finished specification, including page security markings, graphics or figures, user-
definable tables, and indexes (68 percent); and
 synchronizing changes between the RE tool and the formatted document—creating a complete
loop between the two (41 percent).

The most capable tools for both modeling and requirements specification are Cognition Cockpit, Cradle,
and PACE. Reqtify stands out for modeling; GMARC, inteGREAT, MKS Integrity, QPack, Doors,
TraceCloud, and VisibleThread stand out for requirements specification.

Scenario 3
The last scenario is related to organizations seeking a high level of project control and quality assurance.
Achieving these is difficult by any means other than requirements V&V and traceability. We already
addressed V&V; the tools support traceability by

 generating reports comparing current and previous versions when a source document is
updated (57 percent);
 tracing across the tools’ boundaries (57 percent);
 tracing text to graphics (54 percent);
 tracing graphics to graphics (49 percent);
 tracing elements in graphics (43 percent);
 tracing tables and table cells (38 percent); and
 generating reports of traceability attributes—for example, category, number approved or
unapproved, number of changes, and number pending (62 percent).

The most capable tools for both requirements V&V and traceability are CASE Spec, Cognition Cockpit,
Cradle, GMARC, and Reqtify. inteGREAT stands out for traceability.
Discussion
Most tools obtained high or very high scores in requirements elicitation (88 percent), analysis (76 percent),
specification (77 percent), V&V (82 percent), traceability (79 percent), and other capabilities (73 percent).
However, a relevant set obtained medium, low, or very low scores in modeling (42 percent) and
requirements management (39 percent). This information suggests that a margin for improvement still
exists for modeling and requirements management, although we’ve found that generally, current tools
cover RE well.
Moving to more fine-grained concerns, these RE tools don’t extensively support a small percentage of
relevant features. Although they deal well with requirements elicitation, few of them (for example, Caliber
RM, Cognition Cockpit, and Cradle) support ReqIF to fill the gap in effectively sharing, communicating,
and collaborating across different tools. Support to ReqIF is needed because companies usually don’t work
on the same requirements repository or use the same RE tools.1
Other tool capabilities, including tool administrative information, GUI, and data integration features, are
well covered. However, regarding data integration, only a few tools (for example, CASE Spec,
inteGREAT, and MKS Integrity) support database federation. This feature provides users a virtual data
warehouse that eliminates the need to transfer the data, provides access to live data and functions, and
employs a single arbitrarily complex query to efficiently combine data from multiple sources of different
types.
As we noted, these tools don’t completely support requirements management. For instance, they poorly
support the open data model. This model facilitates tool interaction with external components by ensuring a
standard format for the application’s fundamental data structures. This allows runtime access to the
information without requiring a complex protocol. A few tools (for example, Bright Green Projects, PACE,
and ReqMan) provide this functionality by implementing the required features.
With the increasing complexity of software development and products, the need is growing for RE tools
integrated in product lifecycle management and application lifecycle management (PLM/ALM)
architectures.2 The software product lifecycle treats RE in an orderly fashion and is continuously managed
through an appropriate set of tools (for example, MKS Integrity, Polarion ALM, and QPack ALM-suite).

T he RE tools market is changing fast. Classic tools that used to dominate the market are increasingly
complex and difficult to use. Many expensive tools aren't sufficiently open to the tools of other
vendors, such as for modeling or traceability. This encourages newcomers to introduce interesting
capabilities, especially for collaboration. We recommend carefully evaluating the requirements of the RE
tool for your own environment. Be prepared to pay for what you expect. Cheap tools don't deliver
sophisticated features.
In the future, we plan to explore whether current RE tools adequately support features aimed at global
software development environments, and how these tools deal with the difficulties of distributed and
collaborative work.

Acknowledgements
We thank the survey’s participants for their invaluable help. The Pegaso/Pangea project (TIN2009-13718-C02-01-02),
Origin Integrated Project (IDI-2010043 (1-5)), and Englobas Project (PII2I09-0147-8235) have funded this research.

References
1. M.R. Monteiro, C. Ebert, and M. Recknagel, “Improving the Exchange of Requirements and Specifications
between Business Partners,” Proc. 17th IEEE Int’l Requirements Eng. Conf. (RE 09), IEEE CS Press, 2009, pp.
253–260.
2. C. Ebert, “Requirements Engineering: Management”. In: Encyclopedia of Software Eng., P.A. Laplante, ed.,
Taylor & Francis, 2010, pp. 932–948.

Juan Manuel Carrillo is a research assistant at the Software Engineering Research Group, Faculty of
Computer Science, University of Murcia. Contact him at jmcdg1@um.es.

Joaquín Nicolás is an associate professor at the Software Engineering Research Group, Faculty of
 Computer Science, University of Murcia. Contact him at jnr@um.es.

José Luis Fernández is an associate professor at the Software Engineering Research Group, Faculty of
Computer Science, University of Murcia. Contact him at aleman@um.es.

Ambrosio Toval is a full professor at the Software Engineering Research Group, Faculty of Computer
Science, University of Murcia. Contact him at atoval@um.es.

Christof Ebert is managing director at Vector Consulting Services. He is a senior member of the IEEE and
is on IEEE Software’s editorial board. Contact him at christof.ebert@vector.com.

Aurora Vizcaíno is an associate professor in the Alarcos Research Group, Escuela Superior de Informática,
University of Castilla-La Mancha. Contact her at aurora.vizcaino@uclm.es.

View publication stats