Requirements Engineering (2019) 24:523–543

https://doi.org/10.1007/s00766-018-0299-9

ORIGINAL ARTICLE

Sustainability requirements for eLearning systems: a systematic

literature review and analysis
Ahmed D. Alharthi1 · Maria Spichkova1 · Margaret Hamilton1

Received: 22 August 2017 / Accepted: 24 May 2018 / Published online: 31 May 2018
© Springer-Verlag London Ltd., part of Springer Nature 2018

Abstract
eLearning systems have become a very important part of teaching, both as web-based systems for online education and as
auxiliary tools for face-to-face study, where they provide an additional learning support for on-campus learners. To insure
the sustainability of an eLearning system on both individual and social levels, we have to cover many aspects of sustain-
ability requirements: human, technical, economic, and environmental. This paper provides a systematic literature review of
the sustainability meta-requirements for eLearning systems to identify open problems and to present the state of the art of
this research area. We analysed 124 papers, so we identified 18 high-level sustainability requirements for eLearning systems.

Keywords  Sustainability requirements · Requirements engineering · eLearning · Sustainable eLearning systems ·

Sustainability · Systematic literature review

1 Introduction We call a system sustainable if it satisfies the sustain-

ability requirements which cover the related sustainability
Requirements engineering (RE), i.e., requirements elicita- dimensions identified in [52, 115, 116, 122]:
tion, evaluation, specification, and design producing the
functional and non-functional requirements, is one of the – Human sustainability: Individual needs should be pro-
key disciplines in software engineering, as requirements- tected and supported with dignity and in a way that
related errors are often a major cause of the delays in the developments should improve the quality of human life
product delivery and development costs overruns, [82]. A and not threaten human beings;
number of studies have shown that if a software system is – Social sustainability: Relationships of people within
developed without taking into account the sustainability society should be equitable, diverse, connected and
requirements, this system could have negative impacts on democratic;
individual, social, technology, economic, and environment – Technical sustainability: Technology must cope with
sustainability, cf. [11, 80, 103, 116, 143]. This is especially changes and evolution in a fair manner, respecting natu-
important for eLearning systems, as they deal not only with ral resources;
a large amount of teaching data, but also with a large number – Environmental sustainability: Natural resources have to
of users. be protected from human needs and wastes; and
– Economic sustainability: A positive economic value and
capital should be ensured and preserved.

Therefore, sustainability comprehensively covers 1) the con-

* Ahmed D. Alharthi cept of “green in” software systems which means reducing
ahmed.alharthi@rmit.edu.au; adharthi@uqu.edu.sa energy and resources consumption and wastage in processes,
Maria Spichkova and 2) the context of “green by” software systems which
maria.spichkova@rmit.edu.au covers human and economic sustainability are improvements
Margaret Hamilton [21, 81, 115]. One dimension of sustainability cannot be
margaret.hamilton@rmit.edu.au reinforced without taking others into account. Each of the
1 five dimensions is interconnected. For example, supporting
School of Science, RMIT University, Melbourne, Australia

13
Vol.:(0123456789)

524 Requirements Engineering (2019) 24:523–543

more features related to individual and social dimensions Contributions This paper provides a SLR on the research
would increase energy consumption, i.e., influence economic conducted on sustainability meta-requirements which
and environmental dimensions. describe high-level requirements, i.e., a generalised class of
eLearning systems are a special kind of software system, goals, for eLearning systems to analyse the state of the art of
developed to provide a platform for accessible teaching and this research area and to recognise open problems. We iden-
learning, including online access to learning materials and tify and categorise high-level sustainability requirements
online support for learning and teaching. eLearning systems (sustainability meta-requirements) that need to be refined
are commonly composed of modules containing video con- to produce functional and non-functional requirements for
ference, discussion board, assignment and assessment man- sustainable eLearning systems. The identified sustainability
agement, grade book and weekly content cover units, or a meta-requirements are mapped to a software quality model,
different organisational structure. eLearning systems have which includes greenability characteristic.
become a very important part of the learning and teaching Outline The paper is organised as follows. The back-
process, due their flexibility and accessibility for instructors ground research is introduced in Sect. 2. Section 3 presents
and learners: the eLearning system may assist in delivering the methodology of the conducted SLR, and the core find-
knowledge and information any time and everywhere to any- ings are presented in Sect. 5. In Sect. 6, we discuss further
one [24]. When developing an eLearning system, there are suggestions on sustainability meta-requirements that were
a large number of requirements that need to be gathered and not covered in the literature, and propose a mapping from
negotiated with various stakeholders, as well as a large num- the identified meta-requirements to the Software Product
ber of diverse technical, cultural, and/or legal requirements. Quality Model (ISO/IEC 25010). Finally, Sect. 7 discusses
College affordability, which impacts learners, will be threats to validity of this work and Sect. 8 summarises the
raised when educational institutions pay more expanses contributions of the paper.
of an eLearning system that has high maintenance budget
and consumes more resources and energy for long-running
and vice versa. Also, lifelong learning will be supported 2 Background
when eLearning systems have functionality that includes
self-learning path process and integration of personalised In this section, we discuss the definition of eLearning sys-
cloud storages. These functionalities will improve individual tems, as well as how the sustainability of eLearning systems
productivity (human sustainability) and reduce power con- was analysed in the related work.
sumption (environmental sustainability) [8, 12, 39, 57, 58,
78, 79]. 2.1 eLearning systems
Providing the high-level sustainability requirements of
eLearning systems via a systematic literature review (SLR) An eLearning system can be defined as an educational solu-
will cover all aspects and it assist researchers, business ana- tion to deliver knowledge, facilitate learning and improve
lysts, engineers, developers and educational institutions to performance by creating, using and managing appropri-
improve and ensure sustainability not merely in software ate technological processes and resources, cf. [51, 123].
systems, but also in eLearning processes, learners and eLearning systems such as Blackboard and Moodle provide
instructors, natural resources, and institutions and businesses innovative services for learners, instructors, institutions and
models. learning process. Instructors can create modules to organise
This SLR has not been undertaken previously to bring course content by week or units. This course content could
together the five overlapping sustainability dimensions and contain discussions, assignments, quizzes, and learning
requirements of eLearning systems where human and social materials, so learners will engage in learning process either
aspects can be recognised easily in the education context. In on-campus or off-campus. Mridha and Nihlen [100] stated
this context, we can provide as next step of the SLR guide- that in developing countries such as Bangladesh, particu-
line, check-list, and/or references model for sustainable larly in rural areas, eLearning systems allow educational
eLearning systems. For instance, when educational institu- equity for people who cannot afford to pay for private tutors.
tions would shift their eLearning system to sustainable one, Likewise, Stepanyan et al. [143] provided examples of how
they should check the 18 sustainability meta-requirements technological affordances might assist with new approaches
that we extracted and collected from 124 studies cover- for learners to learn. One popular example of an eLearning
ing the five sustainability dimensions. Also, a community system is a Learning Management System (LMS) which
including individuals and societies may benefit from the includes a virtual classroom, collaboration functions and
technological and educational affordability of development, instructor-led courses. As per Dagger et al. [34], a LMS has
operation, and knowledge construction [4]. two types:

13
Requirements Engineering (2019) 24:523–543 525

– Proprietary LMS, e.g., Blackboard and Desire2Learn; development. For example, collaboration, which is part of
and the individual dimension, is not included. Also, the sustain-
– Open-source LMS, e.g., Moodle and Sakai. ability requirements may identify and follow sustainable
software engineering in order to cover all the five dimen-
Furthermore, eLearning systems can be categorised from sions and to be standardised with other software domains.
a management perspective as an Institutional Learning Sustainability is a very complex research area although
Environment (ILE), Personal Learning Environment (PLE, there may be five aspects identified, they overlap and inter-
cf. [6]), and Institutional Personal Learning Environment act. Ideally, they should not be separated and have to be tack-
(IPLE, cf. [24]). An ILE is provided by institutions and is led together under one umbrella because of the overlapping
usually used for instructor-led courses. An example of an aspects as well as to provide a “big picture”. For example,
ILE is a LMS. In contrast to an ILE, the PLE is managed providing eLearning systems with sustainable eLearning
personally and is usually used for learner-led courses. For processes and without reducing energy consumption could
instance, learners use Web 2.0 technology such as blogs to impact rising the electricity costs and increase tuition fees.
create and share knowledge. These could impact on eLearning systems by causing learn-
Casquero et al. [24] suggested the combination of an ILE ers to drop education because of the high cost of college
and PLE as an IPLE which may help institutions and learn- affordability.
ers to share learning resources across multiple administra- For these reasons, we cover all aspects of sustainability
tive and learning contexts. Moreover, LMS vendors recently in eLearning systems to include high-level sustainability
started to adopt PLEs and merge them with their LMS [36]. requirements as the following, for example:
For instance, Blackboard bought CourseSites in order to
provide a PLE for their LMS, and in the meantime Desire- – eLearning systems shall run on green data centre (envi-
2Learn updated its LMS to support social networks such as ronmental sustainability),
Facebook and Google+. – eLearning systems shall share learning content with other
eLearning systems and social network (technical sustain-
2.2 Sustainable eLearning systems ability),
– eLearning system shall provide extension for massive
In order to provide a definition for a sustainable eLearning open online courses for anyone to enrol (human sustain-
system, we have to specify and analyse the corresponding ability),
sustainability requirements. There are many studies focusing – eLearning system shall allow collaboration on a docu-
on the sustainability of eLearning systems, but they usually ment to use real-time co-authoring (social sustainability),
cover only a single aspect of sustainability. For example, and
many researchers have studied individual dimensions [78], – eLearning system shall calculate the return on investment
while other scientists have discussed the economic dimen- formula and power consumption per business transaction
sion [40, 76], and the social dimension [85]. The environ- (economic sustainability).
mental dimension of eLearning system sustainability was
analysed by Dong et al. [39] and Roy et al. [128]. As we see, each aspect of sustainability impacts oth-
In order to provide a sustainability profile for an eLearn- ers and many studies tackle either one or two aspects of
ing system, Stewart and Khare [145] employed the Sustain- susceptibility.
ability Circle Framework, developed by the Global Com-
pact Cities Programme for the urban sustainability profile
of a particular city or region [64]. This framework has four
domains including ecology, economy, culture and politics. 3 Methodology
Each domain has 7 sub-domains in order to assist in assess-
ment through the completion of a survey having 7 questions In the section, we discussed research questions and Sys-
for each domain. The assessment is conducted on a nine- tematic Literature Review (SLR) methodology which was
point scale that ranges from 1 being critical to 9 labelled complemented by snowballing and Non-Systematic Review
vibrant. The framework is based on the colours of traffic (NSR) for additional papers.
lights with critical marked red and vibrant marked green
[64]. The authors proposed this method in order to generate
a clear graphical representation of the sustainability pro- 3.1 Research questions
file for eLearning systems [145]. Even though this adoption
framework could rank the specific nuances in the economic The main aim of this paper is to answer the following
dimension, it needs to be reformulated to fit eLearning research questions:

13

526 Requirements Engineering (2019) 24:523–543

RQ1 What are the requirements for eLearning systems 3.2 Systematic literature review
that cover the sustainability aspects? In order to answer
this question, sustainability requirements are identified We followed the SLR methodology in [72] and [68]. This
by forming a search string to include the following cri- approach consists of the following steps:
teria:
– Planning the review
– sustainability including the three forms as sustain-
ability, sustainable, sustain; – Identification of the need for a review,
– eLearning considering learning, e-learning, eLearn- – Development of a review protocol
ing, electronic learning or distance education terms;
– requirements engineering: within singular or plural – Conducting the review
keywords of requirement; and
– system identification the system may be an environ- – Identification of research
mental or ecosystem, or it may be a learning man- – Selection of primary studies
agement system. – Study quality assessment
– Data extraction
RQ2 How can we classify sustainability requirements – Data synthesis
for eLearning systems from the SE perspective? We
explained the sustainability requirements from a soft- – Reporting the review
ware engineering perspective. To answer RQ2, we need
to consider the eLearning system as software and apply Figure 1 presents a visualisation of our methodology,
the same sustainability requirements to it. including the review process and outcomes. In the first step,
RQ3 Which sustainability requirements are specific to we selected the following primary sources that suggested by
eLearning systems? Sustainability requirements dif- [16] to perform automated and manual searches:
fer from one domain to another, particularly human
(individual and social) sustainability requirements. For – IEEExplore Digital Library1 (automated search),
instance, the lifelong learning requirement of individual – ACM Digital Library2 (automated search),
sustainability is a specific requirement in the educational – Scopus3 (automated search),
software domain [110] while controlling energy con- – Springer Link4 (manual search),
sumption is an environmental requirement in eLearning – Wiley5 (manual search).
and other domains such as the health domain.
For automated search, the following search string was used
We carried out the SLR which has not been previously sys- over the title, abstract and keywords fields of the papers in
tematically analysed the sustainability meta-requirements the digital libraries:
of eLearning systems. We believe that the SLR covering the (sustainability OR sustainable OR sus-
five overlapping sustainability aspects with requirements of tain) AND (requirement OR requirements
eLearning systems is important to investigate previous results OR requirements engineering) AND (learn-
together under one umbrella. The impossibility of reducing all ing OR e-learning OR eLearning OR (dis-
dimensions to a single dimension or tackling a single dimension tance AND education) OR (electronic AND
without consideration of other dimensions is a critical issue to Learning)) AND (system OR systems OR
satisfy the goal of sustainability. We could not assume learners environment OR ecosystem)
as individuals have no or a zero impact on the environment The result after Step 1 was a total of 901 studies, cf.
when we elicit or analyse requirements and vice versa. Thus, Table 1. In Step 2, part of refinement in Fig.  1, the results
this review will benefit researchers and developers. Researchers were refined by removing any papers which did not meet
can identify with important research gaps the relevant work and the following criteria:
have a clear and consistent picture of sustainability require-
ments for eLearning systems instead of a smaller study that
might give a small piece of the whole picture. For developers,
the SLR will provide useful information about the high-level 1
  http://ieeex​plore​.ieee.org.
sustainability requirements. They can use the results in this 2
  http://dl.acm.org.
paper as a reference to develop and select sustainable eLearning 3
  http://scopu​s.com.
systems for fulfilling long-living, meeting stakeholders needs, 4
  http://sprin​gerli​nk.com.
reducing global carbon emissions, and budget savings. 5
  http://onlin​elibr​ary.wiley​.com.

13
Requirements Engineering (2019) 24:523–543 527

Fig. 1  Review process and Step 1

outcomes

Automated Search Manual Search

IEEExplore ACM Scopus Springer Wiley

659
659 papers
papers 242
242 papers
papers

Step 2 901
901 papers
papers

Refinement 1: based on the sustainability, educa

on, Step 4

so ware development and exclusion criteria

Non-systemac review
Step 3 473
473 papers
papers

Refinement 2: based on the inclusion and exclusion criteria Addional studies

Step 5
63
63 papers
papers
13
13 papers
paperrs

Merged list Added to

Step 6 76
76 papers
papers

Perform snowballing procedure

Step 7 124
124 papers
papers

Extract sustainability requirements

Also, a number of papers were excluded if they were pub-

Table 1  Primary sources and number of papers
lished in pedagogical strategies, extended abstracts, editori-
Primary sources Papers als and workshops or tutorials, or duplicated studies (where
IEEExplore 87
the same study was presented in several publications).
ACM 113
The total result after Step 2 was 473 studies on sustain-
Scopus 459
ability, education and/or SE including RE.
Springer: Requirements Engineering Journal 26
In Step 3, we refined the results to remove papers which
Springer: Empirical Software Engineering 39
did not focus on:
Springer: Education and Information Technologies 77
Wiley: Software Practice and Experience 29
– RE phase of software development, or
Wiley: Journal of Software: Evolution and Process 71
– The analysis of sustainability in eLearning or distance
Total 901
education systems or ecosystems or educational software.

We excluded studies on environmental curriculum and peda-

gogy, as well as on sustainability in education as curriculum
– Journals, grey literature (i.e., technical reports) and and pedagogy, which left 63 papers on the four topics:
conference proceedings;
– Publication data between January 2005 and June 2017; – Sustainability,
– Papers related to three main topics (sustainability, educa- – Education,
tion and RE); and – Requirements engineering, and
– Papers written in English. – eLearning systems.

13

528 Requirements Engineering (2019) 24:523–543

3.3 Additional non‑systematic review Table 2  Sustainability meta-requirements for eLearning  systems that

identified
We also added result of a NSR that we conducted on an earlier Dimension Sustainability meta-requirements
stage of our work to identify state of the art mainly focusing on
Individual and Social R1.1 Personalisation
learning and teaching system perspective. Thus, on the NSR,
R1.2 Learner-Centred Features
we included the following:
R1.3 Collaboration
R1.4 Leadership Development
– Three papers [85, 86, 131] have been published in 2003,
R1.5 Privacy and Security
which were out of the selected SLR search range, but they
R1.6 Analysis of the Learning Pro-
provide an important background for the research on sus- gress
tainability requirements for eLearning Systems. R1.7 Reuse of the Learning Materials
– Four papers [6, 40, 76, 145] which were published not in R1.8 Integration with Social Networks
the selected sources. However, the papers [6, 40, 76] are R1.9 Standardisation of the LORs
highly cited having 764, 589, 63 citations respectively. The Technical R2.1 Support of LORs
paper [145] has been published in a new series “ World R2.2 Support of Shared Services
Sustainability” started in 2015 by Springer. R2.3 Software Quality Requirements
– Six further papers which covered sustainability aspects R2.4 Portability
within RE and eLearning, but their title, abstract, and R2.5 Modularity
keywords did not contain the words of our selected Environmental R3.1 Cloud Computing
search string, such as “ requirement”, “ sustainability”, Economic R4.1 Reducing the Cost
etc. The papers [39, 78, 155] have been published at R4.2 Ensuring the Growth
the IEEE, papers [110, 140, 143] are counted by the
Scopus.

As a result of the NSR, 13 further papers were added to

the pool of studies to analyse. In Step 5, we merged the 4 Result of SLR
studies from the systematic and non-systematic parts of
the review. The first author identified the requirements listed in column
2 of Table 2 by examining the 124 papers. The other authors
3.4 Snowballing procedure evaluated this individually by inspection.
Figure  2 presents the distribution of 531 publications
Then, we complemented the SLR in Step 6 with snowballing between 2005 and 2017. The number of studies on sustain-
technique using the reference list of a paper to identified addi- ability (including sustainability of education RE/SE) increased
tional papers. We followed a backward iteration of snowballing by two and a half times over the period between 2005 and
procedure in [162] and refined them based on step 2 and step 3 2017: from 14 studies in 2005 to 30 studies in 2017. There was
criteria. In the first iteration, we looked at papers in the refer- constant interest in research on sustainable eLearning systems,
ence list of the 76 studies, so there were 31 referred papers. ranging from 7 paper in 2005, peaking at 17 in 2007 and fin-
Then, we accomplished a second iteration which resulted in ishing at 4 in July 2017.
14 papers. The 14 papers led to two papers that also referred The three pillars of sustainability requirements for eLearn-
to one more paper as a fourth iteration. Thus, 48 studies were ing systems are sustainability, education and software and
identified during the snowballing procedure added to the requirement engineering, which are represented in Fig. 3. The
merged list. overlaps across these three pillars provide us with four com-
The merged list of 124 studies were analysed in Step 7 to bination aspects:
extract the sustainability requirements and the results will be
discussed in Sect. 5. 1. Sustainability in education,
2. Sustainable SE (with the focus on RE phase of SE),
3. Educational software, and
4. Sustainability requirements for eLearning system which
is the main scope of our research.

13
Requirements Engineering (2019) 24:523–543 529

47
42

40
36
36

35

35
35

35
34
34

34
33

33
32

32

32

32
30

30
30

30

30

30
28

28
27
25

24
23
23

23
23

22

22

21
20
19

17
14

12

12

11

11
9

9
7

4
2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 2017

Sustainability Educaon Soware & Requirements Engineering Sustainable eLearning Systems

Fig. 2  Distribution of the number of studies over the years (2005–2017)

Individual Papers
Requirements
40% 32%
Educaon 30%
11% 20%
Economic Social
Requirements 10% Requirements
29%
Sustainability Educaonal 0%
in Educaon Soware
12 arn ble
ap g
eL taina

ers

ts
4 p in

en

4% 24%
g m
s
e
Su

in ire

Environmental Technical
er u
Su

Requirements Requirements
ne eq
st

gi R

Sustainable
ai

En and

Soware
na

Fig. 4  Dimensions of sustainable eLearning systems (2005–2017)

bi

e
ar
lit

w
y

So

contributions over the years between 2005 and 2017, while

the social and technical dimensions have 29% and 24%
Fig. 3  Distribution of the number of studies over the pillars of sus- respectively. The economic and environmental dimen-
tainability requirements for eLearning systems sions were covered only in 11% and 4% of the studies,
respectively.
5 Analysis of sustainability One of the reasons why the individual dimension has the
meta‑requirements for eLearning systems highest percentage while the environmental dimension has the
lowest percentage of studies is the nature of eLearning sys-
To analyse the Sustainability Meta-Requirements (SMRs) tems: these systems have a very strong impact on the human
being a generalised class needed to be combined with dimension of sustainability, where their impact on the envi-
further information to provide actual requirements for ronmental dimension is perceived as rather small. Neverthe-
eLearning systems, and to answer (RQ1) and (RQ2), the less, the environmental dimension plays an important role in
meta-requirements identified within the review process the development of eLearning systems too. For example, the
were distributed among related sustainability requirements lifelong learning requirement of individual sustainability is a
dimensions as per [115], i.e., among the individual, social, specific requirement in the educational software domain [110]
technical, environmental, and economic sustainability while controlling energy consumption that is an environmen-
dimensions. tal requirement is needed in eLearning and other domains
To illustrate these dimensions and the corresponding such as the health domain. It can be argued that each aspect
studies, Fig.  4 provides a chart with the five dimensions is affected by energy consumption, not only the environmen-
and percentages of 124 related papers. As shown in the tal. Individuals consume power to access and learn from the
chart, individual SMRs comprised of 32% of the require- system. The social aspect consumes power when discussion
ments that were the most significant part of the research boards are accessed, lectures are uploaded.

13

530 Requirements Engineering (2019) 24:523–543

5.1 Individual and social sustainability An example of personalisation, beside customisation con-

requirements tents and layout, eLearning system shall allow learners and
instructors to integrate their private cloud storages, hosting
Individual sustainability focuses on individual needs and websites and web services with their account as well as to
rights users of the system, and in the case of eLearning sys- synchronise their data securely with other eLearning systems
tem, these are instructors, learners, and administrators. We and academic systems. Ossiannilsson and Landgren [110]
identified sex core individual sustainability requirements believed that personalisation features, which reflect learn-
(R1.1-6) within the reviewed studies. ers’ personal demands and preferences, should be crucial for
Social sustainability composes of communities, insti- eLearning systems. One of the important aspects of the fea-
tutions, and individual dimension. It covers societal well- tures might be the connection of an eLearning system with
being as well as availability, and equality of education [5]. their academic and personal networks. Attwell [6] stated
The social sustainability dimension of eLearning systems that the personalisation feature has a possibility to create a
is currently not well-covered in the existing approaches. In personalised learning path according to individual data. For
the reviewed papers, we identified only three social SMRs instance, an eLearning system could provide suggestions
which are related to learning materials and objects (cf. on the learning paths, based on the analysis of the learner’s
R1.7-9 below). progress or demonstrated learning ability, as well as provide
While analysing the individual and social SMRs for a certain content or activity based on the knowledge, skills
eLearning systems, we identified that most of these require- and objectives demonstrated in the corresponding online
ments are heavily correlated to both individual and social quizzes. Thus, having a personalisation feature may help
dimensions, if we take into account not only the first-order learners to increase the efficiency of the training.
but also second-order impacts, as per Berkhout and Hertin R1.1 is an individual-social SMR:
[11] who proposed to distinguish between 3 orders of the Individual dimension: first-order impact.
impacts of information and communication technologies on Social dimension: second-order impact (via networking,
environmental sustainability: efficiency of training, etc.).
R1.2. Learner-centred features: An eLearning system
– first order impacts, producing direct environmental should be aligned to the learner-centred approach.
effects, For instance, eLearning system shall provide self-assess-
– second-order impacts, producing indirect environmental ment rubric. The learner-centred approach puts education
effects, and responsibility and independence in the learner’s hand,
– third-order impacts, producing rebound effects. which also provides a basis for lifelong learning. Accord-
ing to Attwell [6], learners need to improve their occupa-
For this reason, we prefer to analyse both dimensions jointly. tional knowledge and skills by continuing learning during
Theoretically, the following four options are possible: their work-life. Kruchten and Kendall [70, 78] present the
importance of lifelong learning such as retaining learner’s
– Individual SMR: the requirements having first-order employability and develop critical thinking.
impact within the individual dimension, but having no This requirement also could be addressed by providing
impact within the social dimension; open education [47, 61, 114] as well as government and
– Social SMR: the requirements having first-order impact society support [119, 134] in both formal or informal learn-
within the social dimension, but having no impact within ing methods [70]. Thus, as Ossiannilsson and Landgren
the individual dimension; [110] highlighted the support of lifelong learning would
– Individual-social SMR: the requirements having first- become crucial for sustainable eLearning systems.
order impact within the individual dimension, as well as R1.2 is an individual-social SMR:
second-order impact within the social dimension; Individual dimension: first-order impact.
– Social-individual SMR: the requirements having first- Social dimension: second-order impact (via lifelong
order impact within the social dimension, as well as learning, employability, etc.).
second-order impact within the individual dimension. R1.3. Collaboration: An eLearning system should sup-
port collaboration features.
However, in the case of eLearning systems, the first two For example, eLearning system shall provide discus-
options are irrelevant, and in all cases we have to consider sions board that allow users to participate in a conversation
either individual-social or social-individual SMRs. with an entire class or group. Using the discussion board or
R1.1. Personalisation: eLearning systems should support tools for instant feedback between learners and instructors
personalisation features. could improve the learning curve [42, 114]. In the same
way, Pellas [114] highlighted providing real-time feedback

13
Requirements Engineering (2019) 24:523–543 531

could impact learners on their failure or success as well as and import them into courses. One of the crucial success fac-
enhance practice-based tasks. Ossiannilsson and Landgren tors for eLearning ecosystems is the reuse of learning martial,
[110] stated that collaboration is a key element of success cf. [140]. As explained by Stepanyan et al. [143], reusable
in eLearning systems. learning materials may not only reduce the instructors’ work-
The collaboration feature may increase the success load but also offer additional potential for cost-effectiveness.
rate of the learning process as well as having a social- For instance, if one instructor who has a greater technical
transforming potential. Mocigemba [97] stated that it is ability to design course materials shares his work with other
perceived as fast, direct and less bureaucratic. However, instructors, it will lead to time and cost savings [132, 139].
from our point of view the mentioned perception points This requirement is specific to eLearning systems and could
are controversial. be facilitated by a Learning Object Repository (LOR).
R1.3 is an individual-social SMR: R1.7 is a social-individual SMR.
Individual dimension: first-order impact. R1.8. Integration with social networks: eLearning system
Social dimension: second-order impact (via the social- shall provide learner and instructor role optional integration
transforming potential). with social network and a variety of third-party providers.
R1.4. Leadership development: An eLearning systems, for The importance of integrating social networks with aca-
instance, shall provide leadership development through rais- demic activities is explored. Manca and Ranieri [91] dis-
ing awareness of sustainability information and supported cussed the challenges of incorporating social networks into
availability. To make an eLearning system sustainable, peo- teaching and learning such as policy, and cultural and social
ple awareness [80], academic leadership and institutional factors. Wang et al. [159] introduced guidelines to maximise
transformation [143] have to be involved within the strate- the educational potential of social networks in higher edu-
gies of an organisation to provide a sustainable eLearning cation, so integrating social networks features could help
system. personalisation and collaboration. However, maintaining
R1.4 is an individual-social SMR: privacy and security during integration would be crucial.
Individual dimension: first-order impact. R1.8 is a social-individual SMR.
Social dimension: second-order impact (via institutional R1.9. Standardisation of the learning object repositories
transformation). (LORs): eLearning system shall support a variety standards
R1.5. Privacy and security: Learners and instructor in of learning object repositories. Standardisations particularity
eLearning system shall not be placed in breach of National the open standards of LORs have high potential to be inter-
Privacy Act. Stewart and Khare [145] as well as Roy [127] operable with other eLearning systems and adopted by open
pointed out privacy and security as the political dimension source and proprietary eLearning systems [38]. The standers
which should take place in eLearning systems in order to could asset to reuse learning objects for a long-term goal
protect the individual rights. These quality requirements of educational culture change and to benefit instructors to
should be implemented to protect users’ data and profiles as develop and support their own courses [55]. However, there
well as authors’ rights [104, 111, 140, 153]. are still challenges of content and sharing learning objects
R1.5 is an individual-social SMR. such as copyright, quality control, and cultural assimilation
R1.6. Analysis of the learning progress: Learning sys- that should be addressed [47, 55, 126]. Thus, LORs should
tem shall produce the evaluation of individual components, be standardised with regard to ownership, multilingualism
a course and learner performance. Analysing the usage of and learning styles.
users’ behaviour is describing a real individual behaviour R1.9 is a social-individual SMR.
without any influence by person factor for data collection
and statistics [23]. To analyse the usage, there are differ- 5.2 Technical sustainability requirements
ent layers for log file analysis in infrastructure, contents,
tools, and terminal layers [165]. System load, network traf- Technical sustainability includes on reducing negative
fic, learning behaviours, accessed learning resource, and impacts of technology such as consumption and pollution.
learner assessment could be caught automatically and visu- R2.1. Support of learning object repositories: eLearning
alised. Therefore, when the progress of learning is moni- systems shall support Learning Object Repositories (LORs)
tored, measured and analysed by the eLearning system, it which is a specific requirement. To facilitate the reusable learn-
could assist the learners and instructors to improve their ing material requirement, eLearning systems should support
productivity. LORs having learning elements, attributes and content. How-
R1.6 is an individual-social SMR. ever, there are some challenges for LORs such as the LMS
R1.7. Reuse of the learning materials: An obvious exam- support, ownership, copyright and validation of resources as
ple is that eLearning system shall allow instructor to extract discussed by [140]. Another essential point is learning object
course content, assignments, and quizzes from previous terms meta-data that helps users to store, search, reuse learning

13

532 Requirements Engineering (2019) 24:523–543

objects quickly and effectively particularly if there are too course content by week or units. Modularisation in eLearn-
much meta-data and content to search [164]. Thus, sustainable ing systems has two kinds: modular architectural design and
eLearning systems should support different standards in order modular learning design. Firstly, the modular architectural
for LORs to be used by different institutions across the world. design is a concept of designing the system to support inter-
R2.2. Support of shared services: eLearning system shall operability. So, the sustainable eLearning systems function-
support a variety of third-party providers including Adobe Con- ality should be divided into modules in order to alleviate the
nect and Google Calendar. To enable eLearning sustainability, integration in a flexible manner [34, 89].
the corresponding IT services might be shared among universi- The modular architecture may reduce the cost of eLearning
ties and campuses. This may reduce cost and improve services systems and enable interoperability. Secondly, modular learning
[145]. For instance, the Ethiopian government implemented design is an approach to present course materials in a logical,
an eLearning system in medical schools across the MEPI- sequential fashion to guide learners, cf. [151] This approach
Ethiopia consortium including the Addis Ababa, Hawassa and could assist learners only to complete the part of the course that
Haramaya Universities, and the Defense College of Health in is relevant to their needs instead of completing the entire course.
order to enhance the quality and efficiency of medical educa-
tion [158]. Another example, British Columbia (BC) in Canada 5.3 Environmental sustainability requirements
is sharing IT infrastructure and application services with the
BC Post-Secondary sector. Therefore, this requirement might Environmental sustainability contains resource consumption
standardise services to facilitate integration as well as reduce and waste. Therefore, resource use should be reduced dur-
eLearning cost and energy consumption [143, 145]. ing eLearning system operation in order to decrease energy
R2.3. Software quality requirements: For example, and pollution. Many scientists showed that better design of
eLearning system shall sustain quizzes with not less than the system [11], tracking resource use [125], monitoring
5 mints while in the absence of network. Many reviewers physical waste and energy bills [116] could decrease the
asserted that not only availability, and equality of education energy consumption and pollution which lead to protecting
has to be taken into account but also software should meet the natural resources.
quality requirements such as performance, security, usability R3.1. Cloud computing: eLearning system shall deliver
and longevity [5, 89, 107, 143]. These qualities that belong on-demand computing resources over the Internet. Cloud
to non-functional requirements should be satisfied [22, 89]. computing enables sharing resources and infrastructures
Also, the quality requirements contribute to sustainability that not result only in energy efficiency but also in cost effi-
software. For example, if an eLearning system has a high ciency [80, 90]. For example, eLearning systems could be
quality of performance, security and longevity, the system run through a virtual machine that offers large energy sav-
might not be replaced. These requirements lead to the reduc- ings [39, 79]. Hence, cloud computing may leverage eLearn-
tion of energy consumption during new software develop- ing systems to share sources, reduce cost and energy, and
ment, and the protection of peoples’ information and rights. monitor usage [36, 79, 139].
R2.4. Portability: eLearning system shall run on various
devices of a large, medium and small screen. Portability 5.4 Economic sustainability requirements
and optimising graphical design as technical sustainability
may assist people to use eLearning systems everywhere and The reduction of operating cost and the insurance of eco-
with any device [48]. For instance, rural areas in developing nomic growth should be considered [81], besides the effi-
countries that have low-bandwidth network could benefit ciency of software.
from access to an eLearning system to obtain a high-quality R4.1. Reducing the cost: For instance, eLearning system
education from developed countries. To illustrate, the Uni- shall support reusable learning object. Several scientists pro-
versity of Nsukka in Nigeria started to use mobile devices posed a virtual and remote laboratories framework that shows
to receive lectures [51]. Despite the potential of mobility, major economic advantages [25, 56, 95, 139, 142]. For exam-
the optimisation and downsizing of websites need to be ple, electronic engineering learners who use a virtual labora-
improved to assist accessing knowledge, reminders, and tory could overcome classroom and laboratory limitations in
reviews through assorted mobile devices [48]. equipment as well as space. This approach leads to reducing
Although there is rapid technological change, eLearn- the cost of new equipment and maintenance, cf. [95].
ing systems are not yet fully supported. These systems have Virtual and remote laboratories in eLearning systems
the potential to shape learning processes, and they need to have not only an impact in the economic but also within
streamline tools to be portable with various devices such the individual dimension [142], as they influence creativ-
as mobiles, tablets, laptop and desktop computers [6, 132]. ity, teamwork, learning from failures, etc.
R2.5. Modularity: eLearning system shall allow instruc- R4.2. Ensuring growth: eLearning system, for example,
tors to create, modify and delete module for organising shall provide secure remote-access to data laboratory and

13
Requirements Engineering (2019) 24:523–543 533

off-campus library. This economic sustainability require- meta-requirement which has not been covered by the iden-
ment might ensure the growth of the economy. An example tified 124 studies; we strongly believe that it is required for
being claimed by Alrashidi [2] for the economic benefit of developing a sustainable eLearning system, as it is
using eLearning system is that eLearning systems could fundamental for all systems and is also important for
assist people in Saudi Arabia to stop migrants from rural eLearning systems, cf. also [20]. We label it as Proposed
areas to cities. This advantage may lead to developing and Requirement 3.2 (PR3.2) which belongs to the environ-
growing the economy in the learners’ area and to prevent- mental sustainability dimension.
ing the city from increasing population and pollution. PR3.2. Green and sustainable software engineering (SE)
[environmental SMR]:
eLearning system, for instance, shall run on a green data
6 Discussion centre. Green SE could enable developers to design better
methods, metrics and tools to encourage green behaviour
Based on our analysis of the reviewed studies in sustain- [20]. eLearning systems should be based on green and
ability requirements, we observed that although a num- sustainable software development processes such as the
ber of studies provide various solutions to sustainable GREENSOFT Model [103], sustainable business process
eLearning systems, there are still a number of issues that management [12], and/or green and sustainable software
need to be addressed. We propose to include the following model [90]. These models which cover all aspects of green

Table 3  Sustainability meta-requirements for eLearning systems within software product quality that identified
Dimension Sustainability Meta-Requirements G/S Software Product Quality References

Individual and Social R1.1 Personalisation G Usability [6, 19, 27, 27, 41, 53, 54, 65, 87, 94,
95, 110, 112, 117, 120, 121, 148,
150, 152, 161]
R1.2 Learner-Centred Features S Usability, greenability [6, 17, 23, 38, 50, 57, 61, 62, 66, 70,
70, 78, 87, 88, 93, 96, 106, 110, 113,
114, 117, 119, 134, 150, 159]
R1.3 Collaboration G Functional suitability, compatibility [1, 14, 17, 18, 29, 35, 42, 43, 46, 60,
61, 65, 77, 87, 97, 98, 100, 110, 111,
114, 126, 133, 140, 146, 147, 149,
157, 166]
R1.4 Leadership Development G Usability [7, 9, 10, 49, 53, 54, 62, 73, 80, 124,
129, 130, 133, 134, 137, 143, 153,
161]
R1.5 Privacy and Security G Security [44, 104, 140, 145, 153]
R1.6 Analysis of the Learning S Functional suitability, usability [23, 110, 136, 165]
Progress
R1.7 Reuse of the Learning Materi- S Maintainability [30, 43, 47, 53, 76, 109, 132, 136, 139,
als 140, 143, 158]
R1.8 Integration with Social Net- G Compatibility, portability, usability [6, 30, 31, 77, 84, 91, 134, 159]
works
R1.9 Standardisation of the LORs S Compatibility, maintainability [38, 47, 61, 86, 132, 136, 143, 158]
Technical R2.1 Support of LORs S Maintainability [40, 54, 55, 109, 124, 136, 140, 164]
R2.2 Support of Shared Services G Compatibility [94, 98, 143, 145, 158]
R2.3 Software Quality Requirements G All qualities [5, 13, 22, 28, 39, 44, 48, 67, 69, 89,
92, 96, 107, 110, 143, 160]
R2.4 Portability G Portability [6, 37, 48, 71, 89, 99, 112, 132, 135,
148]
R2.5 Modularity G Maintainability [33, 89, 138, 150, 151, 166]
Environmental R3.1 Cloud Computing G Greenability [8, 36, 39, 79, 80, 90, 138, 139, 156,
161, 165]
PR3.2 Green and Sustainable SE * G Greenability [12, 20, 39, 90, 103, 128, 144, 145]
Economic R4.1 Reducing the Cost G Greenability [9, 10, 25, 35, 45, 56, 83, 95, 102, 139,
142]
R4.2 Ensuring the Growth G Greenability [2, 15, 53, 54, 62, 75, 77, 95, 101, 152]

*Sustainability meta-requirement added and proposed by authors

13

534 Requirements Engineering (2019) 24:523–543

and sustainable software engineering might sustain eLearn- – Performance efficiency considers time behaviour,
ing system from cradle-to-grave. resource utilisation, and capacity;
Table 3 illustrates our findings regarding the research – Compatibility has interoperability, and co-existence as
questions. It summarises the meta-requirements identified sub-characteristics;
within the SLR studies as well as the additionally proposed – Usability includes appropriateness recognisability, learn-
meta-requirement (marked by *). The questions ability, operability, user error protection, user interface
aesthetics, and accessibility attributes;
RQ1: 
What are the requirements for eLearning systems – Reliability: comprises of maturity, availability, fault tol-
that cover the sustainability aspects? and erance, and recoverability sub-characteristics;
RQ2: 
How can we classify SMRs for eLearning systems – Security contains confidentiality, integrity, non-repudia-
from the SE perspective? are answered by the first tion, accountability, and authenticity;
two columns of the table. The column G/S is used – Maintainability incorporates modularity, reusability, ana-
to highlight the answer to the question lysability, modifiability, and testability; and
RQ3: 
Which SMRs are specific to eLearning systems? – Portability covers adaptability, installability, and replace-
The meta-requirements marked by S are specific to ability sub-characteristics.
the domain of eLearning systems, where the meta-
requirements marked by G are general, i.e., appli- Calero et al. [20, 22] introduced a new sustainability-related
cable to other domains. characteristic for quality model based on ISO/ 25010:

For example, in eHealth services personalisation features – Greenability the degree to which a product energy and the
are essential and assist to improve eHealth services Hine resources are optimised, and the product can be used over
et al. [59]. On the other hand, learner-centred features, the a long period. This main characteristic includes energy effi-
reuse of the learning materials, learning object repositories ciency, resource optimisation, capacity optimisation, and
belong to the education domain only, and should be seen perdurability sub-characteristics.
as specific requirements (features) of eLearning systems.
Our study has shown that technical, environmental and eco- Similarly, the relationships between software quality and
nomic are general sustainability requirements, as they could environmental sustainability criteria were analysed by
be identified and analysed for any kind of software. Koçak et al. [74]. In our work, we follow the software prod-
Consideration of the main result, the identified SMRs could uct quality model introduced by Calero et al. [20], which
sustain any eLearning system if they cover all SMRs particu- results in the relationships between software quality and
larly the human dimension because the majority of SMRs in SMRs for eLearning systems presented in Table 3. Thus, the
the individual and social dimensions having 41% and 27%, SMRs that are specific for eLearning systems mostly focus
respectively. Nevertheless, those SMRs still have some issues on such qualities like usability, greenability, maintainability,
to provide sustainable eLearning systems. The influence of compatibility, and functional suitability.
one SMR in the individual dimension to another SMR in the
social dimension could reduce sustainability. For example, if
an eLearning system has personalisation features but it does 7 Threats to validity
not support the standardisation of LORs, a learner who set
accessibility preferences in the system cannot access learn- There are some threats that need to be taken into account.
ing materials as well as perform assignments that not support In this section, we follow the classification for the threats to
visual, auditory, and mobility impairments. validity by Wohlin et al. [163], which includes construct,
As the next step of SMRs analysis, we mapped the identi- internal, external and conclusion validities.
fied SMRs of eLearning systems to the elements of the Soft- Construct validity: The search string is the main threat in
ware Product Quality Model (ISO/IEC 25010) in order to constructing this study. The concept of eLearning has been
ease the accomplishment of software quality. ISO/IEC 25010 used differently in many studies. To make sure we cover
[63] is a division of an International Standard for System all these studies, we considered “learning”, “e-learning”,
and Software Product Quality Requirements and Evaluation “ eLearning”, “electronic learning” and “distance education”
(SQuaRE). The ISO/IEC 25010 includes three quality models: terms. Also, we used system, environment, and ecosystem
Product Quality, Data Quality, and Quality in Use. Product terms to ensure all the selected studies are related to the sus-
quality model has eight main characteristics: tainability meta-requirements of eLearning systems. We per-
formed the manual search on three Springer and two Wiley
– Functional suitability includes functional completeness, journals to make sure of high coverage of potentially rele-
correctness, and appropriateness attributes; vant studies. These five journals have high ranking according

13
Requirements Engineering (2019) 24:523–543 535

Table 4  Journal rank used in the SLR as SJR indicator in 2016, RQ2 How can we classify sustainability requirements for
where Q1 is the highest and Q4 the lowest values eLearning systems from the SE perspective?
Journal SJR Quartile RQ3 Which sustainability requirements are specific to
eLearning systems?
Springer: Requirements Engineering Q2
Springer: Empirical Software Engineering Q1
We extracted 18 high-level sustainability requirements
Springer: Education and Information Technologies Q2
(meta-requirements). We classified these meta-requirements
Wiley: Software Practice and Experience Q2
according to the five dimensions of sustainability: human
Wiley: Journal of Software: Evolution and Process Q3
(individual), social, technical, environmental, and eco-
nomic. We also proposed (1) further classification for the
consideration of individual and social sustainability meta-
to SCImago Journal Rank (SJR) indicator,6 see Table 4. In requirements for eLearning systems jointly. As these meta-
addition, we complemented our search with snowballing requirements are heavily correlated to both individual and
procedure in case of our search string might not be sufficient. social dimensions, if we take into account not only the first-
Internal validity: We collaboratively reviewed, discussed order but also second-order impacts. (2) a mapping from
and resolved any conflict during the inclusion and exclusion the identified sustainability meta-requirements of eLearning
steps to reduced personal bias on study understanding. We, systems to the Software Product Quality Model (ISO/IEC
also, have explained our study categorises and others may 25010), extended by the greenability quality.
category them differently. Our analysis demonstrated that the individual dimension
External validity: We executed the search several times, plays the most significant role for eLearning systems, as edu-
to confirm that the exact string can be performed in differ- cation (in both online and traditional face-to-face versions)
ent search engines. For example, we used the three forms of is part of the human dimension, while the role of the envi-
sustain within all the search strings instead of sustain* that ronmental dimension is similar to its role in other software
cannot be used the ACM search engine. Also, we cannot domains. Many high-level requirements (especially within
guarantee exact number of studies if some publishers added the environmental, economic, and technical dimensions) are
papers because of a merger or revisions. not eLearning systems specific. This allows us to identify
Conclusion validity: There is no any threat for any future what aspects could be inherited from the other domains and
replicated study when researchers follow our method and what aspects are domain-specific for eLearning systems.
categories. The results of this work could be considered as a high-
level guideline or recommendations for researchers and
developers. The SLR covers the research areas that were
8 Conclusions not investigated previously, covering the three overlapping
topics of RE, sustainability, and eLearning/education.
This paper provides a systematic literature review and analysis The elaboration of high-level sustainability requirements
of sustainability meta-requirements for eLearning systems. We aims in supporting the practitioners in developing long-liv-
conducted both manual and automated search over the IEE- ing eLearning systems. For example, while implementing
Explore Digital Library, the ACM Digital Library, Scopus, as a discussion board for an eLearning system, the developers
well as over a number of relevant Springer and Wiley journals, have to take into account not only modularity and portability
for the publication years between January 2005 and June 2017. features but also provide the support of existing and future
As a result, 63 related studies were identified. A further 13 learning objects.
studies were identified via an additional non-systematic review Future work This work provides a basis for elaboration of
that reflected in earlier stage of our work focusing on learning an eLearning RE framework proposed .
and teaching perspective to identify state of the art. We com-
plemented the 76 studies with backward snowballing proce- Acknowledgements  Ahmed D. Alharthi is supported by a scholarship
from Umm Al-Qura University, Saudi Arabia.
dure using their reference list. In total, there were 124 studies
being analysed and investigated to achieve our main objective
of answering the following research questions:
Appendix
RQ1 What are the requirements for eLearning systems
that cover the sustainability aspects? See Table 5.

6
  http://www.scima​gojr.com/.

13

536 Requirements Engineering (2019) 24:523–543

Table 5  List of studies for sustainable eLearning systems

No Paper Year Step Characteristics of paper

1 [98] : A Crowdsourcing Approach for Quality Enhancement of eLearning Systems 2017 SLR Case study
2 [161] : Learning for Low Carbon Living: The Potential of Mobile Learning Applications for Built 2017 SLR Empirical
Environment Trades and Professionals in Australia
3 [91] : Implications of social network sites for teaching and learning. Where we are and where we 2017 SLR Literaturereview
want to go
4 [132] : Mind the gap: Enabling online faculty and instructional designers in mapping new models 2017 SLR Comparative
for quality online courses
5 [106] : Mobile serious game proposal for environmental awareness of children 2016 SLR Experimental
6 [152] : Virtual schools: The changing landscape of K-12 education in the US 2016 SLR Theoretical
7 [42] : Students approaches to groupwork in a blended course, associations with perceptions of the 2016 SLR Empirical
online environment and academic achievement–when is learning engaged?
8 [77] : Monitoring social media: Students satisfaction with university administration activities 2016 SLR Empirical
9 [114] : Bolstering the quality and integrity of online collaborative university-level courses via an 2016 SLR Survey
open Sim standalone server in conjunction with sloodle
10 [48] : Systemic Requirements of a Software Engineering Learning Environment 2015 SLR Theoretical
11 [153] : Education and training challenges in the era of Cyber-Physical Systems: beyond traditional 2015 SLR Theoretical
engineering
12 [78] : Lifelong Learning for Lifelong Employment 2015 NSR Commentary
13 [119] : Towards relating delivery methods and examination success: lessons learned from the 2015 SLR Observational
VALO LLP project case study
14 [145] : eLearning and the Sustainable Campus 2015 NSR Conceptual
15 [160] : Supporting the evolution and interoperability of organisational models with e-learning 2015 SLR Theoretical
technologies
16 [113] : The influence of computer self-efficacy, metacognitive self-regulation and self-esteem on 2014 SP Empirical
student engagement in online learning programs: Evidence from the virtual world of Second Life
17 [147] : The four pillar model-Analysing the sustainability of online doctoral programmes 2014 SP Theoretical
18 [32] : Using social media as a tool for business improvement and certification of knowledge workers 2014 SLR Conceptual
19 [155] : Approaches for competencies assessment in open source e-learning environments 2014 NSR Comparative
20 [165] : Big Log Analysis for E-Learning Ecosystem 2014 SLR Empirical
21 [104] : Deploying suitable countermeasures to solve the security problems within an e-learning 2014 SLR Theoretical
environment
22 [19] : Educational edifices need a mobile strategy to fully engage in learning activities 2014 SLR Empirical
23 [159] : Tapping the educational potential of Facebook: Guidelines for use in higher education 2014 SLR Theoretical
24 [69] : Factors influencing technology planning in developing countries: A literature review 2014 SLR Literaturereview
25 [157] : Mobile inquiry learning in Sweden: Development insights on interoperability, extensibility 2014 SLR Case study
and sustainability of the LETS GO software system
26 [164] : Web-based learning object selection software using analytical hierarchy process 2014 SLR Empirical
27 [17] : Studying learning in the worldwide classroom: Research into edX’s first MOOC 2013 SP Empirical
28 [121] : Towards sustainable well-being in SMEs through the web-based learning program of ergo- 2013 SLR Empirical
nomics
29 [100] : E-learning for empowering the rural people in Bangladesh opportunities and challenges 2013 SLR Experimental
30 [134] : Developing the next generation of engineers for intelligent and sustainable manufacturing: A 2013 SLR Case study
case study
31 [137] : Use of sustainable and systematic plan to assess student learning outcomes for non-tradi- 2013 SLR Survey
tional IT students
32 [139] : A model for creating and sustaining information services platform communities: Lessons 2013 SLR Conceptual
learnt from open source software
33 [143] : Sustainable e-learning: toward a coherent body of knowledge 2013 NSR Scoping review
34 [89] : Performance assessment of an e-learning software system for sustainability 2013 SLR Empirical
35 [3] : Integrating Sustainability into the University: Past, Present, and Future 2013 SLR Theoretical
36 [144] : Athabasca University Reduces ICT Carbon Footprint 2012 SP Theoretical
37 [133] : Industry-University Learning Network to create competences for intelligent and sustainable 2012 SLR Case study
manufacturing: A case study

13
Requirements Engineering (2019) 24:523–543 537

Table 5  (continued)
No Paper Year Step Characteristics of paper

38 [117] : Reusability and interoperability in mobile learning: A study of current practices 2012 SLR Conceptual
39 [73] : Soft Power: A Critical Factor for the Effectiveness and Development of a School 2012 SLR Theoretical
40 [150] : Knowledge management in the smart university 2012 SLR Theoretical
41 [62] : Strategies for ongoing professional training of human resources in a globalized economy 2012 SLR Theoretical
42 [166] : A learning, training&mentoring framework (LTM)&the role of serious games to facilitate 2012 SLR Conceptual
sustainable change in the aviation industry
43 [111] : A distributed collaborative system for flexible learning content production and management 2012 SLR Empirical
44 [129] : Sustainability of a university designed and developed media annotation tool to prepare 2012 SLR Case study
learners with skills needed for future employment
45 [8] : Cloud-based online learning platforms 2012 SLR Theoretical
46 [110] : Quality in e-learning - a conceptual framework based on experiences from three interna- 2012 NSR Conceptual
tional benchmarking projects
47 [151] : Online PBL: A route to sustainability education? 2012 SLR Survey
48 [92] : Industry academia collaboration model: The design challenges 2011 SP Conceptual
49 [23] : Analysis of students’ behaviour in e-Learning system 2011 SP Theoretical
50 [120] : Will mobile learning bring a paradigm shift in higher education? 2011 SP Theoretical
51 [54] : Sustaining e-learning innovations 2011 SP Theoretical
52 [138] : Information systems architecture modelling based on loosely coupled structures: An 2011 SLR Conceptual
e-learning use case
53 [67] : A fuzzy logic-based QFD to identify key factors of e-learning design 2011 SLR Theoretical
54 [96] : Critical success attributes of transnational IT education programmes: The client perspective 2011 SLR Empirical
55 [88] : The Function of the University Libraries in Constructing Lifelong Education System 2011 SLR Theoretical
56 [13] : Technological ambidexterity in the management of national infrastructure programmes 2011 SLR Theoretical
57 [94] : Interoperability Requirements for a Sustainable Component to Support Management and 2011 SLR Theoretical
Sharing of Digital Resources
58 [95] : Design of an electronic instrumentation virtual laboratory based on free-open resources 2011 SLR Conceptual
59 [53] : Sustainability factors for e-learning initiatives 2010 SP Empirical
60 [31] : Future-focused learning via online anchored discussion, connecting learners with digital 2010 SP Empirical
artefacts, other learners, and teachers
61 [126] : Why some distance education programs fail while others succeed in a global environment 2010 SP Empirical
62 [18] : Putting ourselves in the big picture: A sustainable approach to project management for 2010 SP Case study
e-learning
63 [35] : A Reference Model for Sustainable E-Learning Service Systems: Experiences with the Joint 2010 SP Case study
University/Teradata Consortium
64 [1] : K-State’s Distributed Learning Commons: Achieving Long-Term Sustainability Through 2010 SLR Case study
Strategic Partnerships
65 [65] : Four requirements for digital case study libraries 2010 SLR Case study
66 [140] : Critical success factors in elearning ecosystems: a qualitative study 2010 NSR Survey
67 [154] : Establishing a videoconferencing infrastructure in the republic of Macedonia as an engineer- 2010 SLR Empirical
ing educational service
68 [15] : Global perspectives: aligning agendas? 2009 SP Theoretical
69 [41] : Advancing the m-learning research agenda for active, experiential learning: Four case studies 2009 SP Empirical
70 [47] : Open educational resources: New possibilities for change and sustainability 2009 SP Empirical
71 [49] : Role of instructional technology in the transformation of higher education 2009 SP Theoretical
72 [45] : Creating, handling and implementing e-learning courses using the Open source tools OLAT 2009 SP Theoretical
and eLML at the University of Zurich
73 [29] : Understanding the sustainability of a virtual community: model development and empirical 2009 SP empirical
test
74 [39] : An E-learning Ecosystem Based on Cloud Computing Infrastructure 2009 NSR Conceptual
75 [56] : On Objectives of Instructional Laboratories, Individual Assessment, and Use of Collaborative 2009 SLR Empirical
Remote Laboratories

13

538 Requirements Engineering (2019) 24:523–543

Table 5  (continued)
No Paper Year Step Characteristics of paper

76 [87] : Making classrooms socio-technical environments for supporting collaborative learning: the 2009 SLR Empirical
role of personal devices and boundary objects
77 [136] : E-Learning and its Impact to the Educational System in the Arab World 2009 SLR Theoretical
78 [112] : Adaptive open mobile learning device for the underserved 2009 SLR Conceptual
79 [66] : Lifelong learning to advance the engineer’s career 2009 SLR Theoretical
80 [124] : Sustainable e-learning, activity theory and professional development 2008 SP Theoretical
81 [128] : Designing low carbon higher education systems: Environmental impacts of campus and 2008 SP Empirical
distance learning systems
82 [93] : What is complexity theory and what are its implications for educational change? 2008 SP Theoretical
83 [38] : Open educational resources and lifelong learning 2008 SP Theoretical
84 [109] : The current status of instructional design theories in relation to today’s authoring systems 2008 SP Theoretical
85 [148] : What drives a successful e-Learning? An empirical investigation of the critical factors influ- 2008 SP Empirical
encing learner satisfaction
86 [71] : Pocket school: Exploring mobile technology as a sustainable literacy education option for 2008 SP Theoretical
underserved indigenous children in Latin America
87 [46] : Engaging E-Learning in Virtual Worlds: Supporting Group Collaboration 2008 SP Theoretical
88 [105] : Institutional perspectives: The challenges of e-learning diffusion 2008 SP Empirical
89 [10] : Strategic Planning for E-Learning in the Workplace 2008 SLR Conceptual
90 [83] : Modelling benefits-oriented costs for technology enhanced learning 2007 SP Conceptual
91 [156] : The future of E-learning: E-learning ecosystem 2007 SP Theoretical
92 [26] : E-learning ecosystem (eles)-a holistic approach for the development of more effective learn- 2007 SP Conceptual
ing environment for small-and-medium sized enterprises (smes)
93 [149] : Building re-configurable blendedlearning arrangements 2007 SP Conceptual
94 [146] : Technology supported learning Tensions between innovation, and control and organisational 2007 SP Theoretical
and professional cultures
95 [99] : Mobile learning: A framework and evaluation 2007 SP Empirical
96 [84] : Pervasive, lifestyle-integrated mobile learning for distance learners: an analysis and unex- 2007 SP Empirical
pected results from a podcasting study
97 [28] : The factors influencing members’ continuance intentions in professional virtual communities 2007 SP Empirical
a longitudinal study
98 [101] : The socio-economic dimensions of ICT-driven educational change 2007 SLR Theoretical
99 [102] : Policy instruments for sustainability-oriented organizational learning 2007 SLR Theoretical
100 [14] : On-line learning networks: Framework and scenarios 2007 SLR Conceptual
101 [6] : Personal Learning Environments-the future of e-Learning? 2007 NSR Commentary
102 [40] : Models for sustainable open educational resources 2007 NSR Theoretical
103 [43] : Sustaining a community computing infrastructure for online teacher professional develop- 2007 SLR Case study
ment: A case study of designing tapped in
104 [118] : Integrating synchronous and asynchronous internet distributed education for maximum 2007 SLR Empirical
effectiveness
105 [76] : Advancing sustainability of open educational resources 2007 NSR Conceptual
106 [50] : Distance learning: Technologies; Enabling learning at own place, own pace, own time 2006 SP Theoretical
107 [61] : Open educational resources: Opportunities and challenges 2006 SP Theoretical
108 [30] : Understanding knowledge sharing in virtual communities: An integration of social capital and 2006 SP Empirical
social cognitive theories
109 [135] : Implementing a university e-learning strategy: levers for change within academic schools 2006 SP Empirical
110 [37] : What makes an open education program sustainable? The case of Connexions 2006 SP Theoretical
111 [44] : eLML, the eLesson Markup Language: Developing Sustainable e-Learning Content Using an 2006 SP Theoretical
Open Source XML Framework
112 [9] : Implementing and sustaining e-learning in the workplace 2006 SLR Theoretical
113 [108] : Whatever Happened to the Social Dimension?“ Aspects of Learning in a Distance-based 2006 SLR Theoretical
Teacher Training Programme”

13
Requirements Engineering (2019) 24:523–543 539

Table 5  (continued)
No Paper Year Step Characteristics of paper

114 [141] : Informatics system comprehension: A learner-centred cognitive approach to networked 2006 SLR Conceptual
thinking
115 [27] : Personalized e-learning system using item response theory 2005 SP Theoretical
116 [55] : Repurposing learning objects: a sustainable alternative? 2005 SP Theoretical
117 [130] : Flying not flapping: a strategic framework for e-learning and pedagogical innovation in 2005 SP Conceptual
higher education institutions
118 [57] : Lifelong learning for equity and social cohesion: A new challenge for Higher Education 2005 SP Theoretical
119 [7] : It’s installed...now get on with it! Looking beyond the software to the cultural change 2005 SP Empirical
120 [60] : Social capital, knowledge management, and sustained superior performance 2005 SP Theoretical
121 [70] : Lifelong learning really matters for elementary education in the 21st century 2005 SLR Theoretical
122 [85] : Reusing online resources: a sustainable approach to e-learning 2003 NSR Theoretical
123 [86] : Supporting sustainable e-learning 2003 NSR Theoretical
124 [131] : Sustainable implementation of e-learning as a change process at universities 2003 NSR Case study

SLR systematic literature review, NSR non-systematic review, SP snowballing procedure

References 14. Bottino RM (2007) On-line learning networks: framework and

scenarios. Educ Inf Technol 12(2):93–105
15. Bourn D, Shiel C (2009) Global perspectives: aligning agendas?
1. Allen DB, Gould RA, Littrell LA, Schillie JE (2010) K-state’s
Environ Educ Res 15(6):661–677
distributed learning commons: achieving long-term sustain-
16. Brereton P, Kitchenham BA, Budgen D, Turner M, Khalil M
ability through strategic partnerships. Coll. Undergrad. Libr.
(2007) Lessons from applying the systematic literature review
17(2):160–176
process within the software engineering domain. J Syst Softw
2. Alrashidi A (2013) An exploration of e-learning benefits for
80(4):571–583
Saudi Arabia: Toward policy reform. PhD thesis, University of
17. Breslow L, Pritchard DE, DeBoer J, Stump GS, Ho AD, Sea-
La Verne
ton DT (2013) Studying learning in the worldwide classroom:
3. Amador F, Oliveira CP (2013) Integrating sustainability into the
research into edx’s first mooc. Res Pract Asses 8:13–25
university: past, present, and future. In: Sustainability assessment
18. Buchan J (2010) Putting ourselves in the big picture: a sustaina-
tools in higher education institutions: mapping trends and good
ble approach to project management for e-learning. Int J E-Learn
practices around the world, Springer, pp 65–78
Distance Educ 24(1):55–76
4. Anderson T, Dron J (2012) Learning technology through three
19. Burton SL, Harris HR, Burrell DN, Brown-Jackson KL, Bessette
generations of technology enhanced distance education peda-
D, McClintock R, Lu S, White YW (2014) Educational edifices
gogy. Eur J Open Distance e-learn 15(2):1–14
need a mobile strategy to fully engage in learning activities. In:
5. Assembly UG (2015) Transforming our world: the 2030 agenda
Implications of social media use in personal and professional
for sustainable development. Technical report, A/RES/70/1, 21
settings, pp 284–309
October
20. Calero C, Piattini M (2015) Green in software engineering.
6. Attwell G (2007) Personal learning environments-the future of
Springer, Berlin
e-learning? eLearning Pap 2(1):1–8
21. Calero C, Piattini M (2017) Puzzling out software sustainability.
7. Bell M, Bell W (2005) It’s installed... now get on with it! looking
Sustain Comput Inf Syst 16:117–124
beyond the software to the cultural change. Br J Educ Technol
22. Calero C, Bertoa MF, Moraga MÁ (2013) Sustainability and
36(4):643–656
quality: icing on the cake. In: Proceedings of the 2nd interna-
8. Bensch S, Rager M (2012) Cloud-based online learning plat-
tional workshop on requirements engineering for sustainable
forms. In: Business information systems workshops, vol 127
systems, CEUR-WS.org
LNBIP, pp 165–176
23. Cápay M, Mesárošová M, Balogh Z (2011) Analysis of students’
9. Berge ZL, Giles L (2006) Implementing and sustaining e-learn-
behaviour in e-learning system. In: Proceedings of the 22nd
ing in the workplace. Int J Inf Commun Technol Educ (IJICTE)
EAEEIE annual conference (EAEEIE), 2011. IEEE, pp 1–6
2(4):64–75
24. Casquero O, Portillo J, Ovelar R, Benito M, Romo J (2010) iPLE
10. Berge ZL, Giles L (2008) Strategic planning for e-learning in
network: an integrated eLearning 2.0 architecture from a univer-
the workplace. In: Adapting information and communication
sity’s perspective. Interact Learn Environ 18(3):293–308
technologies for effective education. IGI Global, pp 257–270
25. Castro-Schez JJ, Redondo MA, Gallardo J, Jurado F et al (2012)
11. Berkhout F, Hertin J (2001) Impacts of information and commu-
Designing and developing software for educative virtual labora-
nication technologies on environmental sustainability: specula-
tories with language processing techniques: lessons learned in
tions and evidence. Report to the OECD, Brighton, p 21
practical experiments. J Res Pract Inf Technol 44(3):289–308
12. Betz S, Caporale T (2014) Sustainable software system engineer-
26. Chang V, Guetl C (2007) E-learning ecosystem (eles)-a holistic
ing. In: The fourth international conference on big data and cloud
approach for the development of more effective learning environ-
computing (BdCloud). IEEE, pp 612–619
ment for small-and-medium sized enterprises (SMES). In: Digi-
13. Bhat JS (2011) Technological ambidexterity in the management
tal EcoSystems and technologies conference, 2007. DEST’07.
of national infrastructure programmes. J Asian Public Policy
Inaugural IEEE-IES. IEEE, pp 420–425
4(3):350–356

13

540 Requirements Engineering (2019) 24:523–543

27. Chen CM, Lee HM, Chen YH (2005) Personalized e-learning 46. Franceschi KG, Lee RM, Hinds D (2008) Engaging e-learning in
system using item response theory. Comput Educ 44(3):237–255 virtual worlds: supporting group collaboration. In: Proceedings
28. Chen IY (2007) The factors influencing members’ continuance of the 41st annual Hawaii international conference on system
intentions in professional virtual communitiesa longitudinal sciences. IEEE, pp 7–7
study. J Inf Sci 33(4):451–467 47. Friesen N (2009) Open educational resources: New possibilities
29. Cheung CM, Lee MK (2009) Understanding the sustainability for change and sustainability. Int Rev Res Open Distrib Learn
of a virtual community: model development and empirical test. 10(5):1–13
J Inf Sci 35(3):279–298 48. Garg K, Varma V (2015) Systemic requirements of a software
30. Chiu CM, Hsu MH, Wang ET (2006) Understanding knowledge engineering learning environment. In: Proceedings of the 8th
sharing in virtual communities: an integration of social capital India software engineering conference, ISEC ’15. ACM, pp
and social cognitive theories. Decis Supp Syst 42(3):1872–1888 147–155
31. Colasante M (2010) Future-focused learning via online anchored 49. Garrison DR, Akyol Z (2009) Role of instructional technology
discussion, connecting learners with digital artefacts, other learn- in the transformation of higher education. J Comput Higher Educ
ers, and teachers. In: Ascilite 2010. Ascilite, pp 211–221 21(1):19
32. Colomo-Palacios R, Messnarz R, Siakas K, Palosi D, Coakley 50. Georgiadou E, Siakas KV (2006) Distance learning: technolo-
D (2014) Using social media as a tool for business improvement gies; enabling learning at own place, own pace, own time. In:
and certification of knowledge workers. J Softw Evol Process Proceedings of the 11th international conference on software pro-
26(9):791–798 cess improvement-research into education and training,(INSPIRE
33. Dagger D, Conlan O, Wade V (2005) Fundamental requirements 2006) learning and teaching issues in software quality, April,
of personalised elearning development environments. In: Rich- Southampton, UK, pp 29–40
ards G (ed) E-Learn: world conference on e-learning in corpo- 51. Ghirardini B (2011) E-learning methodologies A guide for
rate, government, healthcare, and higher education 2005, associa- designing and developing e-learning courses. Food and Agri-
tion for the advancement of computing in education (AACE), pp culture Organization of the United Nations
2746–2754 52. Goodland R (2002) The concept of environmental sustainability.
34. Dagger D, O’Connor A, Lawless S, Walsh E, Wade V (2007) Annu Rev Ecol Syst 26(1):1–24
Service-oriented e-learning platforms: from monolithic systems 53. Gunn C (2010) Sustainability factors for e-learning initiatives.
to flexible services. Internet Comput 11(3):28–35 ALT-J 18(2):89–103
35. Demirkan H, Goul M, Gros M (2010) A reference model for 54. Gunn C (2011) Sustaining e-learning innovations, chang-
sustainable e-learning service systems: experiences with the ing demands, changing directions. Proc Ascilite Hobart
joint university/teradata consortium. Decis Sci J Innov Educ 2011:509–519
8(1):151–189 55. Gunn C, Woodgate S, OGrady W (2005) Repurposing learning
36. Demski J (2012) Rebuilding the lms for the 21st century. Campus objects: a sustainable alternative? ALT-J 13(3):189–200
Technol 25(8):34 56. Gustavsson I, Nilsson K, Zackrisson J, Garcia-Zubia J, Hernan-
37. Dholakia U, King J, Baraniuk R (2006) What makes an open dez-Jayo U, Nafalski A, Nedic Z, Gol O, Machotka J, Pettersson
education program sustainable? the case of connexions. In: Open M, Lago T, Hkansson L (2009) On objectives of instructional
education conference, vol 2340 laboratories, individual assessment, and use of collaborative
38. Dinevski D (2008) Open educational resources and lifelong remote laboratories. IEEE Trans Learn Technol 2(4):263–274
learning. In: 30th international conference on information tech- 57. Halimi S (2005) Lifelong learning for equity and social cohesion:
nology interfaces ITI 2008, 2008. IEEE, pp 117–122 a new challenge for higher education. Lifelong Learn Distance
39. Dong B, Zheng Q, Yang J, Li H, Qiao M (2009) An e-learning Higher Educ Commonw Learn /UNESCO 2005:11–22
ecosystem based on cloud computing infrastructure. In: Interna- 58. Halkias D, Mills GT (2008) Distance education in support of
tional conference on advanced learning technologies. IEEE, pp lifelong learning: the case of the hellas alive web platform in
125–127 building greek language learning communities. In: Proceedings
40. Downes S (2007) Models for sustainable open educational of the 12th international conference on computers. ACM, pp
resources. Interdiscip J E-Learn Learn Objects 3(1):29–44 57–60
41. Dyson LE, Litchfield A, Lawrence E, Raban R, Leijdekkers P 59. Hine N, Petersen F, Pluke M, Sund T (2008) Standardization
(2009) Advancing the m-learning research agenda for active, work on personalized ehealth systems. In: Proceedings of the
experiential learning: four case studies. Aust J Educ Technol 30th annual international conference of the IEEE engineering
25(2):250–267 in medicine and biology society. IEEE, pp 1518–1520
42. Ellis RA (2016) Students approaches to groupwork in a blended 60. Hoffman JJ, Hoelscher ML, Sherif K (2005) Social capital,
course, associations with perceptions of the online environment knowledge management, and sustained superior performance.
and academic achievement-when is learning engaged? Educ Inf J Knowl Manag 9(3):93–100
Technol 21(5):1095–1112 61. Hylén J (2006) Open educational resources: opportunities and
43. Farooq U, Schank P, Harris A, Fusco J, Schlager M (2007) Sus- challenges. In: Proceedings of open education
taining a community computing infrastructure for online teacher 62. Iatagan M (2012) Strategies for ongoing professional training
professional development: a case study of designing tapped in. of human resources in a globalized economy. Human Capital
Comput Support Cooper Work (CSCW) 16(4–5):397–429 and Resources: Developments, Management and Strategies
44. Fisler J, Bleisch S (2006) elml, the elesson markup language: 63. ISO/IEC 25010 (2011) Systems and software engineering
developing ustainable e-learning content using an open source software product quality requirements and evaluation (square)
xml framework. In: In WEBIST 2006-international conference software product quality and system quality in use models
on web information systems and technologies, 11th–13th April 64. James P (2014) Urban sustainability in theory and practice:
2006 (Setubal, Citeseer) circles of sustainability. Routledge, London
45. Fisler J, Schneider F (2009) Creating, handling and implementing 65. Jiang H, Ganoe C, Carroll JM (2010) Four requirements for
e-learning courses using the open source tools olat and elml at digital case study libraries. Educ Inf Technol 15(3):219–236
the university of zurich. In: Proceedings of the world congress 66. Jin G, Law R (2009) Lifelong learning to advance the engi-
on engineering and computer science, vol 1 neer’s career. Ann Conf Can Soc Civil Eng 1:50–59

13
Requirements Engineering (2019) 24:523–543 541

67. Kazancoglu Y, Aksoy M (2011) A fuzzy logic-based qfd to 88. Lizhong T, Aichen Z, Yongjie S (2011) The function of the uni-
identify key factors of e-learning design. Proc Soc Beha Sci versity libraries in constructing lifelong education system. Adv
28:322–327 Intell Soft Comput 109:341–346
68. Keele S (2007) Guidelines for performing systematic literature 89. Mahmood T, Hafeez K (2013) Performance assessment of an
reviews in software engineering. Technical report, Department e-learning software system for sustainability. Int J Qual Serv Sci
of Computer Science, University of Durham 5(2):208–229
69. Keengwe J, Malapile S (2014) Factors influencing technology 90. Mahmoud SS, Ahmad I (2013) A green model for sustainable
planning in developing countries: a literature review. Educ Inf software engineering. Int J Softw Eng Appl 7(4):1738–9984
Technol 19(4):703–712 91. Manca S, Ranieri M (2017) Implications of social network sites
70. Kendall M (2005) Lifelong learning really matters for ele- for teaching and learning. Where we are and where we want to
mentary education in the 21st century. Educ Inf Technol go. Educ Inf Technol 22(2):605–622
10(3):289–296 92. Manuja M, et al (2011) Industry academia collaboration model:
71. Kim P, Miranda T, Olaciregui C (2008) Pocket school: exploring the design challenges. In: 2011 24th IEEE-CS Conference on
mobile technology as a sustainable literacy education option for software engineering education and training (CSEE&T). IEEE,
underserved indigenous children in latin america. Int J Educ Dev pp 111–120
28(4):435–445 93. Mason M (2008) What is complexity theory and what are its
72. Kitchenham B, Dyba T, Jorgensen M (2004) Evidence-based implications for educational change? Educ Philos Theory
software engineering. In: 26th International conference on soft- 40(1):35–49
ware engineering. ACM, pp 273–281 94. Memmel M (2011) Interoperability requirements for a sustain-
73. Ko HM (2012) Soft power: a critical factor for the effectiveness able component to support management and sharing of digital
and development of a school. Lecture Notes in Computer Science resources. In: 1st Workshop on exploring fitness and evolvability
(including subseries Lecture Notes in Artificial Intelligence and of personal learning environments, CEUR-WS.org, pp 25–29
Lecture Notes in Bioinformatics) 7513 LNCS:414–420 95. Meneses G (2011) Design of an electronic instrumentation
74. Koçak SA, Alptekin GI, Bener AB (2015) Integrating environ- virtual laboratory based on free-open resources. In: 2011 6th
mental sustainability in software product quality. In: Proceedings Colombian computing congress (CCC). IEEE, pp 1–6
of the fourth international workshop on requirements engineering 96. Miliszewska I, Sztendur E (2011) Critical success attributes of
for sustainable systems (RE4SuSy), CEUR-WS.org, pp 17–24 transnational it education programmes: the client perspective. J
75. Konting MM (2012) Leadership development for sustainability Inf Technol Educ Res 10(1):123–137
of e-learning. Proc Soc Beh Sci 67:312–321 97. Mocigemba D (2006) Sustainable computing. Poiesis Praxis
76. Koohang A, Harman K (2007) Advancing sustainability of open 4(3):163–184
educational resources. Issues Inf Sci Inf Technol 4:535–544 98. Mohan L, Raman P, Choppella V, Reddy YR (2017) A crowd-
77. Koshkin AP, Rassolov IM, Novikov AV (2016) Monitoring social sourcing approach for quality enhancement of elearning systems.
media: students satisfaction with university administration activi- In: Proceedings of the 10th innovations in software engineering
ties. Educ Inf Technol 22:2499–2522 conference, ISEC ’17, ACM, pp 188–194
78. Kruchten P (2015) Lifelong learning for lifelong employment. 99. Motiwalla LF (2007) Mobile learning: a framework and evalua-
IEEE Softw 32(4):85–87 tion. Comput Educ 49(3):581–596
79. Kumar S, Buyya R (2012) Green cloud computing and environ- 100. Mridha GMN, Erlandsson B, Khan A, Islam M, Sultana N, Reza
mental sustainability. Harnessing green it: principles and prac- S, Srinivas M (2013) E-learning for empowering the rural people
tices, pp 315–339 in bangladesh opportunities and challenges. In: International con-
80. Lago P, Jansen T (2011) Creating environmental awareness in ference on e-learning and e-technologies in education (ICEEE).
service oriented software engineering. In: Service-oriented com- IEEE, pp 323–328
puting. Springer, pp 181–186 101. Müller J, Gil JMS, Hernández F, Giró X, Bosco A (2007) The
81. Lago P, Koçak SA, Crnkovic I, Penzenstadler B (2015) Framing socio-economic dimensions of ict-driven educational change.
sustainability as a property of software quality. Commun ACM Comput Educ 49(4):1175–1188
58(10):70–78 102. Müller M, Siebenhüner B (2007) Policy instruments for sus-
82. van Lamsweerde A (2008) Requirements engineering: From tainability-oriented organizational learning. Bus Strategy Envi-
craft to discipline. In: Proceedings of the 16th ACM SIGSOFT ron 16(3):232–245
international symposium on foundations of software engineering. 103. Naumann S, Dick M, Kern E, Johann T (2011) The GREEN-
ACM, pp 238–249 SOFT model: a reference model for green and sustainable
83. Laurillard D (2007) Modelling benefits-oriented costs for tech- software and its engineering. Sustain Comput Inform Syst
nology enhanced learning. Higher Educ 54(1):21–39 1(4):294–304
84. Lee MJ, Chan A (2007) Pervasive, lifestyle-integrated mobile 104. Neila R, Rabai LBA (2014) Deploying suitable countermeas-
learning for distance learners: an analysis and unexpected results ures to solve the security problems within an e-learning envi-
from a podcasting study. Open Learn 22(3):201–218 ronment. In: Proceedings of the 7th international conference
85. Littlejohn A (2003a) Reusing online resources: a sustainable on security of information and networks, SIN ’14, ACM, pp
approach to e-learning. Psychology Press, London 33–38
86. Littlejohn A (2003b) Supporting sustainable e-learning. Assoc 105. Nichols M (2008) Institutional perspectives: the challenges of
Learn Technol J 11(3):88–102 e-learning diffusion. Br J Educ Technol 39(4):598–609
87. Liu CC, Chung CW, Tao SY (2009) Making classrooms socio- 106. Nunes EP, Luz AR, Lemos EM, Maciel C, dos Anjos AM, Borges
technical environments for supporting collaborative learning: the LC, Nunes C (2016) Mobile serious game proposal for envi-
role of personal devices and boundary objects. In: Proceedings ronmental awareness of children. In: The frontiers in education
of the 9th international conference on computer supported col- conference (FIE), IEEE, pp 1–8
laborative learning, international society of the learning sciences, 107. Nwokediuko A (2012) Advancing e-learning in african native
CSCL’09, pp 320–324 communities: the language factor. In: International conference
on education and e-learning innovations (ICEELI). IEEE, pp 1–4

13

542 Requirements Engineering (2019) 24:523–543

108. Olofsson AD, Lindberg JO (2006) ”Whatever happened to 128. Roy R, Potter S, Yarrow K (2008) Designing low carbon higher
the social dimension?” aspects of learning in a distance-based education systems: environmental impacts of campus and dis-
teacher training programme. Educ Inf Technol 11(1):7–20 tance learning systems. Int J Sustain Higher Educ 9(2):116–130
109. O’Neil AF (2008) The current status of instructional design theo- 129. Ruyters M, Colasante M, Douglas K, Mandarano G (2012) Sus-
ries in relation to today’s authoring systems. Br J Educ Technol tainability of a university designed and developed media annota-
39(2):251–267 tion tool to prepare learners with skills needed for future employ-
110. Ossiannilsson E, Landgren L (2012) Quality in e-learning-a con- ment. ASCILITE–annual conference of the Australian Society
ceptual framework based on experiences from three international for Computers in Tertiary Education
benchmarking projects. J Comput Assist Learn 28(1):42–51 130. Salmon G (2005) Flying not flapping: a strategic framework for
111. Pardo A, Fisteus JA, Kloos CD et al (2012) A distributed col- e-learning and pedagogical innovation in higher education insti-
laborative system for flexible learning content production and tutions. ALT-J 13(3):201–218
management. J Res Pract Inf Technol 44(2):203–221 131. Schoenwald I (2003) Sustainable implementation of e-learning
112. Park CB, Choi H, Kim P (2009) Adaptive open mobile learning as a change process at universities. Online Educ 1:1–6
device for the underserved. In: ICCE’09 international conference 132. Scoppio G, Luyt I (2017) Mind the gap: enabling online faculty
on consumer electronics, digest of technical papers, IEEE and instructional designers in mapping new models for quality
113. Pellas N (2014) The influence of computer self-efficacy, meta- online courses. Educ Inf Technol 22:725–746
cognitive self-regulation and self-esteem on student engagement 133. Secundo G, Passiante G, Romano A (2012) Industry-university
in online learning programs: Evidence from the virtual world learning network to create competences for intelligent and sus-
of second life. Comput Hum Behav 35:157–170. https​ ://doi. tainable manufacturing: a case study. In: International conference
org/10.1016/j.chb.2014.02.048 on information technology based higher education and training
114. Pellas N (2016) Bolstering the quality and integrity of online (ITHET). IEEE, pp 1–8
collaborative university- level courses via an open sim stan- 134. Secundo G, Passiante G, Romano A, Moliterni P (2013) Devel-
dalone server in conjunction with sloodle. Educ Inf Technol oping the next generation of engineers for intelligent and sustain-
21(5):1007–1032 able manufacturing: a case study. Int J Eng Educ 29(1):248–262
115. Penzenstadler B (2014) Infusing green: Requirements engineer- 135. Sharpe R, Benfield G, Francis R (2006) Implementing a uni-
ing for green in and through software systems. In: Proceedings versity e-learning strategy: levers for change within academic
of the third international workshop on requirements engineering schools. Res Learn Technol 14(2):135–151
for sustainable systems, (RE4SuSy), CEUR-WS.org 136. Shehabat IM, Mahdi SA (2009) E-learning and its impact to the
116. Penzenstadler B, Femmer H (2013) A generic model for sustain- educational system in the arab world. In: International conference
ability with process- and product-specific instances. In: Proceed- on information management and engineering. IEEE, pp 220–225
ings of the workshop on green in/by software engineering. ACM, 137. Shen L, LeClair J (2013) Use of sustainable and systematic plan
pp 3–8 to assess student learning outcomes for non-traditional it stu-
117. Pettersson O, Vogel B (2012) Reusability and interoperability dents. In: ASEE annual conference and exposition, American
in mobile learning: a study of current practices. In: The seventh Society for Engineering Education
international conference on wireless. Mobile and ubiquitous tech- 138. Sousa JL (2011) Information systems architecture modeling
nology in education. IEEE, pp 306–310 based on loosely coupled structures: an e-learning use case. In:
118. Pullen JM, Snow C (2007) Integrating synchronous and asyn- 9th International conference on emerging elearning technologies
chronous internet distributed education for maximum effective- and applications (ICETA). IEEE, pp 195–198
ness. Educ Inf Technol 12(3):137–148 139. Sowe SK, Zettsu K, Murakami Y (2013) A model for creating
119. Rahanu H, Georgiadou E, Siakas K, Ekert D, Messanarz R, and sustaining information services platform communities: les-
Abeysinghe G (2015) Towards relating delivery methods and sons learnt from open source software. In: Proceedings of ITU
examination success: lessons learned from the valo llp project kaleidoscope: building sustainable communities (K-2013). IEEE,
case study. J Softw Evol Process 27(8):555–564 pp 1–8
120. Rajasingham L (2011) Will mobile learning bring a paradigm 140. Sridharan B, Deng H, Corbitt B (2010) Critical success factors
shift in higher education? Educ Res Int 2011:1–10. https​://doi. in elearning ecosystems: a qualitative study. J Syst Inf Technol
org/10.1155/2011/52849​51 12(4):263–288
121. Randelin M, Saaranen T, Naumanen P, Louhevaara V (2013) 141. Stechert P (2006) Informatics system comprehension: a learner-
Towards sustainable well-being in smes through the web-based centred cognitive approach to networked thinking. Educ Inf
learning program of ergonomics. Educ Inf Technol 18(1):95–111 Technol 11(3):305–318
122. Razavian M, Procaccianti G, Tamburri DA, et al (2014) Four- 142. Stefanovic M (2013) The objectives, architectures and effects of
dimensional sustainable e-services. In: Proceedings of the 28th distance learning laboratories for industrial engineering educa-
EnviroInfo, Shaker-Verlag AG tion. Comput Educ 69:250–262
123. Richey RC (2008) Reflections on the 2008 AECT definitions of 143. Stepanyan K, Littlejohn A, Margaryan A (2013) Sustainable
the field. TechTrends 52(1):24–25 e-learning: toward a coherent body of knowledge. J Educ Tech-
124. Robertson I (2008) Sustainable e-learning, activity theory and nol Soc 16(2):91–102
professional development. In: Ascilite, ASCILITE (Melbourne, 144. Stewart B, Khare A (2012) Athabasca university reduces ict car-
Australia), pp 819–826 bon footprint. GSTF J Comput 1(4):70–75
125. Roher K, Richardson D (2013) Sustainability requirement pat- 145. Stewart B, Khare A (2015) elearning and the sustainable cam-
terns. In: The third international workshop on requirements pat- pus. In: Transformative approaches to sustainable development
terns (RePa). IEEE, pp 8–11 at universities. Springer, pp 291–305
126. Rovai AP, Downey JR (2010) Why some distance education pro- 146. Stiles M, Yorke J (2007) Technology supported learning tensions
grams fail while others succeed in a global environment. Internet between innovation, and control and organisational and profes-
Higher Educ 13(3):141–147 sional cultures. J Organ Transform Soc Change 3(3):251–267
127. Roy A (2012) eLearning: tool to ensure growth and sustainability 147. Suhonen J, Sutinen E (2014) The four pillar model-analysing
of SMEs. In: Proceedings of the 11th European conference on the sustainability of online doctoral programmes. TechTrends
e-learning. ECEL, p 480 58(4):81–88

13
Requirements Engineering (2019) 24:523–543 543

148. Sun PC, Tsai RJ, Finger G, Chen YY, Yeh D (2008) What 158. Vovides Y, Chale SB, Gadhula R, Kebaetse MB, Nigussie NA,
drives a successful e-learning? an empirical investigation of the Suleman F, Tibyampansha D, Ibrahim GR, Ntabaye M, Frehywot
critical factors influencing learner satisfaction. Comput Educ S et al (2014) A systems approach to implementation of elearning
50(4):1183–1202 in medical education: five mepi schools’ journeys. Acad Med
149. Thomas L, Trapp S (2007) Building re-configurable blend- 89(8):S102–S106
edlearning arrangements. In: Balkan conference in informatics 159. Wang R, Scown P, Urquhart C, Hardman J (2014) Tapping the
(BCI 2007), pp 271–280 educational potential of facebook: guidelines for use in higher
150. Tikhomirova N, Tikhomirov V, Maksimova V, Telnov Y (2012) education. Educ Inf Technol 19(1):21–39
Knowledge management in the smart university. Proc Eur Conf 160. Weichhart G (2015) Supporting the evolution and interoperability
Knowl Manag 2:1172–1178 of organisational models with e-learning technologies. Ann Rev
151. Tomkinson B, Hutt I (2012) Online pbl: a route to sustainability Control 39:118–127
education? Campus-Wide Inf Syst 29(4):291–303 161. Winfree T, Goldacre P, Sherkat M, Graham P, Mendoza A, Miller
152. Toppin IN, Toppin SM (2016) Virtual schools: the chang- T (2017) Learning for low carbon living: the potential of mobile
ing landscape of k-12 education in the us. Educ Inf Technol learning applications for built environment trades and profession-
21(6):1571–1581 als in Australia. Proc Eng 180:1773–1783
153. Törngren M, Bensalem S, McDermid J, Passerone R, Sangio- 162. Wohlin C (2014) Guidelines for snowballing in systematic lit-
vanni-Vincentelli A, Schätz B (2015) Education and training erature studies and a replication in software engineering. In:
challenges in the era of cyber-physical systems: beyond tradi- Proceedings of the 18th international conference on evaluation
tional engineering. In: Proceedings of the WESE’15: workshop and assessment in software engineering, EASE ’14. ACM, pp
on embedded and cyber-physical systems education, WESE’15. 38:1–38:10
ACM. pp 8:1–8:5 163. Wohlin C, Runeson P, Höst M, Ohlsson MC, Regnell B (2012)
154. Trajkovik V, Caporali E, Palmisano E, Valdiserri J (2010) Experimentation in software engineering. Springer, Berlin
Establishing a videoconferencing infrastructure in the republic 164. Yigit T, Isik AH, Ince M (2014) Web-based learning object
of macedonia as an engineering educational service. In: Joint selection software using analytical hierarchy process. Softw IET
International IGIP-SEFI annual conference, SEFI 8(4):174–183
155. Tuparov G, Kostadinova H, Tuparova D, Raykova M (2014) 165. Zheng Q, He H, Ma T, Xue N, Li B, Dong B (2014) Big log
Approaches for competencies assessment in open source e-learn- analysis for e-learning ecosystem. In: The 11th international
ing environments. In: Global engineering education conference conference on e-business engineering. IEEE, pp 258–263
(EDUCON). IEEE, pp 529–532 166. Zon G, Corrigan S, McDonald N, Maij A (2012) A learning,
156. Uden L, Wangsa IT, Damiani E (2007) The future of e-learning: training & mentoring framework (ltm) & the role of serious
E-learning ecosystem. In: 2007 Inaugural IEEE-IES Digital Eco- games to facilitate sustainable change in the aviation industry. In:
Systems and technologies conference, pp 113–117. https​://doi. 11th International probabilistic safety assessment and manage-
org/10.1109/DEST.2007.37195​5 ment conference and the annual european safety and reliability
157. Vogel B, Kurti A, Milrad M, Johansson E, Müller M (2014) conference, vol 6, pp 5044–5051
Mobile inquiry learning in sweden: development insights on
interoperability, extensibility and sustainability of the LETS GO
software system. Educ Technol Soc 17(2):43–57

13