See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/221307308

Research methods in computer science

Conference Paper · September 2011

DOI: 10.1109/ICSM.2011.6080841 · Source: DBLP

CITATIONS READS

14 40,084

1 author:

Serge Demeyer
University of Antwerp
330 PUBLICATIONS 7,982 CITATIONS

SEE PROFILE

All content following this page was uploaded by Serge Demeyer on 16 June 2022.

The user has requested enhancement of the downloaded file.

 Research Methods in Computer Science
Serge Demeyer
University of Antwerp
Department of Mathematics and Computer Science
Middelheimlaan 1 B-2020 ANTWERPEN
http://www.win.ua.ac.be/∼sdemey/

I. O RIGINS OF C OMPUTER S CIENCE R ESEARCH II. P UBLICATION C ULTURE

The computing science field has a long tradition of using
Computer Science as a research discipline has always strug- conference publications as the primary unit of dissemination,
gled with its identity. On the one hand, it is a field deeply which is in contrast with other scientific disciplines (physics,
rooted in mathematics which resulted in strong theories.1 For biology, . . . ) where emphasis is on journal publications. This
example, there is computational complexity theory (turing is a hotly debated issue within the community (see among
machines, the halting problem), database theory (the relational others [8], [9], [10]) and will not change in the foreseeable
model, expresive power of query languages), formal language future.
theory (the chomsky hierarchy, well-formedness, formal se- Indeed, scientific publications drive today’s academic re-
mantics). On the other hand, it is a field deeply rooted in search, yet scientific publications are not a goal in themselves!
engineering which resulted in machines that have completely Rather they are a means towards a goal, namely recording and
warped our society: the von Neumann architecture (the basis disseminating the contributions to human knowledge — the
for digital computers), parallel processors (the new generation proverbial “standing on the shoulders of giants”. Nevertheless,
of multi-core machines), distributed computers (a prerequisite quantifying scientific output is a common practice these days
for the success of the internet and recent phenomena like grid with scientific publications being the most visible aspect.
computing). Consequently, computer science has inherited its Hence there is a tremendous pressure on PhD students to
research methods from the same disciplines: on the one hand, organise a PhD around a collection of peer reviewed articles.
the mathematical approach with axioms, postulates and proofs; A common template is to have (a) several workshop papers
on the other hand the engineering approach with quantification, outlining research idea(s) in the beginning of the process;
measurements and comparison. (b) two conference publications halfway in the PhD process
Software Engineering research in particular has suffered detailing some of the findings; (c) a journal publications
from this identity crisis, and several authors have argued the summarizing the main contribution for archival reference.
need for stronger research methods [1], [2], [3]. Moreover, Given this template, a (computer) scientist is expected to
software engineering research —with its emphasis on pro- write a significant number of peer reviewed papers. Conse-
cesses and team work— must also take into account group quently, a PhD student should understand the peer review
dynamics and cognitive factors, hence borrows research meth- process, in order to increase the chances of success.
ods from sociology and psychology as well [4], [5]. Certainly,
with innovations like distributed development and open source III. G OALS FOR THE T UTORIAL
release, software engineering is at the forefront of introducing This tutorial is aimed at PhD students who want to have a
new communication paradigms, hence is itself a testbed for better grasp on what exactly is “good” research. We explore
experiments in social sciences. the role of research methods in computer science, draw-
Consequently, case studies are a dominant research method ing upon practical examples from empirical approaches in
within software engineering [6]. This should come as no software engineering. Given the need for stronger research
surprise, since case studies are particularly useful to “investi- methods (Section I) and the ongoing pressure on publication
gate a contemporary phenomenon within its real-life context; output (Section II) the tutorial wants PhD students to . . .
when the boundaries between phenomenon and context are • Name and explain different approaches to conduct com-
not clearly evident” [7]. Nevertheless, the term “case study” puter science research (i.e. feasibility study, case study,
is used quite liberally, ranging from small toy examples for comparative study, literature survey).
illustrative purposes to full blown benchmarks with all the re- • Understand the peer reviewing process inherent in aca-
quired threats to validity. Consequently, a PhD student should demic research, including the implications it has for their
understand the full spectrum of what computer scientists refer own research (i.e. writing papers).
to as a “case study”.
ACKNOWLEDGMENT
1 “theory”should be interpreted here in the scientific sense of the word, i.e. This work has been sponsored by the Research Foundation Flan-
an important feature of nature supported by facts gathered over time. ders (FWO) sponsoring a sabbatical leave of Prof. Serge Demeyer.
 R EFERENCES
[1] M. V. Zelkowitz and D. R. Wallace, “Experimental models for validating
technology,” Computer, vol. 31, pp. 23–31, May 1998.
[2] A. Höfer and W. F. Tichy, “Status of empirical research in software
engineering,” in Empirical Software Engineering Issues. Critical Assess-
ment and Future Directions, ser. Lecture Notes in Computer Science,
V. Basili, D. Rombach, K. Schneider, B. Kitchenham, D. Pfahl, and
R. Selby, Eds. Springer Berlin / Heidelberg, 2007, vol. 4336, pp. 10–
19.
[3] M. Shaw, “What makes good research in software engineering?” In-
ternational Journal on Software Tools for Technology Transfer (STTT),
vol. 4, pp. 1–7, 2002.
[4] D. E. Perry, A. A. Porter, and L. G. Votta, “Empirical studies of software
engineering: a roadmap,” in Proceedings of the Conference on The
Future of Software Engineering, ser. ICSE ’00. New York, NY, USA:
ACM, 2000, pp. 345–355.
[5] S. Hanenberg, “Faith, hope, and love: an essay on software science’s
neglect of human factors,” SIGPLAN Notices, vol. 45, pp. 933–946,
2010.
[6] P. Runeson and M. Höst, “Guidelines for conducting and reporting case
study research in software engineering,” Empirical Software Engineer-
ing, vol. 14, pp. 131–164, 2009.
[7] R. K. Yin, Case Study Research: Design and Methods, 3 edition. Sage
Publications, 2002.
[8] R. Andonie and I. Dzitac, “How to write a good paper in computer
science and how will it be measured by ISI web of knowledge,” Inter-
national Journal of Computers, Communications and Control, vol. V,
no. 4, pp. 432–446, 2010.
[9] J. Grudin, “Technology, conferences, and community,” Commun.
ACM, vol. 54, pp. 41–43, February 2011. [Online]. Available:
http://doi.acm.org/10.1145/1897816.1897834
[10] C. Ghezzi, “Reflections on 40+years of software engineering research
observed through ICSE: an insider’s view,” Keynote at ICSE 1999
Conference, 1999.

View publication stats