Prisma For Review of Management Literature Method, Merits, and Limitations An Academic Review
Prisma For Review of Management Literature Method, Merits, and Limitations An Academic Review
MANAGEMENT LITERATURE –
METHOD, MERITS, AND
LIMITATIONS – AN ACADEMIC
REVIEW
Vinaytosh Mishra and Monu Pandey Mishra
ABSTRACT
Preferred Reporting Items for Systematic Reviews and Meta-Analyses
(PRISMA) is a widely accepted guideline for performing a systematic
review (SR) in clinical journals. It not only helps an author to improve the
reporting but also assists reviewers and editors in the critical appraisal of
available SR. These tools help in achieving reproducibility in research, a major
concern in contemporary academic research. But there is a lack of awareness
about the approach among management researchers. This chapter attempts to
fill this gap using a narrative review of reliable online resources and
peer-reviewed articles to discuss the PRISMA guidelines and recent amend-
ments. The chapter further points out the limitations of PRISMA in the review
of management literature and suggests measures to overcome that. This piece
of literature introduces a reader to the basics of a systematic review using
PRISMA as an instrument. One of the significant contributions is to delineate
a seven-step strategy to attain reproducibility in the systematic review. The
chapter is useful for researchers and academicians in the field of social science
and management.
Keywords: Systematic review; review methods; PRISMA; PRISMA exten-
sions; reproducibility; literature review
1. INTRODUCTION
A literature review (LR) is an integral part of academic projects. The foremost
purpose of an LR is to develop a knowledge of the extant research work related
to a particular topic or area of study (Knopf, 2006). Another objective of the
literature review is to present insights in the form of a written report. Webster and
Watson in their seminal work asserts that conducting an LR helps you in not only
building your expertise in a specific area of the research field but also in identi-
fying the research gap. An effective LR helps in the development of theory,
summarizes the knowledge where an overabundance of research exists, and dis-
covers areas where research is required (Webster & Watson, 2002). Since they
wrote this paper with the roadmap of literature review 20 years back other
researchers have contributed to the body of review of the literature (ROL) by
defining the different types of LR (Leidner, 2018; Paré et al., 2015) or how to
make searches more inclusive and well-organized (Bandara et al., 2015; Vom
Brocke et al., 2009). A general framework for the LR is depicted in Fig. 1.
Despite the recent advancement in the arena of LR, two major shortcomings
persist. Foremost, the evidence synthesis fails to instill interest in the reader at the
same time it lacks delineating theoretical contribution. Webster and Watson in
their recent work suggests two measures for refining the procedure of LR. Firstly,
they suggest systematically digitally encoding (SDE) of main knowledge contri-
butions in the form of a graph or networks. Secondly, they propose reviewing
creative literature as a source of inspiration for constructing the theoretical
contributions of the paper. Fig. 2 depicts the publication data graph model
suggested by Watson and Webster (2020) in their seminal work.
The task of creating a data graph model may be intimidating for a researcher
who is new to the concept. The availability of Graphic User Interface (GUI)
based tools for the task may result in wider acceptance of the method soon.
(1) Narrative Literature Review: The main objective behind the narrative LR is
to examine and recapitulate an existing body of literature. To achieve this a
thorough background of the literature is presented in interest to educate,
identify gaps, or spot inconsistencies in the research area. Thus, the narrative
review can not only assist in refining, focusing, and identifying research
questions but also in proposing conceptual and theoretical frameworks
Fig. 2. Publication Data Graph Model. Source: Adopted from (Webster &
Watson, 2020).
128 VINAYTOSH MISHRA AND MONU PANDEY MISHRA
Out of the methods discussed above, the dominant styles used in the review of
management literature are narrative and systematic LR. The narrative review in
the field of social science is suitable for pinpointing the knowledge gaps, whereas
the systematic review is more focused on disseminating the existing information.
Jesson et al. (2011) discuss a continuum of the diverse nuances of these two types
of academic reviews (Fig. 3).
The rest of the chapter is schematized as follows. Section 2 deliberates the
concerns over reproducibility in contemporary research and seven strategies to
achieve it in the literature review. Section 3 discusses PRISMA guidelines and
recent updates in it. The section further lists the limitations of PRISMA and its
extensions of it. The chapter concludes with discussions beyond PRISMA, and
frameworks used for systematic review.
3. REPRODUCIBILITY IN RESEARCH
Reproducible research ensures that if the same analysis is repeated multiple times
the result obtained will be the same. It is a by-product of watchful diligence in the
process of research (Alston & Rick, 2021). An article published in Nature
observes that more than 70% of researchers have attempted and failed to replicate
the research of other academicians, while more than 50% half have been
unsuccessful in replicating the findings of their research (Baker, 2016). This paints
a very grim picture of the state of reproducibility in academic research. Irre-
producibility of research causes grave concern in academia and the management
field is no different. Moreover, irreproducibility restricts the translation of
research into practice as it adversely affects the reliability of the information.
Various measures to achieve reproducibility in the research listed in Table 1
(Shokraneh, 2019).
Discussion about all seven strategies is out of the scope of this chapter and
needs a series of articles to discuss it adequately. The focus of the next section is
to discuss PRISMA reporting guidelines and recent updates. The objective of the
section is to introduce PRISMA guidelines to researchers new to this tool in a
lucid manner. This chapter addresses the following three research questions:
(4) What are extensions of PRISMA useful in the review of the management
literature?
4. RESEARCH METHODOLOGY
This study uses narrative LR to explain the PRISMA guidelines and their
extensions useful in the review of the management of literature. The fundamental
narrative reviews are extremely effective for obtaining a wide perspective on a
subject and are often more comparable to a textbook chapter on an important
topic. One of the drawbacks of this type of review is the bias of the author in the
evidence synthesis. The authors of this study have tried their best to provide
evidence factually and to perform narrative review. This chapter uses the
approach discussed in extant literature (Contandriopoulos et al., 2010; Nasheeda
et al., 2019; Rice et al., 2016).
section discusses the guidelines of PRISMA 2020 and the addition of a more than
one decade-old version of it.
(1) The addition of the abstract writing specification within the newer
guidelines.
(2) Protocol and registration items are moved from the beginning of the method
section to the “Other Information” section with the addition of a sub-item
suggesting authors explain changes to the information presented at the time
of registration in directories such as PROSPERO.
(3) The “Search” item is modified to recommend authors present full search
strategies for all databases, registers and websites searched, not just at least
one database.
(4) “Study selection” items are changed to give more insight into how many
reviewers screened each record and each report retrieved, whether they
worked independently, and if applicable, details of automation tools used in
the process.
(5) The new standard includes a subitem to the “Data items” advising authors
to describe how results were defined, which of these were required, and
methods for selecting a subcategory of findings from included studies.
PRISMA for Review of Management Literature 133
(6) The new guideline splits the “Synthesis of results” into the “Methods” part
into six sub-items and advises authors to illustrate: the processes used to
determine which studies were eligible for each synthesis.
(7) The addition of a sub-item to the “Study selection” item in the Results
section advises the researchers to cite studies that could appear to meet the
inclusion criteria, but which were excluded, and explain why they were not
included for further evidence synthesis.
(8) “Asking authors to summarize the characteristics and risk of bias among
studies contributing to the synthesis; present results of all statistical syn-
theses conducted; present results of any investigations of possible causes of
heterogeneity among study results; and present results of any sensitivity
analyses” (Page et al., 2021).
(9) The inclusion of the latest items recommends researchers describe methods
for results of an evaluation of conviction in the body of evidence for a result.
(10) An additional item recommends authors declare competing interests and
make data, syntax, and code used in the review publicly available.
1 PRISMA for abstracts “The 12-item checklist gives authors a framework for
(PRISMA-A) condensing their systematic review into the essentials for a
Page et al. (2021) journal or conference abstract.” It is updated in PRISMA 2020
statement.
2 PRISMA equity (PRISMA-E) “It guides reporting equity-focused systematic reviews to help
Welch et al. (2012) reviewers identify, extract, and synthesize evidence on equity in
systematic reviews.” the PRISMA-Equity extension was
published in 2012
3 PRISMA for protocols “PRISMA-P was published in 2015 aiming to facilitate the
(PRISMA-P) development and reporting of systematic review protocols.”
Moher et al. (2015)
4 PRISMA for scoping reviews It was published in 2018 to synthesize evidence and evaluate the
(PRISMA-ScR) scope of extant articles in a research area. It also helps in
Tricco et al. (2018) assessing whether a systematic review is required of the topic
at all.
5 PRISMA for searching “The PRISMA extension for searching was published in 2021.
(PRISMA-S) The checklist includes 16 reporting items, each of which is
(Rethlefsen et al., 2021) detailed with exemplar reporting and Rationale.”
6. CONCLUSION
PRISMA guidelines have evolved over time and in the last decades, there have
been many extensions published to cater to the specific need. The use of the
PRISMA protocol not only gives structure to the review process it also helps
other researchers to reproduce the findings of the systematic review. The recent
updates in PRISMA go one step further and provide guidelines for result syn-
thesis and reporting. Even then evidence synthesis is based on the selected liter-
ature deeds of the researcher performing the review. There should be a
mechanism to minimize these biases. The existing guideline for PRISMA asks
authors to summarize the characteristics and risk of bias among studies
contributing to the synthesis. These types of majors keep a reader informed about
the probable biases in the findings of the systematic review. Other tools such as
the assessment of multiple systematic reviews (AMSTAR) are extensively used
for examining the methodological quality of systematic reviews (SR). Again,
AMSTAR is specially designed for randomized controlled trials (RCTs) and its
applicability to SRs of other study designs prevalent in management literature is
arguable.
PRISMA is extensively used in healthcare research but its wide use in man-
agement research is still debatable. The use of traditional narrative reviews is
more common in management literature. There is a need for PRISMA extension
of specific objectives review of management literature. Management researchers
suggest the use of a framework for conducting a review of management literature.
They argue that reviews with a framework have proven to be more acceptable as
they are likely to show a more robust structure (Paul & Criado, 2020). Some of
the frameworks used in the review of literature are ADO (Antecedents, Decisions,
and Outcome), 6 W Framework (who, when, where, how, what, and why), and
TCCM Framework (Theory, Construct, Characteristics and Methodology)
(Callahan, 2014; Paul & Criado, 2020; Paul & Rosado-Serrano, 2019). Although
extant literature cites the distinct advantage of using a framework for the review
of management literature, there is a lack of standardization of these frameworks.
Moreover, developers of these frameworks do not give a clear understanding of
which framework is better in which scenario. One of the reasons behind it may be
these frameworks are suggested and updated by researchers in their capacity and
followed in their academic community. There is a need for a more organized
effort like the PRISMA group around the review of management literature.
REFERENCES
Alston, J. M., & Rick, J. A. (2021). A beginner’s guide to conducting reproducible research. The
Bulletin of the Ecological Society of America, 102(2), 1–14.
Baker, M. (2016). 1,500 scientists lift the lid on reproducibility. Nature, 533(7604), 452–454.
Bandara, W., Furtmueller, E., Gorbacheva, E., Miskon, S., & Beekhuyzen, J. (2015). Achieving rigor
in literature reviews: Insights from qualitative data analysis and tool support. Communications
of the Association for Information Systems, 37(1), 8.
Beller, E., Clark, J., Tsafnat, G., Adams, C., Diehl, H., Lund, H., Ouzzani M, Thayer, K., Thomas, J.,
Turner, T., Xia, J., Robinson, K., & Glasziou, P. (2018). Making progress with the automation
of systematic reviews: Principles of the International Collaboration for the Automation of
Systematic Reviews (ICASR). Systematic Reviews, 7(1), 1–7.
Brocke, J. V., Simons, A., Niehaves, B., Niehaves, B., Reimer, K., Plattfaut, R., & Cleven, A. (2009).
Reconstructing the giant: On the importance of rigor in documenting the literature search process.
CIS 2009 Proceedings Paper 161.
Callahan, J. L. (2014). Writing literature reviews: A reprise and update. https://doi.org/10.1177/
1534484314536705
Contandriopoulos, D., Lemire, M., Denis, J. L., & Tremblay, É. (2010). Knowledge exchange pro-
cesses in organizations and policy arenas: A narrative systematic review of the literature. The
Milbank Quarterly, 88(4), 444–483.
Coughlan, M., Cronin, P., & Ryan, F. (2007). Step-by-step guide to critiquing research. Part 1:
Quantitative research. British Journal of Nursing, 16(11), 658–663.
Jesson, J., Matheson, L., & Lacey, F. M. (2011). Doing your literature review: Traditional and sys-
tematic techniques. Sage Publications Ltd.
Knopf, J. W. (2006). Doing a literature review. PS: Political Science & Politics, 39(1), 127–132.
Koffel, J. B., & Rethlefsen, M. L. (2016). Reproducibility of search strategies is poor in systematic
reviews published in high-impact pediatrics, cardiology, and surgery journals: A cross-sectional
study. PLoS One, 11(9), e0163309.
Lachal, J., Revah-Levy, A., Orri, M., & Moro, M. R. (2017). Meta-synthesis: An original method to
synthesize qualitative literature in psychiatry. Frontiers in Psychiatry, 8, 269.
Leidner, D. E. (2018). Review and theory symbiosis: An introspective retrospective. Journal of the
Association for Information Systems, 19(6), 1.
Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., Altman, D., Antes, G., Atkins, D., Barbour, V.,
Barrowman, N., Berlin, J. A., Clark, J., Clarke, M., Cook, D., D’Amico, R., Deeks, J. J.,
Devereaux, P. J., Dickersin, K., Egger, M., Ernst, E.,... Tugwell, P. (2015). Preferred Reporting
Items for Systematic Review and Meta-Analysis Protocols (PRISMA-P) 2015 statement.
Systematic Reviews, 4(1), 1. https://doi.org/10.1186/2046-4053-4-1
Nasheeda, A., Abdullah, H. B., Krauss, S. E., & Ahmed, N. B. (2019). A narrative systematic review of
life skills education: Effectiveness, research gaps, and priorities. International Journal of
Adolescence and Youth, 24(3), 362–379.
Page, M. J., Altman, D. G., Shamseer, L., McKenzie, J. E., Ahmadzai, N., Wolfe, D., Yazdi, F.,
Catalá-López, F., Tricco, A. C., & Moher, D. (2018). Reproducible research practices are
underused in systematic reviews of biomedical interventions. Journal of Clinical Epidemiology,
94, 8–18.
136 VINAYTOSH MISHRA AND MONU PANDEY MISHRA
Page, M. J., McKenzie, J. E., Bossuyt, P. M., Boutron, I., Hoffmann, T. C., Mulrow, C. D., Shamseer,
L., Tetzlaff, J. M., Akl, E. A., Brennan, S. E., Chou, R., Glanville, J., Grimshaw, J. M.,
Hróbjartsson, A., Lalu, M. M., Li, T., Loder, E. W., Mayo- Wilson, E., McDonald, S.,...
Moher, D. (2021). The PRISMA 2020 statement: An updated guideline for reporting sys-
tematic reviews. Systematic Reviews, 10(1), 1–11.
Page, M. J., & Moher, D. (2017). Evaluations of the uptake and impact of the Preferred Reporting
Items for Systematic reviews and Meta-Analyses (PRISMA) Statement and extensions: A
scoping review. Systematic Reviews, 6(1), 1–14.
Paré, G., Trudel, M. C., Jaana, M., & Kitsiou, S. (2015). Synthesizing information systems knowledge:
A typology of literature reviews. Information & Management, 52(2), 183–199.
Paul, J., & Criado, A. R. (2020). The art of writing a literature review: What do we know and what do
we need to know? International Business Review, 29(4), 101717.
Paul, J., & Rosado-Serrano, A. (2019). Gradual Internationalization vs Born-Global/International new
venture models: A review and research agenda. International Marketing Review, 36(6), 830–858.
Polit, D. F., & Beck, C. T. (2006). The content validity index: Are you sure you know what’s being
reported? Critique and recommendations. Research in Nursing & Health, 29(5), 489–497.
PRISMA. (2022). The PRISMA 2020 Checklist and flow diagram. https://prisma-statement.org/
Rana, S., Raut, S. K., Prashar, S., & Hamid, A. B. A. (2020). Promoting through consumer nostalgia: A con-
ceptual framework and future research agenda. Journal of Promotion Management, 27(2), 211–249.
Rana, S., Raut, S. K., Prashar, S., & Quttainah, M. A. (2022). The transversal of nostalgia from
psychology to marketing: What does it portend for future research? International Journal of
Organizational Analysis, 30(4), 899–932.
Rethlefsen M. L., Kirtley S., Waffenschmidt S., Ayala, A. P., Moher, D., Page, M. J., Koffel, J. B., &
PRISMA-S Group. (2021). PRISMA-S: An extension to the PRISMA statement for reporting
literature searches in systematic reviews. Systematic Reviews, 10(1), 39. https://doi.org/10.1186/
s13643-020-01542-z
Rice, S. M., Purcell, R., De Silva, S., Mawren, D., McGorry, P. D., & Parker, A. G. (2016). The
mental health of elite athletes: A narrative systematic review. Sports Medicine, 46(9),
1333–1353.
Sarkis-Onofre, R., Catalá-López, F., Aromataris, E., & Lockwood, C. (2021). How to properly use the
PRISMA Statement. Systematic Reviews, 10(1), 1–3.
Shokraneh F. (2018, November 26). Reproducible and Replicable Search for Research Methods in
Systematic Reviews. Search Solutions.
Shokraneh, F. (2019). Reproducibility and replicability of systematic reviews. World Journal Meta-
Analysis, 7(3), 66–71.
Stewart, L., Moher, D., & Shekelle, P. (2012). Why prospective registration of systematic reviews
makes sense. Systematic Reviews, 1(1), 1–4.
Tao, K. M., Li, X. Q., Zhou, Q. H., Moher, D., Ling, C. Q., & Yu, W. F. (2011). From QUOROM to
PRISMA: A survey of high-impact medical journals’ instructions to authors and a review of
systematic reviews in anesthesia literature. PLoS One, 6(11), e27611.
Tricco, A. C., Lillie, E., Zarin, W., O’Brien, K. K., Colquhoun, H., Levac, D., Moher, D., Peters, M.
D. J., Horsley, T., Weeks, L., Hempel, S., Chang, C., Elie, A., McGowan, J., Stewart, L.,
Hartling, L., Aldcroft, A., Wilson, M. G., Garritty, C.,... Straus, S. E. (2018). PRISMA
extension for scoping reviews (PRISMA-ScR): Checklist and explanation. Annals of Internal
Medicine, 169(7), 467–473. https://doi.org/10.7326/M18-0850
Watson, R. T., & Webster, J. (2020). Analyzing the past to prepare for the future: Writing a literature
review on a roadmap for release 2.0. Journal of Decision Systems, 29(3), 129–147.
Webster, J., & Watson, R. T. (2002). Analyzing the past to prepare for the future: Writing a literature
review. MIS Quarterly, 26(2), 13–23.
Welch, V., Petticrew, M., Tugwell, P., Moher, D., O’Neill, J., Waters, E., & White, H. (2012).
Guidelines and guidance-PRISMA-equity 2012 extension: Reporting guidelines for systematic
reviews with a focus on health equity. PLoS Medicine, 9(10), 1487.
Williams, R. I., Jr., Clark, L. A., Clark, W. R., & Raffo, D. M. (2021). Re-examining systematic
literature review in management research: Additional benefits and execution protocols.
European Management Journal, 39(4), 521–533.