See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/284790946

Research Design for Mixed Methods: A Triangulation-based Framework and

Roadmap

Article  in  Organizational Research Methods · November 2015

DOI: 10.1177/1094428115610808

CITATIONS READS
66 9,014

3 authors, including:

Scott F. Turner Richard M. Burton

University of South Carolina Duke University
14 PUBLICATIONS   566 CITATIONS    175 PUBLICATIONS   2,915 CITATIONS   

SEE PROFILE SEE PROFILE

Some of the authors of this publication are also working on these related projects:

Justification shift and uncertainty: Why are low-probability near misses underrated against organizational routines? View project

All content following this page was uploaded by Scott F. Turner on 21 December 2015.

The user has requested enhancement of the downloaded file.

Research Design for Mixed Methods: A Triangulation-Based Framework and Roadmap

Scott F. Turner
Darla Moore School of Business
University of South Carolina
Columbia SC 29208
803-777-5973
scott.turner@moore.sc.edu

Laura B. Cardinal
C. T. Bauer College of Business
University of Houston
Houston, TX 77204
713-743-2559
lbcardinal@bauer.uh.edu

Richard M. Burton
Fuqua School of Business
Duke University
Durham, NC 27708
919-660-7700
rmb2@duke.edu

Organizational Research Methods

November 25 2015
Published online before print
 Research Design for Mixed Methods: A Triangulation-Based Framework and Roadmap

Abstract

All methods individually are flawed, but these limitations can be mitigated through mixed methods
research, which combines methodologies to provide better answers to our research questions. In
this study, we develop a research design framework for mixed methods work that is based on the
principles of triangulation. Core elements for the research design framework include theoretical
purpose, i.e., theory development and/or theory testing; and methodological purpose, i.e.,
prioritizing generalizability, precision in control and measurement, and authenticity of context.
From this foundation, we consider how the multiple methodologies are linked together to
accomplish the theoretical purpose, focusing on three types of linking processes: convergent
triangulation, holistic triangulation, and convergent and holistic triangulation. We then consider
the implications of these linking processes for the theory at hand, taking into account the following
theoretical attributes: generality/specificity, simplicity/complexity, and accuracy/inaccuracy.
Based on this research design framework, we develop a roadmap that can serve as a design guide
for organizational scholars conducting mixed methods research studies.

Keywords

research methods, mixed methods, triangulation, validity, research strategies, theory

development, theory testing

Acknowledgements

The authors wish to thank Associate Editor David Ketchen and three anonymous reviewers for
their detailed and thoughtful feedback. Portions of this research were presented at the 75th Annual
Meeting of the Academy of Management and the 35th Annual International Conference of the
Strategic Management Society.
 Research Design for Mixed Methods: A Triangulation-Based Framework and Roadmap

As a basic concept in the social sciences, triangulation refers to using multiple, different
approaches to generate better understanding of a given theory or phenomenon (Burton & Obel,
2011; Singleton & Straits, 1999). While early work in the social sciences focused on the role of
triangulation in measurement, i.e., using different methods to assess construct validity (Campbell
& Fiske, 1959), more recent attention has emphasized how triangulation can contribute to our
understanding more broadly. Mixed methods research is based on the idea of heightened
understanding through methodological triangulation (Denzin, 1970, 2012; Molina-Azorin, 2007;
Torrance, 2012). For this form of triangulation, McGrath (1982) describes the use of multiple
methodological strategies, which represent generic classes of research strategies for gaining
knowledge about a research question; examples include laboratory experiments and field studies.
McGrath (1982) discusses the importance of methodological triangulation from the view that
individually “all research strategies and methods are seriously flawed” (p. 70), but in
combination, the use of multiple methods that do not share the same failings can enhance what is
known about a given research question. In similar fashion, Weick (1969) states, “we typically
need multiple methods, or techniques which are imperfect in different ways. When multiple
methods are applied, the imperfections in each method tend to cancel one another” (p. 21).

Although triangulation can take place within a given methodology, such as using different
subject populations for a lab experiment (e.g., Bateman & Zeithaml, 1989) or using different
platforms for computer simulation (Axtell, Axelrod, Epstein, & Cohen, 1996), mixed methods
research focuses on triangulation that spans multiple methodologies (Denzin, 1970, 2012;
Singleton & Straits, 1999). Jick (1979) argues that methods-spanning forms of triangulation are
on the complex end of a continuum of triangulation design while within-method forms are on the
simple end. For triangulation that spans methodologies, much attention has been directed to
convergent triangulation, which is based on the idea that knowledge develops by obtaining
convergence in substantive findings across a diverse set of methodologies (McGrath, Martin, &
Kulka, 1982). This reflects the view that using multiple methods produces more valid results as
the strengths of one method can offset the limitations of another method (Jick, 1979; Scandura &
Williams, 2000).

While scholarly advice has been offered for many forms of convergent triangulation
(McGrath et al., 1982; Pedhazur & Schmelkin, 1991), less attention has been directed to the
holistic form of triangulation. This represents an important omission because holistic
triangulation offers the promise of capturing “more complete, holistic, and contextual portrayal
of the unit(s) under study” and enriching “our understanding by allowing for new or deeper
dimensions to emerge” (Jick, 1979, pp. 603-604). Relative to convergent triangulation, which
emphasizes knowledge development in the search for agreement, the holistic form asserts that
divergence can be just as important as convergence in the accrual of knowledge. This is based in
part on the idea that certain methods are uniquely capable of “seeing” particular aspects of a
phenomenon or “judging” particular attributes of a theory. 1 Thus, through holistic triangulation,

1
A parallel discussion of convergence versus divergence can be found with respect to performance management in
the organizational behavior literature. For example, we might not expect to find convergence when performance of
an individual’s work is evaluated by peers versus a supervisor because these sources have different vantage points
and can provide qualitatively different types of information. We wish to thank a reviewer for highlighting this
observation.
1
scholars can generate more insight for a theory or develop more complete understandings of a
given phenomenon through the use of multiple methodologies (Denzin, 2012; Gibbert &
Ruigrok, 2010; Shah & Corley, 2006).

While scholars have made the case for the benefits and promise of methodological
triangulation in the organizational sciences (e.g., Jick, 1979; McGrath et al., 1982), in a related
review, Scandura and Williams (2000) found that triangulating across methods is still rare in the
organizational sciences. There are a number of reasons for the scarcity of studies employing
methodological triangulation. One core reason is that there is limited guidance available for
organizational scholars regarding how to design mixed methods research studies and particularly
for how to design holistic triangulation studies. Other important challenges include the time-
consuming nature of the work, the need to assemble and coordinate researcher expertise across
different methodological areas, and the demands associated with publishing mixed methods
research.

Thus, the objective of this article is two-fold: the first objective is to develop a research
design framework for mixed methods work that is based on the principles of triangulation, and
then based on that framework, the second objective is to provide a roadmap to guide
organizational scholars interested in conducting mixed methods research studies. Our emphasis
is on research that combines methods that stand more or less on equal footing, i.e., equivalent
methods, rather than situations in which one method is dominant and another is in a supporting
role (Molina-Azorin, 2007). While triangulation in research methodology can encompass a
number of aspects, such as using multiple sources of data and examining multiple empirical
settings (Scandura & Williams, 2000), as a methods foundation, our study draws from the
research strategies presented in McGrath (1982). In this study, we focus on the following
research strategies: archival records, 2 case studies, computer simulations, experimental
simulations, field experiments, formal theory (mathematical), interviews, lab experiments, and
surveys. In the article, we often use the research strategy and method terms interchangeably.

For the first objective, our research design framework is based on several core elements,
which are captured in Figure 1. The first two elements focus on theoretical and methodological
purpose – that is, the theoretical purpose for the research, and the purposes for the methods that
are utilized in that research. The third element focuses on how the multiple methodologies are
linked together given theoretical and methodological purpose and considers the implications of
the linking process for the theory at hand. 3 For our second objective, we draw from this research
design framework to develop a roadmap that provides guidance for organizational scholars
regarding how to design and implement triangulation-based mixed methods research studies. We
conclude with further examination of two key issues involved in the design and implementation

2
Some scholars have placed emphasis on archival records as a distinct research strategy that involves the use of
archival sources of data (Austin, Scherbaum, & Mahlman, 2002; Scandura & Williams, 2000; Singleton & Straits,
1999). While Scandura and Williams (2000) incorporated this approach under the label of field studies that use
secondary data, we have adopted the archival records label used in Singleton and Straits (1999), given its distinction
from many field studies.
3
Given our emphasis on methods at the level of research strategies (McGrath, 1982) and their relationship with
theoretical purpose, our framework focuses more on design and to a lesser extent on measurement and analysis
among the core methodological topics of design, measurement, and analysis (e.g., Aguinis, Pierce, Bosco, & Muslin,
2009).
2
of mixed methods studies: how to select a set of methodologies that increases validity by
offsetting the limitations inherent to the individual methodologies, and how to overcome
practical challenges involved in conducting mixed methods research. This discussion considers
important assumptions and challenges associated with triangulation-based mixed methods
research that influence whether and how this research can produce heightened understanding.

*** Insert Figure 1 Here ***

The Foundation for Triangulation: Theoretical Purpose and Methodological Purpose

Theoretical purpose. To develop a research design framework for mixed methods research
involving triangulation, we first need to consider the theoretical purpose for the research study,
which often ties closely to the research question (Burton & Obel, 1995; Molina-Azorin, 2007).
Theoretical purpose refers to what researchers aim to accomplish in the study as it pertains to
theory; such aims include theory generation, theory elaboration, and theory testing (Lee,
Mitchell, & Sablynski, 1999). For mixed methods studies that focus on convergent triangulation,
the theoretical purpose may involve assessing the validity of an explanation by conducting
multiple tests using methodologies that are flawed in different ways, or it may involve starting
with one method for theory development, 4 and concluding with another method for theory
testing. Thus, theory testing often plays an important role in convergent triangulation research.
By contrast, for studies focused on holistic triangulation, the theoretical purpose often centers on
the development of theory.

Methodological purpose. Following theoretical purpose, scholars pursuing mixed methods

research need to give careful attention to the purposes for which they are utilizing particular
methods. McGrath (1982, 1995) suggests that scholars select research strategies based on the
extent to which a given method can accomplish several objectives: (1) its ability to maximize
generalizability with respect to populations (e.g., actors), (2) its precision in control and
measurement of variables related to behaviors of interest, and (3) its ability to provide
authenticity of context for the observed behaviors. Accomplishment of these respective
objectives enables researchers to contend with threats to different types of validity. For example,
a method that offers precision in control and measurement of variables is well-suited for
addressing threats to internal validity, which refers to the validity of inferences regarding
whether covariation between an independent and dependent variable results from a causal
relationship, as opposed to the possibility that other extraneous variables are responsible for the
outcome. Alternatively, a method that can maximize generalizability with respect to populations
can contend with threats to external validity, which refers to the validity of inferences about
whether the causal relationship is generalizable to other subject populations, settings, and time
periods (Shadish, Cook, & Campbell, 2002). As such, the selection of research methodologies is
important because it influences the answers to our research questions in that the types of
evidence, nature of claims, and confidence in inferences depend considerably on which methods
are being employed.

4
For parsimony in presentation, we use “theory development” to encompass theory generation and theory
elaboration. In the former case, researchers ideally begin with no theory, while in the latter case, the objective is to
advance preexisting theory; in both cases, though, the objective is development of theory (Eisenhardt, 1989; Lee et
al., 1999).
3
 According to McGrath (1982, 1995), certain research strategies are often well-suited to one
of these purposes; for example, lab experiments enable precision in control and measurement of
variables, while field studies (i.e., case studies) provide an authentic context within which the
behaviors of actors are observed. While this view emphasizes that individual methods tend to
have particular strengths and weaknesses, it is also important to keep in mind that certain
strategies can be utilized to accomplish different methodological purposes. For example,
computer simulations can be used to maximize generalizability with respect to populations, or
they can be used as laboratories for control and manipulation of variables (Burton, 2003; Davis,
Eisenhardt, & Bingham, 2007; Harrison, Lin, Carroll, & Carley, 2007). As another example,
archival records may be used to enhance precision in control and measurement of variables (e.g.,
focusing on measures well-suited for a particular group, firm, or industry), maximize
generalizability with respect to populations (e.g., seeking breadth across groups, firms, or
industries), or capture behaviors of interest that have taken place within an authentic context.
Table 1 highlights which methodological purposes are commonly fulfilled by particular research
strategies.

*** Insert Table 1 Here ***

Overall theoretical purpose (i.e., theory development, theory testing) and methodological
purpose (i.e., generalizability, precision in control/measurement, authenticity of context) provide
an important foundation for triangulation-based research design.

Linking Methods in Light of Theoretical and Methodological Purposes

In the previous section, we described how theoretical and methodological purposes are at the
foundation of triangulation-based research. In this section, we focus on the process of linking,
which is a central way in which scholarly understanding in a given area can be extended through
the use of mixed methods.

In the design of triangulation-based studies, linking refers to the process by which multiple
research strategies are brought together within a research study to realize the theoretical purpose
for that work. We argue that there are three core processes for linking research strategies in
triangulation-based research: (a) linking processes focused on convergent triangulation, (b)
linking processes focused on holistic triangulation, and (c) linking processes focused on
convergent and holistic triangulation. As described next, what distinguishes these processes is
how the set of methodologies relate to the theoretical purpose for the study and how the
researcher leverages methodological purpose in determining the complement of methodologies
utilized in the study.

Linking processes focused on convergent triangulation are commonly of two basic types; one
type focuses on theory testing, with two research strategies being used for the purpose of testing
the same theory, while the other type spans both categories of theoretical purpose, with one
research strategy focused on theory development and one focused on theory testing. 5 For both
types of convergent triangulation, the researcher utilizes multiple methods to assess whether

5
For ease of explanation, we typically refer to linking processes as if two methods were being utilized, although
researchers could clearly use more than two methods in a mixed methods research study.
4
convergent results are observed across the methods.

Linking processes for holistic triangulation are focused on theory development and span at
least two categories of methodological purpose (e.g., prioritizing generalizability, prioritizing
control/measurement, prioritizing authenticity of context). For holistic triangulation, the
researcher utilizes multiple research strategies because he or she expects to learn different and
unique things from them. 6

Linking processes focused on convergent and holistic triangulation span both categories of
theoretical purpose (i.e., theory development, theory testing) and involve at least two categories
of methodological purpose. For convergent and holistic triangulation, the researcher utilizes
multiple research strategies because he or she expects to obtain better understanding through
opportunities for the methods to show convergent results in certain areas and through unique
perspectives/angles that one or more of the individual methods can provide in other areas. Figure
2 provides a number of examples of these linking processes.

*** Insert Figure 2 Here ***

Linking of methodologies in convergent triangulation studies. For studies focused on

convergent triangulation, the objective for the researcher is to demonstrate more valid results by
finding agreement across different research strategies (Jick, 1979; McGrath, 1982). In Figure 2,
linking processes focusing on convergent triangulation are reflected in horizontal arrows within
theory testing (i.e., testing theory using two complementary methods) and in arrows that begin in
theory development and conclude in theory testing (i.e., developing and testing theory).

Good examples of convergent triangulation studies that test theory using two complementary
methods can be found in Bardolet, Fox, and Lovallo (2011) and Morris and Moore (2000).
Bardolet et al. begins with the observation that strategy scholars commonly explain subsidization
across business units in multi-business firms with agency theory (e.g., principal-agent
conflicts/negotiations). The researchers then propose that the cross-subsidization phenomenon
may be explained by a simpler argument that corporate executives have cognitive biases in favor
of even resource allocations. To test this argument, they initially turn to regressions using
archival data, specifically using Compustat data to generate a large dataset that spans many
industries. The researchers found support for their argument with regression analyses of the
archival data, but they were also aware that the archival-based analyses did not allow for
sufficient controls to rule out common explanations based on agency theory. Thus, they next
turned to a series of lab experiments as a complementary method for theory testing, which
offered greater capacity to control for such possibilities and provided consistent results, which
offered greater support for their argument.

As another example of convergent triangulation, Morris and Moore (2000) examined how
learning was influenced by counterfactual thinking, and how organizational accountability
influenced this process. The researchers distinguished between upward counterfactual
6
An alternative process could be one of holistic triangulation with the aim of theory testing. For example, multiple
methods might be used for their unique ability to test different aspects of the same theory (e.g., Flynn & Staw,
2004). We do not consider such a process here because of the emphasis on convergence with theory testing in mixed
methods research (i.e., multiple methods providing different tests for the same question).
5
comparisons, which compare actual events to better possible alternatives, and downward ones,
which foster thinking relative to worse possible alternatives. They argue that upward
counterfactual comparisons promote learning because they focus attention on factors preceding
outcomes that might be changed in order to produce better outcomes, which provides a frame for
learning how to act in similar situations in the future, and they argue that organizational
accountability inhibits upward counterfactual comparisons because it restricts complex and self-
critical thinking. In the study, the researchers tested these predictions through two
complementary methods: (a) statistical analyses using archival data of experienced pilots’
assessments of near-miss aviation incidents and (b) an experimental simulation in which
undergraduate students operate a flight simulator. 7 The analyses initially focused on the archival
data from experienced pilots because it offered an authentic context as described in Table 1,
whereas the researchers described tests of similar processes in prior studies as limited to
academic and lab settings. While these analyses supported their predictions, the regressions
based on archival data could only rule out certain alternative explanations, so the researchers
next turned to an experimental simulation using a flight simulator that afforded greater precision
in control and measurement.

As a good example of the alternative form of convergent triangulation, i.e., developing and
testing theory, consider Cohen and Klepper (1996). The study examines the effect of
organizational size on the allocation of R&D effort between process and product innovation. The
theoretical purpose of the study was first to develop theory through a mathematical model that
explains the influence of size on the allocation of R&D effort and then to test the predictions
derived from the model with regression analyses using archival data. From the view of
methodological purpose, both methods prioritize generalizability with respect to populations of
organizations. Specifically, the researchers develop a theoretical model that they expect to hold
for organizations in a variety of industries, and they test the predictions using archival data from
the Federal Trade Commission that includes businesses in 36 different manufacturing industries.

As another example of this form of convergent triangulation, Grant, Berg, and Cable (2014)
examine how self-reflective job titles affect emotional exhaustion. Beginning with the purpose of
theory development, the researchers conducted a case study at the Make-A-Wish Foundation.
Based on their analyses of interviews, observations, and archival sources, the researchers
discovered that employees’ selecting their own job titles contributed to less emotional
exhaustion, and they proposed three mechanisms (self-verification, psychological safety, and
external rapport) to explain this effect. While the case study suited the methodological purpose
of authenticity of context, it was also considered to be an extreme case of employee burnout,
such that the researchers described selecting an alternative research strategy to test their
inductively-derived theory in a way that could support greater generalizability. Their choice of a
second research strategy – a field experiment – also contributed by offering greater precision in
control and measurement. Thus, they conducted a field experiment in a large health services
organization, which included pre- and post-intervention surveys, and the results supported their
proposition that choosing self-reflective titles reduces emotional exhaustion for employees.

7
McGrath (1995) describes the experimental simulation research strategy as one in which “the researcher attempts
to achieve much of the precision and control of the laboratory experiment but to gain some of the realism (or
apparent realism) of field studies. This is done by concocting a situation or behavior setting or context, as in the
laboratory experiment, but making it as much like some class of actual behavior setting as possible.” (p. 157).
McGrath (1995) also provides the use of flight simulators as a good example of this type of research.
6
However, the researchers found that only two of the three proposed mechanisms – self-
verification and psychological safety – had a significant mediating effect in the field experiment.

Linking of methodologies in holistic triangulation studies. For studies focused on holistic

triangulation, the objective is to obtain fuller understanding through unique insights/perspectives
gained from the different research strategies being utilized; according to Jick (1979), such
triangulation allows for more complete and contextual portrayal of the units under study. In
Figure 2, linking processes focusing on holistic triangulation are reflected in horizontal arrows,
with theory development as the theoretical purpose.

Cardinal, Turner, Fern, and Burton (2011) provide a good example of holistic triangulation.
With the purpose of theory development, the researchers wanted to establish a more
comprehensive understanding of organizing for innovation by exploring a traditional
contingency-based model of product development. They argued that different technological
environments have different features that require different information processing, such that
better performance would be observed from project designs that fit the information-processing
requirements of the environment. The researchers initially turned to computer simulation
methodology given its precision and capacity for manipulation of design and environmental
conditions as outlined in Table 1. The simulations produced unexpected results that only partly
aligned with the propositions, calling into question a traditional contingency perspective of
designing for product development; specifically, rather than finding superior performance for
project designs that fit their respective technological environments, the simulation results
suggested that organizations face substantial performance trade-offs (e.g., speed of development
vs. cost) with project design. The researchers then turned to multiple case studies as a
complementary methodology for authenticity of context that focused on six project development
projects at three firms competing in the three focal technological environments and included 75
interviews with project leaders and team members across the six projects; this methodology
offered authenticity of context for the study participants and enabled the researchers to take a
deeper look into how technological environments affect performance priorities, project design,
and performance.

As another example of holistic triangulation, Anteby (2010) utilized multiple methods for the
purpose of theory development, focusing on the broad question of what makes markets moral,
with particular interest in understanding variations in legitimacy for trades involving the same
class of goods. The empirical setting was trade in cadavers in the State of New York and
involved multiple sources of data, including interviews with participants in the cadaver trade,
direct observations at supplying and recipient organizations, and archival data documenting the
exchange of cadavers in the state of New York. After focusing initially on the interviews and
direct observations for the purpose of understanding how commerce in cadavers worked, Anteby
then turned to the archival data to construct a map that identified which organizations were
involved in supplying and receiving cadavers in New York and documented the flow of cadavers
in the state. The interviews and observations facilitated authenticity of context for the study
participants as well as for the researcher and offered a deep look into how the trade operated; in
turn, the archival data provided a precise accounting of the involved organizations and the flow
of cadavers among them. Through these methods, Anteby was able to develop a practice-based
model of trade legitimacy, focusing the distinctions between proper and improper means of
commerce made by professionals in the cadaver trade.

7
 Lin, Zhao, Ismail, and Carley (2006) also provide a nice example of the holistic triangulation
linking process. The objective of this study was one of theory development, with a particular
focus on how to design organizations for better performance during crises. Drawing from
organizational theory, and specifically using a neo-information processing perspective, the
researchers developed a computer simulation model for predicting the performance of
organizations under crisis conditions. Next, turning to archival data, they collected data on the
relative effectiveness of 80 organizations faced with actual crises; these archival data were used
both to seed the contextual conditions for simulation predictions of organizational performance
and as a source of data for the actual performance of the organizations. Comparisons of the
simulation-predicted organizational performance with the actual-observed organizational
performance pointed to areas where the theory needed to be extended, and the researchers then
returned to the simulation model to conduct a number of “what if” experiments for that purpose.

Linking of methodologies in convergent and holistic triangulation studies. For studies that
encompass both convergent and holistic triangulation, the objective is to assess the validity of a
theory or set of results by examining across the research strategies, including the extent of
agreement across the research strategies (the convergent aspect) and their capacity to offer
unique perspective that can provide a more complete understanding of a phenomenon or theory
(the holistic aspect). On Figure 2, linking processes focusing on convergent and holistic
triangulation are reflected in arrows that span cells vertically and horizontally.

As one example of convergent and holistic triangulation, in their study of knowledge sharing
within networks, Dyer and Hatch (2006) used regression analyses of survey data for theory
testing, followed by theory development through interviews. Drawing on evolutionary
economics, the researchers argued that a buyer can generate competitive advantage by sharing
knowledge with its supplier network. Following development of their argument, the researchers’
initial objective was to test the following prediction: a buyer that transfers more knowledge to its
supplier network will develop the capabilities of their suppliers, such that the operational
relationship between the buyer and its suppliers will be more productive (i.e., creation of relative
competitive advantage). This test was conducted using survey data obtained from US-based
suppliers for Toyota, namely, a methodological approach offering precision in measurement as
well as an authentic context as highlighted in Table 1. Consistent with the principle of
convergent triangulation, the hypothesis was supported by the analyses of survey data as well as
by interviews conducted with employees at US-based suppliers for Toyota. Next, and consistent
with the principle of holistic triangulation, the researchers sought to develop theory to explain
why the created advantage could be sustained given that suppliers could share the knowledge
provided by a focal buyer with its other buyers such that the relative competitive advantage
could dissipate. For this aspect, Dyer and Hatch used qualitative analysis of the interviews with
employees at US-based suppliers for Toyota, which offered an authentic context; this method
provided rich insight and nuance for understanding why the relative competitive advantage could
be sustained.

In addition, Nguyen and Nguyen (2008) provide a good example of convergent and holistic
triangulation in their study of Vietnamese entrepreneurs. The researchers were interested in
examining the psychological values and motivations for establishing a new business, with
particular interest in understanding differences between Vietnamese and Western entrepreneurs.

8
Their initial research strategy was focused on theory testing. Specifically, the researchers used a
survey approach with a sample of 117 Vietnamese entrepreneurs to test three core hypotheses.
The survey was administered in person in an effort to maximize the response rate given cultural
views of research. Only the first hypothesis was fully supported, which focused on
entrepreneurs’ values reflecting both personal and collective orientations. For the second and
third hypotheses, the researchers found that the entrepreneurs did not fully share the values of
their American counterparts as predicted and that neither values nor motives affected venture
success. Next, the researchers turned to in-depth interviews with female Vietnamese
entrepreneurs to understand what motivates these entrepreneurs to start their businesses and also
to learn how those motives change over time. For this part, they employed a grounded study
approach using qualitative field data to develop a conceptual framework. From the view of
convergent triangulation, the researchers found support that was consistent with the survey-based
results, namely, support for non-economic motives such as continuous learning, promotion of
Vietnamese culture, and helping the disabled. In addition, and consistent with the principles of
holistic triangulation, they discovered that over time, economic motives for founding the
business gave way to more altruistic motives after the business had stabilized or was successful.

Ely (1994) provides another example of convergent and holistic triangulation. This study
examined how the representation of women in upper management affects the hierarchical and
peer relationships that form among women in organizations. Using the empirical context of law
firms, the researcher created four matched pairs of law firms, where each pair of organizations
included one firm with male-dominated upper management and one firm with more gender
balance in upper management. Social identity theory and organizational demography research
were drawn upon to develop the arguments and predictions. Ely then identified a sample of 30
women associates from the eight law firms and conducted two interviews with each of the 30
associates. For her objective of theory testing, Ely utilized two different methods focused on the
same sample of associates. First, as a way to preserve authenticity of context for the participants,
the interviews were content analyzed to identify core themes pertaining to the hypotheses, and
regression analyses were performed on the themes related to the hypotheses. Second, Ely turned
to a survey as a way to provide more precise measurement; using the themes emerging from the
interview transcripts, she developed for a survey that was completed by each of the interviewed
associates and then performed regression analysis on the survey data. Consistent with the
principle of convergent triangulation, the results were largely supported across the statistical
analyses of the content-analyzed interviews data and the survey data. In addition, Ely was able to
revisit the original interview transcripts for further interpretation and insight, which is consistent
with the principle of holistic triangulation and contributed to theory development by offering
more depth and nuance for understanding the why behind the focal relationships.

As demonstrated by this set of mixed methods research examples, scholars can link multiple,
complementary methods in different ways to generate better understanding or obtain a more
valid answer to a specific research question. By understanding the different strengths and
weaknesses of particular research strategies as summarized in Table 1, scholars conducting
mixed methods research can offset some of the limitations inherent in a single-method study.

Implications for Theory from Linking Processes

In the previous section, we examined different linking processes by which strategy and

9
organizational scholars triangulate across research strategies when conducting mixed methods
research. In this section, we consider the implications that these processes have for the attributes
of the theory at hand.

In particular, we draw from the work of Thorngate (1976) and Weick (1979), which
considers theoretical explanations in the social sciences with respect to the following attributes:
(a) generality versus specificity in domain scope, (b) simplicity versus complexity in parameters,
and (c) accuracy versus inaccuracy in prediction. The first attribute, generality versus specificity,
refers to the range of phenomena for which the theoretical explanation holds, with general
explanations being preferred to specific ones because they have application across a broader
range of settings. The second attribute, simplicity versus complexity, refers to how parsimonious
the theoretical explanation is, which often reflects how many variables are needed for the
explanation; from this view, parsimonious or simple explanations are considered more elegant as
fewer variables are required to explain a phenomenon. The third attribute, accuracy versus
inaccuracy, refers to how well the theoretical explanation captures the actual state of a given
phenomenon; accuracy is often assessed by how well the predictions derived from the theoretical
explanation correspond with the actual state. Thorngate emphasizes that while it is desirable to
maximize each of these attributes, there are trade-offs among them, which he captures in an
impostulate for theory: “it is impossible for an explanation of social behavior to be
simultaneously general, simple, and accurate” (p. 126). In recognition of these trade-offs, Weick
encourages scholars to accept that “at most only two of those three virtues can be realized” (p.
36) and to focus attention on work that seeks to maximize at most two of the attributes (e.g.,
general and simple) while being particularly cognizant of research by others that addresses the
foregone attribute (e.g., accuracy).

We emphasize the Thorngate/Weick work because it addresses important theoretical

attributes and highlights the presence of inherent trade-offs among them. In addition, the criteria
are applicable for theory even in its earliest stages of development, which is often the case for
mixed methods research studies with the objective of theory development. For evaluation of
more established theory, we highlight that other theoretical frameworks (e.g., Bacharach, 1989)
are also valuable. 8 While Thorngate (1976), Weick (1979), and Bacharach (1989) emphasize
many similar properties, such as generality, parsimony, and predictive accuracy, the latter is a
more comprehensive framework for evaluating established theory according to a range of issues
for variables, constructs, and relationships, and one in which testing plays a prominent role.

Next we examine how the processes that we discussed previously shape attributes of a theory
using several of our examples of linking processes for illustrative purposes.

Convergent triangulation. Proponents of convergent triangulation have stressed that greater

validation for a theory is produced when it has been subject to multiple tests using different
methods and yields consistent results (Denzin, 1970; Jick, 1979; McGrath, 1995). In this study,
we have highlighted two different forms of linking processes focused on convergent
triangulation. The first form focuses on testing theory using two complementary research
8
While the present study focuses on work by Thorngate (1976) and Weick (1979), we encourage those developing
theory in mixed methods research studies to be mindful of the Bacharach (1989) framework; while the focal theory
may not be developed enough for an evaluation based on such a framework, awareness of the corresponding
elements can help in establishing a solid theoretical foundation.
10
strategies.

For example, consider the two research strategies linked in Morris and Moore (2000). Recall
that the researchers were interested in understanding how learning is affected by counterfactual
thinking and how this process is influenced by organizational accountability. For the first test,
they assessed predictions derived from theory using regressions of archival data in which
experienced pilots reflected on near-miss aviation incidents. This method offered an authentic
context for the study participants and helped the researchers to rule out some alternative
explanations and address certain potential confounds (e.g., incident severity). But there were
other aspects that could not be accounted for with the archival data, such as the possibility of
motivational differences between pilots with organizational accountability and those without
such accountability. Thus, Morris and Moore turned to an experimental simulation involving
undergraduate students operating a flight simulator, which enabled them to randomly assign
participants to conditions, and helped them to eliminate plausible alternative explanations and
address the potentially-confounding factors.

From the view of theory attributes, the Morris and Moore (2000) study was one focused on
theory with respect to accuracy (i.e., support for external validity from the archival method and
internal validity from the experimental simulation). But little study evidence was provided to
support the generality of the explanation for other settings as both methods focused on a specific
and similar task: incidents for pilots during real or simulated flight experiences. So the two
research strategies in this study support the accuracy of the theory, with one contributing to
external validity and one to internal validity, but they offer little to support or extend
understanding of the generality of the theoretical explanation.

We highlight that an alternative mix of methods might have supported different theory
attributes. For example, the researchers might have paired the initial archival-based research
strategy with analyses of survey data that were less specific to the particular task and involved a
broader range of participants. If consistent results were found across the two research strategies,
the study would support the accuracy of predictions from the theory based upon greater external
validity and would offer evidence that could support claims of a more general domain for the
theoretical explanation.

The second form of convergent triangulation is one in which the first research strategy is
used for the purpose of theory development, followed by a second that tests predictions derived
from the developed theory. Researchers often employ this triangulation approach as a way of
demonstrating that the developed theory is strong enough to survive an initial round of empirical
scrutiny. As an example, consider the linking approach in Cohen and Klepper (1996). In this
study, the researchers initially developed theory using mathematical modeling, followed by
testing a set of corresponding hypotheses using regressions of archival data that spanned a broad
number of industries. From the view of theory attributes, the study supports a general and simple
theoretical explanation; through the math model, the researchers developed a relatively simple
explanation that they expect to hold for a broad number of industries, and their statistical analysis
of industry-spanning archival data provided evidence to support that the explanation has breadth
in domain scope. However, the study methods and results offered less support with respect to
accuracy; as a research strategy for theory development, mathematical modeling often relies on
strong assumptions (Adner, Polos, Ryall, & Sorenson, 2009; Debreu, 1986; Freese, 1980), and

11
the empirical test using archival data offers limited support for internal validity, including little
in the way of control variables and attempts to account for alternative explanations.

Again, we note that an alternative mix of research strategies could have supported different
attributes for the theoretical explanation. For example, the authors might have followed theory
development using mathematical modeling with a more focused empirical test, such as
regression analyses with archival or survey data that focus on a smaller number of industries for
longer periods of time with more control variables. Such an approach could offer more support in
terms of accuracy through greater internal validity; but it would offer less support for the
generality of the explanation and may suggest the need for more complexity in the explanation to
provide accurate predictions.

Holistic triangulation. The next process, holistic triangulation, involves the use of two or
more research strategies for theory development. In particular, this form of mixed methods
research aims to develop a more complete understanding of a given phenomenon, as when
scholars use methods that are uniquely able to generate or elaborate certain ideas (i.e., increasing
the breadth or depth of an explanation).

Consider Cardinal et al. (2011) as an example. In the study, the researchers set out to explore
a contingency-based model of product development, where project performance was a function
of the fit between technological environment and project design. The researchers initially
examined their propositions using computer simulation, the results of which provided some
support and some contradictions with their initial theory. They then turned to multiple case
studies (Eisenhardt, 1989) to try to better understand how project performance was a function of
technological environment and project design. Rather than supporting a traditional interaction-
based contingency model of product development, the two methods in combination suggested a
mediation-based contingency model, whereby the technological environment influenced
performance priorities that affected project design and project performance. Holistic
understanding was the product of case studies contributing unique insight in terms of how
managers prioritized different performance dimensions in different environments, which affected
project design (i.e., the first part of the mediation model), and the computer simulation method
contributing unique insight in terms of how project designs affected particular dimensions of
project performance (i.e., the second part of the mediation model).

With respect to theory attributes, for the first part of the mediation model, the multiple case
study method contributed to accuracy on the basis of external validity (Eisenhardt & Graebner,
2007), and for the second part of the mediation model, the computer simulation method
contributed to accuracy in terms of internal validity (i.e., precise control, manipulation of
variables). So both methods contributed to accuracy in different ways for different parts of the
model. But as is often the case with theory development from case studies, the resulting
theoretical explanation was complex (Eisenhardt, 1989). Also, while the study offered limited
evidence to support domain generality for the theory, it did offer some support for generality in
that the simulation platform had been validated in a variety of project management settings (Jin
& Levitt, 1996; Levitt, Cohen, Knuz, Nass, Christiansen, & Jin 1994) and the theory was derived
from multiple case studies (Eisenhardt, 1989; Eisenhardt & Graebner, 2007).

As an alternative, the researchers could have linked the research strategies in a different way,

12
with different implications for theory attributes. Rather than using two methods to explore two
different parts of the model, the researchers might have used two methods to consider the entire
model. For example, they could have initially used multiple case study methodology to explore
how technological environment affects performance priorities and project design and
performance, and then as a complement, conducted interviews with project managers focused on
product development across a broad array of industries. In so doing, their study would have
provided greater support for generality in the domain for the explanation; at the same time, the
resulting theory would still be complex (i.e., based on analyses of case studies and analyses of
interviews) and the overall support for accuracy in predication likely unchanged (i.e., the
interviews might provide additional support in terms of external validity but likely less internal
validity without the simulation).

Convergent and holistic triangulation. The third process is convergent and holistic
triangulation, which focuses on using multiple research strategies for both developing and testing
theory. Similar to convergent triangulation, the researchers aim to find areas with consistent
results across the methods, but like holistic triangulation, they also expect that at least one
method will be able to offer unique insight that can extend understanding of the phenomenon.

In thinking about how the linking process for convergent and holistic triangulation affects
theory attributes, recall the Dyer and Hatch (2006) study of how knowledge sharing by buyers
affects the performance of buyer-supplier networks. The researchers initially used regression
analyses of survey data to test their theoretical argument regarding how buyers can generate
competitive advantage by sharing knowledge with their suppliers. Next, they turned to
interviews, first in seeking corroborating evidence for their survey-based findings and second to
explore the related question of how the created competitive advantage is sustained. Overall, the
theoretical explanation for how knowledge sharing led to the creation of competitive advantage
was supported by the survey results and the interviews (convergent triangulation), and qualitative
analysis of the interviews also developed understanding of how that created advantage was
sustained (holistic triangulation). From the perspective of theory attributes, through the
combination of methods, the study supported the accuracy of predictions derived from the theory
(i.e., support for internal validity from the surveys and interviews). But the study provided little
evidence to support the generality of the theoretical explanation, given that both the surveys and
interviews used the same small sample of actors.

As we have suggested earlier, an alternative linking of research strategies could have

different implications for theory attributes. For example, the researchers might have initially used
the interviews to further develop the theoretical explanation, both for creating and sustaining
competitive advantage, and then used regressions of survey data from a broader sample of
participants to test the core elements of the explanation. If so, the study would have greater
support in terms of accuracy in prediction (i.e., internal validity from the interviews and surveys)
and more support for the domain generality of the explanation by testing the predictions with a
wider range of actors (i.e., external validity from the surveys). Further, both research strategies
would have contributed to both aspects of the explanation (i.e., creating and sustaining
competitive advantage). While starting with the interviews may increase the complexity of the
theory, there would also be an opportunity to revisit the interview transcripts following the
survey results for further interpretation, insight, and theory development (similar to Ely, 1994).

13
A Roadmap for Triangulation-Based Mixed Methods Research

In this section, we present a roadmap that can serve a design guide for organizational
scholars seeking to pursue triangulation-based mixed methods research. Figure 3 summarizes the
key steps for this roadmap.

*** Insert Figure 3 Here ***

Step 1: Decide on the Research Question

Like any good research project, an early objective for scholars is to identify an intriguing
research question that has the potential to yield interesting and important insights for theory or
understanding of a phenomenon. Scholars should also have clear understanding of prior work
that has considered the research question, as this can play a critical role in both recognizing and
articulating why the question has good prospects for yielding important contributions for the
field, and oftentimes, deep understanding of prior research enables fruitful evolution and
refinement in the question. From the perspective of research design, particular attention should
be given to the form of the question, e.g., why, what, how. This is an extension of the idea that
particular methods are well-suited for addressing particular forms of research questions; for
example, case study methodology is appropriate for how and why questions (Eisenhardt &
Graebner, 2007; Yin, 1994); in similar fashion, for mixed methods research, particular
combinations of methods are well-suited for producing insight for certain forms of research
questions.

As examples, consider some of the research questions in the exemplar studies we presented
earlier. Cohen and Klepper (1996) highlighted how it is important for policymakers, managers,
and academics to understand what drives the composition of firms’ R&D efforts and focused on
the question of what is the effect of firm size on the composition of R&D across product and
process innovation. In work by Cardinal et al. (2011), the researchers focused attention on the
issue of organizing for product development, examining the specific question of how project
performance is affected by technological environment and project design.

Step 2: Determine Theoretical Intentions

Once scholars have an important research question in mind, their attention should be directed
to determining their intentions with respect to theory. Theoretical intentions encompass both
theoretical purpose and theoretical attributes. For theoretical purpose, the decision is whether the
research at hand is primarily oriented to theory development, theory testing, or both developing
and testing theory. For theoretical attributes, the decision involves determining priorities with
respect to the generalizability, simplicity, and accuracy (GAS) of the theory, given trade-offs
among these attributes (Thorngate, 1976; Weick, 1979). For theory development, this pertains to
the theory that researchers are trying to create or elaborate, while for theory testing, it relates to
whether the theory is being tested in such a way that claims of generality, simplicity, or accuracy
can be supported.

While theoretical purpose has received considerable attention in the organizational sciences,
this is less the case for theoretical attributes, although it is no less important in the pursuit of

14
heightened understanding of a phenomenon or theory. What makes theoretical attributes
particularly important is the presence of often unrecognized or unacknowledged trade-offs
(Thorngate, 1976). Weick (1979, p. 36) states, “failure to accept the inevitable tradeoffs implied
in the GAS formulation seems to be at the heart of many current research problems. Investigators
act as if they can simultaneously accomplish all three aims in their explanations, and that
delusion is at the heart of much trivial, inconclusive research.” Thus, researchers need to
explicitly decide which one or two among the theoretical attributes of generality, accuracy, and
simplicity is their intention for the study.

As examples of theoretical intentions, we return to our set of exemplars. Cohen and Klepper
(1996) stated the first aim clearly as they proposed to develop and test a theory for the effect of
firm size on the allocation of R&D effort between process and product innovation; while their
second aim was more implicit, it was evident that they were seeking to develop a general theory.
Cardinal et al. (2011) stated their first aim as one of theory development, specifically theory
elaboration (Eisenhardt & Graebner, 2007), and they prioritized accuracy in their efforts to
extend and refine extant theory.

Step 3: Select the Process for Triangulation

After determining the theoretical intentions for the focal work, the next step for scholars is to
decide on the process for triangulation, which makes clear which objective(s) they are trying to
accomplish through the use of a mixed methods design. As we have discussed, the three core
processes are convergent triangulation, holistic triangulation, and convergent and holistic
triangulation. For scholars with the intention to develop theory, there are several options. If
focusing solely on theory development, holistic triangulation is the appropriate process. If
researchers are seeking to combine theory development with theory testing, one option is the
form of convergent triangulation where one method is used to develop theory and another
method is used to test the developed theory (e.g., Grant et al., 2014), while the other option is
convergent and holistic triangulation, where one method can develop theory and another method
tests and extends the theory (e.g., Ely, 1994), or one method can test theory while another
method provides a complementary test and then extends the theory (e.g., Dyer & Hatch, 2006).
For scholars focused solely on theory testing, the appropriate process is convergent triangulation,
where multiple methods are used to test the same theory (e.g., Bardolet et al., 2011).

A related issue for researchers is deciding whether to pursue the selected process for
triangulation by linking multiple methods within one research study or by using different
methods across multiple research studies. In the case of linking methods within a single research
study, design decisions are often made in advance of initiating the work, while scholars engaged
in using different methods across research studies frequently make corresponding decisions in
later stages, namely, greater clarity on complementary methods emerges as a research program
evolves and progresses. For scholars considering the question of whether to link methods within
a single study, a key factor should be the theoretical intentions for the study; for example, if the
intention is to develop comprehensive theory, it may require a holistic triangulation process that
is based in linking particular methods within a single research study; similarly, linking within a
study ensures that triangulation actually happens for a given question, recognizing that research
attention on particular problems can be fleeting (Staw, 1982). Scholars should also keep in mind
pragmatic issues with decisions regarding linking methods within a single study, recognizing that

15
there can be value in linking methods that are related in certain ways. For example, conducting
interviews and surveys with a similar sample enables time/resource efficiencies (e.g., Ely, 1994),
and focusing on a similar task for methods involving archival data and design of an experiment
can enable a more efficient and coherent presentation of the ideas to an audience (e.g., Bardolet
et al., 2011).

Step 4: Determine the Mix of Methodologies

As the last step in triangulation-based research design, scholars need to decide which
combination of research strategies will best meet their objectives, taking into account the
research question, theoretical intentions, and process of triangulation. In other words, scholars
should select their methodologies following their identification of the research question(s),
determination of theoretical purpose (i.e., development and/or testing), prioritization among
theoretical attributes (i.e., generality, accuracy, simplicity), and selection of the process for
triangulation (i.e., convergent, holistic, or both). To illustrate, we look at one or two exemplar
studies for each of the triangulation processes.

Convergent triangulation. For processes of convergent triangulation, consider first the

situation where a researcher wants to use one research strategy to develop theory and another to
test the developed theory and has the intention to develop a general theory. Cohen and Klepper
(1996) is good example of such a study. With these objectives, the scholars used mathematical
modeling as a method that is conducive to development of general theory. They then used a
compatible method for testing the developed theory, specifically, conducting regression analysis
on archival data spanning a wide range of industries as a way to provide evidence that can
support the generality of the theory. Second, consider the form of convergent triangulation in
which a researcher wants to use two research strategies for purposes of testing the same theory.
As a good example for this process, recall Bardolet et al. (2011). As the initial research strategy,
the researchers utilized regression analyses of archival data that spanned many industries, which
contributed to external validity and supported the generality of the theory. Next, as a
complementary methodology, they conducted a lab experiment, which supported the accuracy of
the theory through internal validity based in greater precision in control and measurement.

Holistic triangulation. For the process of holistic triangulation, scholars are focusing
exclusively on theory development, but they similarly want to determine the set of research
strategies based upon their research question and theoretical intentions. For example, consider
Cardinal et al. (2011). In this study, the researchers implicitly selected accuracy as their focal
priority and generality as a secondary priority. As one method, they used computer simulation,
which is well suited for theory development and contributes to internal validity (Burton & Obel,
2011; Davis et al., 2007); in turn, their choice of simulation platform (VDT/SimVision) was one
in which the model had been calibrated using a range of actual projects across various industries,
such that the findings enabled some claims as to accuracy as well as generality for the theory. As
the other method, the researchers used case studies, which is also appropriate for theory
development, and because their approach involved multiple case studies, the method offers
support for claims of generality and accuracy for the developed theory (Eisenhardt & Graebner,
2007).

A key point here is that both research strategies were appropriate for the purpose of theory

16
development, but they also were complementary to one another. Specifically, both were helpful
in developing a broad/holistic understanding of how project performance is a function of
technological environment and project design, but they also offered unique contributions, with
the case studies contributing more insight for the first part of the mediation model (i.e., how
managers prioritized performance differently in different technological environments) and the
simulations contributing more for the second part of the model (i.e., how project designs
influenced project performance).

Convergent and holistic triangulation. Similar to the other processes, when scholars are
engaged in convergent and holistic triangulation, they should determine the set of research
strategies in light of their research question and theoretical intentions. With convergent and
holistic triangulation, scholars are focused on both theory development and theory testing,
although not necessarily in that order.

Recall the Dyer and Hatch (2006) study that was primarily oriented to accuracy. The
researchers began with the intention to test predictions from their theory of how buyers create
relative competitive advantage by sharing knowledge with their suppliers, and they used
regression analysis of survey data as a way of assessing the validity of the theory. Then, as a
complement, the researchers turned to interviews. Initially they examined whether the survey
results would be confirmed by the interviews; the researchers then used the interviews to develop
theory regarding a related research question (why the created relative competitive advantage is
sustained).

In sum, this section draws from the triangulation-based research design framework to
identify key steps for designing mixed methods studies. In the final section, we extend this
discussion to consider issues related to the validity of these studies and the practical challenges
involved in conducting them.

Discussion

While scholars have long argued for the benefits of triangulation-based research (Denzin,
1970; Jick, 1979), researchers have been slow to adopt mixed methods in the organizational
sciences (Creswell, 2014; Scandura & Williams, 2000). In our view, a key factor in this slow rate
of adoption is the limited guidance available for organizational scholars in terms of how they can
design research studies using mixed methods to accomplish their goals. In this study, we develop
a research design framework for mixed methods research that is based on the principles of
triangulation. Specifically, this framework draws together two areas that are paramount for
mixed methods research: (a) the methodologically-based work focused on triangulation (e.g.,
Jick, 1979; McGrath, 1982) and (b) the theoretically-oriented research that identifies associated
trade-offs (e.g., Thorngate, 1976; Weick, 1979). 9 Based on the research design framework, we
develop a roadmap that can serve as a design guide for organizational scholars who are interested
in conducting mixed methods research. We see these developments as timely and important as
there are signs of growing interest in mixed methods research in areas like strategic management,
along with evidence that this type of work is impactful (Molina-Azorin, 2012). In this section,

9
We thank a reviewer for highlighting that this integration of the methodological and theoretical represents a
valuable contribution in the triangulation literature.
17
we extend these ideas in two important areas: (1) from a design perspective, we offer further
insight into the core factors that constitute exemplary mixed methods research studies, with
particular attention on selecting an effective mix of research strategies, and (2) from a practical
perspective, we address some of the key challenges associated with conducting mixed methods
research in the organizational sciences.

First, a central issue in the design of mixed methods research is determining the particular set
of methods to address the research question. From a triangulation perspective, scholars like
McGrath (1982, 1995) have stressed the importance of selecting research strategies with
maximum divergence in terms of prioritizing generalizability across populations, precision in
control and measurement, and authenticity of context. But as Jick (1979, p. 604) highlights, the
quality of triangulation-based mixed methods research design hinges on a “buried” assumption:
“the effectiveness of triangulation rests on the premise that the weaknesses in each single method
will be compensated by the counter-balancing strengths of another... although it has always been
observed that each method has assets and liabilities, triangulation purports to exploit the assets
and neutralize, rather than compound, the liabilities.” Thus, thoughtful deliberation is required
when selecting the combination of research strategies for mixed methods work in order to ensure
that the research strategies do not share the same liabilities and that the strengths of one method
serve to offset the weaknesses of another. While guidance for researchers has emphasized
maximum divergence for addressing these issues (McGrath, 1982, 1995), we argue that great
mixed methods research studies strike a productive balance between divergence and
commonality across the methods.

Divergence. From one perspective, researchers need to seek divergence in the set of research
strategies in order to offset vulnerabilities that are inherent to the individual strategies. For
example, consider a scholar who is interested in pursuing a convergent triangulation study that
uses two different methods to test a theoretical explanation for a research question. By choosing
a laboratory experiment as one of the methods, the researcher is well-positioned with respect to
inferences about causality and threats to internal validity, but he or she is subject to considerable
threats in areas like reactive measurement, which refers to the potential influence that the process
of observation may have on a subject, and external validity (McGrath, 1995; Shadish et al., 2002;
Singleton & Straits, 1999). Given these threats, an effective research design would pair the lab
experiment with a research strategy that is strong in the latter areas, such as archival records. 10
We highlight that one of our exemplars – Bardolet et al. (2011) – followed this approach in their
selection of archival records and a lab experiment for their mixed methods study.

As another example, consider a scholar who is interested in conducting a convergent and

holistic triangulation study for the purpose of theory development. If he or she were to select a
case study for one of the research strategies, which offers a number of advantages with respect to
inductive theory development (Gioia, Corley, & Hamilton, 2012; Glaser & Strauss, 1967), the
researcher would be subject to criticism in areas like internal validity (e.g., lack of control),
construct validity (e.g., measurement difficulty), and difficulty in replication (Shadish et al.
2002; Singleton & Straits, 1999). For an effective mixed methods design, the researcher would
be well-served to select a research strategy that in strong in the latter areas and thus capable of

10
With respect to the threat of reactive measurement, a related factor for archival records can be the extent to which
subjects are aware that the records are or are likely to be made public (McGrath, 1995, p. 164).
18
offsetting the key limitations of the case study method. Good examples would be research
strategies based on a survey or experiment (Singleton & Straits, 1999). We highlight that one of
our exemplars, Grant et al. (2014), followed this particular approach in selecting the set of
research strategies for their study. Thus, we concur with scholars’ recommendations that
effective mixed methods designs seek divergence across methods, such that a method that is
subject to a particular validity threat is offset by another that is strong in that respect. But we also
believe that there is considerable value in seeking commonality in order to facilitate
interpretation across the methods.

Commonality. While commonality has received less attention in mixed methods design, it is
no less important. This aspect focuses on the value of making incremental advances in
knowledge accrual by holding certain methodological elements constant. Specifically, while
extant research emphasizes the value of maximizing divergence across research strategies to
offset the limitations inherent to the individual strategies, it is also important to realize the
challenges that this can pose for interpreting the set of results. For example, if a researcher is
pursuing convergent triangulation and finds support for the theory using a particular research
strategy, setting, and sample and finds no support for it using a very different research strategy,
setting, and sample, it may be hard to fully understand the implications for theory. But if the
latter involved a similar setting and sample and varied only the research strategy, then the
researcher is in a better position to interpret potential mixed results and contribute to theory. We
frequently observed this incremental approach to mixed methods research in our exemplar
studies, including the use of the same sample with different research strategies (Dyer & Hatch,
2006; Ely, 1994), and use of the same basic task with different research strategies (Bardolet et
al., 2011; Morris & Moore, 2000).

As a related point, we encourage scholars to keep closely in mind their theoretical intentions
(i.e., generality, simplicity, accuracy) as they determine empirical settings and samples. Rather
than trying to find settings and samples that contribute to all three of the GAS attributes and
running into trade-off challenges, we would encourage researchers to try to create samples that
align with their theoretical intentions and to be explicit regarding these choices in presenting the
research. This is also consistent with the idea that research strategies can have multiple purposes.
For example, depending on the setting and sample, archival records could be drawn upon to
support the generality of a theory (Cohen & Klepper, 1996) or the accuracy of a theory (Morris
& Moore, 2000). Similarly, surveys can be conducted with a broad range of participants when
generalizability of populations is a priority, as McGrath (1982, 1995) suggests. But in our
exemplars, we found that surveys often involved a much narrower range of participants,
frequently using the same or a very similar sample of participants in a pairing of research
strategies (e.g., surveys, interviews); in these instances, while the surveys did not contribute
much to supporting the generalizability of a theory, they offered greater precision in
measurement and control over the field-based methods that they were paired with (Dyer &
Hatch, 2006; Ely, 1994), as well as pragmatic value, for example, time/resource efficiencies.

In sum, we consider how divergence and commonality in the set of methods affect validity
and interpretability and emphasize that effective mixed methods research strikes a productive
balance between divergence and commonality. On the one hand, validity is enhanced through
divergence when scholars select research strategies that diverge from one another in ways that
can offset the inherent vulnerabilities of the individual strategies. Thus, a cautionary note for

19
mixed methods researchers is to ensure they do not select research strategies that are too similar;
otherwise, we end up “driving our triangulation posts into the [same] spot” (McGrath et al.,
1982, p. 111). On the other hand, interpretation is enhanced through commonality when mixed
methods researchers vary only certain methodological elements, such as the research strategy,
while holding other methodological elements constant (e.g., empirical setting, sample). From this
perspective, mixed methods researchers also need to be careful that they do not seek maximal
divergence in their mix of methodological elements and thereby weaken their capacity to
interpret the set of results. This balancing among divergence and commonality points to the need
for future research to give more attention to the benefits and drawbacks of triangulation-based
approaches to mixed methods research, particularly from the view of variability within and
across multiple methodological elements, for example, research strategies, empirical settings,
samples, observers (Denzin, 2012; Scandura & Williams, 2000).

In addition, we highlight that a related issue of importance involves generativity, namely,

how the mix of methods influences the generation of new theoretical insights and uncovering of
novel research questions. One can imagine ways in which greater divergence provides the
heterogeneity and breadth needed for creative insight and development of novel theory;
alternatively, one can consider how greater commonality may enable sharper focus and more
depth of understanding, which fosters the development of new theoretical insight. While outside
the scope of the present study, we encourage future research to consider this important question
of how methodological divergence and commonality influence the development of emergent
theory and identification of important, unexplored research questions.

Second, conducting mixed methods research poses a number of practical constraints for
researchers in the organizational sciences. For one, mixed methods research can be very time-
consuming. With respect to the time constraint, we advise prospective scholars to be intentional
in their selection of research strategies, not just from the perspective of validity and
interpretability but also with respect to time requirements. Fortunately, the latter often go hand in
hand. While we previously advocated for commonality across research strategies for
interpretation value, doing so can also be pragmatic with respect to time conservation. As
examples, using different methods involving a similar or identical sample can save time with
respect to data collection (Dyer & Hatch, 2006; Ely, 1994), and using methods involving the
same type of activity (e.g., Morris & Moore, 2000) can conserve time with respect to background
preparation for conducting the study.

Another practical challenge for mixed methods research is assembling the necessary
expertise for conducting the study. By its very nature, mixed methods research requires expertise
in different methodologies, with well-designed studies frequently involving very different
methods. Given that methodological expertise often involves specialization, a team of
researchers can be required for conducting mixed methods research. Further, in much the same
way as the mix of methods requires a balance of divergence and commonality, the team of
researchers itself needs a mix of experts from different areas, disciplines or schools, but they also
must have overlapping expertise as well to facilitate interpretation. In addition, a spirit of
collaboration and learning is a must.

The publication process can also be challenging for mixed methods research. As Jick (1979,
p. 604) indicated, whether explicitly or implicitly, “journals tend to specialize by methodology

20
thus encouraging purity of method.” When targeting such an outlet for mixed methods research,
scholars need to be cognizant of which methods are favored and capable of clearly articulating
how the focal set of methods can lead to greater insight and understanding for the question at
hand. Further, rather than having the methods stand on equal footing, a researcher may choose to
emphasize the more accepted methodology, which is then complemented by a more novel
method according to the standards of the outlet/field. Along similar lines, Creswell (2014)
advises that scholars take into account the inclination of one’s field to particular methods and
designs.

But publishing mixed methods research can be challenging even in journals that have
policies encouraging diverse and mixed methods. To constitute high-quality mixed methods
research, each method must be done well, and the mix of methods must say something more; in
other words, the methods as a set need to go beyond the separate methods. This can present
practical challenges in multiple ways. For one, in the process of demonstrating that each method
has been done well and saying something more, it is easy to run into page length constraints with
journal articles. As another, while journals may have policies that advocate mixed methods, it is
not always the case that individual reviewers do. In these instances, authors can often find
considerable value from the use of exemplar mixed methods studies, both as guides for framing
and as indicators of legitimacy given the early stage of mixed methods research (Creswell,
2014). In addition, editors clearly play a critical role, both with selecting reviewers and in
guiding the subsequent review process.

Last, even after the studies have been published, mixed methods research can pose
challenges. As Jick (1979, p. 609) observes, triangulating across methodologies makes
replication more challenging, particularly when qualitative data/analyses are involved. While
mixed methods research clearly magnifies the challenges of replication, we note that the extent
likely depends upon study design; for example, some of the challenges with replication are
attenuated when mixed methods studies involve multiple research strategies that center on the
same sample or empirical setting.

With all these challenges and difficulties with mixed methods research, one may ask why
bother. In fact, without an effective research design, it is likely not worth the effort. But if
designed well, mixed methods research studies provide deeper insight and better understanding
for our research questions, relative to single method studies.

Conclusion

While mixed methods offers a powerful way to broaden and deepen our understanding of
phenomena and theories, there has been a slow rate of adoption in the organizational sciences.
Our purpose in this article is two-fold. First, we aim to clarify how scholars use mixed methods
to answer their research questions by developing a research design framework that is based on
the principles of triangulation. Second, we offer guidance for scholars interested in mixed
methods research by providing a corresponding roadmap that begins with the research question
and theoretical intentions and culminates in the selection of methodologies. In doing so, we hope
that this study can help to advance mixed methods research in the organizational sciences.

21
REFERENCES
Adner, R., Polos, L., Ryall, M., & Sorenson, O. (2009). The case for formal theory. Academy of
Management Review, 34, 201-208.
Aguinis, H., Pierce, C. A., Bosco, F. A., & Muslin, I. S. (2009). First decade of Organizational
Research Methods: Trends in design, measurement, and data-analysis topics. Organizational
Research Methods, 12, 69-112.
Anteby, M. (2010). Markets, morals, and practices of trade: Jurisdictional disputes in the U.S.
Commerce in cadavers. Administrative Science Quarterly, 55, 606–638.
Austin, J. T., Scherbaum, C. A., & Mahlman, R. A. (2002). History of research methods in
industrial and organizational psychology: Measurement, design, analysis. In S. G. Rogelberg
(Ed.), Handbook of research methods in industrial and organizational psychology (pp. 1-33).
Oxford, UK: Blackwell.
Axtell, R., Axelrod, E., Epstein, J. M., & Cohen, M. D. (1996). Aligning simulation models: A
case study and results. Computational and Mathematical Organization Theory, 1, 123-142.
Bacharach, S. B. (1989). Organizational theories: Some criteria for evaluation. Academy of
Management Review, 14, 496-515.
Bardolet, D., Fox, C. R., & Lovallo, D. (2011). Corporate capital allocation: A behavioral
perspective. Strategic Management Journal, 32, 1465–1483.
Bateman, T. S., & Zeithaml, C. P. (1989). The psychological context of strategic decisions: A
model and convergent experimental findings. Strategic Management Journal, 10, 59-74.
Burton, R. M. (2003). Computational laboratories for organization science: Questions, validity
and docking. Computational and Mathematical Organization Theory, 9, 91–108.
Burton, R. M., & Obel, B. (1995). The validity of computational models in organization science:
From model realism to purpose of the model. Computational and Mathematical Organization
Theory, 1, 57-72.
Burton, R. M., & Obel, B. (2011). Computational modeling for what-is, what-might-be, and
what-should-be studies – and triangulation. Organization Science, 22, 1195-1202.
Campbell, D. T., & Fiske, D. W. (1959). Convergent and discriminant validation by the
multitrait-multimethod matrix. Psychological Bulletin, 56, 81-105.
Cardinal, L. B., Turner, S. F., Fern, M. J., & Burton, R. M. (2011). Organizing for product
development across technological environments: Performance trade-offs and priorities.
Organization Science, 22, 1000–1025.
Cohen, W. M., & Klepper, S. (1996). Firm size and the nature of innovation within industries:
The case of process and product R&D. The Review of Economics and Statistics, 78, 232–243.
Creswell, J. W. (2014). Research design: Qualitative, quantitative, and mixed methods
approaches. Thousand Oaks, CA: Sage Publications.
Davis, J. P., Eisenhardt, K. M., & Bingham, C. B. (2007). Developing theory through simulation
methods. Academy of Management Review, 32, 480–499.
Debreu, G. (1986). Theoretic models: Mathematical form and economic content. Econometrica,
54, 1259–1270.
Denzin N. K. (1970). The research act: A theoretical introduction to sociological methods.
Chicago, IL: Aldine Publishing Company.
22
Denzin, N. K. (2012). Triangulation 2.0. Journal of Mixed Methods Research, 6, 80-88.
Dyer, J. H., & Hatch, N. W. (2006). Relation-specific capabilities and barriers to knowledge
transfers: Creating advantage through network relationships. Strategic Management Journal,
27, 701–719.
Eisenhardt, K. M. (1989). Building theories from case study research. Academy of Management
Review, 14, 532-550.
Eisenhardt, K. M., & Graebner, M. E. (2007). Theory building from cases: Opportunities and
challenges. Academy of Management Journal, 50, 25–32.
Ely, R. J. (1994). The effects of organizational demographics and social identity on relationships
among professional women. Administrative Science Quarterly, 39, 203–238.
Flynn, F. J., & Staw, B. M. (2004). Lend me your wallets: The effect of charismatic leadership
on external support for an organization. Strategic Management Journal, 25, 309-330.
Freese, L. (1980). Formal theorizing. Annual Review of Sociology, 6, 187–212.
Gibbert, M., & Ruigrok, W. (2010). The “what” and “how” of case study rigor: Three strategies
based on published work. Organizational Research Methods, 13, 71-737.
Gioia, D. A., Corley, K. G., & Hamilton, A. L. (2012). Seeking qualitative rigor in inductive
research: Notes on the Gioia methodology. Organizational Research Methods, 16, 15-31.
Glaser, B. G., & Strauss, A. L. (1967). The discovery of grounded theory: Strategies for
qualitative research. New York: Aldine de Gruyter.
Grant, A. M., Berg, J. M, & Cable, D. M. (2014). Job titles as identity badges: How self-
reflective titles can reduce emotional exhaustion. Academy of Management Journal, 37,
1201-1225.
Harrison, J. R., Lin, Z., Carroll, G. R., & Carley, K. M. (2007). Simulation modeling in
organizational and management research. Academy of Management Review, 32, 1229–1245.
Jick, T. D. (1979). Mixing qualitative and quantitative methods: Triangulation in action.
Administrative Science Quarterly, 24, 602-611.
Jin, Y., & Levitt, R. E. (1996). The virtual design team: A computational model of project
organizations. Computational and Mathematical Organization Theory, 2, 171–195.
Lee, T. W., Mitchell, T. R., & Sablynski, C. J. (1999). Qualitative research in organizational and
vocational psychology, 1979-1999. Journal of Vocational Behavior, 55, 161-187.
Levitt, R. E., Cohen, G. P., Knuz, J. C., Nass, C. I., Christiansen, T., & Jin, Y. (1994), The
‘virtual design team’: Simulating how organization structure and information processing
tools affect team performance. In K. M., Carley & M. J., Prietula, (Eds.), Computational
organization theory (pp. 1-18). New York: Lawrence Erlbaum Associates.
Lin, Z., Zhao, X., Ismail, K. M., & Carley, K. M. (2006). Organizational design and restructuring
in response to crises: Lessons from computational modeling and real-world cases.
Organization Science, 17, 598–618.
McGrath, J. E. (1982). Dilemmatic: The study of research choices and dilemmas. In J. E.
McGrath, J. Martin, & R. A. Kulka (Eds.), Judgment calls in research (pp. 69-102). Beverly
Hills, CA: Sage Publications.
McGrath, J. E. (1995). Methodology matters: Doing research in the behavioral and social
sciences. In R. M. Backer (Eds.), Readings in human-computer interaction: Toward the year
23
 2000 (pp. 152–169). Burlington, MA: Morgan Kaufman.
McGrath, J. E., Martin, J., & Kulka R. A. (1982). Some quasi-rules for judgment calls in
research. In J. E. McGrath, J. Martin, R. A. Kulka (Eds.), Judgment calls in research (pp.
103-118). Beverly Hills, CA: Sage Publications.
Molina-Azorin, J. F. (2007). Mixed methods in strategy research: Applications and implications
in the resource-based view. Research Methodology in Strategy and Management, 4, 37-73.
Molina-Azorin, J. F. (2012). Mixed methods research in strategic management: Impact and
applications. Organizational Research Methods, 15, 33–56.
Morris, M. W., & Moore, P. C. (2000). The lessons we (don’t) learn: Counterfactual thinking
and organizational accountability after a close call. Administrative Science Quarterly, 45,
737–765.
Nguyen, H. D., & Nguyen, N. T. (2008). Examining personal values and entrepreneurial motives
of Vietnamese entrepreneurs in the 21st century: Two empirical studies. African and Asian
Studies, 7, 141-171.
Pedhazur, E. J, & Schmelkin, L. (1991). Measurement, design and analysis: An integrated
approach. New York, NY: Psychology Press.
Scandura, T. A., & Williams, E. A. (2000). Research methodology in management: Current
practices, trends, and implications for future research. Academy of Management Journal, 43,
1248-1264.
Shadish, W. R., Cook, T. D., & Campbell, D. T. (2002). Experimental and quasi-experimental
designs for generalized causal inference. Boston: Houghton-Mifflin Company.
Shah, S. K., & Corley, K. G. (2006). Building better theory by bridging the quantitative-
qualitative divide. Journal of Management Studies, 43, 1821-1835.
Singleton, R., & Straits, B. C. (1999). Approaches to social research. Oxford, UK: Oxford
University Press.
Staw, B. M. (1982). Some judgments on the judgment calls approach. In J. E. McGrath, J.
Martin, R. A. Ku, J. E. Martin, R. A. Kulka (Eds.), Judgment calls in research (pp. 119-127).
Beverly Hills, CA: Sage Publications.
Thorngate, W. (1976). Possible limits on a science of social behavior. In J. H. Strickland, F. E.
Aboud, & K. J. Gergen (Eds.), Social psychology in transition (pp. 121-139). New York,
NY: Plenum Press.
Torrance, H. (2012). Triangulation, respondent validation, and democratic participation in mixed
methods research. Journal of Mixed Methods Research, 6, 111-123.
Weick, K. E. (1969). The social psychology of organizing (first edition). Menlo Park, CA:
Addison-Wesley Publishing Company.
Weick, K. E. (1979). The social psychology of organizing (second edition). Menlo Park, CA:
Addison-Wesley Publishing Company.
Yin, R. K. (1994). Case study research: Design and methods (second edition). Beverly Hills,
CA: Sage Publications.

24
Table 1. Methodological Purposes Fulfilled by Particular Research Strategies

Research
Methodological Purpose within Mixed Methods Research Design
Strategy
Archival records can be used in a variety of ways. As a strategy, it can be
Archival effective in maximizing generalizability with respect to populations,
Records enhancing precision in control/measurement of variables, and/or capturing
behaviors that have taken place in an authentic context
Case studies are well-suited for capturing behaviors that have taken place in
an authentic context. Although this method is typically not well-suited for
Case Study
maximizing generalizability with respect to populations, the use of multiple
case studies allows for more claims regarding generalizability
Computational simulation is well-suited for enhancing precision in
Computer
control/measurement of variables and can be effective in maximizing
Simulation
generalizability with respect to populations
Experimental simulation can be effective in enhancing precision in
Experimental
control/measurement of variables and capturing behaviors that have taken
Simulation
place in an authentic context
Field experiments can be effective in enhancing precision in
Field
control/measurement of variables and capturing behaviors that have taken
Experiment
place in an authentic context
Formal theory through mathematical models is well-suited for enhancing
Formal Theory
precision in control/measurement of variables and can be effective in
(Mathematical)
maximizing generalizability with respect to populations
Interviews are well-suited for capturing behaviors that have taken place in
Interviews
an authentic context
Lab Lab experiments are well-suited for precision in control/measurement of
Experiment variables
Surveys can be effective in precision in control/measurement of variables
Survey
and capturing behaviors that have taken place in an authentic context

25
26
27
 28

View publication stats