Sampling and Data Collection Means (Methods)

Implications of a Sample Design

A sample design is a definite plan for obtaining a sample from a given population.
It refers to the technique or the procedure the researcher would adopt in selecting
items for the sample. Sample design may as well lay down the number of items to
be included in the sample i.e., the size of the sample. Sample design is determined
before data are collected. There are many sample designs from which a researcher
can choose. Some designs are relatively more precise and easier to apply than others.
Researcher must select/prepare a sample design which should be reliable and
appropriate for his research study.
While developing a sampling design, the researcher must pay attention to the
following points:
(i) Type of universe: The first step in developing any sample design is to clearly
define the set of objects, technically called the Universe, to be studied. The universe
can be finite or infinite. In finite universe the number of items is certain, but in case
of an infinite universe the number of items is infinite, i.e., we cannot have any idea
about the total number of items. The population of a city, the number of workers in
a factory and the like are examples of finite universes, whereas the number of stars
in the sky, listeners of a specific radio programme, throwing of a dice etc. are
examples of infinite universes.
(ii) Sampling unit: A decision has to be taken concerning a sampling unit before
selecting sample. Sampling unit may be a geographical one such as state, district,
village, etc., or a construction unit such as house, flat, etc., or it may be a social unit
such as family, club, school, etc., or it may be an individual. The researcher will
have to decide one or more of such units that he has to select for his study.
(iii) Source list: It is also known as ‘sampling frame’ from which sample is to be
drawn. It contains the names of all items of a universe (in case of finite universe
only). If source list is not available, researcher has to prepare it. Such a list should
be comprehensive, correct, reliable and appropriate. It is extremely important for the
source list to be as representative of the population as possible.
(iv) Size of sample: This refers to the number of items to be selected from the
universe to constitute a sample. This a major problem before a researcher. The size
1
of sample should neither be excessively large, nor too small. It should be optimum.
An optimum sample is one which fulfills the requirements of efficiency,
representativeness, reliability and flexibility.
While deciding the size of sample, researcher must determine the desired precision
as also an acceptable confidence level for the estimate. The size of population
variance needs to be considered as in case of larger variance usually a bigger sample
is needed. The size of population must be kept in view for this also limits the sample
size. The parameters of interest in a research study must be kept in view, while
deciding the size of the sample. Costs too dictate the size of sample that we can draw.
As such, budgetary constraint must invariably be taken into consideration when we
decide the sample size.
(v) Budgetary constraint: Cost considerations, from practical point of view, have
a major impact upon decisions relating to not only the size of the sample but also to
the type of sample. This fact can even lead to the use of a non-probability sample.
(vi) Sampling procedure: Finally, the researcher must decide the type of sample he
will use i.e., he must decide about the technique to be used in selecting the items for
the sample. In fact, this technique or procedure stands for the sample design itself.
There are several sample designs (explained in the pages that follow) out of which
the researcher must choose one for his study. Obviously, he must select that design
which, for a given sample size and for a given cost, has a smaller sampling error.
Different Types of Sample Designs
There are different types of sample designs based on two factors viz., the
representation basis and the element selection technique. On the representation basis,
the sample may be probability sampling or it may be non-probability sampling.
Probability sampling is based on the concept of random selection, whereas non-
probability sampling is ‘non-random’ sampling. On element selection basis, the
sample may be either unrestricted or restricted. When each sample element is drawn
individually from the population at large, then the sample so drawn is known as
‘unrestricted sample’, whereas all other forms of sampling are covered under the
term ‘restricted sampling’.
The sample designs are basically of two types viz., non-probability sampling and
probability sampling. We take up these two designs separately.
Non-probability sampling: Non-probability sampling is that sampling procedure
which does not afford any basis for estimating the probability that each item in the
2
population has of being included in the sample. Non-probability sampling is also
known by different names such as deliberate sampling, purposive sampling and
judgement sampling. In this type of sampling, items for the sample are selected
deliberately by the researcher; his choice concerning the items remains supreme. In
other words, under non-probability sampling the organisers of the inquiry
purposively choose the particular units of the universe for constituting a sample on
the basis that the small mass that they so select out of a huge one will be typical or
representative of the whole. For instance, if economic conditions of people living in
a state are to be studied, a few towns and villages may be purposively selected for
intensive study on the principle that they can be representative of the entire state.
Thus, the judgement of the organisers of the study plays an important part in this
sampling design.
In such a design, personal element has a great chance of entering into the selection
of the sample. The investigator may select a sample which shall yield results
favourable to his point of view and if that happens, the entire inquiry may get
vitiated. Thus, there is always the danger of bias entering into this type of sampling
technique. But in the investigators are impartial, work without bias and have the
necessary experience so as to take sound judgement, the results obtained from an
analysis of deliberately selected sample may be tolerably reliable. However, in such
a sampling, there is no assurance that every element has some specifiable chance of
being included. Sampling error in this type of sampling cannot be estimated and the
element of bias, great or small, is always there. As such this sampling design in rarely
adopted in large inquires of importance. However, in small inquiries and researches
by individuals, this design may be adopted because of the relative advantage of time
and money inherent in this method of sampling.
Quota sampling is also an example of non-probability sampling. Under quota
sampling the interviewers are simply given quotas to be filled from the different
strata, with some restrictions on how they are to be filled. In other words, the actual
selection of the items for the sample is left to the interviewer’s discretion. This type
of sampling is very convenient and is relatively inexpensive. But the samples so
selected certainly do not possess the characteristic of random samples. Quota
samples are essentially judgement samples and inferences drawn on their basis are
not amenable to statistical treatment in a formal way.

3
Probability sampling: Probability sampling is also known as ‘random sampling’ or
‘chance sampling’. Under this sampling design, every item of the universe has an
equal chance of inclusion in the sample. It is, so to say, a lottery method in which
individual units are picked up from the whole group not deliberately but by some
mechanical process. Here it is blind chance alone that determines whether one item
or the other is selected. The results obtained from probability or random sampling
can be assured in terms of probability i.e., we can measure the errors of estimation
or the significance of results obtained from a random sample, and this fact brings out
the superiority of random sampling design over the deliberate sampling design.
Random sampling ensures the law of Statistical Regularity which states that if on an
average the sample chosen is a random one, the sample will have the same
composition and characteristics as the universe. This is the reason why random
sampling is considered as the best technique of selecting a representative sample.
Random sampling from a finite population refers to that method of sample selection
which gives each possible sample combination an equal probability of being picked
up and each item in the entire population to have an equal chance of being included
in the sample. This applies to sampling without replacement i.e., once an item is
selected for the sample, it cannot appear in the sample again (Sampling with
replacement is used less frequently in which procedure the element selected for the
sample is returned to the population before the next element is selected. In such a
situation the same element could appear twice in the same sample before the second
element is chosen).
In brief, the implications of random sampling (or simple random sampling) are:
(a) It gives each element in the population an equal probability of getting into the
sample; and all choices are independent of one another.
(b) It gives each possible sample combination an equal probability of being chosen.

More specifically, the important types of sampling designs are as follows:

(i) Deliberate sampling: Deliberate sampling is also known as purposive or non-
probability sampling. This sampling method involves purposive or deliberate
selection of particular units of the universe for constituting a sample which
represents the universe. When population elements are selected for inclusion in the
sample based on the ease of access, it can be called convenience sampling. If a
researcher wishes to secure data from, say, gasoline buyers, he may select a fixed
4
number of petrol stations and may conduct interviews at these stations. This would
be an example of convenience sample of gasoline buyers. At times such a procedure
may give very biased results particularly when the population is not homogeneous.
On the other hand, in judgement sampling the researcher’s judgement is used for
selecting items which he considers as representative of the population. For example,
a judgement sample of college students might be taken to secure reactions to a new
method of teaching. Judgement sampling is used quite frequently in qualitative
research where the desire happens to be to develop hypotheses rather than to
generalise to larger populations.
(ii) Simple random sampling: This type of sampling is also known as chance
sampling or probability sampling where each and every item in the population has
an equal chance of inclusion in the sample and each one of the possible samples, in
case of finite universe, has the same probability of being selected. For example, if
we have to select a sample of 300 items from a universe of 15,000 items, then we
can put the names or numbers of all the 15,000 items on slips of paper and conduct
a lottery. Using the random number tables is another method of random sampling.
To select the sample, each item is assigned a number from 1 to 15,000. Then, 300
five-digit random numbers are selected from the table. To do this we select some
random starting point and then a systematic pattern is used in proceeding through
the table. We might start in the 4th row, second column and proceed down the
column to the bottom of the table and then move to the top of the next column to the
right. When a number exceeds the limit of the numbers in the frame, in our case over
15,000, it is simply passed over and the next number selected that does fall within
the relevant range. Since the numbers were placed in the table in a completely
random fashion, the resulting sample is random. This procedure gives each item an
equal probability of being selected. In case of infinite population, the selection of
each item in a random sample is controlled by the same probability and that
successive selections are independent of one another.
(iii) Systematic sampling: In some instances, the most practical way of sampling is
to select every 15th name on a list, every 10th house on one side of a street and so
on. Sampling of this type is known as systematic sampling. An element of
randomness is usually introduced into this kind of sampling by using random
numbers to pick up the unit with which to start. This procedure is useful when
sampling frame is available in the form of a list. In such a design the selection
process starts by picking some random point in the list and then every nth element
is selected until the desired number is secured.

5
 (iv) Stratified sampling: If the population from which a sample is to be drawn does
not constitute a homogeneous group, then stratified sampling technique is applied so
as to obtain a representative sample. In this technique, the population is stratified
into a number of nonoverlapping subpopulations or strata and sample items are
selected from each stratum. If the items selected from each stratum is based on
simple random sampling the entire procedure, first stratification and then simple
random sampling, is known as stratified random sampling.
(v) Quota sampling: In stratified sampling the cost of taking random samples from
individual strata is often so expensive that interviewers are simply given quota to be
filled from different strata, the actual selection of items for sample being left to the
interviewer’s judgement. This is called quota sampling. The size of the quota for
each stratum is generally proportionate to the size of that stratum in the population.
Quota sampling is thus an important form of non-probability sampling. Quota
samples generally happen to be judgement samples rather than random samples.
(vii) Multi-stage sampling: This technique is meant for big inquiries extending to a
considerably large geographical area like an entire country. Under multi-stage
sampling the first stage may be to select large primary sampling units such as states,
then districts, then towns and finally certain families within towns. If the technique
of random-sampling is applied at all stages, the sampling procedure is described as
multi-stage random sampling.

Designing a Quantitative Study

1. Experimental and Quasi-Experimental Research
Randomization is usually viewed as one of the hallmarks of experimental research.
Design types can range from truly experimental (with random assignment) to what
is known as quasi-experimental (without random assignment). Clearly, some design
types are more prototypical of one end of the range than the other. In this section we
deal with both types, beginning with those that include random assignment of
individuals.
A typical experimental study usually uses comparison or control groups to
investigate research questions. Many second language research studies involve a
comparison between two or more groups. This is known as a between-groups design.
This comparison can be made in one of two ways: two or more groups with different
treatments; or two or more groups, one of which, the control group, receives no
treatment.
6
a. Comparison Group Design
In a comparison group design, participants are randomly assigned to one of the
groups, with treatment (the independent variable) differing between or among the
groups.
Example: A researcher wants to investigate whether aural input or input through
conversational interaction yields better L2 learning.
• Group 1: Hears a text with input containing the target structure.
• Group 2: Interacts with someone who provides input on the target structure.
Assuming a pretest/posttest design (see later discussion), the results of the two
groups would be compared, with inferences being made as to the more appropriate
method of providing information to learners. In comparison research, more treatment
groups can be added to the study if the research question is elaborated. The following
example suggests a slightly different research question with a more elaborate design:
Example: A researcher wants to investigate to what extent aural input, input through
conversational interaction, or a combination of aural and conversational input yields
better L2 learning.
• Group 1: Listens to a text containing the target structure.
• Group 2: Interacts with someone who provides input on the target structure.
• Group 3: Receives some input through listening and some through interaction.
Were this researcher to add yet another question, the design could have a fourth
group:
Example: A researcher wants to investigate to what extent aural input, input through
conversational interaction, a combination with aural input followed by interaction,
or a combination with interaction followed by aural input yields better learning.
• Group 1: Hears a text with input containing the target structure.
• Group 2: Interacts with someone who provides input on the target structure.
• Group 3: Receives some input first through listening and then through interaction.
• Group 4: Receives some input first through interaction and then through listening.

7
b. Control Group Design
The second standard type of experimental design is a control group design. This is
similar to the comparison group design, with the important difference that one group
does not receive the treatment. The control group would typically take the same
pretest and posttest as would the experimental groups, but would not have the same
treatment in between tests. For control groups, some researchers may want to
provide some language activity or input (of course, different from the treatment) in
which the participants are doing something else with language. This is to ensure that
it was the treatment, not the mere fact of doing something, that led to any change.
One aspect that all researchers grapple with in second language research is how to
control for outside exposure to the language. This is much easier to control in a
foreign language environment than in a second language environment.
In a foreign language setting, control for exposure can often be accomplished simply
by ensuring that the particular language focus is not covered in the syllabus during
the time of treatment. Another way to prevent external input influencing the results
of the study is not to have long periods of time between testing sessions (although
there are instances when long periods of time are desirable, as in delayed posttests
when testing longer-term effects of treatment), or, minimally to be able to argue that
if there is additional exposure, the groups are equivalent in this regard. Each
researcher must be cognizant of the problem and must determine how to deal with it
in an appropriate manner.
In sum, a true experimental design will have some form of comparison between
groups. The groups will differ in terms of some manipulation of the independent
variable to examine the effect of manipulation on the dependent variable.
Assignment will be random, or as random as possible, to avoid threats to internal
validity caused by participant characteristics.
2. Measuring the Effect of Treatment
There are a number of ways of designing a study using the design characteristics
discussed earlier. We focus on two: pretest /posttest (with and without delayed
posttests), and posttest only.
a. Pretest/Posttest Design
In many second language studies, participants are given a pretest to ensure
comparability of the participant groups prior to their treatment, and a posttest to

8
measure the effects of treatment. We must take into consideration the need to ensure
that our measures are assessing what we intend them to assess. Once we have
determined that our measures are indeed appropriate for our research question, there
is a further question to be addressed: Is the pretest comparable in difficulty to the
posttest? If the pretest turned out to be more difficult than the posttest, participants
might demonstrate artificially greater improvement; if the pretest turned out to be
easier, participants might demonstrate artificially less improvement. One solution to
this problem is in the form of a study that randomly assigned some sentences to the
pretest and others to the posttest. In this way, the threat of test bias is reduced.
In a pretest/posttest design, researchers can determine the immediate effect of
treatment. However, clearly, the real question for studies of second language
learning is to address to what extent a treatment truly resulted in learning. From what
we know about learning, it is a process that may begin with a particular treatment,
but it is not always clear that the effects of that learning are long-lasting. To measure
the longer-term effects, researchers often want to include delayed posttests in
addition to the immediate posttests. In fact, this is becoming increasingly common
in second language research. With delayed posttests, a test comparable to the posttest
(and pretest, if there is one) is administered one or more times at predetermined times
after the treatment. Often this is 1 week following the first posttest and then 2 weeks
later and even 2 or 3 months later. The advantage of delayed posttests is that one
gets a wider snapshot of treatment effects; the disadvantages are that there is a
greater likelihood of losing participants, extra-experimental exposure will be greater,
and there will be maturation. It is important to make an a priori principled decision
as to how many posttests a participant must take before eliminating data.
b. Posttest-Only Design
In some instances, it is undesirable to give a pretest because the pretest itself might
alert participants to what the treatment concerns. In most cases, we do not want our
participants to know or guess the purposes of the treatment. It is important to
recognize that there are limitations to this type of design, the main one being that a
researcher cannot be sure if there is initial comparability of groups. This becomes an
issue particularly when measuring improvement as a result of the manipulation of
an independent variable. In posttest-only designs the focus of study is usually
performance and not development. Following are suggestions for ways to address
the issue of lack of initial comparability measures, so that one can make assumptions
about group comparability:

9
• Match learners on other measures that one can gain from a background
questionnaire, for example:
• Gender.
• Age.
• Years of language study.
• Language(s) spoken in the home.
• Class grades.
• Teachers' ratings.
• Placement test.
• Base the investigation on language features that have not previously appeared in
the syllabus or in the textbook (relevant for foreign language environments).
• Use low-frequency words for vocabulary studies.

Gathering Qualitative Data

As noted earlier, a wide variety of different techniques are used in the collection of
qualitative data. As with all methods, the advantages and disadvantages of each
technique should be taken into consideration when deciding how to address a
specific research question.
Here we present an overview of some of the most commonly used qualitative data
collection methods, including:
• Ethnographies
• Case studies
• Interviews
• Observational techniques
As discussed earlier, because there is little general agreement in the field about what
constitutes qualitative research, some of the data collection techniques we discuss
now are associated with more "descriptive" than truly "qualitative" methods by some
researchers. For example, Brown (2003) categorized interviews and questionnaires
as part of survey-based research, a distinct category from qualitative and quantitative
research, which he referred to as "interpretive and statistical methods." Also, given
10
that some of the data collection methods described here are associated with particular
contexts or overlap with each other. Each approach and method can be seen as
contributing its own piece of the puzzle in qualitative researchers' attempts to obtain
rich, detailed, participant-oriented pictures of the phenomena under study.
1. Ethnographies
Although there has been much recent debate concerning the nature of ethnography,
it can be said from a second language research perspective that ethnographic
research aims "to describe and interpret the cultural behavior, including
communicative behavior, of a group" (Johnson, 1992, p. 134) as well as "to give an
oriented description of the cultural practices of individuals" (Ramanathan &
Atkinson, 1999, p. 49), or, in other words, to carry out research from the
participants3 point of view, using categories relevant to a particular group and
cultural system.
This focus on group behavior and the cultural patterns underlying that behavior is
one of the key principles of ethnography identified by Watson- Gegeo in her (1988)
review article. Another important principle of ethnographic research is the holistic
approach taken to describing and explaining a particular pattern in relation to a whole
system of patterns.
Hence, ethnography can be viewed as a qualitative research method that generally
focuses on the group rather than on the individual, stresses the importance of
situating the study within the larger sociocultural context, and strives to present an
emic perspective of the phenomena under investigation, with the categories and
codes for analysis being derived from the data themselves rather than being imposed
from the outside.
Ethnographic approaches have been used in a very broad range of second language
research contexts, ranging from ethnographies of schools and language programs to
personal accounts and narratives or life histories of learning and teaching (e.g., Duff,
2002; Pavlenko & Lantolf, 2000), home-school discontinuities among Native
American children (e.g., Macias, 1987; Philips, 1972, 1983), bilingual language use
outside educational settings (Goldstein, 1997), cultural and ideological processes in
first and second language learning (King, 2000), and research on specific aspects of
the L2 process, such as second language writing in different cultural contexts (e.g.,
Atkinson & Ramanathan, 1995; Carson & Nelson, 1996). However, as Johnson
(1992) noted, one of the main uses of ethnographic research in the second language
context "has been to inform us about the ways that students' cultural experiences in
11
home and community compare with the culture of the schools, universities, and
communities where they study, and the implications of these differences for second
language and culture learning" (p. 135).
A well-known piece of research of this nature, an ethnography of first language
communication, was carried out by Shirley Brice Heath (1983), who spent a decade
living in what she described as two working class communities in the Carolinas; in
her terms, one black community, Trackton, and one white community, Roadville.
Using data from these working- class communities, as well as middle-class school-
oriented black and white families in the town, Heath's research focused on how the
children in her study learned to use language, how their uses of language were related
to their literacy, and how their use of language in the home interacted with how they
used print. She showed how the different ways of learning language interacted with
the children's integration into academic life. The working-class communities held
different expectations and different usage patterns, as well as different attitudes to
the mainstream families and schools in terms of uses of language, including what
Heath referred to as "ritualized uses of language," such as the assignment of labels
to objects, response to questions whose answers were already known to the
questioner (often known as display questions), and recitation of discrete points of
factual material separated from context.
In summary, the language and literacy practices of middle-class families mirrored
expectations in the school. These differences in home language and literacy practices
had implications for academic success. The goal of such research is to be detailed,
holistic, and situated in context with a focus on exploring how complex factors
interact.
Ethnographies can profitably make use of methods specifically designed to tap into
participants' perspectives and, as such, they often involve or are overlapped with the
use of observations, interviews, diaries, and other means of data collection that are
discussed in more detail later in this chapter.
a. Advantages
One advantage of using an ethnographic approach is that the research questions
employed in these studies can be dynamic, subject to constant revision, and refined
as the research continues to uncover new knowledge. For example, an ethnographer
studying second language writing classrooms may enter the research process with
the aim of describing the patterns of interaction between teachers and students and
illustrating how those patterns are related to the writing process. However, over the
12
course of many classroom observations, analyses of student essays, and interviews
with both students and teachers, the researcher may alter the focus of the study and
begin concentrating on the types of feedback that are provided by both teachers and
students.
Ethnographic approaches are particularly valuable when not enough is known about
the context or situation to establish narrowly defined questions or develop formal
hypotheses. For example, why does a particular group of heritage language learners
do poorly when learning in formally instructed foreign language classroom settings?
An ethnographic approach to this question could examine the context, the attitude of
the teacher and the students, the influence of home and social groups, and so on in
an attempt to uncover information relevant to addressing the question. If the
researcher shares the heritage language background, participant observation could
be used; that is, the researcher might be able to participate in the language classes or
share social occasions in which the language is used in some way. Because
ethnographies typically employ multiple methods for gathering data, such as
participant observations and open-ended interviews as well as written products,
ethnographic research may be able to provide a holistic, culturally grounded, and
perspective of the phenomena under investigation.
b. Caveats
In embarking on an ethnographic study, researchers need to be aware of some
potential challenges and sensitive issues. First of all, ethnographies involve intensive
research over an extended period of time. They require a commitment to long-term
data collection, detailed and continuous record keeping, and repeated and careful
analysis of data obtained from multiple sources. It is also important for the researcher
to realize that ethnographic approaches to research may create potential conflicts
between the researcher's roles as an observer and a participant. If the researcher
participates in an event he or she is observing, this may leave little time for the
carefully detailed field notes that ethnographies may require. This can be rectified
to a certain extent by audio and video tape recording. However, and more seriously,
the researcher's participation may change the nature of the event. Researchers thus
need to be aware of how they can supplement and triangulate ethnographic data
obtained through participant observation, and they must carefully consider how their
dual roles might influence the data collected.
In addition to these practical concerns, there are theoretical issues that the researcher
should take into consideration. First of all, it has been argued that an ethnographer's

13
focus on describing a culture is problematic, because "there is no such thing as a
social group that is not constantly destabilized by both outside influences and
personal idiosyncrasy and agency" (Ramanathan & Atkinson, 1999, p. 45). In its
strong form, this criticism implies that any attempt to describe a group is to some
extent misguided on the part of the ethnographer.
A second theoretical concern about ethnographies concerns the act of writing up the
research. Because research reports adhere to certain (culturally influenced) standards
of writing, the otherwise accurate picture an ethnographer has recorded may come
out skewed. In other words, the very act of transcribing the events that were observed
may inevitably entail a misrepresentation of them. Finally, it is often difficult to
generalize the findings of ethnographic research to other problems or settings
because of the highly specific nature of such work.
2. Case Studies
Like ethnographies, case studies generally aim to provide a holistic description of
language learning or use within a specific population and setting. However, whereas
ethnographies focus on cultural patterns within groups, case studies tend to provide
detailed descriptions of specific learners (or sometimes classes) within their learning
setting. Case studies are also usually associated with a longitudinal approach, in
which observations of the phenomena under investigation are made at periodic
intervals for an extended period of time.
Case studies have been used in a wide variety of second language research studies.
One well-known longitudinal case study investigating the development of L2
communicative competence is Schmidt's (1983) study of Wes, an ESL learner. Wes
was a 33-year-old native speaker of Japanese who had little formal instruction in
English. Schmidt studied Wes' language development over a 3-year period when he
was residing in Japan but visited Hawaii, the research site, regularly on business.
The study focused on a small number of grammatical features, including plural s,
third-person singular 5, and regular past tense. Schmidt transcribed conversations
between Wes and friends and also transcribed monologues that he asked Wes to
produce while at home in Japan. Although Wes attained relatively high levels of
pragmatic ability and acculturation (e.g., in the use of formulae such as "So, what's
new?" and "Whaddya know?"), he had very limited improvement in terms of
linguistic accuracy for the grammatical forms over the 3 years of the study, thus
providing evidence for the separability of linguistic and pragmatic competence.

14
Another well-known case study is Ellis' (1984) investigation of two child learners in
an L2 context. J was a 10-year-old Portuguese boy, and R was an 11-year-old
Pakistani boy. Both children were learning ESL in London in a program that catered
to new arrivals with the aim of preparing them for transfer to high schools. Ellis
investigated the learning patterns of the two children in an instructed context, as
opposed to the naturalistic context in which Schmidt had studied Wes' language
development. In order to examine the learners' use of requests, Ellis visited the
classrooms regularly, writing down the samples of requests the two children
produced. Ellis' analysis documented different stages in the children's use of requests
over time, also noting their tendency to use formulae. In another second language
case study in which the focus was the role of collaborative interaction in second
language development, Ohta (1995) examined the language of two intermediate
learners of Japanese as a foreign language, finding that their patterns of interaction
facilitated a form of scaffolding, or assisted help, that benefited both learners.
Obviously, cases can also be individual classes or schools.
a. Advantages
One main advantage of case studies is that they allow the researcher to focus on the
individual in a way that is rarely possible in group research. As Johnson (1993)
noted, "[I] too often, because of the nature of correlational, survey, and experimental
research, and their privileged status in L2 research, very little is learned about
individual language learners, teachers, or classes.
Case studies stand in sharp contrast to these approaches by providing insights into
the complexities of particular cases in their particular contexts" (p. 7). In addition,
case studies can be conducted with more than one individual learner or more than
one existing group of learners for the purpose of comparing and contrasting their
behaviors within their particular context. Case studies clearly have the potential for
rich contextualization that can shed light on the complexities of the second language
learning process.
b. Caveats
An essential point to bear in mind with case studies, however, is that the researcher
must be careful about the generalizations drawn from the study. Although this is true
for all forms of research, it is particularly pertinent to case studies, which often
employ only a few participants who are not randomly chosen. For this reason, any
generalizations from the individual or small group (or classroom) to the larger
population of second language learners must be made tentatively and with extreme
15
caution. From a single case study, it may be difficult to recognize idiosyncrasies as
such, with the potential that they are misinterpreted as typical language learning
behavior. To address this concern, the findings from multiple longitudinal case
studies can be combined to help researchers draw firmer conclusions fromtheir
research. For example, Wray (2001) summarized 14 case studies that focused on the
role of formulaic sequences in child second language acquisition. The cases involved
21 children (12 girls and 9 boys), aged approximately 2 to 10. Based on these
multiple case studies, Wray argued that children express themselves holistically in a
second language by employing formulaic sequences. In short, case studies may
provide valuable insights into certain aspects of second language learning, but single
case studies are not easily generalizable.

3. Interviews
A number of different interview types can be employed to gather data for qualitative
research. As noted in the introduction to this chapter, interviews are often associated
with survey-based research, as well as being a technique used by many qualitative
researchers. In structured (also known as standardized) interviews, researchers
usually ask an identical set of questions of all respondents. Structured interviews
resemble verbal questionnaires and allow researchers to compare answers from
different participants. Less rigid are semi-structured interviews, in which the
researcher uses a written list of questions as a guide, while still having the freedom
to digress and probe for more information. In unstructured interviews, on the other
hand, no list of questions is used. Instead, interviewers develop and adapt their own
questions, helping respondents to open up and express themselves in their own terms
and at their own speed.
Unstructured interviews are more similar to natural conversations, and the outcomes
are not limited by the researcher's preconceived ideas about the area of interest.
Some interviews can also be based around a stimulus—for example, a completed
questionnaire or a videotape of a lesson. Focus-group sessions are related to such
interviews, and usually involve several participants in a group discussion, often with
a facilitator whose goal it is to keep the group discussion targeted on specific topics,
again often using a stimulus for discussion, such as a videotape or previously elicited
data.

16
a. Advantages
Interviews can allow researchers to investigate phenomena that are not directly
observable, such as learners' self-reported perceptions or attitudes. Also, because
interviews are interactive, researchers can elicit additional data if initial answers are
vague, incomplete, off-topic, or not specific enough. Another advantage of
interviews is that they can be used to elicit data from learners who are not
comfortable in other modes. For example, some learners are more at ease speaking
than writing and are more likely to provide extended answers in a conversational
format. Depending on the research question and the resources available, interviews
can also be conducted in the learner's Ll, thus removing concerns about the
proficiency of the learner impacting the quality and quantity of the data provided.
b. Caveats
Researchers must also take note of the potential drawbacks of interviews. For
example, Hall and Rist (1999) made the point that interviews may involve "selective
recall, self-delusion, perceptual distortions, memory loss from the respondent, and
subjectivity in the researcher's recording and interpreting of the data" (pp. 297-298).
Multiple interviews—that is, interviewing the same subject more than once, or
interviewing many different subjects—is one potential means of addressing such
issues. Another concern is that good interviewing is a skill. It may not be easy for
novice researchers to conduct unstructured interviews without practice and / or
training in drawing participants out, encouraging them to express themselves, and
gathering valuable data on the area of interest. Given that participants' attitudes
toward other people can impact what they say, there is also the danger of the so-
called halo effect. This refers to what happens when interviewees pick up cues from
the researcher related to what they think the researcher wants them to say, thus
potentially influencing their responses. In addition to these concerns, the possibility
of cross-cultural pragmatic failure exists. Some questions may be considered
inappropriate in particular cultures, and because of the different connotations words
carry in different linguistic and cultural contexts, miscommunications may arise.
To address some of these concerns, the following suggestions may be useful in
interviewing:
• Be sensitive to (and / or match the interviewer's characteristics with) the age,
gender, and cultural background of the interviewee.

17
• Encourage open-ended discussion—for example, by keeping silent, or by saying
'Anything else?" rather than accepting a first answer as the interviewee's final and
complete response to a question.
• Develop skills in anticipating and addressing communication problems.
• Try to make the interviewee as comfortable as possible. This can be done by
conducting the interview in a familiar place, beginning with small talk to relax the
interviewee, and/ or using the Ll if a communication problem arises or if the
interviewee so prefers.
• Place the key questions in the middle of the interview, because the interviewee may
be nervous in the beginning and tired by the end.
• Mirror the interviewee's responses by repeating them neutrally to provide an
opportunity for reflection and further input.
4. Observations
As Mason (1996) noted, observation usually refers to "methods of generating data
which involve the researcher immersing [him or herself] in a research setting, and
systematically observing dimensions of that setting, interactions, relationships,
actions, events, and so on, within it" (p. 60). When collecting data using
observational techniques, researchers aim to provide careful descriptions of learners'
activities without unduly influencing the events in which the learners are engaged.
The data are often collected through some combination of field notes (which can
involve detailed impressions of the researcher's intuitions, impressions, and even
questions as they emerge) and audio or visual recordings (which allow the researcher
to analyze language use in greater depth later and to involve outside researchers in
the consideration of the data). In second language research, observations have been
used in a wide variety of studies, ranging from naturalistic studies to the rather more
common classroom observations.
Different types of observations can be identified, depending on their degree of
structure. In highly structured observations, the researcher often utilizes a detailed
checklist or rating scale. In a complex L2 environment such as the language school,
workplace, or community, a structured observation can facilitate the recording of
details such as when, where, and how often certain types of phenomena occur,
allowing the researcher to compare behaviors across research contexts in a
principled manner. In less structured observations, the researcher may rely on field

18
notes for detailed descriptions of the phenomena being observed, or transcripts of
tapes of those events.
a. Conducting Classroom Observations
Obtrusive Observers. Observation etiquette may initially seem secondary to the
more practical nuts and bolts of carrying out a thorough observation, but conforming
to good observation etiquette is very important. Any observer in the classroom runs
the risk of being an obtrusive observer, which can be problematic for research. An
obtrusive observer's presence may be felt in the classroom to the extent that the
events observed cannot be said to be fully representative of the class in its typical
behavior, and therefore the observation data may have limited validity.
An obtrusive observer may also be problematic for the instructor and students in
terms of compromising the quality of the lesson, preventing instructors from
delivering the lesson to the best of their ability and, consequently,
preventing the students from learning to the best of theirs. For example, younger
learners in particular can become very easily distracted by observers. They may be
interested in the recording equipment and may pay more attention to a new person
in their classroom with a digital video or voice recorder than to their instructor.
The Hawthorne Effect. Another potential problem for observational research is the
so-called Hawthorne effect, also discussed in chapters 4 and 6. This effect was first
described by observers at the Hawthorne, Chicago branch of the Western Electric
Company (Brown, 1954; Mayo, 1933). When the observers were present, the
productivity of workers increased regardless of whether or not there were positive
changes in working conditions.
The workers were apparently happy to receive attention from researchers who
expressed an interest in them by observing them, and this impacted their behavior.
Accordingly, whereas in observational research it may be difficult to be sure that the
observed classes are the same as they would be without the observation, in controlled
research it may be difficult to separate Hawthorne effects from experimental
variables. Although Hawthorne effects in management have been queried by some
(e.g., Adair, 1984), including those who have pointed to the small number of
participants in the original study, educationalists usually make efforts to take such
effects into account when conducting observations. For example, as mentioned in
chapter 5, Mellow et al. (1996) argued that time-series designs are particularly useful
for investigating and evaluating different approaches to second language teaching,

19
pointing out that one of their benefits is that they may reduce the Hawthorne effect
as students and teachers begin to feel more comfortable and natural about being
observed.
Objectivity and Subjectivity. Classroom observations are not only conducted by
researchers external to the school or educational environment. Instructors often
observe each other's classes for professional development as well as for research
purposes, and they may also carry out observations of their own classes, usually
using audio or videotapes to assist with this process.
This brings us to another consideration that needs to be taken into account when
conducting observations: namely the level of objectivity or subjectivity of the
observer. The level and impact of this on the study is often debated and needs to be
clearly recognized and reported in research. Whereas objectivity is typically valued
in second language research, particularly in experimental work, both objectivity and
subjectivity have their respective roles in research on second language learning.
Therefore, in classroom studies, it is necessary for researchers to both strive for
objectivity and also be aware of the subjective elements in that effort—for example,
in how they gather data, analyze data, and report the results of analyses.
Obtaining Permission to Observe and Enlisting the Help of the Instructor.
In addition to keeping issues of objectivity and subjectivity in mind, there are several
further precautions for observers to take into account. First of all, it is important to
obtain the permission of the instructor to observe the class well in advance of the
scheduled observation(s). This is not only a professional courtesy, but may also help
the instructor to lessen any impact of the observation on lesson planning and
implementation.
When working with schools and language programs, the researcher should not
assume that the permission of administrators indicates that individual classroom
instructors have been informed and that their instructional schedules have been
considered. It is important to contact the instructor in advance in order to obtain
consent and to negotiate the schedule and observation process. It is also important
to seek the instructor's input about matters such as when to arrive. For example,
arriving a little before the learners, or at the same time, or during an activity when
their attention will be focused elsewhere, can all be options for lessening the impact
of the observer.

20
The instructor may also have the best idea about where the observer can sit in the
classroom so as to be minimally intrusive. Some instructors prefer observers to sit
in the back or off to the side of the class, and some may recommend that researchers
begin coming to class several days before they conduct the research in order to
habituate the students to their presence. Instructors may feel that by minimizing the
presence of the observer, their students will not become distracted by note taking or
the direction of the observer's gaze or equipment, and will concentrate on the lesson.
For some classes, students may be used to the occasional presence of a supervisor or
instructor-trainer, and little explanation will need to be provided. However, in other
classes, especially those early in the semester or program, students may never have
experienced an observer before. For these classes, depending on the research
problem being studied, it might be possible to introduce the researcher into the
classroom so that they participate in the instructional activities and are seen as an
instructor's aide. Another reason to be sure to have made personal contact with the
instructor beforehand is that having more than one observer in the classroom at
anytime could be disruptive for the instruction, and the instructor is usually in the
best position to know the schedule for who plans to observe when. If the observation
is to be ongoing, it may also be wise to ask the instructor for feedback after the class
in case they would prefer something to be done differently the next time around.
Murphy (1992) recommended that observers who are there for research purposes
keep in mind that their role is not to judge, evaluate, criticize, or offer constructive
advice, and also that if asked by the learners what they are doing in the classroom,
observers keep their responses as short as possible.
Debriefing the Instructor. It is also important as part of the negotiation surrounding
the observation process to debrief the instructor about the research findings or the
content of the observation notes or scheme.
Timing is also an important consideration here. For example, researchers might
provide instructors with a copy of their notes after each lesson or arrange a time to
meet in order to discuss the research. By keeping the observation process as
transparent and interactive as possible, researchers can often establish a more
trusting and cooperative relationship with instructors.
Of course, in some cases, the instructors may be the focus of the research, or it may
unduly influence the research if they are kept continually debriefed. In these cases,
it may be preferable to make such contact after the project has been completed.

21
Finally, of course, it is always important to thank the instructor for allowing the
observation, together with any other parties who have been helpful to the research,
including both administrators and students. It can be easy to overlook such simple
things, but in fostering good relationships between instructors and future researchers,
expressing courtesy and appreciation is important. As part of this process, when
research is published with acknowledgments made to schools, principals,
instructors, and students, it is helpful to send copies of the publication to the schools
and instructors because they may not have access to the same journals and
publications as the researcher. Following is a helpful checklist to consider in setting
up observations:
• Contact the classroom instructor (in person if possible).
• Determine the schedule for observation.
• Negotiate the observer's role in the classroom, including regular pre-visits, arrival
time, introductions, and seating arrangements.
• Debrief the instructor (either during or after the observational period) on the
findings of the study.
• Clearly express appreciation to the instructor, students, and administration.
b. Advantages
Observations are useful in that they provide the researcher with the opportunity to
collect large amounts of rich data on the participants' behavior and actions within a
particular context. Over time and repeated observations, the researcher can gain a
deeper and more multilayered understanding of the participants and their context.
c. Caveats
Observations typically do not allow the researcher access to the participants3
motivation for their behaviors and actions. For this reason, observations may be most
useful when combined with one or more of the other methods.
However, perhaps the most serious concern is the "observer's paradox" (Labov,
1972). This refers to the fact that although the goal of most observational research is
to collect data as unobtrusively as possible, the presence of an observer can influence
the linguistic behavior of those being observed. There is also some possibility of the
Hawthorne effect (discussed above), which may occur when learners perform better
due to positive feelings at being included in a study. Simply put, if learners realize

22
that they are under observation, their performances might improve because of the
fact of that observation.
To minimize these threats, researchers should consider the ways in which they may
influence an L2 setting and take steps to mitigate the effect of their presence. For
example, if the goal of a study involves observing the use of a second language
among immigrants in their workplace, researchers may try to blend into the
background of the workplace to make the participants more accustomed to their
presence. Another less obtrusive option is participant observation, by which
researchers are members of the group they are observing. They play a dual role of
observing while fully participating in activities with other group members. Although
participant observation can limit the effects of the observer's paradox, it can also be
difficult to both observe and participate, as discussed previously. Participant
observation is generally most feasible in adult learning contexts where the researcher
can easily blend in—for example, in conversation or language exchange clubs.
Ethical issues related to participant observations also need to be considered.

23