Research Methodology

Defining Interviews:

Each research strategy contains one or more data generation methods. We'll look now at the
first of these — interviews, An interview is a particular kind of conversation between people. It
has a set of assumptions (normally unspoken) that do not apply to ‘normal’ conversations.
Usually, one person has a purpose for undertaking the interview: they want to gain information
from the other(s). This means that the discussion does not occur by chance, but has been
planned in some way by the researcher. They usually have an agenda - particular issues they
want to find out about — so the discussion topics do not occur arbitrarily or randomly, with
both sides free to choose topics at will. Instead, the researcher will steer the discussion onto
their topics of interest. Since the researcher normally guides the discussion, it does not have a
free-flowing form like other conversations. Rather, there is tacit agreement that, at least at the
beginning of the interview if not all the way through, the researcher has the right to control
both the agenda and the proceedings and will ask most of the questions. A research interview is
nor carried out covertly, for example, by secretly recording discussions, but is openly a meeting
aiming to produce material for research purposes, and the interviewee knows this and agrees
with it. It is also understood that the interviewee’s words can normally be treated as ‘on the
record’, so they can be used by the researcher later. Only if the interviewee specifies that some
responses are not to be made publicly available are they treated as ‘off the record’. Interviews
can be suitable data generation methods when a researcher wants to:

 obtain detailed information;

 ask questions that are complex, or open-ended, or whose order and logic might need to
be different for different people;
 explore emotions, experiences or feelings that cannot easily be observed or described
via pre-defined questionnaire responses;
 investigate sensitive issues, or privileged information, that respondents might not be
willing write about on paper for a researcher that they have not met.

Interviews are much used in case studies and ethnographies, but can be used in other
strategies too. For example, they can be used in surveys, that is, interviewing people rather
than asking them to complete a questionnaire. They are also often used to ‘top-and-tail’ a
survey strategy — interviews are used to elicit themes that are then included in a
questionnaire, and follow-up interviews are used to obtain more detail about some
questionnaire responses. Similarly they can ‘top-and-tail’ a design and creation strategy by
generating data for a requirements specification and eliciting user feedback on a finished
design. Although often one-to-one, interviews can also be undertaken with a group of
respondents, as in focus groups that are used by politicians and market researchers.

1
Research Methodology

Planning and Conducting Interviews:

Because we all regularly carry out conversations with others, it is tempting to think that we can
all naturally carry out interviews. However, successful research interviews need planning and a
particular set of skills, as explained below. This section concentrates on one-to-one interviews.
The following section explains additional issues that arise in group interviews.

Types of interviews:

Interviews can be divided into three types, so you need to decide which type you will use:
structured, semi-structured or unstructured interviews.

 Structured interviews: These use pre-determined, standardized, identical questions for

every interviewee. You read out the questions and note your interviewee’s responses,
often using pre-coded answers. Although there will be some social Interviews is)
interaction between you and the respondent, such as if the respondent asks for and
receives clarification, you do not really engage in a conversation and it is important that
you read out all the questions in the same way and note the answers without comment,
otherwise you might indicate your own views to the interviewee. In effect, you are
asking your interviewees to complete a questionnaire, but you are doing the writing for
them (or even entering their responses straight into a computer, if you use a laptop).The
remainder of this chapter concentrates on the other two types of interview.

 Semi-structured interviews: You still have a list of themes to be covered and questions
you want to ask, but you are willing to change the order of questions depending on the
flow of the ‘conversation’ and you might ask additional questions if your interviewee
brings up issues you had not prepared questions for. The interviewees are able to speak
with more detail on the issues you raise, and introduce issues of their own that they
think relevant to your themes.

 Unstructured interviews: the researcher has less control. You start things off by
introducing a topic and then let the interviewees develop their ideas, talking freely
about events, behaviour or beliefs, while you try not to interrupt and are as un-intrusive
as possible.

Both semi-structured and unstructured interviews allow interviewees to ‘speak their minds’ and
so are used where the primary purpose is ‘discovery’, rather than ‘checking’. They are therefore
used for in-depth investigations, especially those aimed at exploring personal accounts and

2
Research Methodology

feelings. They are not useful, though, for situations where you want to draw research
conclusions that are generalizations about the whole population (whatever this may be in
relation to your research topic), because you will not have responses about the same topics
from all your interviewees, and the time required means that you will usually only have a small
number of cases from which to draw conclusions.

 Internet-based Interviews: Although there has been limited use so far, the Internet
offers exciting possibilities for online individual and group interviews. Unless Voice-over-
Internet telephony is being used, the interview is not spoken, but comprises a set of
written questions and answers. Clearly this changes our understanding of ‘interview’.
Because responses have to be typed, they are likely to be shorter and less expansive
than in a face-to-face interview. In online interviews, we also lose the richness of
gestures and facial expressions and the context of a face-to-face interview. Researchers
also have to either take on trust that their informants are who they say they are, or do
some detective work to try and establish whether they are really qualified to speak on
the subject. Similarly, informants need to be convinced about the identity and honesty
of a researcher they have not met. However, online interviews can be cheaper than
face-to-face interviews because there are no travel costs, they allow the researcher to
reach informants across the world (or at least those people that have email access and
the technical expertise), there are no tapes to transcribe and the data is already typed
ready for analysis, Opinion is still divided about whether online interviews are
appropriate when the information sought requires the establishment of good inter-
personal relationships, Some maintain that such intimacy cannot be achieved online
because there are insufficient social cues for people to establish the human ‘presence’
of each other. Others argue that warm relationships can and do develop online, with a
rapid increase in intimacy between people who were strangers a short while ago. (For a
discussion of these two viewpoints, see Mann & Stewart, 2000.)

The simplest form of online interviews is via email to named individuals. An email can be sent
introducing the researcher and the research topic and the recipient can be invited to answer
some questions. Once those answers are received, the researcher follows up with more
questions and a ‘conversation’ can ensue. Be careful if you are sending out the same questions
via email to several people who do not know the identity of the other participants — use the
‘Bcc’ (blind copy) facility of your emailing software. For a group interview, all questions and
responses sent to the whole group.
Alternatively, synchronous communication can be used, t+ is, real-time chat, where what one
person types is immediately visible to everyone else. Some companies now offer virtual

3
Research Methodology

facilities for online focus groups with web-based ‘rooms’ for discussion sessions, and technical
backup can also be provided for a fee.
If you do not know the names and email addresses of the people you want to invite to
participate in your online interviews, you will have to appeal for volunteers via newsgroups or
mailing lists - remember to obtain the permission of the list-owner or gatekeeper first.

You should be familiar with some of the shorthand notation used online (for example, LOL:
laughs out loud, IMHO: in my humble opinion), but try not to use them yourself in case your
interviewee does not know them. Similarly you should recognize the emoticons in common
usage such as ;-) and :-(, but again use them carefully yourself because some people find them
irritating. Remember, also, that different cultures might not know the emoticons, or have their
own system (for example, Japan — see Mann and Stewart [2000] for examples).
Because they cannot see you nodding your head and looking interested, interviewees may need
reassurance from you that they are supplying the kind of information you need, Remember,
too, that participants can suddenly disappear - you do not know whether their emails have
been lost in cyberspace, or they have decided to withdraw from your research, or they are ill, or
...? Bear in mind that, currently at least, there is an expectation that email is answered quickly.
This means that people will expect to hear back from you speedily, regardless of any other work
or deadlines you might have. Online interviews can therefore be very demanding, and you will
need to plan how you will schedule your online interview work alongside the rest of your
commitments.

Defining Observations:

To observe means ‘to watch’ and ‘to pay attention to’. Observing is something most of us do a
lot of the time: seeing, hearing, noting, analysing, forming theories, making inferences,
imposing meaning. Even something simple such as walking down the street involves us:
 watching what others are doing;
 trying to predict which way the oncoming people will move so that we don’t bump into
them;
 noting what they’re wearing; deciding if we’ve seen someone we know or someone who
might pose a risk to us
 deducing why the harassed-looking woman is shouting at the little girl ...

... and we’ve still barely covered 100 metres!

Researchers use observation as a data generation method to find out what people actually do,
rather than what they report they do when questioned. Often observation

4
Research Methodology

Highly systematic observations of pre- observations of anything and

defined types of events everything
Narrow concentration on particular Broad focus
type of event
Observer takes no part in the Observer participates fully in the
proceedings. Proceedings
Fact of observations taking place is Fact of observations taking place is
known to all known to none except the
researcher
No explanation, or false explanation. Full explanation given for the
given for presence of observer presence of the observer—researcher
researcher
short duration -— could be as lite as Long duration - possibly years
5 minutes
Record-keeping uses only simple note Record-keeping uses technology
Taking (e.g. audio tape, camera, stop watch,
two-way mirror, computer program)
No feedback given afterwards to the Full feedback given after word the
observed observed

Figure . Different kinds of observation

involves looking, but it can involve senses other than sight: hearing, smelling, to watching and
tasting. For example, a researcher might look at the seating pattern in a group meeting, feel the
texture or comfort of different chairs, listen to people's contributions (both the words and the
tone), watch the body language, smell when people are becoming hot and bothered, even taste
the orange juice that is handed round to help people coal down. Observation as a data
generation method can be used within any of the research strategies discussed in this book. It
can be used in academic research to study, for example, behaviour at meetings or on a
production line, or interactions between end users and staff on an IT help-desk, or what
students are really doing during a programming tutorial in a laboratory. It can also be used to
study the behaviour of inanimate objects, such as software programs or computer-controlled
devices.

5
Research Methodology

There is a wide range of approaches to observation. They can be analysed by placing them on a
number of spectrums, as above Figure shows. In research approaches using observation is
between systematic observation and participant observation. These are explained and
discussed in the two sections that follow.

Planning and Conducting Systematic Observation

Systematic observation is where you decide in advance the particular type of events you want
to observe, and use a pre-designed schedule to note their frequency or duration. In other
words, you work with a pre-defined system of observations. This usually involves counting or
timing, so leads to the generation of quantitative data. For example:
 observing a group meeting -— number and type of contributions made by each member
of the group;
 observing a queue at a university IT help-desk — time of arrival of each student, time
student reaches head of queue, time taken to deal with the student’s query;
 sample of people - observing everything one person does for a given time period, then
switching to another person for the same length of time, and so on. You could devise a
schedule to observe, for example:
 frequency of events — counting how often the categories on the observation schedule
occur in a given time period;
 timing of events -- recording how long instances of events take, for example, the time
taken for. Computer program to process a given amount of data;
 events at a given time — logging everything that is happening at a specified time and
repeating after « pre-defined interval (for example, every 15 minutes).

For example, Figure shows an observation schedule used hourly each day for a week by
technicians monitoring university computer labs.

Choosing or designing a schedule

By looking in the literature or on the web, you may find that someone has already designed a
schedule that you could use. For example, many researchers have used a pre-defined schedule
to observe the types of contribution people make to meetings

6
Research Methodology

Fig. Observation schedule for monitoring computer laboratories

(for example, making a new suggestion, acting as peace-maker, being obstructive and critical,
summarizing ideas, drawing in other group members). If you can’t find an ‘off-the-shelf’
schedule in the IS and computing literature, try the psychology literature, which has a long
tradition of research based on observations, or, for more technical observations, the
engineering literature.
If you have to design your own schedule, it’s often worth initially spending time in the situation
you are interested in, observing what goes on, before deciding exactly what kinds of events or
activities to focus on. For all schedules, the items to be studied must be easily defined and
obvious — you do not want to have to spend time deciding whether an action is really taking
place. The items must also be relevant to your research objectives — there is no point
generating data you are not going to use. You also need to make sure that your list of
categories includes all possibilities - you do not want to notice relevant activities for which you
have no place on your schedule. Your categories must also be unambiguous and not overlap
with each other, so you can easily and quickly decide whether an item fits one category or
another. Finally, the things to be observed must be easy to record — for example, it is easier to
record isolated events than simultaneous ones, It is important to do a pilot study first, to see
whether your observation schedule works, You should include space in the schedule to note
contextual factors that might be relevant to the situation. For example, a schedule to observe
people in a meeting (Figure) includes a seating plan showing who was sitting where, and space
to note such things as the time of day (if the meeting is late afternoon people may be more
tired and irritable than at the start of the working day), the room temperature, background
noise, even the weather outside the window. This contextual information may be needed to
help explain the observations made.

7
Research Methodology

Planning and Conducting Participant Observation

In participant observation, the researcher takes part in the situation under study, so that it can
be experienced from the point of view of the others in that setting. This can be overt — people
know that you are carrying out research into what they do. Or it can be covert — people think
you are a ‘normal’ person, not a researcher. Rather than using a pre-defined observation
schedule, the researcher notes down as much a possible about what occurs, producing a rich
description of life in the setting. Your main piece of equipment is therefore yourself. You use
your own senses - seeing, hearing, and so onto experience and reflect on what people do in
your chosen situation. Participant observation is often associated with ethnography, where the
researcher tries to experience life in the setting from the inside, to gain understanding about
what people do, why they do it, and the meanings they assign to activities.

Types of participation

We'll look at four types of participation: complete observer, complete participant, participant-
observer and practitioner-researcher.
A complete observer is present in the setting either overtly or covertly, observing everything
that occurs, but takes no other part in the proceedings. For example, a researcher might sit in
on a teacher’s class to watch everything that occurs, but take no part in the lesson and class
activities.
 At the other extreme, a complete participant uses covert observation and tries to
become a member of the group being researched, to see the group’s world from the
inside. Of course, to fit into the research setting as a true participant, you require the
necessary credentials. You may need to be of the right sex and age, and you may need
particular skills and qualifications. For example, many readers of this book could quickly
acquire the skills to participate as a supermarket shelf-stacker or car park attendant, but
few will have the time to gain the necessary credentials to participate in the working
lives of accountants or lawyers.
 A participant-observer shadows someone, and can be used if you don’t have the
necessary credentials to be a complete participant. You follow people as they go about
their lives or jobs, observing the activities and interactions, taking part where you can
for example, making the coffee, doing some photocopying). If you gain the people’s
trust, so that they accept your presence and even forget that you are there sometimes,
you can learn about what they do and how they feel about it.
 A practitioner-researcher is someone who already has a job and decides to put on a
researcher’s ‘hat’ as well and investigate their own work organization. Benefits of this
approach are that you don’t have to negotiate access into the setting ~ you are already
8
Research Methodology

there, and you don’t have to spend a lot of time becoming familiar with it. However,
such research does bring its own difficulties. You will probably need to obtain
permission to research into your own organization, from both your bosses and
colleagues — you do not want to risk losing your job if they find out only by accident.
You must also somehow make yourself aware of the assumptions and pre-conceptions
that you have about your own job and organization, otherwise you will overlook issues
that an outsider would observe. You must also find the time to both continue to do your
own job and also to be a researcher. Further difficulties are discussed below.

Process

Participant observation is time-consuming. The longer you can spend in a situation, the more
you are likely to learn. Start by being non-selective in what you observe — try to observe
everything that goes on rather than starting with pre-conceived ideas about what you are going
to observe. In this way, you get a feel for the situation.
Later, as things emerge that seem particularly significant to you, you can make more focused
observations, concentrating on what seems important or strange, and start to form a tentative
theoretical model of what is occurring. You might then look for unexpected or contradictory
events, things that ought not to happen according to your emerging theory, meaning your
theory needs revising. You should also look for the issues and problem that the participants see
as important, so that you get an understanding of their views and beliefs. Remember that your
purpose is to develop a theory about what is occurring — not just to tell a story about your
time in the situation along the lines of ‘What I did in the school-holidays’.

Field notes

Participant observation relies on the observer being able to make field notes — you
can’t possibly remember everything that you observe. You need to make notes as soon as you
can after you have noticed something relevant. If you are a covert observer, this may mean you
have to make frequent trips to the toilet to have the privacy to make notes! As well as noting
what you have observed, you should note down your thoughts on the research process (for
example, difficulties in forming a relationship with someone, or plans about what else to study)
and your emerging analysis (for example, how what you are observing and experiencing relates
to your research questions). You should also include reflections on yourself as a researcher (for
example, how you feel, how you might be affecting the situation).

9
Research Methodology

Rules of the game

Even if people have given permission to be observed, they may not realize the rules of the
game, that is, that everything they do and say might be recorded. For example, a group of
systems developers might be happy for you to record what they do and say during official
meetings at work, but not over the canteen table at lunchtime. Even if they say, ‘this is not to
be used’, you have still heard the comment and cannot just erase it from your memory. When
obtaining permission to observe, you should point out to people that you are interested in all
aspects of their work or activities, not just the ‘official’ parts. You might need to remind them
again later, once they have become used to your presence. But sometimes you and your
conscience will have to weigh up whether you want to remind them you’re observing, because
reminding them may stop them doing precisely the things you are interested in.

Dangers and difficulties

Covert researchers risk being discovered by the people they are observing, as mentioned
already. This could put you in physical danger. There are also other dangers and difficulties you
may have to face when doing participant observation. One of the biggest hazards is ‘going
native’. You lose your sense of detachment as an observer and identify fully with the other
participants in the situation. No longer do things seem strange to you, they just seem like the
natural way to do things. Its vital that you do not forget your research purpose and do not lose
your researcher’s independent way of seeing things. However, being required to operate at two
levels — as a participant and a detached observer — is stressful. When you have come to know
others in a situation, and possibly formed friendships with some, it can be difficult to separate
yourself from them and write dispassionately about what you have observed and learnt. It can
feel like a violation of their trust, causing stress for you. You should arrange to have regular
meetings with your supervisor or a research support group to help you cope with your dual
existence as participant and researcher.

Practitioner-researchers, as already explained, have to learn to recognize and discard their

assumptions and pre-conceptions, to see themselves and their organizations as others would. If
they are able to make the familiar seem strange, they can reveal and question the professional
practices and priorities in their organization. However, there is a danger that once the research
is over, they can’t make the strange become familiar again, and can’t go back to being a
‘normal’ practitioner. Practitioner researchers can therefore end up unhappy and stressed in
their jobs. The feelings experienced have even teen compared to those of a priest, mullah or
rabbi losing his faith. Again you should look to your supervisor or a support group to help you
cope.
10
Research Methodology

Finally if you are participating in a group that is acting unlawfully, you too may be expected to
behave unlawfully. If caught, the plea of being a researcher is no defence.

Validity

For participant observation, there is often only one researcher, you. This means readers will
wonder why they should take your word that what you claim to have observed would also have
been observed by any other reasonable person, and you have not been biased. In other words,
is your work and its outcomes valid? You can’t guarantee that another person would observe
the same as you. Each of us has:
 selective recall: our minds remember some things and forget others;
 selective perception: our minds notice some things and simply ignore others;
 accentuated perception: our minds are particularly sensitive to some things, based on
previous experiences or our current state, and so take more notice of some things than
others would.
Being selective and partisan is an inevitable part of most research. It’s just that we become
more aware of the problem in research based on personal observation because it is so
dependent on the observer as the main piece of ‘equipment’. You can strengthen your
claim to validity by using:
 Verbatim quotations from the people in the setting, rather than summarizing what was
said in your own words. Giving the actual words used reassures readers that you did
indeed hear them.
 Triangulation of data or methods You can observe several people doing the same thing,
to see if they each do it in the same way, and you can use, say, interviews, to see if your
findings derived from observation are confirmed by the participants.
 Reflexivity — you should constantly reflect upon yourself in the situation: how you
might be affecting it, what you are taking for granted, assumptions you are making, and
so on. Some of these reflections should be included when you write up your research, so
that readers can judge whether they would observe the same as you if they were to
enter the same setting.

The Internet and Observation

The scope for Internet-based observation is limited because mostly we can only ‘see’ the words
people place on the Internet, or observe which webpages they visit. Given current technology,
we cannot smell, touch or taste via the Internet. There is the possibility of hearing via Internet
telephony — this has been little used so far in research, but is expanding rapidly, so it might be
used more in future. Up to now, researchers have concentrated on people’s written linguistic
11
Research Methodology

behaviour. By observing the interactions in a newsgroup, chatroom or mailing list, researchers

investigate computer-mediated communication. You could be a complete participant in a
newsgroup, chatroom or mailing list by reading, interpreting and replying to messages. Because
no one can see you, it does not matter what you look like. You can pretend to be a different age
or gender. If people accept you as a legitimate member of that newsgroup, you are carrying out
complete participant observation. However, you must consider the ethics of this. If other
members eventually discover that you are a researcher and have been observing them covertly,
and possibly deceiving them, they may feel harmed and that their privacy has been infringed.
You could, instead, introduce yourself as a researcher and ask the group’s permission to
observe and analyse their postings, as a participant-observer. You will need to explain your
researcher role at the beginning of each of your own postings, because any new member of the
group will not know about it. Be prepared for angry reactions. Many newsgroup members
consider the group’s online interactions as private (even though they are publicly available on
the Internet), and there does seem to be a suspicion of, and even antagonism towards,
academic researchers in cyberspace. Alternatively you could lurk, not posting, merely observing
— a covert complete observer. But then you are back with the ethical problems of covert
observation.
Other ways of observing people using Internet technology are also possible. A webcam could be
used to observe a setting, and the images sent over the Internet to a researcher who could be
anywhere in the world. The participants might be aware of the camera’s presence, and could
choose when to switch it off, or they might be unaware. leading to ethical concerns about
covert surveillance. Web-server logs, that monitor where website visitors come from and which
pages they visit, and for how long, can also be analysed to learn more about people’s web
behaviour. Be wary of making unwarranted conclusions from the data in such a log. For
example, if it shows that someone spent a long time on a particular webpage, it does not
necessarily mean that they found that page interesting. They could have had a slow computer
or network connection, or they could have turned away to speak to someone or make a cup of
coffee. Researchers are still discussing and evolving guidelines for Internet ethics and how and
when researchers should obtain consent.

12