ý Ċ

Menu Search

You are signedInstitution

inin
toas:
Sage Research Methods à
Sign
Get o"-campus Ă
my pro!le
access
East Texas A&MSigned in
Home
University

Summary
Sign in here to
Ă
access your reading
Have you created a personal pro!le?
orWith
createinstitutional
a pro!le
searches thataccess
so and
lists,Login
you can Isave
alerts. can:
saved

SIGN
clips, playlists and OUT
searches
ĕ View or download all content
VIEW MY PROFILEmy institution
has access to. SIGN IN
Contents SIGN IN/REGISTER
VIEW MY LISTS
ACCESS VIA YOUR INSTITUTION
Subject index
I don’tSIGN
haveOUT
a pro!le

Want With
to tryyour
Sagefree
Learning
pro!leResources
you can: !rst?
á BACK TO TABLE OF CONTENTS

SignCreate
ĕ up foryour
a free trial and
reading experience all Sage
lists
Learning Resources has to o"er.
Collecting, Collating and Conversing with Data
ĕ Save your searches

ĕ CreateSTART
searchFREE
alerts
30 DAY TRIAL Ā
By: Mary McAteer
In: Action Research in Education
You must have a CREATE
valid academic
PROFILEemail address to
Chapter DOI: https://doi.org/10.4135/9781473913967.n5
sign up.

Subject: Education
Need help? Contact Sage
Keywords: action research; case studies; critical incidents; critical incidents; data
Want to try Sage Learning Resources !rst?
collection; diaries;
Tools
interviews; pupils; pupils;
On this page
questionnaires; research questions;
teaching; teaching Sign up for a free trial and experience all Sage

č Show page numbers

Collecting, Collating and Conversing with Data

This chapter explores data; its nature, collection, collation and meaning. Case
:
 studies and exemplars are used throughout the chapter to illustrate and explore
the ways in which we identify appropriate data, plan and undertake its collection,
collation and analysis. Through these discussions, we also address the practicalities,
ethical issues, intellectual rigour and integrity of action research.

The importance of early data collection, the initial reconnaissance phase, is also
emphasised in this chapter, as is the initial analysis process. Through this, we begin
to develop an understanding of the concept of data-driven, theory-generative
approaches in research.

What Counts as Data?

On starting an action research, or indeed any other research project, for many
practitioners, this question is one of the !rst asked once a focus has been clari!ed,
and forms a key part of the early planning process. It is important for me to clarify
my use of terminology in respect of the word ‘data’. While for some people this
word has resonances of the type of ‘hard data’ produced through large-scale
surveys or other quantitative means only, I use it to mean all those items of
information gathered in the course of a research project. Further, I use the word
‘evidence’ in relation to the use of data. The argument built around a particular set
of data is what constitutes the ‘evidence’ for any claims made by the research.

Deciding what counts as data, and then which data to collect, will help guide the
planning of the project, negotiation of access to the !eld, and appropriate
timescales to be drawn up. For the beginning action researcher, this question may
not be as simple as it !rst appears. Chapter 2 introduced some models of the action
research process and it is possibly at this stage of a project that closer conversation
with these models becomes important. As an iterative process of review, re#ect,
:
 observe and act, the data collection phase of an action research project is neither
epistemologically nor temporally removed from the analysis or the action in the
project. Indeed, for action research as a project which often has its genesis in a
‘hunch’, or a ‘general idea’, the very notion of ‘data’ within the project is not only
potentially challenging at the start, but almost certainly changes conceptually and
substantively during the life of the project. Given that action research is also a data-
driven approach to research, the questions relating to data are vitally important in
getting the project o" to a good start.

Quantitative and Qualitative

In order to e"ectively address these questions so that data collection, collation and
analysis can be properly planned, it is imperative to return to the research question
itself. The nature of the data required in any research project is highly dependent
on the actual research question. This might seem a rather obvious, indeed
super#uous, statement but experience of supervising countless projects has shown
me that many inexperienced researchers come to their projects with !xed notions
of how they will collect their data. First, cultural orientation to ‘scienti!c’ and
‘objective’ concepts cause many to view the process of data collection as leading to
quanti!able, objective measures of the phenomena in question. Secondly, emerging
understanding of social science methodologies suggests to many new researchers
that most projects in this broad genre use questionnaire and interview as the
predominant data collection devices. This is often coupled with an assumption that
the qualitative data produced from a questionnaire will be converted to numeric
form, and thus made more legitimate. Qualitative data such as interview transcripts
may be seen as sources of less reliable evidence, and it is not unusual to see action
research assignments for postgraduate programmes include summary tables of
questionnaire responses in the text, which are then explained and analysed, while
:
 interview transcripts are found in the appendices, and barely referred to in the main
text. In some way, the initial ‘sense’ of the research question can get lost at this
stage, and the whole process turn into a more mechanistic one than it might be. It is
also likely that the data presented will not have actually addressed the research
question. Questions about children's enjoyment of their work may be answered
with data showing improved test scores (which may be proxy measures, but cannot
be assumed to be so). Qualitative data will often be identi!ed as relevant, but is
often dealt with in a quanti!cation framework. The propensity to interpret a range
of well-produced graphs and charts as the e"ective presentation of research
!ndings is probably somewhere in most of us.

Further, and it is worth introducing a cautionary note here, ‘qualitative’ and

‘quantitative’ are sometimes used as descriptors of research approaches or
methodologies (often thought of as almost polar opposites in their philosophies). I
do not subscribe to that view, but instead consider these words to be descriptors of
data types. As such, any research approach can make use of both types of data.
Action research is no exception, and indeed a quantitative data set can provide a
very useful backdrop to a study, illustrating perhaps a broader signi!cance of your
own research question. The more qualitative data can then be used to provide
insight into the situation.

Let us start exploring the ‘data’ question in a little more detail by introducing the
story of Judy, a pastoral leader in a secondary school, to illustrate the stages in her
project, and the ways in which data were generated and dealt with.

Judy and the Behaviour Problems

Case Study Part 1

:
 Judy was an experienced teacher working in a mixed-sex secondary school in an
inner-city area. Indicators of deprivation were a little above the average, but not
signi!cantly so, and on the whole the school had a good reputation. The school had
undergone a reorganisation in its pastoral structure, and single-sex, relatively small
tutor groups operated in Key Stage 3. While this had initially caused some
consternation among sta", now in its second year of operation it was felt that
behaviour problems had subsided and those which persisted were more e"ectively
managed. Early indications suggested that achievement was also improving. Judy
had a tutor group of 15 year 8 boys who seemed to be the focus of every negative
conversation in the classroom. During the autumn term, they had had more
disciplinary and behaviour incidents logged than the rest of the year group in total.
Their attendance was beginning to drop o" also and, even in her own lessons with
them (geography), they had begun to be troublesome. The typical pattern revolved
around persistent, but often low-level disruption such as calling out, ‘messing
around’ with other pupils, causing minor distractions and, at times, ‘answering back’.
The boys were of average ability, one was receiving support because of dyslexia,
and one had mild attention de!cit hyperactivity disorder (ADHD) for which he
received some in-class support on a daily basis. All teachers in the school had
attended INSET training in relation to general behaviour management, and in
relation to speci!c needs, such as ADHD, children on the autistic spectrum and
dyslexia.

At the start of her action research project Bernadette formulated her research
question as follows:

‘What can I do to help improve the behaviour of the boys in my tutor group?’

Commentary
:
 Judy has clearly formulated a typical action research question and, following Elliott's
model, decides that her starting point is to undertake a reconnaissance. Other
authors describe this as scoping or mapping out the !eld. In the words of McNi"
and Whitehead (2005) the researcher is asking ‘What do I already know about the
issue?’ This is an important starting point, and indicates the need to consider all the
possible sources of information about the matter in question. To reiterate the
words of Popper, ‘ We must !rst get better acquainted with the problem .’ (Popper,
1972: 260) It is important to recognise and remember that the articulation of a
hunch or perception needs further exploration before it can be translated into a
legitimate research focus. A feeling on the part of a practitioner that something may
not be quite right, or in some way problematic, is not necessarily grounded in ‘fact’.
It could arguably be the case that a practitioner such as Judy is feeling somewhat
sensitive about her group's behaviour, and therefore her initial hunch is as much a
manifestation of personal sensitivity or security as it is of any reality. To ameliorate
this, and to justify her engagement on a research project, it is imperative that some
further substantiating evidence is sought. In this case, Judy, in order to consolidate
(or otherwise) her own perception of the problem, has also consulted school
records such as behaviour logs and attendance records. Together with her own
hunch, they convince her that there is an issue worth exploring. What the records
do not do, however, is help her understand what the nature of that problem or
issue is. They do not get her better acquainted with her problem.

When considering sources of data through which a practitioner might do this, it

must be remembered that there may be many di"ering perspectives on an issue,
and it is necessary to get a wide range of views in order to capture as full a picture
as possible. This does not necessarily involve selecting a large number of
respondents or participants. If they all represent a similar perspective, then your
evidence could still be rather skewed. What is important is that you consider what
:
 those sources of information and data might be, and deliberately seek a wide range.

The action researcher is trying to delve into what is happening or what has
happened and, as such, needs to ensure that this initial reconnaissance phase
frames the rest of the project e"ectively and accurately, and is informed by as
complete a range of perspectives as possible.

Reflection

Consider Judy's situation, and make a list of sub-questions she might ask in relation
to her focus. Some starting points might be:

What is the nature of the unacceptable behaviour?

When and how often does it occur?
Who is involved?
Is there any pattern?

For each of these (and others you identify) compile a list of the sources and types of
data that might help her get a better understanding of her problem. Remember
that it is important for her to consider the problem from all angles. Consider also,
for each source of data, the pragmatics of accessing and collecting them. Will Judy
!nd it easy or di$cult to access the data? Will she need any special permission?
Ethical matters will be discussed in more detail later in the chapter, but at this stage,
list and start to re#ect on what you think the ethical issues might be in relation to
each source.

Do you think there are any ethical issues apart from those associated with data?

Case Study Part 2

Judy spends some time re#ecting on her initial question. Having attended many
:
 behaviour management training events in her 15 years’ teaching, she has a range of
resources at her !ngertips and has become adept at using them. This time,
however, her own initial pause for re#ection makes her consider the
appropriateness of using some ‘o"-the-shelf’ behaviour management scheme. She
decides to keep a re#ective diary for a two-week period before deciding on anything
further. Previous experiences in using approaches have been variously successful,
but her re#ections also make her realise that they have often been most e"ective
when she has adapted them, or used some of their principles tailored to her own
situation. She is initially unsure what exactly she should record in the diary, but
decides that if she records quite widely at !rst, then perhaps the signi!cance of her
notes may emerge later. She begins therefore to note particular events, brief
accounts of incidents in class, or reported to her, recording an account that is more
than just an event log. She notes thoughts, re#ections, questions and feelings
relating to the events. Although she usually records all school-related events
electronically, she feels that a dedicated notebook would allow her more scope in
terms of keeping a re#ective diary. She ensures that each entry is carefully dated
(and with the time, if appropriate), and leaves plenty of space for later re#ections,
thoughts and questions as they may arise.

Commentary

Judy has begun to realise the importance of re#ection in action research. Day (1993)
considers re#ection to be a necessary (though not su$cient) condition for
professional learning and development, and in discussing its signi!cance, identi!es
that it provokes ‘confrontation’ of that learning in a way that informs future action.
The Ghaye and Ghaye (1998) model of re#ection was introduced in Chapter 2. Other
such models, such as those of Smyth (1991), Gri$ths and Tann (1991) and Rolfe et
al. (2001), provide a range of frameworks and approaches, and likewise provide an
:
 imperative to action. Later in the chapter we discuss the nature and structure of the
re#ective diary in more detail.

During the two-week diary-writing period Judy captures stories from practice,
anecdotes from the classroom and sta"room, near-verbatim transcripts of pupil
and colleague discussions, and a record of her own thoughts and feelings over the
period. In short, she has begun to build up a story of what is happening, in a more
systematic manner than she had previously done. She is beginning to gain an
insight into the nature of the behaviour problem, the perspectives of pupils and
colleagues, and her own response in relation to it. As yet, however, she is unsure
whether or how it can be called ‘data’. Reading over the entries for the past two
weeks, like many new researchers, she !nds that her questions are of the type,
‘That's all very well, but is it data? Can it be used in my project? If so, how?’

It can be tempting to think that such anecdotal evidence is inadmissible in ‘real’

research. Vandenbroucke (2006), however, in his discussion of the strength of study
designs in relation to their intended outcomes, identi!es anecdotal case study and
report as the most e"ective design in studies whose focus is discovery and
explanation. Although writing in the !eld of medical studies, I would suggest that his
analysis is germane in most contexts. Suggesting that ‘odd observations … in data or
the literature’ will spark new ideas, he identi!es this as the source of further
analysis and discovery.

Thus, Judy's own diary of anecdotal evidence is indeed part of the legitimate data
for her project. Not only will it help her in building the narrative of her project, but
quotations from it can be used to illustrate her own perception of events as they
were uncovered, her responses to them and her developing understanding of their
importance. Alongside other key data sources, her diary will form part of a well-
rounded evidence base for the project. It will also of course contain more than
:
 anecdotes, as is discussed in the next section of this chapter.

Having now listed some the sub-questions that Judy might ask, and the means by
which she might gather relevant data, let us now further consider the range of data
sources that may be useful to her and to action researchers in general. The list of
possible sources can include, but is not restricted to:

re#ective diaries and critical incident approaches

interviews
questionnaires
document analysis
observations of practice.

The rest of this chapter provides an overview of data collection techniques,

exploring their appropriateness in particular situations, and discussing pragmatic
issues around them, and some possible ethical considerations.

Data Sources

The intensely personal nature of an action research project, with its focus on
re#ective practice, is perhaps a key di"erence between it and other forms of
practice-based research. For this reason, I intend to discuss the use of self-re#ective
means of data collection !rst, and in more detail than other data collection
methods. Of primary importance in this set of methods is the re#ective diary. I
begin by discussing the diary, its use and approaches to its analysis, and also
introduce the relationship between critical incident techniques and diary-keeping.
While Judy has already begun keeping a diary, a more fully developed strategy for
this is likely to be of bene!t throughout the project. Throughout this section on data
collection I draw on examples from my own research, using them to illustrate some
of the key issues raised.
:
 The Reflective Diary

The case study has drawn attention to Judy's use of a re#ective diary in her
research. Almost universally used in action research projects, it can form both the
narrative and chronology of the project, as well as provide a thinking space. The
format of this diary is entirely up to the researcher. What is important, is that the
format is easily accessed and used, that the researcher feels comfortable with it,
and that it is used on a regular basis.

For many action researchers, a key question at the start is ‘What should I write in
the diary?’ As in many action research questions, there is no right answer. However,
that is not to say that there are no guidelines. It is, of course, important that if the
diary is to provide a chronology of events, it is completed (and dated) on a regular
basis with descriptions of events, as they happen. It is a good habit to write
something in the diary at least once each day, even if it is just something relatively
short. These daily entries can also form prompts for re#ection, both in the moment
and later. In this way, they help develop understanding and analysis. Moon (2006:
26–36) suggests that:

Journals slow the pace of learning

Journals increase the sense of ownership of learning
Journals acknowledge the role of emotion in learning
Journals give learners an experience of dealing with ill-structured material
Journals encourage metacognition (learning about one's own process of
learning)
Journals encourage learning through the process of writing.

Regular diary writing keeps the research question, and thoughts about it,
continuously to the fore in the researcher's mind and, as such, provides the
opportunity for personal, and indeed emotional engagement with the research
topic and process. Saunders (2004) suggests that the evidence base in considering
:
 practice is a living process built around practical experience. Citing Dadds (1995),
she describes the process as passionate enquiry. In the text ‘Passionate enquiry and
school development’, Dadds herself suggests that Vicki's enquiry (Vicki is a teacher
whose story is related in the text) is informed as much by feelings as by thoughts.
The diary can be a safe place in which to record those feelings.

The regular writing of a diary is signi!cant also in its contribution to increasing what
Mason (2002) calls ‘sensitivity’ within the research. Despite our familiarity with our
own practice, the keeping of a diary and other self-re#ection/self-study techniques
almost always reveal something previously hidden to us. In short, the diary can be
seen as a way of producing a running commentary on the project, to record
observations and, crucially, a space in which to begin the process of theorising
practice in order to develop contextualised understanding of speci!c problems or
concerns.

The use of such techniques in conjunction with other sources of practice

observation data (such as that from other adults, or pupils in the room) can provide
a valuable means of triangulation, that is, comparing and contrasting di"erent
perspectives (either from di"erent sources or di"erent data collection tools) in
order to !nd points of agreement, points of disagreement or areas of incomplete
understanding or further interest; all possible pointers to the formulation of a new
action research cycle.

Using Diary Data

While diary-keeping is fundamentally a private and personal business, the research

diary will also contain elements that the writer may wish to make more public. The
!nal report may include some citation from the content of the diary, but it is often
the analysis of the diary that features most strongly in the report. I have chosen not
:
 to discuss at this point the way in which diary entries may be analysed, but rather
keep it for later in the chapter, relating it also to other sources of unstructured, text-
based data in the project, and at this stage focus on where it can be useful in the
project.

Excerpts from the diary can, on occasion, feature in the !nal report. One example
might be to show how thinking has changed over the course of the project. The
example below is from my own diary entry during my DPhil study. Having spent
some time wondering why so few girls were choosing to study A-level physics, and
what I, as a classroom teacher, could do to make the subject more attractive, I was
uncovering perspectives that really challenged my initial understanding. I had a
reluctance at the time to simplify (as I thought it would be) my focus to one simply
of feminism, and felt that to do so, would in fact be an ‘easy’ way through a much
bigger philosophical problem. As I analysed interview data with a range of students I
began to see a strong theme of the mathematicisation of physics coming through,
and a student perception of both mathematics and physics as lacking in creativity,
and de!ned by non-negotiable rules and a somewhat Boolean approach to logic.
From my perspective, this was highly signi!cant, and con!rmed my own perception
that it was not a simple ‘girls don't like physics’ matter. It was more than that.
Couched in the epistemology of the subjects themselves, in their very nature, the
analytic framework was one of philosophy in the wider sense. What did this
evidence mean for me as a teacher? Did physics have to have this epistemology?
Was this an essential part of its nature? Had it always been?

I then undertook what could be described as a conversation between my data and

‘the literature’, trying to understand this theme of rules and logic, and, through the
history of science and science education, began to understand the provenance of
the curriculum I was teaching at the time. My diary entry re#ected the change in my
understanding that followed my analysis and reading (and in fact, writing it in my
:
 diary was also the place where I ‘worked it all out’, or had conversations in my head),
and raised questions about my own practice:

Example 1

I've been reading some contemporary literature on women in science. It seems that
the whole thing about logic, and mathematical rules, and no creativity, is in fact a
gender issue. I never thought of it that way. It seems that girls and boys deal with
rules di"erently, and that the way physics presents them suits boys rather than
girls! That seems peculiar to me; I'd never thought of it that way, but there does
seem to be a lot of fairly recent research indicating that the way the content, and
the methods of science are presented to students is biased in favour of boys. I
wonder if that's the case? I suppose that's really two questions. Do boys and girls
see logic and rules di"erently, and is science education really based on that type of
model? What would I see if I examined my own classroom?

Using the diary entry in this form, almost as a ‘pause for re#ection’ in the narrative
of the project, not only identi!ed the questions that had become important for the
next phase of my action research, but gave some insight into the way in which they
had arisen. It was, in essence, an auto-dialogic process, in which I began to make
meaning from data, informed by wider consideration and reading.

While it is probably unusual for relatively large extracts from a diary to make their
way into a report, especially a report which is fairly short in length, it is not at all
uncommon to see action research reports draw on diaries to illustrate changes in
thinking, near verbatim reports from practice (such as an exchange with a colleague
or pupil), or an a"ective response to a particular situation. A further example from
my DPhil diary illustrates this:
:
 Example 2

My diary entry of late August 1999 indicates that ‘talking things over’ is still
something I value very much in terms of helping me clarify my thinking and makes
me think too about how little the children really get the chance to do this in their
learning.

This realisation was in fact for me, what Tripp (1993) and others would call a critical
incident, which he describes as things that mark ‘signi!cant turning points or
changes in a person’ (1993: 8). What, then, makes an incident ‘critical? While a
number of writers have addressed this, they mainly draw on the early work of Tripp
(1993) and Woods (1993), who both emphasise the extent to which their
identi!cation is based on the judgement of the researcher after the incident has
occurred. In my experience, researchers have found this a very helpful construct in
the analytic rereading of re#ective diaries. The identi!cation of critical incidents in
their practice, or in their thinking, can be powerful stimuli for further research or
action.

Tripp (1993: 8) suggests that critical incidents are not

things which exist independently of an observer … awaiting discovery like

gold nuggets or desert islands, but … are created. Incidents happen but
critical incidents are produced by the way we look at a situation: a critical
incident is an interpretation of the signi!cance of an event. To take
something as a critical incident is a value judgement we make, and the
basis of the judgement is the signi!cance we attach to the meaning of the
incident.

Woods (1993: 357) suggests that critical incidents are ‘unplanned, unanticipated and
:
 uncontrolled. They are #ash-points that illuminate in an electrifying instant some
key problematic aspect of the teacher's role and which contain, in the same instant,
the solution’.

As such, and of necessity, a critical incident only becomes critical in retrospect. Their
identi!cation happens only on re#ection. They are not classi!ed as ‘critical’ due to
any drama or sensationalism attached to them. Rather, their criticality is based on
the justi!cation, signi!cance, or meaning given to them by participants (Angelides,
2001). A re#ective diary can provide a rich seam of potentially critical incidents and
their analysis can signi!cantly inform the progress of an action research project.

I have found Tripp's guidelines for critical incident analysis supportive and
epistemologically helpful. In short, these can be presented as follows:

1 Describe an incident from recent professional practice, choosing something

interesting, annoying, inspiring, thought-provoking or typical.
2 Suggest an initial explanatory response to the incident (including your
emotional response).
3 Ask questions which delve deeper into the meanings behind the incident, for
example, di"erent ways of thinking about it. Keep asking why; explore your
dilemma, consider personal theories and values that in#uence your
judgement.
4 Consider the implications of the incident and your deeper understanding of
it on your future practice. (Adapted from Tripp, 1993)

This approach (and a simple template derived from it) has been of use to many
action researchers trying to make sense of diary entries. For others, critical incident
analysis becomes a deliberate choice of method in its own right. These researchers
often choose to use a template, such as that in Figure 4.1, to help them capture and
initially explore such incidents.

Figure 4.1 Critical Incident template

:
 ð Download Image as .jpg
:
 Reflection

Re#ect on your own practice in recent weeks. Has anything happened which you
now, in retrospect, might class as ‘critical’? Use the template (based on Tripp's work)
to record and re#ect on your own development.

Interviews

Interviews often form the mainstay of the data for action research projects.
Although time-consuming to carry out, they are particularly useful in helping the
researcher to ‘get inside’ the story. On the whole, action research interviews tend to
be of the ‘semi-structured’ variety. By this, I mean that the researcher will normally
have some guideline questions around issues they want to explore, but will have
#exibility in how the questions are worded, in how they lead to follow-up questions
and in the order in which they are asked. The aim of the semi-structured interview
is to allow both the researcher and the participant the freedom through which to
explore an honest and authentic account. The more skilled the interviewer, the
more the ‘structure’ can be relaxed, and the interview can become more of a
conversation. However, it does take well developed skill on the part of the
interviewer to ensure that such unstructured interviews remain focused on the
main research question(s).

Becoming skilled in interview technique is usually as a result of experience. Early

attempts at interviewing are often characterised by a more structured and
somewhat formal approach. For this reason, I usually advise my own
students/mentees to plan a period of time between the !rst and second interview
they undertake so that they can re#ect on and learn from the process itself.
Regardless, however, of the skill and experience of the interviewer, there are some
:
 key points to consider in preparing to interview participants. Among those that are
particularly pertinent to school-based researchers are:

selection of interview participants

location and time of the interview
relationship between interviewer and interviewee
ensuring encouragement during the interview without asking leading
questions
capturing the content of the interview.

These are in fact ethical considerations as well as pragmatic ones. Let us address
each brie#y in turn.

Selection of Interview Participants

Two things are of central importance in selecting participants for your interviews.
First, a return to the initial research question is important, so that in remembering
exactly what it is you want to !nd out, you begin to form a picture of who might be
able to contribute useful information. For most educational researchers this will
mean that interview participants can come from colleagues, pupils, those you
manage or who manage you, and other professionals working in your setting. It is
tempting to include groups such as parents or school governors in this list but,
given the challenges you might face in undertaking such interviews, consideration
should be given as to whether these voices are needed in relation to the research
question. If your question relates to improving your own teaching of a particular
subject or topic, then while the parent voice in relation to a pupil's natural interest
in the subject might be of some interest, it is unlikely to provide you with the data
that will allow you to e"ectively understand your own practice better and improve it
within your context. However, if your question relates to motivating children to
regularly complete homework tasks, then the parent voice will probably be essential
:
 to your study.

Secondly, it is necessary to remember that there will be a range of viewpoints and

perspectives on any issue and, as a researcher, you must try to ensure that voices
from all these perspectives, if possible, make their way into your data set. The
process of ‘triangulation’ is explored in many texts, the key de!nitions indicating its
role in ensuring validity of the data. Derived from the language of navigation, its
main articulation is the comparison of data gathered by di"erent methods and/or
di"erent sources. Thus, the researcher actively seeking out participants for
interview who can provide di"ering perspectives is engaging in one aspect of the
triangulation process. In this way the strength of claims made can be improved and,
at the same time, points of disagreement #agged as areas of incomplete
understanding; hence the development of continued exploration.

Throughout the selection process, however, pragmatic issues must be

remembered. It may be that one particular person could bring a very important
perspective into the research but, on asking them, they decline to take part. This is a
very real issue, and sometimes a risk, in designing a project of this nature, and must
be thought through carefully at the planning stage. If, for example, a person
thought to have a vital voice declines to participate in the research, is it possible to
get that perspective from an alternative source? If not, will its absence compromise
the quality, or even the viability, of the research?

For this reason, the practical matters around planning a range of respondents, and
gaining their consent to be interviewed, is vital early in the process. If key personnel
do not wish to participate, then it is much better to know this at an early stage.

Location and Time of the Interview

:
 It is to be expected the researcher is deeply interested in the project. It is not
necessarily the case that others will share this passion. When participants agree to
take part in a research project they are giving freely of their time as well as making a
contribution to the project itself. Care should be taken in planning a location that
will both show appreciation of this and set an atmosphere that will put respondents
at ease. Privacy, protection against interruption and the comfort of both parties will
all contribute to a more productive interview. The relationship between the
interviewer and interviewee is also important in choosing an appropriate location.
The use of his classroom, by a teacher to interview a pupil may highlight a power
imbalance and make the pupil feel uncomfortable. Likewise, a senior leader
interviewing a junior member of sta" would be better choosing a neutral venue
rather than their own o$ce. Power relationships are an important and ethical
consideration in any research, and perhaps even more so in action research which
claims to be a democratic and democratising process.

Relationship Between Interviewer and Interviewee

In action research, where the research is situated within the researcher's own
practice, there will inevitably be issues of power relationships between the
interviewer and interviewee. These may emanate from teacher–pupil or sta"
hierarchy situations, and must be taken into account in the conduct of an interview.

The teacher–pupil issue is also potentially problematic for school-based

researchers. Inevitably, it raises questions of whether or not the pupil has actually
felt they can refuse to participate, and what e"ect their participation may have on
their life within the school. Will they feel able to respond truthfully? Will they !nd it
hard to understand the nature and purpose of the research? Will they feel
uncomfortable when asked probing questions?
:
 There are issues, of course, for the teacher also. Will they be able to help the pupil
feel comfortable and unthreatened? Will the pupil tell ‘the truth’ and produce useful
data? In some ways, these issues can be addressed though careful planning of the
location for the interview. Conducting it at a teacher's desk, with the teacher at one
side and the pupil at the other, is unlikely to facilitate frank and open dialogue
during the interview. Likewise, choosing a location in the school that may have
negative associations for pupils may not be wise either. If it is possible to use a
room that is comfortable, not associated with normal teaching or disciplinary
functions, and unlikely to be interrupted, then the quality of the interview may be
improved.

While these examples seem speci!c to teachers and pupils, it is not di$cult to
extrapolate from them to other situations where there may be a power imbalance,
and it is the responsibility of the researcher to consider this at the planning stage,
and take any steps possible in order to mitigate the issues that may arise.

Ensuring Encouragement During the Interview without Asking Leading Questions

For the novice researcher, this can be a particular challenge. Nervousness can make
the interviewer stick rigidly to a script, with the e"ect that he or she may ask little in
the way of follow-up, or probing questions. Doing this can reduce the interview to
something more like an oral questionnaire, losing the rich data that it has the
potential to generate. On the other hand, it is possible that the interjection of
probing questions, if not managed carefully, can serve to move the interview in a
particular direction, betraying the researcher's own interpretation or agenda, and in
this way skew the information. Hasty interjections can exacerbate this and one
technique that may be helpful is to seek clari!cation through rephrasing what has
been said, thus con!rming what is meant but also gaining some thinking time in
:
 which to develop a more considered probing question. ‘So can I just con!rm that
what you mean is …?’ helps both in reducing the potential for misinterpretation, and
also the rushed move to the next stage in the interview.

Capturing the Content of the Interview

Interviews have the potential to produce large quantities of unstructured, free-text

data and, as such, a key decision accompanying the decision to undertake
interviews is that of how to capture and record the interview. The options available
include note-taking during the interview, writing up immediately after the interview
and audio-recording during the interview. My own preference in this matter, and
indeed my practice, is to audio-record if at all possible. This must be done with the
permission of the interviewee, but in my experience it is unusual for participants to
refuse. After an initial feeling of self-consciousness, they generally stop noticing a
small recording device. Audio-recording obviates the need to focus on the recording
process, and allows the interviewer to interact more thoughtfully with the
interviewee content. In this way, both the interviewer and interviewee can feel more
relaxed, the interviewer can listen more sensitively to the responses, and the
interviewee can feel that the #ow of what they are saying is not constantly broken
by the interviewer pausing periodically to take notes.

However, this obvious bene!t also has the potential to generate more text than
might have been the case, and so the problem of how to later deal with the audio-
recording is one to consider. Should the interviewer make notes from the
recording? Should a full transcript be produced? Should the services of a transcriber
be used? Many texts deal with the pros and cons of each, and of course the decision
made is often one of pragmatism rather than philosophy. What is important is that
we return to the notion of action research as a data-driven, theory-generative
:
 methodology, recognising that it is perhaps more important to consider how you, as
a researcher, might get to know this data in the best way possible. If you are
fortunate enough to have the services of someone to transcribe interviews for you,
then it is imperative that time is taken to read, reread and otherwise become
immersed in the data.

As a !nal word on this matter, it must be remembered that the write-ups from the
interview (whether in note form or as full transcriptions), like all data, remain the
property of the respondent/participant, and should therefore be agreed as an
accurate record or data set, and also, agreed in relation to their !nal use.

Questionnaires

If there is a need for data that can be quanti!ed, for example, in response to a
question about how widespread a particular perception is, or the extent to which
colleagues have used a particular resource in a range of situations, then a potent
and cost-e"ective way is through the use of questionnaires. This is particularly the
case in school-based research as questionnaires can be administered and collected
with little e"ort on the part of the researcher. There is no requirement for
participants to post the return document and, in general, response rates are high.
For reasons such as these, many practitioner-researchers use questionnaires as a
main source of data collection.

The design of the questionnaire may, however, be more time-consuming than

initially considered. Common problems that all researchers can run into include:

lack of clarity in their own mind in relation to what they hope the
questionnaire will elicit
ambiguity in the wording of the questions
lack of planning in terms of how the data will be collated and analysed
:
 bias in the framing of the questions
unreasonable expectations in relation to what can be learned from the
responses
a presentation and layout of the document that may not be user-friendly
response categories which fail to meet the responses people want or need to
make.

The use of non-speci!c language is an important consideration in writing a

question. It is highly likely, for example, that the use of words such as ‘regularly’, will
be open to a range of di"ering interpretations and therefore answers to questions
along the lines of ‘do you regularly undertake extra study in your own time?’ with
the option of a ‘yes’ or ‘no’ answer will generate relatively meaningless data. It is
much better to give speci!c measured intervals in cases like these, so that the
question might be reformulated as follows:

Do you undertake extra study in your own time:

a. Once a month or less often

b. 2–4 times per month
c. At least once a week
d. More than 4 times per week

Reflection

Do you think this question would now tell a teacher all she or he wanted to know
about a student's study habits? Do you think a further question might help? What
would you ask?

Some problems of lack of clarity can be addressed through piloting the

questionnaire before it goes live. Getting a number of people (of a similar
demographic to the intended recipients) to answer it, and even provide feedback,
:
 can seem time-consuming, but in the long run, is time well spent. If, in a set of 10
questions, three are unclear to most of the pilot group, then it makes sense to
either discount or change those questions.

The layout and presentation of the questionnaire is an important and often

overlooked consideration. Endless pages of poorly laid out questions are unlikely to
look inviting to potential participants. Likewise, an abundance of free-response
questions make the process look like hard work.

Having decided on a set of questions, and determined that some free-response

questions are vital to the quality of the information possible, the matter of how
much space to leave for those free-response questions that you consider necessary
is also important. The issue of free-response versus coded response questions is
one which you also need to consider in relation to the validity of the questionnaire.
On the positive side, using pre-coded responses sets up an e"ective system for data
collation and thus aids analysis. On the other hand, if your pre-selected responses
are not appropriate for the needs of the participants, then the approximations they
use in making their selected response may not give an accurate data set. This can
be further complicated if responses to open-ended questions seem inconsistent
with those gained from the coded responses.

As a researcher, these questions need to be carefully considered at the start of the

project, and for those undertaking school-based action research, a further layer of
complication is added.

In many large-scale surveys, the data collection document will ask for some
biographical/demographical information in order that appropriate generalisations
can be made from the !ndings. Names are not normally asked for, as there will be
no need to consider the relevance of individual responses. In the small-scale action
research projects undertaken by most education practitioners, no such
:
 generalisations will be made. Further, in many cases, the questionnaire may be a
precursor to follow-up interviews and, as such, there needs to be some
identi!cation on the questionnaire. This raises many issues for the action
researcher. Will participants be as willing to participate if they are asked to put their
name on the form? Will those who do participate answer as truthfully as they might
have done on an anonymous document? How can con!dentiality be assured? Again,
and particularly important in educational workplaces, there are issues of power
relationships and ethics. Again also, there are no right answers. There is, however, a
need for the researcher not only to consider these issues, but to formulate a
justi!ed and justi!able response to them. What will you, as an action researcher, do
in order to deal with these concerns?

Document Analysis

In any practice-based research project there are likely to be documents available

which will give both historical and contextual information to the researcher. These
documents are of two broad types. In the !rst case, there will be a range of
documents produced externally to the setting, such as legislative and policy
documents (which can illustrate not only guidance for practice, but also some of the
reasons behind its articulation), national and regional achievement and other data,
against which schools and settings are often benchmarked. Documents produced
within the setting, such as school or setting-based policies, minutes of meetings,
schemes of work, reports of inspections and so forth, can all provide a history of the
current situation and, as such, help provide a rationale and historical narrative for
the chosen research project.

Observations (of Practice)

:
 In certain settings, the use of interview, questionnaire and other data collection
approaches may be di$cult to implement e"ectively. With small children, for
example, the lack of reading, writing and oral communication skills may be
suggestive of a more observational approach to data capture. Approaches to this
range from the very structured Flanders Interaction Analysis Categories (FIAC)
(1970) to the more unstructured naturalistic approaches described by Miles and
Huberman where the researcher tries to acquire an ‘empathetic understanding’ of
the situation as perceived by ‘local actors’ as if ‘from the inside’ (Miles and
Huberman, 1994: 6).

As with all decisions of method, the purpose of the data collection should inform
the choice. If, for example, a teacher is interested in reviewing the nature and
quantity of classroom talk, then the FIAC structure will be a focused and highly
appropriate tool. A tool like this can also be adapted for observations of other types
of interactions, as can be seen in the exploration of interaction and communication
between adults and children on the autistic spectrum undertaken by McAteer and
Wilkinson (2009), where Melanie, a teacher in the school, had developed her own
speci!c adaption of the FIAC schedule to suit the needs of her own project which
had involved developing a customised adult–child interaction training approach.
Knowing that she needed to conduct highly structured observations of the training
sessions, she adapted the FIAC schedule to take account of the speci!cs of her
situation. (It is worth reading her account to see her re#ections on using this
approach.)

If, on the other hand, the observation is more to see ‘in general’ what interactions,
conversations and behaviours are happening, then a more unstructured and
naturalistic approach will be the most appropriate. While this approach helps to
overcome what McKernan (1997: 115) describes as the ‘problems with interaction
analysis categories’, indicating that they ‘do not take su$cient account of the
:
 substantive content of the behavior or message’, it is an approach which has the
potential to yield large quantities of disparate data. As Punch (1998) explains,
undertaking observations of this nature necessitates a lengthy familiarisation with
the data, and a willingness and ability to develop conceptual categories in a more
grounded approach. There is also a very practical question for the researcher
choosing this. How will they actually capture and record the observations? Will they
write notes as they observe? Will they ask someone else to ? (in the knowledge that
someone else may not be as attuned to the subtleties of the situation as might be
hoped, and additionally, will add a further interpretation layer to the data).

Whichever approach to observation is used, my advice is to video-record the

observed session, and conduct the observation afterwards. The challenge of
accurately recording observations in-the-minute is demanding, and risks oversights
which may be signi!cant. Further, in recording and re-watching the session, the
researcher develops a greater familiarisation with the data, which is in itself an
initial analytic device.

Other Data Collection Methods

While the diary, interview and questionnaire often form the mainstay of many
action research projects, they are by no means the only methods. The challenges of
working with very young children, or those for whom communication or
understanding may be a problem, may make interviews and questionnaires
inappropriate. Further, the research question itself may suggest a type of
knowledge or information that these methods cannot adequately provide. For this
reason, the following techniques may also form part of the action researcher's
armoury:
:
 focus groups
photographs
children's work/drawings.

The list is not exhaustive. Dadds and Hart (2001) use the term ‘methodological
inventiveness’ to describe the ways in which action researchers may approach their
endeavours. I will not explore each in detail as, on the whole, they draw on similar
recording and analytic techniques as those methods already dealt with.

The use of photographs in research of any type is both potent and potentially
problematic. While it is a technique often used in community and participatory
action research projects where it can provide a means to both document and
analyse lives and experiences, schools and other educational settings are often
cautious about allowing its use. For this reason, and because it is a major discussion
in its own right, I will do no more here than #ag up its potential, but caution about
its ‘unapproved’ use in schools and settings.

The use of children's drawings, again potent, but less problematic, will feature in
relation to data analysis in the next section.

Analysis and Presentation of Data

If there is one underlying theme throughout this section, it is that of familiarity with
the data. It can probably be best expressed as a set of questions. How familiar are
you with your data? Are you steeped in it? Immersed in it? Quite simply, the better
you know the data, the better the analysis.

In the !rst part of this section I focus more on the processes of dealing with free-
text and unstructured data than on dealing with more structured data, often
produced as a result of closed-response questions on questionnaires.
:
 Because the design of closed responses is in itself an analytic device, it is normal
practice to use the response categories as a means for collating and presenting the
data collected. The use of tables or various diagrammatic or graphical presentations
can be used to make the data accessible to the reader, and also to aid initial
analysis. On a cautionary note, however, the facility on even the most basic software
packages to produce a range of output types is often a temptation to approach the
presentation question as one of technical Information and Communication
Technology (ICT) use, rather than one of meaning-making and communication.
Knowing your own data so well can make it hard to assess whether or not your
chosen presentation is best for the ‘cold’ reader. Getting some honest feedback
from a critical friend is always of bene!t here. It is crucial that underpinning every
analysis and presentation decision is the question, ‘What does it mean and how can
I communicate it e"ectively?’

Analysing Unstructured or Free Text Data

Given the unstructured and discursive nature of a diary (or other free-
response/unstructured data such as that produced by naturalistic observation or
unstructured interview), it is vital to consider carefully just how its contents can be
analysed. Options for analysing such data include discourse analysis, conversation
analysis (both of which deal in di"erent ways with language and its construction)
and thematic analysis (which looks for patterns of meaning). Of these, thematic
analysis is both more accessible to practitioner-researchers in that it is not
dependent on specialised theory, and is more relevant in that the purpose of the
research is usually to uncover meaning. I brie#y now address the way in which such
thematic analysis can be undertaken and signpost the appropriateness of this
process in the analysis of interview and other free-response data (such as from
free-response questions on a questionnaire).
:
 Of primary importance in the analysis is immersion in the data. Being familiar with
the contents of the diary (or interview or open-ended question response in
questionnaires) is of crucial importance in helping identify units of meaning within
it. On a practical level, it is useful if the data can be collated in a way that gives space
for annotation, identi!cation of areas of interest, colour-coding or cross-
referencing. Having the notes in electronic format helps signi!cantly with this, and
while some people opt to use software programs, in my experience, they are more
helpful for larger-scale projects and the time and e"ort spent in learning them and
setting them up for the project may be better spent getting started on the task
manually. It is helpful to undertake this annotation process at least twice, using the
combing of data in this way to produce the key themes that will inform the analysis.

Having completed this annotation, units of commonality or units of meaning should

start to become evident. From these, broader categories can be formed and, hence,
themes for analysis. These stages are iterative in nature, and subsequent attempts
may yield di"erent outputs to previous attempts. It is not unusual for researchers to
underestimate the amount of time needed for such an exercise, bearing in mind
that it may take a number of iterations before some degree of consistency is
reached. The use of a critical friend or research partner can be helpful here and aid
clarity of thought, providing a useful cross-check on possible interpretations.
Likewise, cross-checking emerging meanings with other participants can aid both
the reliability and the validity of the interpretation. Finally, having decided on the
themes, pieces of evidence can then be assigned to each theme so that the end-
product is in the form of a thematic ‘picture’ of practice.

This inductive process results in what is known as a theory-generative (as opposed

to a theory-testing) analysis. Action research is often described as a theorising
methodology in that it does not seek to test practice against externally generated
theories, which often results in the exposition of what Carr (1980) described as ‘a
:
 gap between theory and practice’. These gaps, he suggests, are endemic to the ‘view
that educational theory can be produced from within theoretical and practical
contexts di"erent from the theoretical and practical context within which it is
supposed to apply’ (Carr, 1980: 65). The use of a theorising methodology should
produce a theory of practice which is congruent with that practice.

Analysing Non-Text Data

Data produced from non-text sources such as images and drawings may also need
to be analysed. In many ways, the process is similar to that for text-based data. I will
illustrate this with an example, again from my own DPhil study.

Example 3

Having become aware of the masculine epistemology of mathematics and physics I

wanted to further explore just what image pupils had of scientists. Given that most
of their experience of having been taught science was with women teachers (they
were year 9 pupils in secondary school), I opted to use the simple Draw A Scientist
Test (DAST) (see Mason et al. 1991), in order to see just what their perceptions of
scientists were. I analysed the drawings produced according to the following
categories drawn up by Chambers (1983):

male
white
laboratory coat
eyeglasses
facial hair
symbols of research
symbols of knowledge
products of technology
captions.
:
 I interpreted facial hair as any kind of hairstyle which could be thought of as
abnormal (generally wild, unruly and spilling over the face). Symbols of research
were interpreted as items of standard laboratory equipment, for example test
tubes, Bunsen burners, bottles, batteries, etc. Symbols of knowledge were
interpreted as phrases, comments, or equations designating knowledge or
discovery, while products of technology are interpreted as computers or electronic
sensing, control or logging devices.

Table 4.1 summarises the responses of these year 9 students to the DAST.

Table 4.1 Year 9 Draw a Scientist Test

Characteristic Frequency % occurrence

Male 19 90.48

White 21 100.00

Laboratory coat 9 42.86

Spectacles 13 61.9

Facial hair 15 71.43

Symbols of research 9 42.86

Symbols of knowledge 2 9.52

Products of technology 0 0.00

Captions 12 57.14

ð Download Table as xlsx

:
 ð Download Table as csv
ð Download Table as pdf
From the DAST exercise, it was evident that these students perceived scientists as
white, spectacle-wearing males with strange or eccentric hairstyles. The use of an
existing codifying system (Chambers, 1983) was helpful to me in this case, but it
would have been possible also to derive these categories from the data themselves,
in a manner similar to that outlined above. Presenting the data in tabular form was,
I felt, a simple way to help me and my readers make meaning. A particular strength
in using this data collection method, however, was that it gave the pupils freedom
to draw what they might have found di$cult to express (particularly to a science
teacher), allowed the collection of a large data set in a relatively time-e$cient way,
and produced something that I felt was more powerful than if I had extracted some
of these descriptors from interviews or focus groups with them. From my
perspective, the image each child produced was a very powerful articulation of the
fact that science was seen as somehow ‘outside’ their !eld of normality. Further, on
discussing this later with the children, I realised, with some surprise, that many of
the children viewed their science teachers more as members of the set ‘teachers’
than the set ‘scientists’. A science teacher was considered much more similar to a
music or geography teacher than to a scientist. The method therefore produced
data that provoked further questions, a key feature of a theorising, data-driven
methodology.

Ethics in Action Research

This is a complex matter and one which I hope to raise as a matter for ongoing
contextualised re#ection, rather than as something which is ‘dealt with’ in this
:
 chapter. It is perhaps more important to raise awareness of the nature of ethical
issues in research, to develop a questioning approach to ethics, than it is to produce
a set of ‘ethical guidelines’. Too often dealt with as a set of procedures for
‘permission’ and ‘anonymity’, the matter can be reduced to both a simplistic and,
more importantly, an unethical one. There are various sets of ethical guidelines
available for researchers and, depending on the context of your research, there
may be an in-house policy for ethical approval. Whichever guidelines you consult, it
is important to treat them as a set of prompts for question and re#ection, rather
than a tick-list, often explained as ‘I ensured that my research followed the ethical
guidelines as outlined by “Key-Text”’. When reading someone's report of their
research, I, like many of my colleagues, !nd this troubling. It tells me nothing
speci!c about the problems they faced in their research, nor the ways in which they
made decisions about those problems. Indeed, it seems to imply that, in having
read such guidelines, researchers get ‘right answers’ to matters of ethics, and thus
conduct ethical research. Thomas (2009: 147) sums up the highly complex nature of
ethical decision-making as follows, ‘ethical principles encompass some decisions
and dilemmas that pit not just right against wrong, but balance one right action
against another right action, taking into account the possibly con#icting interests of
the parties involved. What is right for me, may not be right for you’. The ethical
researcher must be aware of this complexity and able to make a decision that is
predicated on an exploration of these matters, both justi!able and justi!ed.

For many education practitioners the BERA's Ethical Guidelines for Educational
Research (2011) is a key guide. Framed as an ethic of respect for

the person
knowledge
democratic values
the quality of academic research
academic freedom
:
 it explores and unpicks the meaning and signi!cance of each of these within a range
of di"erent research contexts. If we re#ect on one of these brie#y in the context of a
piece of small-scale practitioner research, then we can see how a more
problematising approach to ethical frameworks can be used.

Reflection

Returning to the case of a teacher opting to interview a pupil as part of a classroom-

based action research study, and considering the power relationships that exist,
what steps do you think can be taken to clearly show that the teacher has operated
within a framework of respect for the person?

(How) Do you think she can also ensure that she herself can be respected during
the process?

Consider the ways in which the information from the interview will be used. In what
ways might that inform her decision about ‘respect for the person’? Can you see any
compromises or choices she might have to make? How might she justify such
compromises?

Summary

This chapter attempts to bring together some key issues around the business of
‘dealing with data’, taking us from a starting point of the nature of data, through to
questions of analysis, presentation, quality and ethics. Hopefully, having read this,
you will have begun to be aware that while there are many guidelines around the
decisions you make in a research project, the blind following of guidelines is not in
itself su$cient to ensure ‘good research’. Indeed, underpinning all action research is
:
 a question of ‘ought’, rather than ‘can’, putting the onus on the researcher to
consider not simply the operation of guidelines or procedures, but also their
signi!cance in moral and ethical terms.

Further Reading

Burton, D. and Bartlett, S. (2009) Key Issues for Education Researchers. London:
Sage.

Thomas, G. (2009) How to do your research project. London: Sage.

These two books come to mind immediately as having strong and well-focused
chapters on data collection methods. While not written speci!cally as action
research texts, like many general education research methods books they provide
very useful chapters on matters of data and their analysis.

Prosser, J. (ed.) (1995) Image-based Research: A Sourcebook for Qualitative

Researchers. London: Falmer Press.

If the bulk of your research data is based on images or photographs, then a text
such as this may be useful for you. While it does not have an action research focus,
nor even an education focus, it will be helpful in exploring options in relation to this
type of data.

Additional online resources can be found at:

http://www.sagepub.co.uk.proxy.tamuc.edu/beraseries.sp

Chapter 3
á Getting Started on an Action Research
Project
:
 Chapter 5
â
Dealing with the Literature

Read next

More like this

Sage Recommends
We found other relevant content for you on other Sage platforms.

About
About Sage Publishing

About Sage Research Methods

Accessibility

Author Guidelines AI/LLM

CCPA
:
 Permissions

CCPA – Do Not Sell My Personal Information

Privacy policy

Terms of use

What's new

Information for
Faculty

Frequently Asked Questions

Librarians

Trial Login

Stay in Touch
Contact us

Copyright © 2025 by Sage Publications

: