Unit 6

OBSERVATIONAL DESIGN
Research design in psychology

Julio Imbernon de Alvaro

Psychology degree
INDEX
CONTENTS
1. INTRODUCTION
2. CHARACTERISTICS:
2.1. Level of Structuring of the Situation
2.2. Degrees of Participation
2.3. Phases of an Observational Study
3. OBSERVATION INSTRUMENTS
4. SAMPLING AND RECORDING
5. OBSERVATION MEASURES
6. SOURCES OF ERROR AND FORMS OF CONTROL IN OBSERVATION
7. DESIGNS: TYPOLOGY OF OBSERVATIONAL STUDIES
2
1. Introduction

3
1. Introduction

4
How would you categorize this?

5
6
 2. Characteristicas
Degrees of structuring
Naturalistic: Completely natural situation, in its
usual environment, and without any modification by
the researcher.

Example: To what degree we

adress
Recording the first forms of mother-baby modification?
communication after eating at home or The teacher
observes the development of their class.

Recording teh first forms of communication but the

researcher is speaking with the mother and guiding
her

7
 2. Characteristicas
Degrees of structuring No
Naturalistic: Completely natural situation, in its interaction
usual environment, and without any modification by
the researcher.

Example:
No
Recording the first forms of mother-baby
communication after eating at home or The teacher
intervention
observes the development of their class.

Non-aware

8
 2. Characteristicas
Degrees of structuring

Semi-structured: Mainly in
comparative studies where it is
convenient to ensure some elements of
consistency between situations. Example:
Comparing mother-child interaction with
Down syndrome and without Down
Interaction?
syndrome with instructions for mothers
on games or toys to use.
Comparing mother-child interaction with
Down syndrome and without Down syndrome
without instructions for mothers on games or
toys to use. 9
2. Characteristicas
Degrees of structuring

Semi-structured: Mainly in
comparative studies where it is
convenient to ensure some elements of
consistency between situations.
Example: Comparing mother-child Interaction
interaction with Down syndrome and
without Down syndrome with
instructions for mothers on games or
toys to use.

10
 2. Characteristicas
Degrees of structuring

Structured: More rigid, where the researcher can

"intervene" systematically or introduce substantial
modifications in the situation.

Can you give me an example where there is not

-Intervention

-Sistematical modifications

-Whats the value of the measuring then?

11
 2. Characteristicas
Degrees of structuring

Structured: More rigid, where the researcher can

"intervene" systematically or introduce substantial
modifications in the situation. Example: Evaluation of
attachment in children (the parent, following the
researcher's guidelines, at a certain point leaves the
room where they are with their young child, and then re-
enters: the child's reactions inform about the degree of
attachment). Example: Creating a reading competence
program and applying it in one classroom but not in
another.

12
2. Characteristics
CHARACTERISTICS

13
2. Characteristics
External or non-participant observation:
• The observer is separate from the situation
under study, with very little or no
interaction with the participants being
studied.
• These observation conditions facilitate the
objectivity of the observer in their recording
decisions and avoid the risks of interference
in the phenomenon under study.
• This is the most commonly used form of
observation in studies conducted with a
quantitative strategy.
14
2. Characteristics

Participant observation:
• The observer integrates into the
dynamics of the phenomenon under
study, participating in the activities of
the group or individual to establish
direct contact with their reality.
• This is the form of observation typical of
qualitative methodology, whose
objective is to deepen the
understanding of the phenomenon
under study.

15
2. What kind of observation would be this?

16
 2. Characteristics
Self-observation:
• Self-observation in research refers to a method where individuals observe and
record their own behaviors, thoughts, feelings, or experiences. It involves
individuals monitoring themselves in various situations or contexts and
documenting their observations.
• This approach is often used in psychological research, self-help techniques, and
therapeutic interventions.
• Self-observation can provide valuable insights into one's own behavior patterns,
cognitive processes, emotional responses, and overall mental states.
• The potential interference of being both an active subject (observer) and the
object of observation can be a source of bias in the data, so its use is more
common as a source of information for diagnosis and monitoring of an
intervention (where clinical significance outweighs statistical significance and
replicability) than for basic research.
17
 2. Characteristics
EXAMPLE: Let's say a researcher is interested in understanding the impact of stress on
academic performance among college students. The researcher decides to conduct a self-
observation study where students track their own stress levels and academic performance over
the course of a semester.

Each participant in the study would be given a journal or a mobile app where they can record
their daily stress levels using a self-assessment scale (e.g., from 1 to 10) and any relevant
stressors they encountered throughout the day (e.g., exams, deadlines, personal issues).
Additionally, participants would track their academic performance by recording their grades or
progress on assignments, exams, and other coursework. They may also note any factors that they
believe influenced their academic performance, such as study habits, sleep quality, or
extracurricular activities.

At the end of the semester, the researcher would collect and analyze the self-observation data to
examine patterns and correlations between stress levels and academic performance among the
participants. This self-observation approach allows individuals to directly report on their own
experiences and provides rich, firsthand data for the research study.
18
 2. Characteristics
How would I be able to give a solid foundation to my method?

Combine with audio recordings or external Is this valid?

feedback
• It’s possible to employ a “theoretical” framework with expert
Assessment of what behaviors are supposed to be reported or
not
• The option of employing recording devices and employ
interevaluator agreements measures, is a possibility Is this replicable?

Employement of line bases

• Using deliberatedly a “pilot” phase where the subject is given a

period to get used to taking measures is usually recommended
• Extreme values or aquiescent may be a bias for self-report, with Could I apply
a wider measure range, the “sensibility” of the subject may control techniques?
decrease

19
2. Characteristics

20
2. Characteristics
Observation instrument
• For every procedure we will probably need a different instrument

Sampling procedure
• Which is our data sample, the subject, the behaviour or the units?
Are we designing a
reliable method? Registre procedure
• How are we going to gather the data?

Study design
• Which options do we have for an observational design?

Measurement parametres
• Nature of the observations

Assesment of data quality

REPORT
21
2. Characteristics
2.3. Stages of an observational study: Specific decisions
of this type of observational methodology

22
2. Characteristics
2.3. Stages of an observational study: Specific decisions of this type
of observational methodology

23
2. Characteristics
2.3. Stages of an observational study: Specific decisions of this type
of observational methodology

24
2. Characteristics
2.3. Stages of an observational study: Specific decisions
of this type of observational methodology

25
3. Observation Instruments
The most commonly used observation instruments are three

I want to measure ascethic

behaviour
• What is asceticism?

I know that the AED

exists…
• Where do we go from here?
26
3. Observation Instruments
The most commonly used observation instruments are three: 1st field
work, 2nd category system, and 3rd instruments that combine both.

1st: Field work:

• It is especially used when there is no theoretical framework for the
selection of behaviors and one wants to approach the observation task
with an open and flexible system, which also allows attending
simultaneously to various aspects of an event.
• It is based on the initial proposal of relevant criteria to consider and a list
of levels, or an open catalog of behaviors, related to each of these criteria.
• But if there is a theoretical framework that guides the selection of
relevant behaviors, a category system is constructed, in which there is an
explicit and complete list of behaviors that must be recorded.
27
EXAMPLE: AED in aescetic secluded zen monk
Field work
I dont have a DSM-V at hand neither have an specific theoretical background
so I will elaborate a detailed list on the “possible” behaviors of interest regarding the
objective of my study, sample characteristics and desired behavior of analysis. What
criteria is relevant to report? Stimming like behavior, social interactions, reported
disociative subjective states and strict routine (patterns of diet, sleep, praying…) FIELD
WORK

If I have the chanche to use a DSM-V

-is this a category system?
-Why?

28
3. Observation Instruments
The most commonly used observation instruments are three: 1st field
work, 2nd category system, and 3rd instruments that combine both.
2nd: Category System

• It is an organized way of describing behaviors and/or events to be observed and

recorded, which includes a system of identification and classification of those behaviors
or events through the definition of categories. This classification of behaviors assigned to
different categories is the way we operationalize constructs in an observational study.
• The category system allows us to face the registration task having selected and defined
the relevant behaviors for our study and on which the observer should focus their
attention.
• It is not easy for a category system created for one study to be directly applicable to
another. It should be tested in pilot situations, reviewed, and adapted, if necessary, to
the needs of the new study. In fact, it is common for researchers to create a specific
category system for the research objectives.
29
3. Observation Instruments
The most commonly used observation instruments are three: 1st field
format, 2nd category system, and 3rd instruments that combine both.
2nd: Category System
In the process of creating the category system, the researcher must make two basic and
interconnected decisions:

a) Determine the dimensions or levels of behavior. The behavior of interest can be

considered from a single dimension or from multiple dimensions. For example, if we want to
analyze the dynamics of interactions that occur in decision-making meetings of department
heads in a company, we can consider a single dimension (e.g., the type of verbal interventions
made by its members) or more than one dimension (e.g., verbal interventions and also their
non-verbal behavior, such as gestures or postures).

b) Determine the unit of behavior. To analyze behavior, which is a continuous flow, we need
to segment it to delineate the elements that constitute it and that we want to record.
30
EXAMPLE: AED in aescetic secluded zen monk
Category system: Dimension and Units

Dimension

Units

31
3. Observation Instruments
The most commonly used observation instruments are three: 1st field
work, 2nd category system, and 3rd instruments that combine both.
Research Question: How do students interact during group discussions in a
classroom setting?
Category System: 2. Non-verbal Interactions:
1. Verbal Interactions: - Eye Contact
- Asking Questions - Body Language (e.g., nodding, shaking head)
- Answering Questions - Gestures (e.g., pointing, thumbs up)
- Providing Feedback
- Agreeing
- Disagreeing Setting: High school classroom during group discussion sessions.
- Clarifying Participants: Students (group members) and teacher (facilitator).
- Summarizing Procedure:
1. Observer(s) sit inconspicuously in the back of the classroom.
Observational Design: 2. Using the category system, observers record instances of verbal and non-
verbal interactions exhibited by students during the group discussion.
3. Observations are conducted for multiple group discussion sessions over a
specified period 32
3. Observation Instruments
The most commonly used observation instruments are three: 1st field
format, 2nd category system, and 3rd instruments that combine both.
Research Question: How do students interact during group discussions
in a classroom setting?

Analysis:
- The recorded data is then analyzed to identify patterns of interaction among students,
such as which students are more active in asking questions or providing feedback, how
often students agree or disagree with each other, and the level of engagement during the
discussions.

Conclusion:
- The findings from the observational study provide insights into the dynamics of student
interactions during group discussions, which can inform teaching strategies aimed at
promoting collaborative learning and effective communication skills among students.
33
3. Observation Instruments
The most commonly used observation instruments are three: 1st field
format, 2nd category system, and 3rd instruments that combine both.
Instruments that combines field format and category system
Mixed combined strategies may apply register entries or register of all behaviors per unit of
time (every 10 seconds) with a coding grid for specifical target behavior. Observational tools
like the HOISAN may be critically usefull and are actually employed to predict behavior on
complex analysis, like for example sports analysis based on movement (stochastic) or
ethology

What else would

we need?

34
 EXAMPLE: AED in aescetic secluded zen monk
Observation period, availability subperiod and session
If by any chance I start measuring the behavior of these monks on the 15th of
may after being transported to the secluded montains where they are and I finish the
research the 18th of may, when I leave, the observation period will be from the 15th to
the 18th. It is the time frame where I have been able to gather information about them.

There were times where we were able to gather information, this will be callled availability
subperiod, and times where we weren’t able (while sleeping, while having a shower, while
preparing the equipment) where we weren’t able to gather information.

BUT the, only moment that we were recording actively with our equipment and categorical
system is the SESSION.

35
 4. Sampling and recording
Related...
• Let's start by clarifying the concept of a
session. We begin with the notion of
an observation session as a period of
time during which the observer
systematically records the behaviors
under study.

• Observation period: It is the temporal

space in which (due to the conditions
of the context at those times, what
may or may not happen, etc.) it would
make sense to record the individual's
behavior based on the study's
objectives. 36
4. Sampling and recording
Related...
• In most studies, the researcher must accept that only certain subperiods (parts of the
observation period) are viable based on the capabilities of the observed participant
and/or the observer.

• Once the viable subperiods are identified, the researcher must determine the specific
time frame for data recording, that is, the session, delineating its duration and the criteria
for its start and end.

37
 EXAMPLE: AED in aescetic secluded zen monk
Sampling and recording
During my 3 days visit, I could
observe them from 0800 to 2300, but the only
moments I am interested in will be a sesión
from 09 to 1400. Now I want to develop a AED
Category behavior
replicable method so other people observe
their very own ascetic monks through the same
Social
procedure. I need to define the procedure to Dimension interactions
gather information, regarding
• The specific units of behavior I am going to Unit Mutism
note down (1 stereotyped movement, 1
prayer, 1…)
“Monk nods
• The way that I am going to look for these Record at 09:31:27”

units in the session 38

 EXAMPLE: AED in aescetic secluded zen monk
Sampling and recording
SAMPLING

DAY 1: As we want to keep track of

all that could happen we will record specific
units of behavior when they happen, so the
sampling will be based on the frequency or
outcome. Everytime a monk does something
Every relevant action that the sampling
that is AED related, we will keep track entries RECORD will be called a record,
as we store this meaningfull unit of
DAY 3: This method requires me to behavior to study later
look at the monks for 10h per day and I have
a migraine, so we will take 5 minutes of every
10 to record all they do in order. Afterwards
39
we will select the relevant behavior
4. Sampling and recording
Record When do we start?

• The basic unit of data or information that is being saved,

adressed or “recorded”
• This individual units of behavior are documented on specefic
categories

Sampling
• The portion that is selected to observe may be a specific
outcome (goal), a time event (10 seconds) or a episode
(interaction) When do we measure?
• As I am gathering a collective number of things with specific
characteristics I will call it a sample

40
 4. Sampling and recording
When do we start?
Sampling Recording Information

• Specific • How are we • Frequency

procedure that going to gather (Amount)
will adress information • Length
when will we • Which • Order
observe and information is
who relevant

When do we measure?

41
Could you make an example of
Intersessional vs Intrasessional

42
4. Sampling and recording
I want to know if the jumping rope, the truth or dare, the…

Who has this information?

• I have the theoretical criteria that is
going to make choose one specific
operative criteria, but where am I
going to get my information?
From the sample
• Why would we choose the sample
due to their specific characteristics?

43
4. Sampling and recording
4.1. Sampling Procedures
We will proceed to select the sample of participants.
• In this type of study, the most common sampling strategy is expert judgment
sampling: participants are selected because they possess certain characteristics
that are relevant to the study's objectives.

• Units of study or sampling units: these units can be individuals, dyads or

groups, etc. However, sampling in observational methodology goes beyond
selecting participants; it has its own peculiarities.

expert Between- Within-

judgment session Session
sampling sampling Sampling
44
4. Sampling and recording
4.1. Sampling Procedures
4.1.1. Between-Session Sampling: Criteria for Starting and Ending Observation
Sessions
Between-session sampling allows us to determine
when observation should take place.
• Fixed-selection sampling: this is the
simplest system for determining the timing and
location of observation sessions and involves the
application of a fixed criterion, which can be
temporal, behavioral, or based on events or
situations.
 Observation sessions always occur at the same time
of day or coincide with specific activities (e.g.,
observation sessions are conducted during the
central 10 minutes of morning school recess). 45
4. Sampling and recording
4.1. Sampling Procedures
4.1.1. Between-Session Sampling: Criteria for Starting and Ending Observation
Sessions
EXAMPLE
Let's consider a research project focusing on
studying the behavior of monks during
lunchtime. The researcher decides to observe
them during their lunch break every day The
fixed-selection sampling method here involves
selecting the same time period (e.g., 12:00 PM
to 1:00 PM) each day as the session for
observation. This approach ensures
consistency and allows for comparisons across
different days.
46
 4. Sampling and recording
4.1. Sampling Procedures
4.1.1. Between-Session Sampling: LIMITATIONS
1. Increased risk of missing important behaviors or events: Since observations are spaced out
over time, there is a higher chance of missing critical behaviors or events that occur between
sessions.
2. Difficulty in capturing variability: Behaviors may vary over time, and sampling at discrete
intervals may not accurately capture this variability, leading to a loss of information.
3. Potential bias in sampling: Depending on the chosen intervals and times for sampling, certain
behaviors or events may be overrepresented or underrepresented, introducing bias into the data.
4. Increased burden on participants and observers: Participants may find it burdensome to have
observations spread out over multiple sessions, and observers may face challenges in maintaining
consistency and accuracy across sessions.
5. Limited understanding of short-term dynamics: It may provide insights into longer-term trends
but may not capture the nuances of short-term dynamics or interactions. 47
4. Sampling and recording
4.1. Sampling Procedures

• Random Selection Sampling: When there are no determinative theoretical

criteria for moments and situations, the start and end of observation sessions should be
scheduled to ensure the representativeness of the recorded behavior.
 In these cases, random selection contributes to achieving this representativeness and can be
applied in several ways.
 It refers to the technique of randomly selecting moments or situations for observation
without bias.
 In this method, the moments or situations for observation are chosen randomly within the
defined observation period.
 This ensures that each possible moment or situation has an equal chance of being selected
for observation, helping to avoid biases in data collection and improve the
representativeness of the observed sample.

48
4. Sampling and recording
4.1. Sampling Procedures
The limitations of random selection sampling in an observational design include:

1. Limited control over when or where observations occur: researchers have less control over
the specific timing or context of the observations. This can make it challenging to ensure that
observations capture relevant behaviors or events.
2. Potential for missed opportunities: Random selection sampling may result in missed
opportunities to observe important behaviors or events if they occur during unselected
moments or situations. This can lead to incomplete or biased data collection.
3. Time and resource constraints: It may require significant time and resources to implement,
particularly if researchers need to observe over a large number of moments or situations to
achieve adequate representation. This can be impractical in some research settings.
4. Difficulty in replicating findings: Since random selection sampling relies on chance, replicating
the same observations in future studies may be challenging. This can limit the ability to confirm
the reliability and validity of study findings.
49
 4. Sampling and recording
4.1. Sampling Procedures
Let’s say that before understanding the possible
relationship between AED and monk behavior we want to run
a pilot considering which are all the possible behaviors that
the monks may do, so we can actually develop a category
system.

Instead of recording for 24h straight we will select at random,

a representative sample of behavior for each hour, lets say 10
minutes. Every hour we will roll a dice and the first time it
lands on the 3 we will start recording.

50
 4. Sampling and recording
4.1. Sampling Procedures
If all the information I am going to gather is going
to be taken in only one session of observation (one single
event is going to be named within session sampling), the
most ideal would be to keep the camera rolling and record all
possible behaviors.

As we may want to address the behavior of every participant,

how could I look at two people at the same time and record
their information?

51
4. Sampling and recording
4.1. Sampling Procedures
4.1.2. Within-Session Sampling
It involves selecting specific moments or intervals within a single observation
session to collect data. This approach allows researchers to focus on particular
events, behaviors, or conditions of interest within a continuous observation period.

• For example, if researchers are studying classroom behavior, they might use within-session
sampling to observe student engagement during specific activities like group discussions or
individual work periods within a single class session.

• By targeting specific moments or intervals, researchers can gather detailed data on

behaviors or events of interest while still maintaining the context provided by observing
within a single session. This method can help researchers efficiently collect data on specific
aspects of behavior without the need for multiple observation sessions.

52
4. Sampling and recording
4.1. Sampling Procedures
4.1.2. Within-Session Sampling

Focal Sampling:
 Focal sampling is a method used in observational research to study the
behavior of a specific individual or a predetermined focal point within a
group over a specified period of time.
 The observer focuses attention exclusively on the chosen individual or target
area, recording all instances of behavior or events related to it.
 Other group members will be observed (becoming focal subjects) during
another part of the session or in subsequent sessions.
 This method allows for in-depth analysis of the behavior patterns,
interactions, and activities of the focal subject or area, providing insights into
their role within the group or environment being studied.

53
4.2. Recording procedures

The act of recording observed behavior can take

place at the same time (or almost) as the behavior
occurs when real-time observation is used. This is
made possible by the prior training of observers,
the practical design of coding sheets, or the use of
recording instruments that facilitate the task.
However, nowadays, given the wide availability of
recording technology, most research relies on
recording observation sessions, so that the actual
task of recording or coding behaviors is carried out
later, through video review.

54
4.2. Recording procedures

Time-sampling recording (recording is done only if the behavior has occurred or not at
predetermined time points): The session is divided into short, equal-duration time
intervals that mark the recording moments. Example: The observer pays full attention
to mentally register any behaviors from the category system that occurred during a 10-
second interval. After that interval, the observer stops observing momentarily to write
down everything on the coding sheet, then resumes observation for another 10
seconds, and so on.

Event-based recording (also known as behavior-transition recording): The observer

must record every occurrence of the behavioral categories specified in the category
system, along with the order in which they occur. This method allows the recording of
frequency, duration, and sequence of occurrence. 55
5. Observational measures
Depending on how we take the measures we will have
The occurrence of a behavior simply tells us whether that behavior has appeared or not
during the recorded time interval or the observation session.
The frequency of each behavior is obtained by counting the number of times that
particular behavior occurs during an observation session.
The duration tells us the units of time (usually in seconds or fractions) taken up by that
behavior during each of its occurrences.
The total duration of the behavior is obtained by simply adding up the durations of
each time the behavior appears, and it tells us the total amount of observation time
that the behavior occupied.

56
6. Sources of error
The observer
The observer plays a fundamental role in an
observational study. Possible errors in the
application of the observation instrument (due to
How could you solve
inadequate interpretation of the definitions) or
this?
errors in recording (due to lack of attention to
certain elements) directly affect the study's data.
Proper training of observers in the use of the
study’s specific category system is the main way
to control these sources of error.

57
 Cohen’s Kappa inter-observer coefficient

To adress the
reliability between coders
or observers the amount
of relative agreement
and the possibility that it
was done by chance is
considered. This is a
viable countermeasure to
tackle the possible bias
in our observers.
58
 6. Sources of error
The participant
The participant: If participants know they are being observed,
there is a risk of reactivity. The awareness of being observed may
lead them to modify their behavior, either consciously and
deliberately or involuntarily, resulting in behavior that differs How could you solve
this?
from what would have occurred spontaneously. This can be a
source of error that negatively affects the validity of the study’s
conclusions. To avoid this bias, we try to minimize reactivity risks
as much as possible employing a baseline or designing a phase
that “distracts” from the critical event. Clearly, the ideal situation
in this regard is the observation of naïve individuals (those
unaware they are being observed), typically achieved through the
use of hidden recording devices.
59
 7. Types of studies

Observational Time dimension

units • Cross sectional
• Ideographic • Longitudinal
• Nomotetic

Behavioral Behavioral
dimensions indicators
• Unidimensional • Static
• Multidimensional • Dynamic

60
7. Types of studies

According to the units being observed, we distinguish

between idiographic studies, based on a single participant
or a natural unit such as a dyad or family, and nomothetic
studies, which are based on the study of a group or groups
of participants.

According to the temporality of the measurement

moments, there is a distinction between cross-sectional
studies, based on a single moment of measurement, and
longitudinal studies, in which multiple measurement points
are taken in order to analyze the evolution or change in
behavior. 61
7. Types of studies

According to the number of levels or dimensions of behavior analyzed, we identify

unidimensional studies, in which only one dimension of behavioral analysis is recorded
and analyzed, and multidimensional studies, when several dimensions of behavioral
analysis are taken into account.

According to the behavioral indicators to be analyzed, studies are identified as

extensive studies when they are based solely on static behavioral indicators (such as
frequency and/or duration and their derived measures), and intensive studies if, in
addition to these, they include dynamic behavioral indicators such as transition
frequencies and other sequential data of behavior.

62
Differences between

Sample

Registre

Units

Dimension
63
BIBLIOGRAPHIC
REFERENCES
Labarca, A. (s.f.). “La técnica de la observación”. [Archivo PDF].
https://docplayer.es/19214037-La-tecnica-de-observacion-en-la-sala-de-
clases.html

64
 Julio Imbernon de Alvaro
jimbernon@um.es

UCAM Universidad Católica de Murcia

© UCAM