Development of a tool for assessing preterm infants

Sonia R. B D'Souza, Vijay Kumar, Leslie Edward Lewis

Abstract: The aim of this study was to validate the content and check the reliability of a tool to
assess the behavior of the preterm infants aged 28 - 33 weeks of gestational Age following a
Developmental Supportive Care (DSC) Intervention. A multidisciplinary panel of seven national and
international experts was selected to assess, review, and suggest modifications for the tool. The tool
had two subsystems. One was the autonomic/visceral system consisting of four domains namely
colour, respiration, visceral signs and neurophysiological responses. The second was the state and
attention-interaction system, consisting of three domains namely state regulation, orientation to
auditory stimulus and alertness. A validity questionnaire was keyed to the individual items clustered
under the two subsystems and the initial draft of the tool was mailed to the content experts along with
the objectives, aim and the blueprint of the tool. Experts were from the field of Pediatrics, Neonatology,
Developmental Psychology, Perinatal nursing and Neonatal nursing. Data were analyzed using
descriptive statistics. The Content Validity Index (CVI) for each item as well as the overall CVI were
calculated. All the seven experts completed the validity questionnaires. The experts generally agreed
that the items of the initial draft were clearly worded, appropriate and relevant. The tool obtained a high
overall CVI score of 0.95, yielding a valid tool for assessing behavior of the preterm infants. To check the
reliability of the tool, two raters observed 40 preterm infants for duration of 30 minutes following the
provision of 30 minutes of the DSC intervention. The inter-rater reliability was checked using Intra class
correlation coefficients (ICC). Calculation was done using SPSS (Ver.15.0), South East Asia, Bangalore
for raw scores obtained by two independent observers for the two subsystems namely the autonomic
and state and attention-interaction systems and their domains in the tool. The 'R' value obtained for the
autonomic subsystem was 0.96 whereas the 'R' obtained for the state and attention-interaction
subsystem was 0.92, which indicates high reliability for both the subsystems of the tool. The findings
reported suggest that the tool has potential for use as both a clinical as well as a research instrument to
assess behaviors of preterm infants who are provided DSC interventions.

Keywords
Behavior of preterm infant, content validity
index, reliability
Correspondence at
Mrs. Sonia R.B D'Souza

Associate Professor,
Department of OBG Nursing,
Manipal College of Nursing Manipal,
Manipal University, Manipal, Karnataka

Introduction
There has been improved survival of
preterm infants of very young gestational age
due to the advancing technology and
improved evidence informed perinatal care.
In the bargain of survival, preterm infants are
exposed to numerous hazardous
environmental stimuli especially excessive
sound in the NICU.

Nursing and Midwifery Research Journal, Vol-10, No.3, July 2014

91

To minimize the effects of the

environmental stimuli, Developmental
Supportive Care (DSC) interventions (E.g.
music therapy, light touch, massage,
vestibular stimulation etc.) are encouraged
to be provided for the preterm infants
admitted to the NICU. DSC interventions are
employed in the NICU for providing the
necessary stimulation and for alleviating
stress experienced by these infants.
However, it is also necessary that the
behavior of the preterm infants be observed
following the DSC interventions.
As, Als1 states that if the interventions
meant to provide stimulation to the preterm
infants are appropriate for the preterm infant
in terms of complexity, intensity and timing,
the preterm infant has strategies available to
him to actively move towards that
stimulation, take it in, and make use of it for
his own development. If stimulation on the
other hand is currently inappropriate for the
infant, the infant is thought to avoid and
defend against that stimulation. If the level of
input and information is currently
appropriate for the infant, he maintains a
balanced and well-regulated behavioral
1
modulation.
In the recent times, worldwide there is a
paradigm shift in the focus of care and the
interventions employed for the preterm
infants from mere survival to optimal
development. DSC interventions, not only
support the NICU medical care but also
contribute to the optimal development of
preterm infants.2 However, it is essential to
know whether this care is appropriate to the

preterm infants because it is necessary to

identify stressors experienced by preterm
infants during these interventions. The
stressors experienced are communicated by
preterm infants through their behavior. Each
behavior has a meaning assigned to it,
though the behaviors exhibited by preterm
infants may not be as clear and distinct as
term infants. A valid and a reliable tool was
thus necessary to assess the behavior of the
preterm infants as well as to determine the
behavior modulation any DSC interventions
could bring about in the preterm infants.
Objective
To develop a valid and a reliable tool for
assessing behavior of the preterm infants
28 - 33 weeks of gestational Age after
provision of DSC interventions in the NICU.
Materials and Methods
For the present study, the tool
development proceeded through a
development stage. To fulfill this, an
extensive review of the literature was
undertaken to identify articles pertaining to
assessment of behavior of the preterm
infants. The domains of the tool were
developed based on the Als Synactive
1
model of neonatal behavioral organization.
The tool consists of two subsystems i.e, the
autonomic/visceral subsystems and state
organization and attention-interaction
subsystem. The tool and the scoring of the
tool is provided in table 1 & 2.

Nursing and Midwifery Research Journal, Vol-10, No.3, July 2014

92

Table 1: Assessment of preterm infant behavior :autonomic/ visceral system

Score Colour

Respiration

Visceral

Neurophysiological
responses

Pale, Cyanotic

Gasping, frequent
apnea, unstable
respiratory rate

Vomits feed,
feed intolerance

Flaccid on
stimulation

Pink but changes Occasional apnea,

rapidly with slow unstable
recovery, not
respiratory rate
returning to
good color

Bowel
Movement grunt
and strain

Abnormal jerks,
Twitch

Pink

Gastrooesophageal Stable
Reflux (GER) Nil

Regular, stable
respiratory rate

Table 2: Assessment of preterm infant Behavior:

State system and attention-interaction system
Score State regulation

Orientation to
auditory stimulus

Alertness

Intense crying which

is rhythmic with
irregular breathing

Does not focus on or

follow stimulus

Rarely or never
responsive to direct
stimulation

Active awake state with

infant fussing but not
crying but stressed and
hyper alert

brightness with stimulus,

may focus and follow
briefly with jerky
eye movements

When alert, responsivity

brief and variable, may be
delayed

Awake alert

Focuses on stimulus and

follows with smooth
continuous head
movement

Always alert in best

periods, stimulation
always elicits alerting
and orientating.

Nursing and Midwifery Research Journal, Vol-10, No.3, July 2014

93

The tool has a point system scoring i.e.,

from zero to two for the autonomic/ visceral
items expressed in domains viz. colour,
respiration, visceral signals and
neurophysiological responses. The state and
attention/interaction system has items to
assess preterm neonates in the early
postnatal period with domains like state
regulation, alertness and orientation to
auditory stimulus. In the autonomic/visceral
subsystem, the maximum total score for
each domain in the tool is 8. If the infant
obtains a score of >5-8 for all the four
domains in the subsystem, the preterm
infant is considered to show normal
behavioral response'.But, if the infant scores
2-4 then it is considered to be showing
^suspected abnormal behavioral response'
and if scores obtained <1 then the infant is
showing ^definite abnormal behavioral
response'. Similarly, in the state system and
attention-interaction subsystem the
maximum total score for each domain of the
tool i.e. state regulation, orientation to
auditory stimulus and alertness is 6. If the
preterm infant obtains a score of >4-6 for all
the three domains in the subsystem, the
preterm infant is considered to show normal
behavioral response', but if the infant scores
2-3 then it is considered to be showing
^suspected abnormal behavioral response'
and if scores obtained <1 then the infant is
showing definite abnormal behavioral
response' respectively.
Validation of tool:
Following this, the research team
proceeded with the judgment-quantification
stage where a panel of experts was requested
to evaluate the tool for item content, clarity of
items and comprehensiveness of items and

rate items of relevance according to the

content domain. For this study, an
interdisciplinary panel of seven national and
international experts was chosen by the
research team, which included researchers
and expert clinicians in Pediatrics,
Neonatology, Developmental Psychologist,
Perinatal nursing and Neonatal nursing.
Cover letters explaining the purpose of
the tool along with background, aim of the
study and instructions were provided to the
panel of experts. Informed consent was
obtained from the experts and the researcher
explained the process to the panel of experts
to ensure understanding of the process. The
panel was asked to review the items in the
tool. All the respondents completed all
questions on the questionnaire. Some
respondents provided narrative feedback to
some questions posed. The content experts
(2 women, 5 men) had an average of above
10 years of experience in their respective
disciplines (Pediatrics, Neonatogy,
development psychology, perinatal nursing
and neonatal nursing).
Subsequent to this, the completed
questionnaires by experts' were inferred.
There were no major modifications
suggested by the experts for the initial draft
of the tool. So, the tool had only one round of
content validity. Further, the research team
tallied the results of the experts based on the
degree to which the experts agreed on the
accuracy, relevance and appropriateness of
the content clustered under the domains of
each of the two subsystems of the tool.
Subsequently, the Content Validity Index
(CVI) was calculated for determining content
validity agreement between the experts for all
the domains of the tool. (Table 3 & 4)

Nursing and Midwifery Research Journal, Vol-10, No.3, July 2014

94

Table 3: Content Validity Index (CVI) of the autonomic/visceral subsystem of the tool in
terms of accuracy, relevance and appropriateness
n=7
Score Domains
0
1
2
0
1
2
0
1
2
0
1
2

Colour
Pale, Cyanotic
Pink but changes rapidly with slow recovery
Pink
Respiration
Gasping, frequent apnea, unstable respiratory rate
Occasional apnea, unstable respiratory rate
Regular, stable respiratory rate
VISCERAL
Vomits feeds, feed intolerance
Bowel movement grunt and strain
Gastro-Oesophageal Reflux (GER) Nil
NEUROPHYSIOLOGICAL RESPONSES
Flaccid on stimulation
Abnormal jerks, Twitch
Stable

Number of
experts agree

Number of expert
did not agree

CVI

6
7
7

1
0
0

85.71
100
100

7
6
7

0
1
0

100
85.71
100

7
6
7

0
1
0

100
85.71
100

7
6
7

0
1
0

100
85.71
100

Table 4: Content Validity Index (CVI) of the state and attention-interaction subsystem of
the tool in terms of accuracy, relevance and appropriateness
n=7
Score Domains

0
1
2
0
1
2
0
1
2

STATE REGULATION
Intense crying which is rhythmic with irregular breathing
Active awake state with infant fussing but not
crying but stressed and hyper alert
Awake alert but quiet or Deep sleep with
regular breathing
ORIENTATION TO AUDITORY STIMULUS
Does not focus on or follow stimulus
Brightness with stimulus, may focus and follow
briefly with jerky eye movements
Focuses on stimulus and follows with smooth
continuous head movement
ALERTNESS
Rarely or never responsive to direct stimulation
When alert, responsivity brief and variable,
may be delayed
Always alert in best periods, stimulation always

Number of
experts agree

Number of expert
did not agree

CVI

7
7

0
0

100
100

85.71

6
7

1
0

85.71
100

100

7
7

0
0

100
100

71.43

elicits alerting and orientating (reworded as 'active alert')

Nursing and Midwifery Research Journal, Vol-10, No.3, July 2014

95

Content Validity Index (CVI) of the tool:

The CVI from the panel of experts for
the domains of the autonomic/visceral
subsystem of the tool is shown in table 3. The
CVIs for the four domains of the
autonomic/visceral subsystem were above
0.80 indicating adequate agreement; and for
eight domains the CVIs of 1.00 indicate 100
percent agreement between the experts or
raters. The overall CVI from the panel of
experts for the autonomic/visceral
subsystem yielded a 0.95. Since the overall
CVI is above 0.90, it indicates good
agreement between the experts for all the
domains of this subsystem of the tool.
The CVI from the panel of experts for
the domains of the state and attentioninteraction subsystem of the tool is shown in
table 4. The CVI for two domains of the state
and attention-interaction subsystem was
above 0.80 indicating adequate agreement;
and for six domains the CVI of 1.00 indicated
100 percent agreement between the experts.
Even though the CVI for one domain on
alertness in the state and attentioninteraction subsystem of the tool was 71.43,
since two experts disagreed on complexity
of the wording' of the domain, the overall CVI
from the panel of experts for the state and
attention-interaction subsystem yielded a
CVI of 0.94. Since the overall CVI is above
0.90, it indicates good agreement between
the experts.
The disagreement between the experts
for one domain namely alertness' was due to
the complexity of the wording' of the rubric.
The experts suggested that the rubric should
be reworded as active alert' instead of the
initial wording of the rubric i.e., Always alert
in best periods/stimulation always elicits
alerting and orientating'. The suggestion was
incorporated in the final form of the tool and

the rubric was reworded as per suggestions

of the experts. Further the overall CVI for
both the subsystems of the tool yielded 0.95
showing good agreement between the
experts. A high overall CVI of 0.95
demonstrates the validity of the tool.
3
Lynn has stated that the CVI obtained
from a panel of six experts or more should be
more than 0.70. The research team also
maintained that the CVI obtained for all
domains of the subsystems of the tool
should be no lower than 0.70 for deeming the
tool as acceptable and valid in terms of
content validity.
Reliability of the tool:
The purpose of the present study was
also to examine systematically the reliability
of the tool developed to assess preterm
infants following provision of the DSC
intervention. For measuring reliability, a
group of 40 preterm infants of 28 33 weeks
of gestation age and whose chronological
age was first week of life were observed for
30 minutes after provision of DSC
intervention for 30 minutes. The two raters,
each of them were registered nurses (both
female) and were working in the Neonatal
Intensive Care Unit (NICU) as staff nurses
and each having an average of eight years of
clinical working experience in the NICU. The
raters simultaneously observed the preterm
infants after provision of DSC intervention
for a period of 30 minutes to provide an
estimate of inter-rater reliability.
Approval was taken from Manipal
University Ethical Committee prior to the
commencement of the study. Criteria for
inclusion of the preterm infants required one
or more of the following: preterm infants who
were 28-33 weeks of gestational Age,
belonging to weight for gestational age' like
appropriate for gestational age (AGA) and

Nursing and Midwifery Research Journal, Vol-10, No.3, July 2014

96

small for gestational age (SGA), cared for in

the open care system with an overhead
radiant warmer and consented to be included
in the study by proxy consent i.e. parental
informed written consent. Forty preterm
infants, who participated in this study, were
admitted to the tertiary level NICU at the time
of assessment. Parental informed proxy
consent was taken before commencement of
the study.
Criteria for exclusion were: preterm
infants who were belonging to weight for
gestational age at birth' i.e., large for
gestational age (LGA), having congenital
malformations, dysmorphic features,
chromosomal abnormalities and surgical
complications, preterm infants receiving
incubator care, having congenital sepsis
(confirmed by blood culture at CA of 1-3
days), exposed to Chorioamnionitis
(confirmed in the antenatal record), having
Central Nervous System (CNS) infection and
any other high-risk condition identified by
the NICU staff or the preterm infant's
physician. All preterm infants were assessed
during the first week of life.
Two raters participated in the collection
of the reliability data. The research team
trained both the raters for examining and
observing the behavior of the preterm infants
using the tool. One rater was designated as
the primary rater and the other rater was
designated as the secondary rater. Each rater
rated the tool independent of the other rater
for the duration of 30 minutes of observation.
When the observations were done on 40
preterm infants during the first week of life,
following the provision of DSC intervention,
the research team ensured that the raters did
not discuss the scoring with each other as
the test proceeded. The observations were
performed following 30 minutes of provision

of DSC intervention. The raters observed the

preterm infants kept in the open care system
following the provision of DSC intervention
i.e. for 30 minutes and recorded their
observations. The DSC intervention in this
study was recorded Indian Classical music
which was provided for duration of 30
minutes.
The Tool could be administered and
scored in less than 15-20 minutes, which
was well within the designated observation
period set by the research team viz. 30
minutes duration after provision of 30
minutes of DSC intervention.
Results: Reliability of the tool:
In this study, the consistency estimates
of Interrater or inter-observer reliability was
the intra-class correlation (R) which
provides an estimate of the relation between
the scores of two raters for the paired scores
on observation (inter-observer reliability).
The Pearson product-moment
correlation coefficient calculates the degree
of consistency between independent raters.
Values approaching +1 or -1 indicate that the
raters are following a consistent pattern,
whereas values close to zero indicate that it
would be almost impossible to predict the
rating of one rater given the rating of the
other rater. The minimal acceptable level of
4
R' is 0.60.
Calculation of ICC coefficient 'R' for the
present study was done using SPSS
(Ver.15.0), South East Asia, Bangalore for
raw scores obtained during observation of
preterm infants following the DSC
intervention by two independent observers
for the two subsystems namely the
autonomic and state and attentioninteraction systems and their domains in the
tool. Following tabulation of the raw scores

Nursing and Midwifery Research Journal, Vol-10, No.3, July 2014

97

for each of the domains and their

subsystems on a contingency table, the ICC
was calculated. The 'R' value obtained for the
autonomic subsystem was 0.98 whereas the
R' obtained for the state and attentioninteraction subsystem was 0.92, which
indicates high reliability for both the
subsystems of the tool.
Discussion:
DSC interventions provide
developmental support to the preterm
infants. But, to ensure that the DSC
interventions help promotion of
developmental support to the preterm
infants and not induce any stress to the
preterm infant, a valid and reliable tool was
necessary. A valid and a reliable tool for
assessing the behavior of preterm infants is
an absolute necessity. The tool developed in
this study assessed the preterm infants
following provision of a DSC intervention.
The present study shows that the tool
developed is a valid and reliable tool for
assessing preterm infants after provision of a
DSC intervention.
The present tool was developed based
on certain domains of ^The Assessment for
Preterm Infants' Behavior'(APIB) developed
by Al et al.5 APIB was developed for use with
preterm and ill full-term newborn infants to
identify both neurobehavioral competencies
and threshold to stress. The tool for
assessment of preterm infant behavior in this
study does not study neurobehavioral
comprentencies, but determines stress
behaviors as well as self regulation behaviors
of preterm infants following a DSC
intervention. The research team had also
reviewed and adapted some domains of
behavioral assessment from the Neonatal
Behavioral Assessment Scale (NBAS)

developed by Brazelton ,which assesses the

behaviors of term infants. Certain domains of
the state organization and the attentioninteraction system of the tool which includes
state regulation, orientation to auditory
stimulus and alertness, were adapted from
an assessment program to assess preterm
neonates in the early postnatal period.6 This
was the first tool of its kind, which was
specifically developed to assess immediate
behaviors of the preterm infants following a
DSC intervention. All in all, the tool
developed, demonstrated good content
validity and reliability. But, the reliability
testing of the tool was done on only 40
preterm infants after provision of the DSC
intervention. Clearly, a study with a larger
sample size is needed to determine if the
reliability trends obtained for the subsystems
of the tool in the present study are consistent
and stable when using the tool further.
Another important limitation of this
study, however, was the fact that though
interdisciplinary panel of experts were
chosen to validate the content of the tool, the
content experts were predominantly from the
field of Pediatrics and Neonatology whereas
there was little representation of experts
from other fields of health care providing
care to the preterm infants, since there were
four clinicians belonging to Pediatrics and
Neonatology whereas one each representing
perinatal nursing, neonatal nursing and
developmental Psychology. The experts
from the field of Pediatrics and Neonatology
who were chosen could have primarily given
their opinions based on their medical
expertise. This shortcoming minimizes the
generalizability of these opinions. Finally, the
value of the experts' opinions about the tool
would have strengthened considerably, had
the experts been given the opportunity to

Nursing and Midwifery Research Journal, Vol-10, No.3, July 2014

98

actually administer the tool rather than to

merely review the tool, though this is not
essentially needed for validating the tool.
The findings reported suggest that a
valid and reliable tool for neonatal
assessment was prepared which has
potential for use as both a clinical and a
research instrument by Pediatricians,

Neonatologists, Neonatal nurses, Perinatal

nurses, Developmental Psychologists,
Pediatric physical therapists, and other
health professionals who are working with
preterm infants aged 28 - 33 weeks of
gestational age
subjected to DSC
interventions in the NICU.

References

4.

Multon KD, Interrater reliability:

Encyclopedia of Research Design.2012;
A v a i l a b l e
f r o m :
http://www.bibb.k12.ga.us/cms/lib01/GA0100
0598/Centricity/Domain/105/Interrater%20Rel
iability%20Article_1.pdf

5.

Als H, Lester BM, Tronick E, Brazelton

TB. Toward a research instrument for the
Assessment of Preterm Infants' Behavior
(APIB). In: Fitzgerald HE, Lester BM, Yogman
MW, editors. Theory and research in behavioral
Pediatrics. New York. Plenum; 1982.

6.

Brazelton TB. Neonatal Behavioral

Assessment Scale: Clinics in developmental
medicine. London: Spastics International;
1984.

7.

Carmichael K, Burns Y, Gray P,

O'Callaghan M. Neuromotorbehavioural
assessment of preterm infants at risk for
impaired development. Aust J Physiother.
1997; 43: 101-7.

1.

2.

3.

AlsH. A synactive model of neonatal

behavioral organization: Framework for the
assessment and support of neurobehavioral
development of premature infants and their
parents in the environment of the NICU.
In:Sweeney JK. Physical and Occupational
Therapy in Pediatrics. 6thed. New York.
Haworth Press; 1986.
Symington A, Pinelli J. Developmental
care for promoting development and
preventing morbidity in preterm infants.
Cochrane Database of Systematic Reviews
[Internet] 2006. Issue 2. Art. No.: CD001814.
DOI: 10.1002/14651858.CD1814.pub2.
Ly n n M . D e t e r m i n a t i o n a n d
Quantification of Content Validity. Nursing
Research.1986; 35: 38285.

Nursing and Midwifery Research Journal, Vol-10, No.3, July 2014

99