NSS SCORING GUIDE

INTRODUCTION

The Narrative Scoring Scheme (NSS) is an assessment tool that provides an index of the student’s ability
to produce a structurally sound and coherent narrative. This scoring procedure combines many of the
abstract categories of Story Grammar, adding features of cohesion, connecting events, rationale for
characters’ behavior, and referencing. Each of the scoring categories has explicit examples to establish
scoring criteria, reducing the abstractness of the story grammar categories.

Samples from the following SALT reference databases have all been coded for NSS:
• Narrative Story Retell database consisting of samples from typically-developing students in grades
P-6 whose primary language is English.
• Bilingual Spanish/English Story Retell database consisting of Spanish and English samples from
typically-developing bilingual students in grades K-3 who are native Spanish-speakers learning
English as a second language.
• Bilingual Spanish/English Unique Story database consisting of Spanish and English samples from
typically-developing bilingual students in grades K-3 who are native Spanish-speakers learning
English as a second language.
• Monolingual Spanish Story Retell database consisting of Spanish samples from typically-
developing students in grades K-3 who are native Spanish-speakers.

These databases can be utilized to compare a student’s narrative skills to those of age-matched peers.
Clinicians can compare individual characteristics of the NSS or the composite score using the database.
The narrative retell task may be repeated to assess progress of story retell skills.

SCORING GUIDELINES

Assigning NSS Scores

The NSS is scored using a 0 - 5 point scale. 5 points are given for “proficient” use, 3 points for
“emerging” use, and 1 point for “minimal” or “immature” use. Scores of 2 and 4 are undefined and
require scorer’s judgment. A score of 0 is given for speaker errors. Examples include: telling the wrong
story, conversing with examiner, not completing/refusing task, using wrong language creating inability
of scorer to comprehend story in target language, abandoned utterances, unintelligibility, poor
performance, and/or if components of the rubric are entirely imitated. A score of NA (non-applicable) is
given for mechanical/examiner/operator errors. Examples include: interference from background noise,
issues with recording (cut-offs, interruptions), examiner quitting before target speaker does, examiner
not following protocol, and examiner asking overly specific or leading questions rather than open-ended
questions or prompts. The scores for each characteristic can be considered individually or combined into
a total composite score (highest possible score being 35).

Refer to the following rubric for guidance when assigning scores to each of the NSS characteristics of a
narrative language sample. Examples are provided on the NSS documents for each story retell narrative.

SALT Software, LLC revised 9/2017

NSS Scoring Guide 2|Page

NSS SCORING RUBRIC

Characteristic Proficient (5) Emerging (3) Minimal/Immature (1)

1) Setting: 1) Setting:
- States general place and - States general setting but
provides some detail provides no detail.
about the setting (e.g., - Description or elements of
reference to the time of setting are given
Introduction the setting, daytime, intermittently through - Launches into story
The presence, absence, bedtime, season). story. with no attempt to
and qualitative - Setting elements are - May provide description of provide the setting
depiction of character stated at appropriate specific element of
and setting
or introduce the
place in story. setting, e.g., the frog is characters.
components.
in the jar.
2) Characters:
- Main characters are 2) Characters:
introduced with some - Characters of story are
description or detail mentioned with no
provided. detail or description.
- Main character(s) and all
supporting character(s)
are mentioned.
- Inconsistent mention
- Discriminates between - Both main and active
Character of involved or active
main and supporting supporting characters
Development characters.
characters, e.g., more are mentioned.
The acknowledgment of - Character(s)
description of, or - Main characters are not
characters and their necessary for
emphasis upon, main clearly distinguished
significance throughout advancing the plot
the story. character(s). from supporting
are not present.
- Narrates in first person characters.
using character voice,
e.g., “You get out of my
tree”, said the owl.
- Mental states of main and
Mental and supporting characters
Emotional States are expressed when - Some use of evident - No use of mental
Score based on the necessary for plot mental state words to state words to
vocabulary used to
development and develop character(s). develop
convey charter
emotions and through
advancement. character(s).
processes. - A variety of mental state
words are used.
Referencing/
Listener Awareness
Scores based on the
- Excessive use of
consistent and accurate - Provides necessary pronouns.
use of antecedents and antecedents to - No verbal clarifiers
- Inconsistent use of
clarifiers throughout pronouns. used.
referents/antecedents.
the story. Use of - References are clear - Speaker is unaware
correct pronouns and throughout story. that listener is
proper names should confused.
be considered when
scoring.

SALT Software, LLC revised 9/2017

NSS Scoring Guide 3|Page

- Resolution(s) or
reaction(s) stated
- Under developed with no mention of
Conflict/Resolution description of cause or conflict.
and Event/Reaction conflict/resolutions and OR
Scores based on the - Clearly states all event/reactions critical - Conflict (s) or event(s)
presence or absence of conflict/resolutions and to advancing the plot of mentioned without
conflict/resolutions and
event/reactions critical the story. resolution.
event/reactions
to advancing the plot of OR OR
required to express the
story as well as how the story. - Not all conflict/resolutions - Many
thoroughly each was or event/reactions conflict/resolutions
described. critical to advancing the or event/reactions
plot are present critical to advancing
the plot are not
present.
- Events follow a logical
order.
- Excessive detail or
emphasis provided on
- Events follow a logical
minor events leading the
Cohesion order.
listener astray.
Scores based on the - Critical events are included
OR
sequence of, details while less emphasis is - No use of smooth
- Transitions to next event
given to, and placed on minor events. transitions.
transitions between unclear.
- Smooth transitions are
each event. OR
provided between
- Minimal detail given for
events.
critical events.
OR
- Equal emphasis on all
events.
Conclusion - Story is clearly wrapped up
- Specific event is
Scores are based on the using general concluding - Stops narrating and
concluded, but no
conclusion of the final statements such as “and listener may need
general statement made
event as well as the they were together to ask if that is the
wrap up of the entire
as to the conclusion of
again happy as could end.
story. the whole story.
be”.

Scoring: Each characteristic receives a scaled score 0-5. Proficient characteristics=5, Emerging=3, Minimal/Immature=1.
Scores in between are undefined, use judgment. Scores of 0 and NA are defined below. A composite is scored by adding the
total of the characteristic scores. Highest score possible=35.
* A score of 0 is given for TARGET SPEAKER errors (i.e., telling the wrong story, conversing with examiner, not
completing/refusing task, abandoned utterances, unintelligibility, poor performance, components of rubric are given in
imitation-only manner).
* A score of NA (non-applicable) is given for MECHANICAL/EXAMINER/OPERATOR errors (i.e., interference from background
noise, issues with recording, examiner quitting before target speaker does, examiner not following protocol, examiner asking
overly specific or leading questions rather than using open-ended questions or prompts.

SALT Software, LLC revised 9/2017

NSS Scoring Guide 4|Page

Helpful Scoring Tips

• Be familiar with the narrated story. It is recommended that the scorer have a copy of the story to
reference while scoring.
• Print the narrative transcript.
• Read the transcript as fluidly/inclusively as possible, ignoring SALT transcription codes.
• Write comments and circle or flag key words/utterances such as mental state words or difficulty
with referents and pronouns.
• For each characteristic, review the NSS before assigning a score. Read the criteria along the
continuum of points. Determine what is present in the transcript and score accordingly. This will
insure better intra- and inter-rater reliability.
• Many of the characteristics are distributed across the entire narrative. They do not occur at one
static point within the story. The scoring of these characteristics must take into account the story
as a whole.
• Frequently review what constitutes a score of 0 or NA. Explanations are given at the bottom of the
NSS scoring rubric.
• Proficiency in assigning scores will develop with experience.

USING SALT TO ENTER NSS SCORES

Use Edit menu → Insert Template → Narrative Scoring Scheme to insert the NSS plus line template at
the bottom of your transcript. Then type the individual scores after each label.

NSS Template Example of NSS Scoring

+ Introduction: + Introduction: 3
+ CharacterDev: + CharacterDev: 2
+ MentalStates: + MentalStates: 2
+ Referencing: + Referencing: 2
+ ConflictRes: + ConflictRes: 1
+ Cohesion: + Cohesion: 3
+ Conclusion: + Conclusion: 2

ANALYZING THE NSS SCORES

• Use the Analyze menu → Narrative Scoring Scheme report to list each individual NSS score along
with the composite score.

• Use the Database menu → Narrative Scoring Scheme to list each individual NSS score along with
the composite score. Scores are listed for your transcript and for the selected database samples.

TRYING IT OUT

The free online training course, 1502: NSS – Narrative Scoring Scheme, has practice transcripts.
Compare your scores to those of our trained transcribers.

SALT Software, LLC revised 9/2017

NSS Scoring Guide 5|Page

REFERENCES

Kay-Raining Bird, E., Cleave, P., Squires, B., Cormier, P., Chaisson, S., Roach, M., Patenaude, T., &
Rushbrook-Dickey, T. (2013, November). Narrative and Expository Discourse in French-English Bilingual
Children. Poster presented at the Annual Convention of the American Speech-Language-Hearing
Association, Chicago, IL.

Miller, J., Andriacchi, K., DiVall-Rayan, J., Lien, P. (2003). Narrative Scoring Scheme.

Peterson, D., Gillam, S., & Gillam, R. (2008). Emerging procedures in narrative assessment: The Index of
Narrative Complexity. Topics in Language Disorders, 28, 115-130.

Stein, N., & Glenn, C. (1979). An analysis of story comprehension in elementary school children. In R.
Freedle (Ed.), New directions in discourse processing (pp. 53-120). Noorwood, NJ: Ablex.

Stein, N., & Glenn, C. (1982). Children’s concept of time: The development of a story schema. In W.J.
Friedman (Ed.), The developmental psychology of time (pp. 255-282). New York: Academic Press.

SALT Software, LLC revised 9/2017