Test-Retest Reliability of Objective.

This study examined the testr-retest reliability of

the legibility portion ofthe Evaluation Tool ofChildren's
Handwriting-Manuscript (ETCH-M).
the Evaluation Tool of Method. The sample consisted 0/31 first-grade and
second-grade students with handwriting dysfunction. The
Children's Handwriting- ETCH-M was administered two times, 1 week apart, to
participants. The primary investigator acted as sole rater
andfOllowed standard scoring procedures.
Manuscript Results. The reliability coefficients were. 63 fOr total
numeral legibility, .77 fOr total letter legibility, and .71 fOr
total word legibility. Individual task reliability coefficients
Susan M. Diekema, Jean Deitz, Susan J. were generally lower and rangedfrom .20 (near-point
copy) to .76 (alphabet uppercase).
Amundson
Conclusion. lOtalletter, total word, and uppercase
letter legibility were more stable than total numerallegibil-
Key Words: reliability of tests • school-based ity scores and other individual tasks scores. When evaluat-
ing handwritingfor a child, it is important to consider
occupational therapy
ETCH legibility scores as only one aspect ofa comprehen-
sive evaluation.

T
he school-based occupational therapy evaluation
focuses on a student's performance in the school
environment. Typical areas of school-based eval-
uation and intervention are (a) mobility and transitions,
(b) handling of classroom materials, (c) functional writ-
ten communication, (d) activities of daily living, (e)
school routines, and (f) socialization. A problem with
handwriting is one of the most common reasons for an
occupational therapy referral in school-age children
(Oliver, 1990; Reisman, 1991). Handwriting is impor-
tant for the child in the school setting for many reasons:
The amount of time devoted to paper-and-pencil tasks is
high (McHale & Cermak, 1992); reduced handwriting
skills may lead to an inability to keep up with written
Susan M. Diekema, MS, OTR, is Occupational Therapist, Har-
borview Medical Center, Seattle, Washington. At the time of work and in taking notes during class time (Graham &
this study, she was Graduate Student, Department of Rehab- Miller, 1980); poor handwriting may affect persistence,
ilitarion Medicine, University of Washington, Seattle, Wash- motivation, and sense of efficacy (MacArthur & Gra-
ington. (Mailing address: 18140 60th Place NE, Seattle, ham, 1987); and reduced legibility of a student's work
Washington 98155) may result in lower grades (Briggs, 1970, 1980).
Handwriting problems may originate from difficul-
Jean Deitz, PhD, OTR, FAOTA, is Professor, Department of ties in cognitive, psychosocial, sensorimotor, fine motor,
Rehabilitation Medicine, University ofWashingron, Seattle,
or auditory or visual processing skills. A full evaluation
Washington.
should include discussion with the teacher regarding con-
Susan J. Amundson, MS, OTR, is Executive Director of a pedi- cerns and observations; review of the student's education-
atric private practice, Homer, Alaska, and Clinical Instructor, al cumulative file; classroom observation of the student
Department of Rehabilitation Medicine, University of Wash- performing school activities, specifically a writing assign-
ingron, Seattle, Washington. ment; and formal and informal assessments (Benbow,
Hanft, & Marsh, 1992). The school-based therapist
This article was acceptedfOr publication October 17, 1997.
depends on available standardized assessments and indi-

248 April1998, Volume 52, Number 4

Downloaded from http://ajot.aota.org on 03/29/2019 Terms of use: http://AOTA.org/terms
 vidualized classroom observations to evaluate children, of letter and numeral legibility of the six individual
provide classroom consultation, and develop intervention handwriting tasks of the ETCH-M?
programs. Standardized assessments often are used in 2. What is the test-retest reliability of the total letter
determining eligibility for children to receive services, legibility score (combination ofTasks la, lb, III, IV,
documenting change, and serving as a communication V, VI)?
tool among members of the educational team (Bonder, 3. What is the test-retest reliability of the total nu-
1989). merallegibility score (combination ofTasks II and
Several handwriting assessments exist. Each varies V)?
greatly in its approach to the rype of writing task, writing 4. What is the test-retest reliability of the total word
style (manuscript or cursive), scoring criteria, and age legibility score (combination of Tasks III, IV, V,
level. Handwriting legibility can be measured on a global VI)?
level for readability by comparing the child's writing sam- 5. What are the magnitudes of difference between test
ples to samples in the assessment manual or on a more and retest scores for each of the following ETCH
specific level for the individual components of handwrit- scores: total letter legibility, total numeral legibility,
ing (e.g., slant, spacing, letter size). There is great variabil- total word legibility, and the six individual writing
ity in the definition of legibility and the approach used to tasks?
measure it (Graham, 1986). One definition of legibility is
the ability to recognize a letter, number, or word easily Method
and correctly outside of the context of the word, sentence, Participants
or phrase (Hasbrouck, Tindal, & Parker, 1994). Because
A convenience sample of 31 children (24 boys, 7 girls)
each practitioner evaluating legibility of handwriting
with identified handwriting performance deficits was test-
brings a different historical experience regarding letter,
ed. The sample was taken from seven elementary schools
word, and numeral recognition, determining legibility is a
in the Seattle Public School District and Bellevue Public
subjective process.
School District and consisted of children in first grade (n
Because handwriting is an important student occupa-
= 18) and second grade (n = 13). Participants were not
tion, Amundson (1995) developed the Evaluation Tool of
excluded if they were receiving special education services,
Children's Handwriting (ETCH) to fill the need for a
provided they also participated within the regular educa-
comprehensive protocol for evaluating handwriting and
tion classroom. Inclusion criteria were (a) the child had
the challenges inherent in measuring legibility. This hand-
to be identified by a teacher or therapist as having a
writing test examines manuscript and cursive handwriting
handwriting performance skill deficit in the classroom;
legibility and speed-the foundations for functional writ- (b) spoken language had to be the child's primary means
ten communication. Well-defined standards are used to of communication; and (c) English had to be the prima-
determine legibility and provide the school-based thera- ry language spoken in the home. Further, each child had
pist with useful information for program planning as well to be able to sustain attention for a 20-min seated activity
as documenting changes in performance over time. The as verified by his or her teacher. The two school districts
ETCH contains two separate segments for handwriting were chosen in order to include variety in socioeconomic
tasks: one for manuscript (ETCH-M) and one for cur- and ethnic background. Before initiation of the study,
sive. human subjects approval was obtained from the Univer-
The ETCH-M is the focus of the current study. An sity of WashingtOn Human Subject Committee and from
interrater reliability study for this measure has been com- the participating school districts. Participants belonged to
pleted by the test developer, and results are included in the following ethnic groups: 15 Caucasian, 7 African-
the examiner's manual. An understanding of the test- American, 3 Hispanic, 3 Asian, 2 Native American, and 1
retest reliability is also important so that the therapist can East Indian.
evaluate the expected stability of test scores over time
when there is no intervention. This understanding facili- Instrument
tates score interpretation by the therapist. Thus, the pur-
The ETCH-M (Amundson, 1995) was designed to evalu-
pose of this study was to examine the stability of the
ate a student's handwriting legibility and rate of writing. It
ETCH-M legibility scores. This was accomplished by
requires the student to complete a variety of writing tasks
using the ETCH-M to test and retest a sample of first-
similar to those expected in a classroom. It evaluates six
grade and second-grade students in order to address the
areas of handwriting: (a) alphabet production in uppercase
following research questions:
and lowercase from memory, (b) numeral writing from
1. What are the test-retest reliabilities for measures memory, (c) near-point copying, (d) far-point copying, (e)

The American Journal ofOccupational Therapy 249

Downloaded from http://ajot.aota.org on 03/29/2019 Terms of use: http://AOTA.org/terms
 Table 1 school-age children to gain experience in administering
Task Descriptions and Potential Raw Score the test, but these children were not included in the study.
Ranges on the ETCH-M
To further control for potential error in test administra-
roeential Row Score R~~gc
tion, a test procedural agreement checklist was developed
Task De,cri~_n_ Leewrs Words Numerals
on the basis of the srandardized procedures in the ETCH
la. Alphabet lowercase (a to z) from 0-26 NA A
memory examiner's manual. During every lOth or 12th test, a sec-
lb. Alphabee uppercase (A ro Z) from 0-26 NA NA ond person sat in an unobtrusive locarion during admin-
memory
II. Numeral wrieing (l ro 12) from mem- NA NA 0-12 isrrarion and recorded procedural correctness with the
ory procedural agreemenr checklist. There was a tOral of five
Ill. Near-point copy (5 words, 18 !erters) 0-18 0-5 NA
procedural checks with procedural agreemem, ranging
from cask sheee
IV. Far-point copy (5 words, 18 leeeers) 0-l8 0-5 NA from 91 % to 100%.
from wall chare
The ETCH-M was administered ro the 31 partici-
V. Diceation 05 leeeers ond numerals 0-10 0-2 0-5
roeal) by verbal direceion pants on two separare occasions, using a 7-day rime inter-
VI. Sentence composieion (5 or more Varies Varies NA val. Anastasi (1988) reported rhat short intervals between
words) cr~a(ive wriring
tesr and rerest are appropriare with young children for
-N~t-;-AJI raw sc~;~~~onverre~percentage equivalents for daea analysis
(0-100%). All eosks ore eimed and recorded in Ieccers per minuee, excepe dic- whom discernible developmemal changes can be expect-
rarion, which is noe eimed. ETCH-M = Evaluaeion Tool for Children's ed within a limited rime frame. The study began during
Handwrieing-Manuscript; NA = noe applicable.
the second half of the school year so rhat the students in
first grade would be exposed to the teaching of rhe entire
dictation, and (0 sentence composition. This criterion-ref- alphabet in rhe classroom. The test was administered
erenced tOol has standard administration and scoring pro- individually in a quiet area of each participanr's school
cedures, is designed to be administered individually, and between 8:30 a.m. and 12:00 p.m. The participant was
requires 20 min to 30 min to complete. Scoring focuses seared ar the same desk for test and rerest. The partici-
on legibility, rate of writing, components of legibility, and pam used a sharpened standard number 2 pencil with
biomechanical aspects of handwriting. For legibility scor- eraser during the tesr and received a colorful pencil after
ing, if a letter, number, or word is individually determined each testing. Each wriring sample was coded with a num-
to be readable, it is counted as a correct item on the score ber. Because testing occurred over a span of 2 months, a
sheet. The number of illegible items are subtracted from low code number could be a tesr or a rerest score sheet.
the tOtal number of items. The difference is then convert- The primary invesrigator acted as rhe sale rater of
ed to a percentage score (see Table 1 for task descriptions the 62 score sheets and was blind borh to the partici-
and potential ranges of raw scores). Qualitative information pants' idemiries and to whether a particular score sheet
is recorded on the basis of the legibility components, reflected a rest or rerest administration. She scored the
including letter formation, letter size, horizontal align- tesr score sheets in randomly selected groups of 10 after
ment, and spacing. The score sheet also allows for data col- all 62 rests had been adminisrered. The fact thar a code
lection related to the student's mechanical aspects of pencil number could be high or low for test and retest and the
grasp pattern, pencil position, pencil pressure, and in-hand addition of rime between test administrarion and rest rat-
manipulation of the pencil along with classroom observa- ing combined to reduce the risk of the primary investiga-
tions. For purposes of this study, only the specific legibility tor's ability to identifY rhe parricipant.
percentages were evaluated for test-retest reliability. Before raring the score sheets, and periodically through-
A chapter in the ETCH examiner's manual assists the out rhe rating process (after every 10th score sheer), the
examiner in learning the complex scoring criteria before primary investigator checked scoring competency ac-
scoring actual test protOcols (Amundson, 1995). A self- cording to rhe examiner's manual instructions. In all
study tutOrial and scoring competency quizzes provide cases, her scoring competence was at or above the 90%
training in scoring. An examiner is expected to achieve a standard described in rhe manual. This periodic reresting
scoring competency of 90% on the trial tests in the man- of competency was an attempr to control rater drifr
ual before attempting to score children's performances in (Goldstein & Tupper, 1987).
the practice setting.
Data Analysis
Procedure
Descriptive sraristics were examined for tesr and retesr for
Each student was administered the ETCH-M by the pri- individual tasks and tOral scores. Raw scores were convert-
mary investigatOr, who followed the examiner's manual ed to percemages as insrructed in rhe examiner's manual.
guidelines. Before the study, the primary investigatOr Nexr, for each individual writing task and the total legi-
administered the ETCH-M to five early e1ementary- bility scores, rhe use of paramerric versus nonparametric

250 April 1998, Volume 52, Number 4

Downloaded from http://ajot.aota.org on 03/29/2019 Terms of use: http://AOTA.org/terms
 statistics was determined after examining scatterplots, his- composition task, total numeral legibility). The maxi-
tograms, measures of central tendency and score variabili- mum change between test and retest means on the indi-
ty, and kurtosis and skewness scores. When data were vidual tasks was .06 for dictation. Second, according to
approximately normally distributed and other assump- Wilcoxon signed rank tests, no significant differences
tions for the use of parametric statistics were met, intra- were found between test and retest scores.
class correlation coefficients (ICCs) were computed. The Magnitudes of difference for the letter legibiliry of
ICC was chosen because it is the preferred statistic for individual tasks and letter, numeral, and word legibiliry of
estimating the reliabiliry of repeated measures in addition total tasks with and without the composition task are
to being sensitive to systematic measurement error when reported in Table 4. An example of how to read and inter-
data are approximately normally distributed (Baumgart- pret Table 4 is as follows: For total letters, the findings
ner, 1989). In situations where data did not meet the indicated that 1 participant's scores did not change be-
assumptions for use of parametric statistics, Spearman tween test and retest; 14 participants' scores increased by
rank order correlations were used. It is possible with 1% to 10%; 10 participan tS' scores decreased by 1% to
Spearman rank order correlations to have a high correla- 10%; and 3 participants' scores either increased or de-
tion between test and retest when test scores are either creased by 11 % to 20%.
consistently higher or consistently lower than retest
scores. To check for such systematic differences between Discussion
test and retest scores, Wilcoxon signed rank tests were Overall, test-retest reliabiliry of handwriting legibiliry as
used. For both the Spearman rank order correlations and measured by the ETCH-M, using a 7-day interval, was
ICC, a test-retest reliabiliry coeffIcient at or above .80 lower than desirable for this group of first-grade and sec-
was chosen as the desired level. ond-grade students with identified handwriting deficits.
Finally, test-retest agreement for individual and total Reliabiliry coefficients were. 77 for total letter legibiliry,
legibiliry scores was addressed through magnitudes of dif- .71 for total word legibiliry, and .63 for total numeral
ference between test and retest scores. This was done by legibiliry. Individual task reliabiliry coefficients were gen-
fIrst subtracting retest percentage scores on the ETCH-M erally lower and ranged from .20 (near-point copy) ro
from test percentage scores. Then, the number of scores .76 (alphabet uppercase). With the exception of alphabet
for each legibiliry task reflecting no change and specific uppercase, individual task scores appeared less stable over
magnitudes of either positive or negative change were time than did the total numeral and total word legibiliry
reported. A positive change might reflect a practice or scores. Therefore, these individual task scores are not rec-
learning effect, whereas a negative change might suggest ommended for use in determining eligibiliry for services
that students became bored with the test on second ad- or in documenting change over time.
mll1lstratlOn. Results were reported with the ICC for those data
that were approximately normally distributed. The ICC is
Results sensitive to systematic error (e.g., systematic improve-
Descriptive statistics for letter legibiliry of individual tasks ments in scores due to maturation or practice). Because
are reported in Table 2, and letter, numeral, and word leg- this is a more robust statistic than the Pearson product-
ibiliry of total scores are reported in Table 3. Reliabiliry moment correlation, the reliabiliry outcomes in this study
coefficients for the individual tasks ranged from .20 for may appear lower than those in similar studies where
the near-point copy task to .76 for the alphabet uppercase other statistics were used.
task. For the total scores, the reliabiliry coefficients ranged The magnitudes of difference (see Table 4) illustrate
from .63 for the total numerallegibiliry task to .77 for the two strengths. First, results revealed no substantial prac-
total letter legibiliry with composition task. For those tice effect overall. Second, no test-retest scores changed
tasks where it was necessary to use Spearman rank order more than 20% for total letter legibiliry scores. This sug-
correlations because the data were not approximately nor- gests that if a child's total letter legibiliry score changes
mally distri bu ted, analyses were done to check for sys- more than 20% between test and retest, then it likely is
tematic error. First, examination of differences between attributable to clinical change and not measurement error.
means of test and retest outcomes indicated that there was In contrast, the individual task of sentence composition
no improvement of greater than .06 on the individual showed the highest score variabiliry, with 32% of the par-
writing tasks and .02 on the total legibiliry scores. For ticipants changing more than 20% between test and re-
three individual writing tasks, the mean score stayed the test. This large variabiliry between test and retest on the
same (near-point copy) or decreased slightly on the retest composition task may be due to the complicated and
(numeral writing, far-point copy). In addition, two total spontaneous nature of being asked to generate a five-plus
score means decreased (total letter legibiliry excluding the word sentence. The scoring of legibiliry for this cask is

The American Journal o/Occupational Therapy 251

Downloaded from http://ajot.aota.org on 03/29/2019 Terms of use: http://AOTA.org/terms
 Table 2
Descriptive Statistics and Test-Retest Reliability Coefficients for ETCH-M Individual Writing Tasks
Low Score/ Reliability
Task M Median SD ~ghS~ - -Coefficienr
- _...
Ia. Alphabet lowetcase
Test .73 .81 .20 .12/1.00 .64 a
Retest .74 .77 .18 .15/0.96
lb. Alphabet uppercase
Test .69 .69 .18 .23/1.00 .76 b
Retest .72 .77 .19 .12/1.00
II. Numeral writing
Test .85 .92 .14 .5011.00 .63 a
Rerest .83 .92 .18 .33/1.00
Ill. Near-poinr copy
Test .88 .89 .09 .67/1.00 .20 a
Retest .88 .89 .11 .5611.00
IV. Far-poinr copy
Test .88 .89 .11 .61/1.00 .63 a
Retest .86 .89 .13 .56/1.00
V. Dictation
Test .70 .73 .23 .13/1.00 .68 a
Rerest .76 .80 .22 .00/1.00
VI. Senrence composition
Test .74 .78 .21 .0611.00 .33 a
Retest .79 .80 .23 .00/1.00
._--_.
Note. All descriptive statistics are converted from raw scores and reported in percentage format. The above tasks refet to individual letter at number legibility,
not word legibility. ETCH-M = Evaluarion Tool for Children's Handwriting-Manuscript.
aSpearman rank order correlation coefficient. b1ntraclass correlation coefficient.

also more complex because of the high degree of variation Wisconsin, and Minnesota. Neither study examined chil-
of sentences and words chosen by each participant. dren with identified handwriting problems, and each used
Test-retest reliability for the ETCH-M was lower the Pearson product-moment correlation.
than desired for test development but within the range of Several factors may explain the low test-retest relia-
other assessment tools measuring children's handwriting bility coefficients of this study. First, ceiling effects may
performance. Using a handwriting assessment tool they have had an impact on some individual task scores and
developed, Ziviani and Elkins (1984) reported a range total scores. Twenty percent or more of the participants
from .48 to .84 of test-retest reliability when examining scored 100% on either the test or retest for three individ-
the handwriting of 575 Australian children between 7 ual task scores (near-point copy, far-point copy, numeral
and 14 years of age in regular education classrooms. On writing) and for one total task score (total numeral legi-
the Minnesota Handwriting Test-Research Version, Reis- bility).
man (1993) found a range of .58 to .94 for test-retest Second, the scoring of handwriting legibility is inher-
reliability on a convenience sample of students in Maine, ently subjective. Whereas some other handwriting tools

Table 3
Descriptive Statistics and Test-Retest Reliability Coefficients for Total Scores on the ETCH-M
Low Score/ Reliability
Letter and Numeral Total n /VI Median SD High Sco.'::-- Coefficient
Total letter legibility, excluding Task VIc
- -- -- -- ----
31 .69 .24 .11/1.00
Test 31 .71 .75 .11/1.00
Retest 31 .78 .75 .25
Total letter legibility .77 .81 .26 08/1.00
Test 28 .80 .14 .00/100
Retest 28 .78 .13 .35/0.97
Total numeral legibility .79 .81 .36/0.97 .71 a
Test 31 .79 .10
Retest 31 .83 .11 .53/0.96
Total word legibility .81 .88 .50/0.99 .77 b
Test 28 .88 .15
Retest 28 .67 .20 .47/1.00
Total word legibility, excluding Task VIc .69 .73 .35/1.00 .63 a
Test 31 .76 .23
Retest
--- - - - - - ---- - - ---- ---
Note. All descriptive statistics are converted from raw scores and reported in percentage format. ETCH-M = Evaluation Tool for Children's Handwriting-Manuscript.
aSpearman rank order correlation coefficient. blnrraclass correlation coefficient. cTask VI is the senrence composition task that three children refused to perform,
thus lowering the sample size to 28.

252 April 1998, Volume 52, Number 4

Downloaded from http://ajot.aota.org on 03/29/2019 Terms of use: http://AOTA.org/terms
 Table 4
Magnitudes of Difference Between Test and Retest Percentage Scores on the ETCH-M
_ __ __ _ __ _ ~~t_S_'_p_re_ad _
Negative ~ Positive. _ - - - - -
Task 50--41 40-31 30-21 20-11 10-1 0 1-10 11-20 21-30 31--40 41-58
- -- --- _.-- - - -
Lowercase letters 1 1 3 7 2 lOG 1
Uppercase letters I 4 7 5 G 3 5
Numerals 2 I I 9 7 G 3 2
Near-point copy 8 G 5 2 7 2
Far-point copy 3 3 8 10 4 1 2
Dictation 4 5 5 7 G 1 3
Composition (n = 28) 2 2 7 1 4 5 I 4
Toralletters (n = 28) 2 10 1 14 I
Total letters, excluding Task VI 2 10 4 12 3
Total numbers 3 3 G 5 5 G 2
Total words (n = 28) 1 3 7 4 5 4 2
Total words, excluding Task VI 3 5 11 5 3 2
Note. N = 31 ex-~ept where otherwise indica·t~d. ETCH-M = Evaluation T~ol for Children's H-andwriting-Manuscri"p-;; Task Vl = composj:-Cri'-on-ra-'-sk-.- - - -

have more detailed analytical rating systems requiring situations could have interfered with achieving the partic-
transparent overlays, rulers, and complex scoring criteria ipants' best effortS on the retest. Although no predictable
for each individual letter's legibility (Reisman, 1993; rise or fall occurred, retesting could have affected some
Tseng & Cermak, 1991; Ziviani & Elkins, 1984), the participants' performance.
ETCH focuses on legibility with a more global measure- Fifth, the sample of children identified to have hand-
ment method. The ETCH targets legibility in terms of writing deficits likely included those with varying degrees
readability of the student's handwriting, supported by of attention deficits, sensory processing problems, neuro-
specific scoring criteria of letters, numerals, and words. To muscular impairments, auditory processing problems,
determine letter legibility, the therapist must compare the language difficulties, and learning deficits. Children with
child's handwriting to illegible and legible models found these underlying problems are likely to have difficulty in
in the ETCH examiner's manual. The handwriting mod- following instructions, attending to task, and completing
els do not cover the inexhaustible number of possible let- tasks; all are necessary for testing performance. This study
ter formations produced by children; thus, the examiner attempted to control for attention deficits by specifYing
is required to score the letter as correct if there is any within the inclusion criteria that the child must be able to
question regarding its being legible. Because of the nature sustain attention for a 20-min seated activity. Even so,
of this scoring method, subjectivity increases, and a there may have been unidentified subgroups in the sam-
greater possibility for inconsistencies in scoring exist. In ple who performed better or worse on the retest because
this study, the primary investigator scored the entire sam- of attention difficulties. Although inclusion of these chil-
ple in an attempt to control for scoring variability. dren may have compromised the results, their inclusion
Third, because of the complex nature of handwriting was appropriate because the ETCH was designed for this
skills, the quality of a child's handwriting performance target population.
tends to vary over time. The development of handwriting Finally, young children exhibit a high level of vari-
ability involves the refinement, synthesis, and integration ability in performance, particularly in early elementary
of sensorimotor, cognitive, and language abilities (Ziviani, school years when handwriting is being taught and mas-
1995) and is one of the more complex tasks performed by tered. Ziviani and Elkins (1984) found that younger chil-
young children as they enter school. This complexity can dren (grade 3) demonstrated lower test-retest correlations
result in substantial variations in handwriting performance than older children (grade 6). They concluded that older
from day to day for the same child (Herrick, 1960). In children generally show more consistency in handwriting
addition, children referred to occupational therapy are legibility over time than do younger children. Because of
usually suspect of an unstable or immature neurological the high referral rate of early elementary school children
system, which can add even more discrepancy berween to occupational therapy, the variability of their perfor-
their handwriting performances over time. In the current mance, and the target population of the ETCH-M, first-
srudy, an effort to minimize the influence of environmen- grade and second-grade students were selected for the
tal factors was made by controlling the time of day, day of current study.
week, and test location. During testing and retesting, a few participants ap-
Fourth, in the retest session, some participants ap- peared to have difftculty with completing the composi-
peared to begin the test before listening to the complete tion task. Several were unable to produce words in a sen-
directions, and a few resisted performing the retest. These tence format, and three refused to attempt the task,

The American Journal ofOccupational Therapy 253

Downloaded from http://ajot.aota.org on 03/29/2019 Terms of use: http://AOTA.org/terms
 claiming that it was "too hatd." Although sentence com- Directions for Future Research
position is commonly required in the classroom, these
Replication of this study with a larger sample would
participants' inability to produce words could have affect-
improve generalizability. A comparison study is needed of
ed teSt-retest reliability outcomes. Total scores were com-
the test-retest reliabilities for first-grade students with
pared with and without the composition task and are
those for second-grade students on the ETCH-M because
reported in Table 4. The correlation coefficients did not
of the apparent variability in early elementary school years.
vaty sufficiently in the combined first-grade and second-
A reliability study of the cursive segment of the ETCH
grade population to support the hypothesis that the sen-
also is needed. Possibly another test-retest reliability study
tence composition task lowered the total scores.
could address the legibility components (i.e., spacing,
In addition to the legibility percentage scores exam-
horiwntal alignment, letter formation, size), the rate of
ined here, the ETCH also examines writing speed, the
writing, and the biomechanical aspects of handwriting as
impact of legibility components (i.e., spacing, size, hori-
assessed by this tool.
wntal alignment, letter formation) on readability, pencil
manipulation, hand preference, grasp pattern, pencil Conclusion
management, and classroom observation. Recommen- Findings from this study suggest that caution should be
dations for intervention evolve from a combination of used when reporting legibility percentages of the ETCH-
these factors. Although scores from a standardized test are
M, specifically with first-grade and second-grade students
useful, the occupational therapist gathers information
with identified handwriting deficits. As with other hand-
from several sources before recommending intervention.
writing assessment tools, test-retest reliability is lower
than desired for pediatric standardized tests. The com-
Clinical Implications
plexity of handwriting for early elementary school chil-
When evaluating a student's handwriting, it is important dren, the subjectivity of readability, and the high variabili-
to consider ETCH legibility scores as only one aspect of a ty of performance among children with immature or
comprehensive evaluation. Direct classroom observation unstable neurological systems may contribute to the insta-
of the student involved in a writing assignment, parent bility of handwriting performance over time. Percentage
and teacher discussions, classroom samples, and an educa- scores for total letter legibility, total word legibility, and
tional cumulative file review are imperative to view the uppercase letter legibility were more stable than total
student's occupation of writing in context. The use of the numeral legibility scores and legibility for the other indi-
ETCH and other data-gathering techniques may allow vidual tasks. Therapists are encouraged to use total letter
the team to gain a comprehensive pierure of a student's and total word percentages as the most reliable scores over
handwriting abilities as well as formulate a blueprint for time for the combined first-grade and second-grade pop-
intervention. ulation. To reflect "best pediatric practice," standardized
testing should be combined with contextual data-gather-
Study Strengths and Limitations ing techniques....
This study had several strengths. First, the sample repre-
Acknowledgments
sented the target population of the ETCH: children with
We thank the srudems, parems, teachers, and occupational therapists
handwriting problems. Second, the sample consisted of a of the Bellevue and Seattle Public School Districrs who assisted and
heterogeneous ethnic mix. Third, time of day, test loca- parricipared in the study. This scudy was supporred, in parr, by Gram
tion, and a 7-day interval were consistent for test and No. 84.029F, "Preparation of Related Service Personnel: Preservice
Training of Occupational Thetapy to Provide Services to Childten
retest for each participant. Fourth, one person adminis- wirh Emotional and Behavioral Disordets," ftom the U.S. Deparrmem
tered and scored the tests to maintain consistency. Fifth, of Education, Office of Special Education and Rehabilitative Services.
steps also were taken to minimize score variance due to This study was conducred in partial fulfillmell[ of the first
aurhor's requirements for a master of science degree, Deparrmem of
rater error. When the primary investigator scored each test,
Rehabiliration Medicine, Universiry of Washington, Seattle, Wash-
she was blind to the participant's identity and whether a ingron.
particular score sheet was from a test or retest administra-
tion. Additionally, she rechecked scoring competency peri- References
odically throughout the study. Amundson, S. J. (1995). Evaluation tool ofchildren '5 handwrit-
The limitations ro the study were the small sample ing. Homer, AK: OT KlDS.
size and small number of school distriers used. However, Anastasi, A. (1988). Psychological testing. New York: MacMillan.
Baumgartner, T. (1989). Notm-reference measuremem: Relia-
the ethnic mix and proportionally high number of class-
biJiry. In M. Safrit & T. Wood (Eds.), Measurement concepts in physi-
rooms from which children were chosen help to increase cal education and exercise science (pp. 45-72). Champaign, IL: Human
the generalizability of the results. Kinetics.

254
Downloaded from http://ajot.aota.org on 03/29/2019 Terms of use: http://AOTA.org/terms April 1998, Volume 52, Number 4
 Benbow, M., Hann, B., & Marsh, D. (1992). Handwriting in Herrick, V. (1960). Handwriting and children's wtiting. Ele-
the classroom: Improving written communication. In C. B. Royeen mentary English, 37, 248-256.
(Ed.), AOTA self-study series: Classroom applications for school-based MacArthut, c., & Graham, S. (1987). Learning disabled stu-
practice. Rockville, MD: American Occupational Therapy Association. dents' composing with three merhods: Handwriting, dictation, and
Bonder, B. R. (1989). Planning the initial version. In L. J. Miller word processing. Journal ofSpecial Education, 21(3), 22--42.
(Ed.), Developing nonn-referenced standardized tests (pp. 15--42). New McHale, K., & Cermak, S. A. (1992). Fine motor activities in
York: Haworth. e1ementaty school: Preliminary findings and provisional implications
Briggs, D. (1970). The influence of handwriting on assessment. for children with fine motor problems. American Journal of Occupa-
Educational Research, 13(]), 50-55. tional Therapy, 46, 898-903.
Briggs, D. (1980). A study of the influence of handwriting upon Oliver, C. E. (1990). A sensorimoror program for improving
grades using examination scripts. Educational Review, 32, 185-193. wriring readiness skills in elementary-age children. American Journal of
Goldstein, P., & Tupper, D. (1987). Quantitative and qualita- Occupational Therapy, 44, 111-116.
tive measurement of subtle neurobehavioral deficit. In D. Tupper Reisman,]. E. (1991). Brief or New-Poor handwriting: Who
(Ed.), Soft neurological signs (pp. 45-69). New York: Grune & Strat- is referred) American Journal ofOccupational Therapy, 45, 849-852.
ton. Reisman, J. E. (1993). Development and reliability of rhe
Graham, S. (1986). A review of handwriring scales and factors research version of rhe Minnesota Handwriting Test. Physical and
rhat contribute ro variability in handwriting scores. Journal ofSchool Occupational Therapy in Pediatrics, 13(2),41-55.
Psychology, 24(1),63-71. Tseng, M. H., & Cermak, S. A. (1991). The evaluarion of
Graham,S., & Miller, L. (1980). Handwriring research and handwriting in children. Sensory Integration Quarterly, 19(4),2-6.
practice: A unified approach. Focus on E'(ceptional Children, 13(2), Ziviani, J. (1995). The development of graphomotor skills. In
1-16. A. Henderson & C. Pehoski (Eds.), Hand function in the child: Foun-
Hasbrouck, J. E., Tindal, G., & Parker, R. I. (J 994, Winrer). datiom/orremediation (pp. 184-193). St. Louis, MO: Mosby.
Objecrive procedures for scoring student's wriring. Teaching Excep- Ziviani, J., & Elkins, J. (J 984). An evaluation of bandwriting
tional Children, 18-22. performance. EduCiltional Review, 36, 249-261.

The Essence Of play:

A-Child Ocoupa Ion •

Edited by Barbara £ Chandler. MOT, orR

This interesting explanation of the nature of play includes theoretical and practical applfGations for occupational
therapy practitioners. Individual chapters by experts help readers understand playas a universal occupation and
'how it can be used as a therapeutic medium for infants and children. Presents play from a neurological Viewpoint
as well as an anthropological perspective written by well-known anthropOlogists Wendy Wood, PhD, OTR/L, and
Reba Anderson, PhD, OTR, FAOTA. 220 pages, indexed, 1997.

Table of Contents
• Playas an Occupation and Indicator of Health • Play Things: Toypse, A'c<;essibiHty,andAdaptation
• Insights From the Play of Nonhuman Primates • Playmates: Social Interaction inEarlyand
• The Anthropological Study of Play Middle Childhood
• Infant Play • Play Environments: AnOccupatfonalTherapy
• What Happens When We Play? Perspective
A Neurodevelopmental Explanation • Play as Treatmenfand TreatnlentThro,ugh Play

Order #11 -$35 _AOTA member $45 nonmember ,

To order, call 1-800-SAY-AOlA (AOTA members),
301-652-2682 (nonmembers), or 1-800-377-8555 (TDD users).
Shipping and handling additional.
_ _
,."1,
A ' n '~A,T'"'''';'''''
Oc:cuJ»ttional TheraI')'
_ _ Assod"tion, Inc.

The American Journal ofOccupational Therapy 255

Downloaded from http://ajot.aota.org on 03/29/2019 Terms of use: http://AOTA.org/terms