Visit https://ebooknice.

com to download the full version and

explore more ebooks

(Ebook) Usability Testing of Medical Devices by

Michael E. Wiklund, P.E., Jonathan Kendler, Allison
Y. Strochlic ISBN 9781439811832, 1439811830

_____ Click the link below to download _____

https://ebooknice.com/product/usability-testing-of-
medical-devices-2088184

Explore and download more ebooks at ebooknice.com

Here are some recommended products that might interest you.
You can download now and explore!

(Ebook) Usability testing of medical devices by Kendler, Jonathan;

Strochlic, Allison Y.; Wiklund, Michael E ISBN 9781466595897,
1466595892

https://ebooknice.com/product/usability-testing-of-medical-
devices-5394112

ebooknice.com

(Ebook) Biota Grow 2C gather 2C cook by Loucas, Jason; Viles, James

ISBN 9781459699816, 9781743365571, 9781925268492, 1459699815,
1743365578, 1925268497

https://ebooknice.com/product/biota-grow-2c-gather-2c-cook-6661374

ebooknice.com

(Ebook) Matematik 5000+ Kurs 2c Lärobok by Lena Alfredsson, Hans

Heikne, Sanna Bodemyr ISBN 9789127456600, 9127456609

https://ebooknice.com/product/matematik-5000-kurs-2c-larobok-23848312

ebooknice.com

(Ebook) SAT II Success MATH 1C and 2C 2002 (Peterson's SAT II Success)

by Peterson's ISBN 9780768906677, 0768906679

https://ebooknice.com/product/sat-ii-success-
math-1c-and-2c-2002-peterson-s-sat-ii-success-1722018

ebooknice.com
(Ebook) Master SAT II Math 1c and 2c 4th ed (Arco Master the SAT
Subject Test: Math Levels 1 & 2) by Arco ISBN 9780768923049,
0768923042

https://ebooknice.com/product/master-sat-ii-math-1c-and-2c-4th-ed-
arco-master-the-sat-subject-test-math-levels-1-2-2326094

ebooknice.com

(Ebook) Cambridge IGCSE and O Level History Workbook 2C - Depth Study:

the United States, 1919-41 2nd Edition by Benjamin Harrison ISBN
9781398375147, 9781398375048, 1398375144, 1398375047

https://ebooknice.com/product/cambridge-igcse-and-o-level-history-
workbook-2c-depth-study-the-united-states-1919-41-2nd-edition-53538044

ebooknice.com

(Ebook) Designing Usability into Medical Products by Wiklund, Michael

E. ISBN 9780849328435, 9781420038088, 0849328438, 1420038087

https://ebooknice.com/product/designing-usability-into-medical-
products-4647436

ebooknice.com

(Ebook) User Interface Requirements for Medical Devices: Driving

Toward Safe, Effective, and Satisfying Products by Specification by
Michael Wiklund; Erin Davis; Alexandria Trombley ISBN 9780367457938,
0367457938
https://ebooknice.com/product/user-interface-requirements-for-medical-
devices-driving-toward-safe-effective-and-satisfying-products-by-
specification-38322854
ebooknice.com

(Ebook) Handbook of Human Factors in Medical Device Design by

Matthew Bret Weinger, Michael E. Wiklund, Daryle Jean Gardner-Bonneau
ISBN 9780805856279, 9781420063516, 0805856277, 1420063510

https://ebooknice.com/product/handbook-of-human-factors-in-medical-
device-design-2395734

ebooknice.com
Usability Testing of Medical Devices 1st Edition Michael
E. Wiklund Digital Instant Download
Author(s): Michael E. Wiklund, P.E., Jonathan Kendler, Allison Y. Strochlic
ISBN(s): 9781439811832, 1439811830
Edition: 1
File Details: PDF, 9.44 MB
Year: 2010
Language: english
 USABILITY TESTING
OF MEDICAL DEVICES
 USABILITY TESTING
OF MEDICAL DEVICES

MICHAEL E. WIKLUND
JONATHAN KENDLER
ALLISON S. YALE

Boca Raton London New York

CRC Press is an imprint of the

Taylor & Francis Group, an informa business
CRC Press
Taylor & Francis Group
6000 Broken Sound Parkway NW, Suite 300
Boca Raton, FL 33487-2742

© 2011 by Taylor and Francis Group, LLC

CRC Press is an imprint of Taylor & Francis Group, an Informa business

No claim to original U.S. Government works

Printed in the United States of America on acid-free paper

10 9 8 7 6 5 4 3 2 1

International Standard Book Number: 978-1-4398-1183-2 (Hardback)

This book contains information obtained from authentic and highly regarded sources. Reasonable efforts have been made
to publish reliable data and information, but the author and publisher cannot assume responsibility for the validity of all
materials or the consequences of their use. The authors and publishers have attempted to trace the copyright holders of all
material reproduced in this publication and apologize to copyright holders if permission to publish in this form has not
been obtained. If any copyright material has not been acknowledged please write and let us know so we may rectify in any
future reprint.

Except as permitted under U.S. Copyright Law, no part of this book may be reprinted, reproduced, transmitted, or utilized in
any form by any electronic, mechanical, or other means, now known or hereafter invented, including photocopying, micro-
filming, and recording, or in any information storage or retrieval system, without written permission from the publishers.

For permission to photocopy or use material electronically from this work, please access www.copyright.com (http://www.
copyright.com/) or contact the Copyright Clearance Center, Inc. (CCC), 222 Rosewood Drive, Danvers, MA 01923, 978-750-
8400. CCC is a not-for-profit organization that provides licenses and registration for a variety of users. For organizations that
have been granted a photocopy license by the CCC, a separate system of payment has been arranged.

Trademark Notice: Product or corporate names may be trademarks or registered trademarks, and are used only for identi-
fication and explanation without intent to infringe.

Library of Congress Cataloging‑in‑Publication Data

Wiklund, Michael E.
Usability testing of medical devices / Michael E. Wiklund, Jonathan Kendler, Allison Y. Strochlic.
p. ; cm.
Includes bibliographical references and index.
Summary: “Informative, practical, and engaging, this handbook covers how to conduct usability
tests of medical devices. Recognizing that the intended readers, including marketers, engineers, and
regulatory affairs specialists, are busy and disinclined to read lengthy textbooks, this book has been
carefully designed to be concise and visual, allowing readers to read it all in one sitting or jump from
one section to another as needed. The book provides a general understanding of usability testing and
reviews key concepts. It highlights the challenges of validating that protects against dangerous errors
that could lead to patient injury and death”--Provided by publisher.
ISBN 978-1-4398-1183-2 (hardcover : alk. paper)
1. Medical instruments and apparatus--Testing. I. Kendler, Jonathan. II. Strochlic, Allison Y. III.
Title.
[DNLM: 1. Equipment and Supplies. 2. Materials Testing. W 26 W663u 2011]

R856.4.W55 2011
610.28’4--dc22 2010025006

Visit the Taylor & Francis Web site at

http://www.taylorandfrancis.com

and the CRC Press Web site at

http://www.crcpress.com
Contents
Acknowledgments......................................................................................................xi
How to Use This Book............................................................................................ xiii
The Limitations of Our Advice................................................................................ xv
Who Could Use This Book?...................................................................................xvii
About the Authors....................................................................................................xix

Chapter 1 Introduction........................................................................................... 1
What Is Usability Testing?.................................................................... 2
What Is a Medical Device?...................................................................7
Class I: General Controls.................................................................8
Class II: Special Controls.................................................................9
Class III: Premarket Approval..........................................................9
Why Conduct Usability Tests of Medical Devices?............................ 10
What Are Common Regulator Comments on Test Plans?.................. 12
Is Usability Testing of Medical Devices Required?............................ 16
Do You Have to Test Minor Design Changes?.................................... 19
How Do You Defend Usability Testing Methods to Market
Researchers?........................................................................................ 21
Notes.................................................................................................... 22

Chapter 2 Risk Management and Usability Testing............................................ 25

What Is the Relationship between Usability Testing and Risk
Management?......................................................................................26
Can Usability Testing Identify Use-Related Hazards?........................ 28
What Is a Dangerous Use Error?......................................................... 30
Is Usability Testing a Reliable Way to Assess the Likelihood
That a Dangerous Use Error Will Occur?........................................... 35
Notes.................................................................................................... 36

Chapter 3 The Commercial Imperative............................................................... 39

How Does Testing Affect the Development Schedule?.......................40
Does Usability Testing Offer Liability Protection?............................. 43
Can You Develop Marketing Claims Based on Test Results?.............46
Note..................................................................................................... 48

Chapter 4 Testing Costs....................................................................................... 49

What Should a Request for Quotation for Usability Testing Include?.... 50
What Does a Usability Test Cost?....................................................... 54
What Is the Return on Investment?.....................................................60

v
vi Contents

Chapter 5 Anatomy of a Usability Test................................................................ 63

What Are the Common Elements of a Usability Test?.......................64
What Is the Proper Duration of a Test Session?.................................. 70
Do You Have to Be a Usability Specialist to Conduct a Test?............ 73
Does It Take a “Brain Surgeon” to Evaluate Medical Devices?......... 75
Why Test if You Cannot Change the Design?..................................... 79
How Do You Set Expectations?.......................................................... 81
What Can Postpone a Usability Test?.................................................84
Note..................................................................................................... 88

Chapter 6 Types of Tests...................................................................................... 89

What Is the Difference between Formative and Summative
Usability Testing?................................................................................90
What Is a Benchmark Usability Test?................................................. 93
What Is an “Out-of-the-Box” Usability Test?..................................... 96
Can a Test Session Include More Than One Participant?................... 98
Can You Conduct a Group Test?....................................................... 101
How Do You Conduct a “Quick-and-Dirty” Usability Test?............ 104
Notes.................................................................................................. 105

Chapter 7 Writing a Test Plan............................................................................ 107

What Should a Test Plan Include?..................................................... 108
Does Usability Matter to Regulators?............................................... 110
Do Usability Test Plans Require Institutional Review Board
Approval?.......................................................................................... 114
How Do You Protect Intellectual Property?..................................... 118
During Test Planning.................................................................... 118
During Recruiting........................................................................ 118
During the Usability Test............................................................. 118
Notes.................................................................................................. 119

Chapter 8 Choosing a Participant Sample and Recruiting Participants............ 121

What Is an Appropriate Sample Size?............................................... 122
Can Advisory Panel Members Play a Role in Usability Tests?......... 124
Should Children Participate in Usability Tests?................................ 126
Should Seniors Participate in Usability Tests?.................................. 129
How Do You Conduct a Usability Test Involving People with
Impairments?..................................................................................... 132
How Do You Recruit Test Participants?............................................ 137
Set an Appropriate Compensation Level...................................... 137
Ensure a Good Cross-Section....................................................... 138
Make the Activity Sound Worthwhile.......................................... 139
Avoid Frauds................................................................................. 139
Contents vii

How Do You Recruit Physicians?..................................................... 141

How Do You Recruit Nurses?........................................................... 143
How Do You Prevent No-Shows?...................................................... 145
How Do You Recruit Laypersons?.................................................... 147
Notes.................................................................................................. 149

Chapter 9 Test Environments............................................................................. 151

What Is the Benefit of Testing in a Medical Environment
Simulator?......................................................................................... 152
How Do You Test in Actual Use Environments?.............................. 156
Should You Test in a Participant’s Workplace?................................. 160
Can You Conduct a Usability Test over the Web?............................. 164
Can You Test a Device While It Is in Actual Use?........................... 168
What if a “Device” Cannot Be Moved?............................................ 170

Chapter 10 Adding Realism................................................................................. 173

Why and How Do You Distract Test Participants?........................... 174
What Use Is a Mannequin?............................................................... 177
What Role Can a Standardized Patient Play?................................... 181
How Do You Simulate Invasive Procedures?.................................... 183
How Do You Simulate Blood?.......................................................... 186
How Do You Simulate Skin and Injections?..................................... 189
How Do You Simulate Impairments?............................................... 192
How Do You Simulate Hardware Interactions?................................ 197
How Do You Simulate Other Medical Devices?............................... 199
Notes.................................................................................................. 201

Chapter 11 Selecting Tasks.................................................................................. 203

Do You Have to Test Everything?.....................................................204
What Tasks Should Test Participants Perform?................................206
Why Focus on Potentially Dangerous Tasks?...................................209
How Do You Choose Tasks When Evaluating Use Safety?.............. 211
Should Tests Include Maintenance and Service Tasks?.................... 213
Can You Test Long-Term Usability?................................................. 215
How Do You Test Alarms?................................................................ 218
How Do You Test Warning Labels?.................................................. 220
How Do You Test Instructions for Use?............................................ 223
How Do You Test Symbols?.............................................................. 226
How Do You Test Legibility?............................................................ 229
How Do You Evaluate Packaging?.................................................... 235
How Do You Test the Appeal of a Device?....................................... 238
Notes..................................................................................................240
viii Contents

Chapter 12 Conducting the Test........................................................................... 243

What Is the Value of Pilot Testing?...................................................244
Who Should Observe the Test Sessions?...........................................246
What Kinds of Usability Problems Arise during a Usability Test?.. 250
What Can Go Wrong before, during, and after a Test?..................... 256
What Risk Do Test Personnel Assume?............................................ 259
Are There Times When the Testing Staff Should Be All Female
or All Male?....................................................................................... 262
Should User Interface Designers Conduct Usability Tests of
Their Own Designs?..........................................................................264
When and How Should You Assist Test Participants?......................266
Can You Modify a Test in Progress?................................................. 269
Can You Reliably Detect Use Errors?............................................... 272
Can You Give Test Participants Training?........................................ 274
Should You Provide Access to Learning Tools?............................... 278
Notes.................................................................................................. 281

Chapter 13 Interacting with Participants............................................................. 283

When Is It Appropriate to Ask Participants to Think Aloud?..........284
What Is the Proper Way to Pose a Question?.................................... 287
Is There a Place for Humor in a Usability Test?............................... 289
How Do You Minimize Participant Fatigue?.................................... 291
How Do You Protect Participants from Harm?................................. 293
What If the Test Participant Gets Hurt?............................................ 296
Notes.................................................................................................. 298

Chapter 14 Documenting the Test........................................................................ 299

What Data Should You Collect?........................................................300
What Use Are Task Times?...............................................................304
What Is a Good Way to Video Record a Session?............................306
How Do You Video Record Participants’ Interactions with a
Moving Device?.................................................................................309

Chapter 15 Analyzing Test Data.......................................................................... 311

What Kind of Statistical Analyses Are Most Useful?....................... 312
Case 1........................................................................................... 312
Case 2........................................................................................... 312
Case 3........................................................................................... 312
Case 4........................................................................................... 313
Case 5........................................................................................... 313
How Do You Handle Outliers?.......................................................... 317
Note................................................................................................... 319
Contents ix

Chapter 16 Reporting Results.............................................................................. 321

What Makes a Good Test Report?.................................................... 322
Should Test Reports Include Design Recommendations?................. 326
Can Usability Test Results Be Misleading?...................................... 329
How Do You Deliver Bad News?...................................................... 332
Example 1..................................................................................... 333
Example 2..................................................................................... 333
How Do You Explain a Lack of Statistical Significance?................. 334
What Makes a Good Highlight Video?............................................. 336
Notes.................................................................................................. 338

Chapter 17 Validation Testing............................................................................. 339

How Does Design Validation Differ from Design Verification?.......340
Design Verification.......................................................................340
Design Validation.........................................................................340
Can a Clinical Trial Supplant Summative Usability Testing?.......... 342
Usability Evaluations during Clinical Use...................................344
Can You Conduct a Usability Test in Parallel with a Clinical
Trial?..................................................................................................346
Can You Conduct a Summative Usability Test without
Conducting a Formative Usability Test?........................................... 348
Notes.................................................................................................. 349
Resources............................................................................................................... 351
Books and Reports............................................................................ 352
U.S. Food and Drug Administration (FDA) Publications................. 352
Standards........................................................................................... 353
Web Sites........................................................................................... 353
Webinars on CD................................................................................ 353
U.S. Courses...................................................................................... 354
Tools.................................................................................................. 354
Index....................................................................................................................... 355
Acknowledgments
We interviewed the following human factors, engineering, and design profession-
als to help identify the topics we should address in the book (i.e., define the book’s
“user requirements”):

• Marianne Boschelli, human factors engineering specialist, User Interface IQ

• Jason Bush, human factors principal scientist, Roche Diagnostics
• Tony Easty, chair, Management Committee, Centre for Global eHealth
Innovation; assistant professor, University of Toronto; senior director, med-
ical engineering, University Health Network
• Torsten Gruchmann, chief executive officer, Use-Lab
• Peter Hegi, director of product management, St. Jude Medical
• Edward Israelski, human factors program manager, Abbott
• Wayne Menzie, director of technology and clinical development, Echo
Therapeutics
• Paul Mohr, principal engineer, Intuitive Surgical Incorporated
• Jennifer Nichols, senior product manager, Philips Healthcare
• Kyle Outlaw, staff engineer, Codman, a Johnson & Johnson company
• Gerald Panitz, lead system designer, Dräger
• Susan Proulx, president, Med-E.R.R.S.
• J. B. Risk, senior product manager, Joerns Healthcare
• Gary Searle, manager, research and development, BD Medical
• Mahesh Seetharaman, senior software engineer, Optiscan
• Eric Smith, director for user experience and design research, Eclipse
Product Development
• Anita Stenquist, manager—human factors engineering, Gambro Renal
Products
• Paul Upham, senior manager, BD Medical
• Matthew Weinger, director, Center for Perioperative Research in Quality,
Vanderbilt University

Stephanie Barnes (usability engineering consultant, Stephanie Barnes Inc.) served

the essential and demanding role of “alpha” reader, reviewing all of the content of
the book with an eye toward enhancing its usefulness and readability as well as giv-
ing our opinions a sanity check.
Our workmates Jon Tilliss, Maya Jackson, and Peter Carstensen helped us
administer numerous usability tests that ultimately led to the insights that we share
in this book.
Many past and present clients granted us permission to use photos appearing in
this book from the usability tests we conducted on their behalf (all photos without
source lines were provided and copyrighted by Wiklund Research and Design or
the authors).

xi
xii Acknowledgments

Michael Slaughter of CRC Press gave the project the green light, recognizing
the potential of a book focused squarely on the needs of medical device developers
facing the challenge of conducting effective usability tests. Jessica Vakili of Taylor
& Francis provided us with excellent direction and editorial support.
Last and most important, our beloved families and friends gave us the encourage-
ment and free time to write this book.
Thanks to all of you.

Michael, Jonathan, and Allison

How to Use This Book
This book does not seek to replace other good
works on the subject of usability testing, such
as Dumas and Redish’s A Practical Guide to
Usability Testing (1999), Rubin and Chisnell’s
Handbook of Usability Testing (2008), or the
helpful usability Web site of the U.S. gov-
ernment (http://www.usability.gov). Rather,
it seeks to help readers take what they might
have already learned about usability testing
from other resources and tailor it to the evalua-
tion of medical devices and software.
As human factors specialists who have
conducted literally thousands of test sessions
involving medical devices used by physicians,
nurses, therapists, technicians, and patients,
we believe we have some important lessons
and tips to share. Therefore, we wrote the kind
of book we would have liked to use when we
started testing medical devices, which explains
why it has so many pictures and keeps things
simple.
We doubt that many will choose to read
the book cover to cover in a marathon session,
such as one might consume a Danielle Steele
or Stephen King novel. There is no protagonist,
antagonist, or surprise ending. The book simply
tries to answer the myriad questions that medical device manufacturers face when
they test the usability of their devices, and we do so in an orderly, readable manner.
There is no story to spoil if you want to jump among the topics.
That said, we present the content in a reasonably logical order. It starts with a cur-
sory review of human factors engineering and how usability testing fits in this area.
It continues with a review of the government regulations and industry standards that
have motivated many medical device manufacturers to conduct usability tests. Then,
the book covers the nitty-gritty of planning, conducting, and reporting the results of
a usability test.
As you read the book, keep in mind that usability tests are like snowflakes, mean-
ing that each is unique. One hundred usability specialists working independently
could take 100 different approaches to testing a dialysis machine, for example. Of
course, their approaches would have considerable methodological overlap, but there
would also be meaningful differences in approach that the practitioners would ener-
getically defend as the best given the circumstances.

xiii
xiv How to Use This Book

So, we suggest drawing as much insight as possible from this book and other
resources and confidently approaching usability testing in your own unique way.
After all, the point is not to conduct an academically perfect usability test. Instead,
the point is to collect the best possible insights from a usability test so that you and
your development team can make your medical device as safe, effective, and appeal-
ing as possible.
The Limitations of Our Advice
This book offers our best and most sincere advice on a wide range of usability testing
topics, and advice is the key word. This is not a physics textbook replete with prov-
able laws and equations. While force is demonstrably equal to the mass of an object
times its acceleration (F = ma), the field of human factors lacks an equivalently exact
means to calculate usability. Consequently, our advice is hardly the last word on any
particular topic. Instead, consider it a starting point or a complement to other usabil-
ity specialists’ opinions and your own opinions and judgment.
Our suggestions and recommendations stem from over 35 combined years of
usability testing experience. However, we recognize that our professional colleagues
might have different experiences and consider some of our advice controversial or
even dead wrong. This is the nature of any text that tries to share knowledge on a
substantially subjective topic that has been the focus of decades rather than centuries
of study and practice.
Please recognize that some of our advice has a limited shelf life. Regulations and
accepted practices pertaining to usability testing of medical devices and software
are likely to change over time, thereby making our advice dated. So, please check
our recommendations against the most up-to-date requirements. We developed this
book’s content from 2008 to 2010.
Here are a few more legal statements intended to protect you, us (the authors), and
the publisher:

• Readers who choose to use the information and recommendations provided

in this book do so at their own risk and discretion.
• The authors and publisher make no warranties, express or implied, regard-
ing the information and recommendations contained in this book.
• Under no circumstances shall the reader hold the authors or publisher
responsible for any damage resulting from the application of the informa-
tion and recommendations contained in this book.

With these disclaimers behind us, we hope you enjoy our book and find its con-
tents helpful, applicable, and thought provoking.

xv
Who Could Use This Book?
This book should be a good resource if you have an interest, need, or direct role in
conducting a usability test of a medical device (or system) or if you are presently
studying the topic. Here is a potential list of professionals and role players who might
find themselves in such a position:

• Biomedical engineers or biomedical technicians

• Cultural anthropologists
• Electrical engineers
• Ethnographers
• Human factors engineers, usability specialists, ergonomists
• Industrial designers, product designers
• Industrial engineers, manufacturing engineers
• Information architects
• Instructors and students
• Marketing researchers, marketing managers
• Mechanical engineers
• Medical device inventors
• Medical device regulators
• Program managers, program planners
• Purchasing managers
• Regulatory affairs specialists
• Risk managers
• Software user interface programmers
• Technical writers
• User interface designers, user interface experience planners

xvii
About the Authors
The authors are colleagues at Wiklund Research & Design Incorporated (Concord,
MA, USA), a consulting firm that provides user research, user interface design, and
evaluation services primarily to medical device manufacturers. Each author is formally
trained in human factors engineering (HFE) and frequently conducts usability tests of
medical devices and software to identify opportunities for design improvement and val-
idate their use safety for regulatory approval purposes. In 2008, the authors foresaw the
need for detailed guidance on how to conduct usability tests of medical devices, noting
the sharp global increase in the number of companies that have chosen to focus more
attention on HFE and are compelled to practice it to meet regulators’ expectations.

Michael E. Wiklund
Michael has worked in the HFE profession for over 25
years as a consultant and educator.
He received his master’s degree in engineering design
(specializing in HFE) from Tufts University, where he
has subsequently taught user interface design for over 20
years. He has a professional engineering license and is a
board certified human factors professional.
He joined the profession in the mid-1980s, a time
when microprocessor technology started to change
the fundamental nature of medical technologies.
Originally trained to make machines safe and user
friendly, his early work was focused on “knobs and dials” but soon transitioned to
making software user interfaces more comprehensible to users. Today, he helps opti-
mize the design of hardware, software, and hybrid devices as well learning tools,
such as quick reference guides, user manuals, and online resources.
In 1997, the U.S. Food and Drug Administration (FDA) invited Michael to write
a guide to applying HFE in medical device development in a manner that was con-
sistent with the (then) new guidance of the agency on the topic. Later, the FDA
provided the guide to the Human Factors Engineering Committee of the Association
for the Advancement of Medical Instrumentation (AAMI), which used it as a basis
for writing AAMI HE74:2001, Human Factors Design Process for Medical Devices
Development. AAMI HE74:2001 then became the basis for the current standard of
the International Electrotechnical Commission (IEC) on the topic (IEC 62366).
In 2005, Michael cofounded Wiklund Research & Design Incorporated with the
goal of providing comprehensive HFE services to industry—medical device manu-
facturers in particular. In the ensuing years, the firm has provided user research, user
interface development, and usability testing services to over 50 clients based in mul-
tiple countries. Wiklund Research & Design has also helped its clients plan and build
human factors programs, delivered workshops on HFE-related topics, and served as

xix
xx About the Authors

an expert witness on medical use error-related cases. As the president of the com-
pany, Michael seeks to deliver cutting-edge user interface research and design ser-
vices and leads many user research-and-design evaluation projects each year.
Michael’s books include Usability in Practice (editor), Medical Device and
Equipment Design, Designing Usability into Medical Products (coauthor), and
Handbook of Human Factors in Medical Device Design (co-editor). He has pub-
lished over 60 articles in Medical Device & Diagnostic Industry (MD&DI) maga-
zine that promote the application of HFE in medical device development and provide
practical tips. He has been an invited speaker at multiple professional conferences
and universities, where he has described HFE as an imperative in the medical indus-
try and a path toward ensuring device safety and commercial success owing to its
effectiveness, usability, and appeal.
He has served as a voting member of the AAMI Human Factors Engineering
Committee for over 15 years. He has also served on the Human Factors Committee of
the IEC and as chair of the Industrial Designers Society of America, Medical Section.

Jonathan Kendler
Jonathan has worked in the HFE profession since
receiving his bachelor of fine arts degree in visual
design from the School of the Museum of Fine Arts,
Boston. He received his master’s degree in human fac-
tors in information design from Bentley College (now
Bentley University). Accordingly, he brings a strong
artistic sensibility to his HFE work.
Also a cofounder of Wiklund Research & Design
Incorporated, Jonathan has a strong interest in ensur-
ing the usability of medical technology. As the design
director of the company, he is routinely involved in
developing “clean sheet” user interfaces for medical devices as well as enhanc-
ing existing designs that need “refreshing.” Clients characterize his user interface
designs as “intuitive and attractive,” bringing attention to critical information and
controls. His design portfolio includes medical devices ranging from small, hand-
held devices to room-size diagnostic scanners.
Virtually all of Jonathan’s user interface design work is informed by user research
and formative usability testing, which he often conducts personally to get close to
the intended users and deeply understand opportunities for design improvement. He
believes that this level of active involvement by a user interface designer in evaluating
personal work is beneficial but requires absolute discipline to maintain objectivity.
Since 2006, Jonathan has co-taught applied software user interface design at
Tufts University and delivered HFE workshops to medical and nonmedical clients.
In 2009, he delivered major segments of a well-attended AAMI-sponsored webinar
on HFE in medical device development.
About the Authors xxi

Allison Y. Strochlic
Allison received her bachelor of science degree in HFE
from Tufts University. She joined Wiklund Research
& Design shortly after receiving her degree and now
serves as a managing human factors specialist. She
has continued her HFE studies at Bentley University.
Allison has accumulated literally thousands
of usability testing hours, most involving medical
devices. She has a passion for making participants
feel at ease during a usability test, enabling them to
perform tasks as naturally as possible so that she and
her colleagues can identify the strengths and short-
comings of a medical device, revealing opportunities for design improvement. Her
usability testing projects have taken her all across the United States as well as to
Europe and Asia. As such, she has become particularly adept at extracting useful
findings from test sessions involving interpreters as well as test sessions conducted
remotely (i.e., via telephone or the Web).
Allison has been active in the New England chapter of the Human Factors and
Ergonomics Society, serving most recently on its board. She has delivered multi-
ple presentations to industry and academic audiences on effective usability testing
methods.
1 Introduction

1
2 Usability Testing of Medical Devices

What Is Usability Testing?

Usability testing calls for representative users to perform representative tasks as a
means to reveal the interactive strengths and opportunities for improvement of a
device. You can think of the activity as pressure testing or debugging the user interface
of a device in terms of how it serves the users’ needs, a critical need being safe opera-
tion. Tests may focus on early design concept models, more advanced prototypes, and
even production units. A two-person team usually collaborates to run test sessions
with one participant at a time. Good practice calls for preparing a detailed usability
test plan and report that can be added to the design history file of a device.

Usability testing is a means to determine whether a given medical device will

meet its intended users’ needs and preferences. By extension, it is a way to judge if a
medical device is either resistant to or vulnerable to dangerous use errors that could
lead to user or patient injury or death.
In its classic form, a usability test takes place in a special-purpose facility—a
usability test laboratory—where test administrators can direct test activities from
within one room while interested parties observe from an adjacent room via a one-
way mirror. In practice, however, you can conduct a usability test in a wide range
of environments, including equipment storage rooms, nurses’ lounges, conference
rooms, hotel suites, focus group facilities, medical simulators, and actual clinical
settings such as an operating room.
The purpose of any usability test is to have test participants perform tasks with
the given medical device, be it an early prototype, working model, production-equiv-
alent device, or marketable device. If the medical device were a patient monitor, test
participants might connect a simulated patient’s sensor leads to the monitor, print an

Figure 1.1 (See color insert following page 202.) A conventional usability testing lab
equipped with a one-way mirror.
Introduction 3

Figure 1.2 (See color insert following page 202.) Scenes from usability tests of vari-
ous medical devices.
4 Usability Testing of Medical Devices

electrocardiogram (ECG) tracing, “shoot” a cardiac output measurement, and adjust

the systolic and diastolic blood pressure alarm limits. If the medical device were an
endoscope, test participants might place the endoscope into a simulated digestive
tract, move the scope through the esophagus and into the stomach and up to the pylo-
ric sphincter (valve), and then place the scope in a retrograde orientation to visualize
the lower esophageal sphincter. If the medical device were an insulin pump, test par-
ticipants might program a basal rate profile calling for different insulin delivery rates
at each hour of the day, look up the carbohydrate content of a baked potato, deliver an
eight-unit bolus before mealtime, and upload a month’s worth of data to a computer
for subsequent trend analysis. Importantly, the insulin pump would not be attached
to the test participant (as it would be to an end user, who is using the device to admin-
ister insulin). Rather, tasks involving insulin delivery would be simulated, and if the
participant needed to fill the device with insulin, inactive fluid such as saline or plain
water would typically be used in its place. As suggested by the examples, usability
testing of medical devices typically does not involve actual patients receiving treat-
ment or taking active medications.
While test participants perform tasks, test personnel—typically a test administra-
tor and note taker—observe intensively to determine how the medical device facili-
tates or hinders task completion. In addition to documenting observed use errors,
test personnel might record data such as task times, test participants’ comments, and
various subjective design attribute ratings, such as ease and speed of use (see “What
Data Should You Collect?” in Chapter 14).
If you are testing a fairly simple device, test sessions might breeze by in as little as
30 minutes. However, most test sessions last between one and two hours, providing
enough time to properly orient the test participant to the test environment, purposes,
and ground rules; to perform hands-on tasks; and to interview the test participant
about the strengths and opportunities for improvement of the design, for example.
A half-day test session is not unreasonable if the device under evaluation requires
one individual to perform an extensive number of tasks (e.g., unpacking, assembling,
calibrating, operating [in multiple modes], and servicing) (See “What Is the Proper
Duration of a Test Session?” in Chapter 5 for more information about determining
the appropriate test session length.)
Usability specialists (or allied professionals responsible for conducting the test)
write detailed test plans to guide effective, consistent, and objective design assess-
ments. After completing a test, analyzing the data, and developing findings, the test
administrator reports his or her findings with the required level of detail and formal-
ity. A sometimes-lengthy narrative test report that describes the purpose, approach,
and participants of the test and presents an analysis of the data, findings, and recom-
mendations is a common final product that medical device developers can add to
their design history file and submit to regulators.
Medical device developers are well served to conduct formative usability tests
“early and often” during device development to assess design alternatives and iden-
tify opportunities for design improvement. Later in the design process, developers
are essentially required to conduct a summative usability test to demonstrate that
their medical devices are safe to use from an interaction design standpoint. During
either type of test, users’ interactions with the given medical device might proceed
Introduction 5

Figure 1.3 (See color insert following page 202.) A sample user interface structure
with a task sequence shown.

smoothly, suggesting that the design is on the right track or even ready for market
introduction. Conversely, testing might reveal usability problems that could, should,
or must be corrected prior to the release of the device.
Usability tests usually involve a small number of test participants compared to
market research studies and clinical trials, for example. An informal test involving
just a few test participants can be productive. However, sample sizes in the range of
8 to 25 test participants are the norm (see “What Is an Appropriate Sample Size?” in
Chapter 8), the mode being around 12–15. No matter the population sample size, the
key is to get the right test participants. This means recruiting a sample of test partici-
pants who represent a good cross section of the people who will actually use the given
medical device. That said, usability specialists sometimes skew the sample so that it
includes an above-average proportion of people with limitations (i.e., impairments)
that could affect users’ ability to use the device. Skewing the sample in this way helps
usability specialists detect potentially hazardous use errors that unimpaired users
might not necessarily commit. Moreover, taking such an approach helps to determine
the accessibility and usability of a medical device by people with impairments.
All sorts of usability problems can arise during a usability test (see “What Kinds
of Usability Problems Arise during a Usability Test?” in Chapter 12). For example,
it is not unusual to see test participants go down the wrong path within a software
screen hierarchy because menu options are poorly worded or because information
and controls of interest are oddly placed. Sometimes, test participants get stuck on a
task because on-screen or printed instructions are incomplete, incorrect, or unclear.
Also, test participants might press the wrong button because they misinterpreted its
iconic label or because it was small and too close to other buttons.
Plenty of good things can happen during a usability test as well. For example, test
participants might correctly set up a device for use on their first try without train-
ing—a harbinger of good usability across the spectrum of possible user tasks. They
might execute a therapeutic procedure in the exact order prescribed by the on-screen
prompts. And, referring to a quick reference guide, test participants might properly
6 Usability Testing of Medical Devices

interpret an on-screen and audible alarm and quickly perform the troubleshooting
steps required to resolve the underlying problem.
Accordingly, usability testing is about discovering the good and bad aspects of
a user interface for the purposes of design refinement and validation. Programmers
might think of usability testing as a method of debugging a user interface from a
user interaction standpoint. Mechanical engineers might liken usability testing to
pressure testing or metaphorically dropping a user interface onto a concrete floor
from a considerable height. And, begging your pardon for one more comparison,
we liken usability testing a user interface to a doctor giving a patient a physical—an
inspection that usually shows most things are normal (i.e., in order) but highlights a
few areas for improvement.
Introduction 7

What Is a Medical Device?

A medical device is a product used to diagnose, treat, or monitor a medical condi-
tion. Given this broad definition, regulators group medical devices into different
classes based on the complexity and inherent potential of a given device to cause
patient harm. Depending on the class of a given medical device, more or less human
factors engineering will be warranted.

We all have a general understanding of the term medical device. A medical device
is something that physicians, doctors, nurses, technicians, and even laypersons use to
diagnose, treat, or monitor a medical condition. Moreover, we think of a device as a
physical item that might also incorporate a software user interface. Medical devices
vary widely in terms of their size and purpose.
A syringe and a magnetic resonance imaging (MRI) scanner are both medical
devices. So are exam gloves and cardiopulmonary bypass machines. However, as
will be discussed, medical devices fall into different classes. You can conduct a
usability test of virtually any medical device. However, manufacturers of Class II
and Class III devices are likely to invest more efforts into usability testing because
their devices have a greater potential to harm someone if operated improperly.
The Food and Drug Administration (FDA) defines a medical device as follows:
An instrument, apparatus, implement, machine, contrivance, implant, in vitro reagent,
or other similar or related article, including a component part, or accessory which is:
• recognized in the official National Formulary, or the United States Pharmacopoeia,
or any supplement to them
• intended for use in the diagnosis of disease or other conditions, or in the cure,
mitigation, treatment, or prevention of disease, in man or other animals
• intended to affect the structure or any function of the body of man or other ani-
mals, and which does not achieve any of its primary intended purposes through
chemical action within or on the body of man or other animals and which is not
dependent upon being metabolized for the achievement of any of its primary
intended purposes1

In Council Directive 93/42/EEC, the European Union offers the following definition:
“Medical device” means any instrument, apparatus, appliance, material or other arti-
cle, whether used alone or in combination, including the software necessary for its
proper application intended by the manufacturer to be used for human beings for the
purpose of:
• diagnosis, prevention, monitoring, treatment or alleviation of disease
• diagnosis, monitoring, treatment, alleviation of or compensation for an injury
or handicap
• investigation, replacement or modification of the anatomy or of a physiologi-
cal process
• control of conception
and which does not achieve its principal intended action in or on the human body by
pharmacological, immunological or metabolic means, but which may be assisted in its
function by such means.2
8 Usability Testing of Medical Devices

Figure 1.4 (See color insert following page 202.) Medical devices vary widely in terms
of shape, size, function, complexity, and usage. Photos (clockwise from top-left) courtesy of
Industrial Design Consultancy, 3M, David Ivison, BrokenSphere, HEYER Medical AG, and
Waisman Laboratory for Brain Imaging and Behavior.

The FDA recognizes three medical device classes:3

Class I: General Controls

“Class I devices are subject to the least regulatory control. They present minimal
potential for harm to the user and are often simpler in design than Class II or Class
III devices. Class I devices are subject to “general controls,” as are Class II and Class
III devices.
“General controls include:

1. Establishment of registration of companies, which are required to register

under 21 Code of Federal Regulations (CFR) Part 807.20, such as manufac-
turers, distributors, repackagers, and relabelers.
2. Medical device listing with FDA of devices to be marketed.
3. Manufacturing devices in accordance with the good manufacturing prac-
tices (GMP) in 21 CFR Part 820.
4. Labeling devices in accordance with labeling regulations in 21 CFR Part
801 or 809.
5. Submission of a premarket notification [510(k)] before marketing a device.

“Examples of Class I devices include elastic bandages, examination gloves, and

handheld surgical instruments. Most Class I devices are exempt from the premarket
notification and/or the GMP regulation.”
Introduction 9

Class II: Special Controls

“Class II devices are those for which general controls alone are insufficient to ensure
safety and effectiveness, and existing methods are available to provide such assur-
ances. In addition to complying with general controls, Class II devices are subject
to special controls. . . . Special controls may include special labeling requirements,
mandatory performance standards, and postmarket surveillance.
“Examples of Class II devices include powered wheelchairs, infusion pumps, and
surgical drapes.”

Class III: Premarket Approval

“Class III is the most stringent regulatory category for devices. Class III devices
are those for which insufficient information exists to ensure safety and effectiveness
solely through general or special controls.
“Class III devices are usually those that support or sustain human life, are of
substantial importance in preventing impairment of human health, or that present a
potential, unreasonable risk of illness or injury.
“Premarket approval is the required process of scientific review to ensure the safety
and effectiveness of Class III devices. Not all Class III devices require an approved
premarket approval application to be marketed. Class III devices that are equivalent to
devices legally marketed before May 28, 1976, may be marketed through the premar-
ket notification [510(k)] process until the FDA has published a requirement for manu-
facturers of that generic type of device to submit premarket approval data.
“Class III devices that require an approved premarket approval application to be
marketed are those:

1. Regulated as new devices prior to May 28, 1976, also called transitional
devices.
2. Devices found not substantially equivalent to devices marketed prior to
May 28, 1976.
3. Class III preamendment devices that, by regulation in 21 CFR, require a
premarket approval application.

“Examples of Class III devices that require a premarket approval include replace-
ment heart valves, silicone gel-filled breast implants, and implanted cerebella
stimulators.
“Class III devices that can be marketed with a premarket notification 510(k) are
those:

• Postamendment (i.e., introduced to the U.S. market after May 28, 1976)
Class III devices that are substantially equivalent to preamendment (i.e.,
introduced into the U.S. market before May 28, 1976) Class III devices and
for which the regulation calling for the premarket approval application has
not been published in 21 CFR
“Examples of Class III devices that currently require a premarket notification
include implantable pacemaker pulse generators and endosseous implants.”4
10 Usability Testing of Medical Devices

Why Conduct Usability Tests of Medical Devices?

Usability testing helps reveal opportunities to make medical devices easier, safer,
and more efficient and pleasant to use. These improved interactive qualities benefit
nearly everyone associated with a given medical device, especially the manufac-
turer, end user (i.e., caregiver), and patient.

The most profound reason to conduct usability tests of medical devices is to pro-
tect people from injury and death due to use errors. Too many people have been
injured or killed because someone pressed a wrong button, misread a number, mis-
placed a component, skipped a step, or overlooked a warning message when using a
medical device, for example. And, while usability testing will not catch every design
shortcoming that could lead to a dangerous use error, it will catch many of them.
Therefore, usability testing should be considered a moral imperative as well as a de
facto regulatory requirement and commercially advantageous.
Usability testing has many beneficiaries:

Manufacturers. Usability testing can lead to user interface design refine-

ments that are likely to increase device sales, engender customer loyalty,
reduce the demand for customer support (e.g., calls to a hotline), extend the
life span of a device, and reduce the chance of product liability claims. In
short, it is good for business.
Customers. Usability testing benefits customers such as hospitals, clinics, pri-
vate medical practices, and ambulance services in myriad ways. Easy-to-
use devices make workers more productive, improve worker satisfaction,
reduce training and support costs, and improve patient care.
Caregivers. Usability testing also benefits caregivers (e.g., physicians, nurses,
therapists, technicians, maintainers). Design improvements made as a result
of usability testing are likely to make a device easier to learn and use, reduce
the need for support, and empower caregivers to do their best work. Usable
devices can even speed up work and enable caregivers to go home on time.
Patients. Finally and most important, usability testing benefits patients because
they are less likely to be injured or killed by user interface shortcomings
that induce caregivers to err. Sadly, thousands of people die each year due

Figure 1.5 Usability testing benefits many people in many ways. Center photo courtesy
of Barwon Health.
Introduction 11

Why Does the FDA Suggest Conducting Usability Tests?

The U.S. FDA recognizes usability testing as one of the methods manufacturers should
use to generate design inputs and, moreover, to validate the design of a device. The FDA
dictated that “Design validation shall ensure that devices conform to defined user needs
and intended uses and shall include testing of production units under actual or simulated
use conditions.”6 The FDA further discussed the importance of human factors engineering
and usability testing in Do It By Design,7 a publication that defined usability testing as “a
test of either an actual device or an advanced prototype with a fully functional user inter-
face. Data obtained includes user performance (time, errors, and accuracy) and subjective
responses of test participants” (p. 42). Another FDA publication, Medical Device Use-Safety:
Incorporating Human Factors Engineering into Risk Management,8 described usability test-
ing as a tool to identify potential use-related hazards.

to medical errors involving devices. For example, infusion pump program-

ming errors (e.g., entering the number 80 instead of 8.0) have led to so many
deaths that the industry coined the expression “death by decimal.”5 The
application of human factors engineering and usability testing in device
development helps reduce the use error rate and limit the consequences of
use errors that do occur.

Another reason to conduct usability tests of medical devices—closely related to

preventing patient injuries and deaths—is to meet the device regulators’ expecta-
tions. In short, usability testing is the predominant means to validate that medical
devices meet users’ needs and are not subject to dangerous use errors. We address
this topic extensively in “What Is the Relationship between Usability Testing and
Risk Management?” in Chapter 2.
12 Usability Testing of Medical Devices

What Are Common Regulator Comments on Test Plans?

Regulators encourage medical device manufacturers to conduct usability tests, and
therefore prepare test plans, that focus on the riskiest user tasks. From a regulatory
perspective, the ideal test plan will raise confidence that the ensuing usability test will
reveal user interface design flaws that could lead to dangerous user errors, if any exist.
Test plans that effectively link usability testing and risk management instill such confi-
dence. Testing activities that are important but do not relate directly to device safety,
such as evaluations focused chiefly on usability and appeal, should be marked as such.

Medical device manufacturers might choose to seek feedback on their usabil-

ity test plans from regulators before proceeding with a summative usability test.
For example, the Food and Drug Administration (FDA) might review a usability
test plan on request and provide official comments via teleconference and letter,
for example. Undoubtedly, responding appropriately to the feedback increases the
chance that the regulatory agency will accept the revised usability testing approach.
Of course, accepting the testing approach has little to do with accepting the test find-
ings as evidence that the design is valid (i.e., safe for use).
Below is a sample of the feedback that manufacturers have received over the past
few years via discussions with and through letters from regulators. Note that we have
commingled comments on test plans and reports because they really address the same
methodology issues in either a prospective versus retrospective manner, respectively.
Caveat: We have paraphrased and, in some cases, expanded the feedback for clar-
ity sake. As such, the feedback is indirect and should not be regarded as regula-
tory policy. Moreover, various regulators might have different views on the issues
addressed. Therefore, you should regard the feedback presented below as simply
informative.

• Finding new use errors. Hypothesize the use errors that might occur dur-
ing each task and consolidate them into an inspection checklist that the
test administrators will use to evaluate participants’ interactions during the
usability test. Include the checklist as an appendix in the test plan.
• Prioritizing. Identify and prioritize directed tasks based on risk analysis
results.
• Relating tasks to risk analysis results. Create a table delineating the iden-
tified risks and associated, directed tasks to show that usability test partici-
pants will perform the riskiest tasks (i.e., tasks subject to use errors that are
most likely to cause harm). Also demonstrate that participants will perform
tasks that serve to assess the effectiveness of risk mitigations such as pro-
tective design features, labels, warnings, and instructions for use.
• Including secondary tasks. Testing should include tasks such as cleaning,
maintaining, and storing a device if these tasks are pertinent to the device’s
safe use.
• Describe how you will evaluate the critical aspects of user interactions
without having participants actually deliver or receive treatment using the
device.
Introduction 13

• Involving representative users. Describe how you will recruit a suffi-

ciently diverse sample of prospective users, including “worst-case users,”
such as marginally trained or even untrained users who might choose or be
directed to use the device, and users with certain impairments.
• Involving “low functioning” users. Include “low functioning” individu-
als in the user population sample. Recruiting only “high functioning”
individuals will not produce a representative sample of the intended user
population.
• Involving people with low language proficiency. Include individuals who
are less proficient in the device’s selected language (e.g., English), noting
that the device might be used by individuals who have low proficiency in
the selected language.
• Company employees serving as test participants. Avoid using company
employees as participants in usability test.
• Providing training. Fully explain the need for and nature of any training
that you plan to deliver to test participants.
• Providing prototype training. If a training program has not yet been
established, it is acceptable to deliver what you consider to be an appropri-
ate level of training.
• Access to training/learning materials. Test participants should be pro-
vided access to the training and instructional materials that would normally
be available to them in an actual use scenario.
• Allowing training benefits to decay. There should be a delay between
training and testing that might, in a realistic manner, result in some “decay”
in the knowledge and skills attained during training. The length of the delay
should be based on real-world use scenarios.
• Population sample size. Include an appropriate size sample from each user
group (e.g., ≥ 15 people per group for a summative usability test). Regulators
seem less concerned about test sample size, although a minimum of 15 to 25
participants appears to be a good working number, subject to increase if the
intended user population has segments with widely differing capabilities
and use the given device in distinctive ways (see “What Is an Appropriate
Sample Size?” in Chapter 8 for more information about selecting an appro-
priate sample size). Be sure your plan includes a sample size rationale.
Regulators also seem less concerned about the test team members’ usability
testing credentials and experience, focusing more attention on whether the
team is proposing a high-quality testing approach.
• Identifying outliers. Establish criteria for declaring a test participant as an
“outlier” (see “How Do You Handle Outliers?” in Chapter 15) whose data
should be excluded from posttest analyses. If providing participants with
training before the usability test, establish criteria for disqualifying a test
participant from participating in the subsequent usability test if he or she
is unable to use the given medical device. For example, if a nurse-trainer
determined that a current home dialysis patient—a candidate usability test
participant—would not be able to safely use a dialysis machine at home,
14 Usability Testing of Medical Devices

such an individual would not be an appropriate participant for a test of such

a device.
• Collecting data unrelated to use safety. Delineate the type of data you
plan to collect and how you will analyze it to draw conclusions regarding
the use safety of a given device. Be sure to differentiate between data you
are collecting for the sake of validation (e.g., observed use errors, anecdotal
comments related to device use safety) and to serve commercial interests
(e.g., subjective ease of use and satisfaction ratings).
• Tracking difficulties and close calls. In addition to describing how you
will detect and document use errors, describe how you will detect and
document operational difficulties and close calls (i.e., cases in which users
almost committed a user error).
• Value of subjective ratings. Ease of use ratings are supportive background
information but not—on their own—a basis for validation. Meanwhile,
subjective data such as ease of use ratings can help identify the occurrence
and nature of close calls.
• Value of clinical findings. Clinical test results are valuable but not a
replacement for usability test results. You need to conduct a usability test
that focuses specifically on use-related risks, and then (if appropriate, such
as in the case of infusion pumps) follow-up with usability studies conducted
in the context of clinical use.
• Value of task times. Tasks times are only relevant when the speed of task
performance is critical to safety, such as when a delay in treatment could
place a patient at risk.
• Focusing on production-equivalent devices. Summative testing should be
performed on a production-equivalent device, not an incomplete prototype
or computer-based simulation.
• Analyzing use failures. Summarize how use errors will be addressed en
route to determining if the device needs to be modified to reduce the likeli-
hood of associated risks to an acceptable level.
• Protecting human subjects. Outline how you will ensure human subjects
protection (see “How Do You Protect Participants from Harm?” in Chapter
13), including how you plan to protect participants from physical and emo-
tional harm, minimize risks to the participant, and deidentify the test data.
• Performing tasks accurately. Explain how the test environment, scenarios,
and directed tasks are reasonably representative of actual use conditions.
• Ensuring a realistic workflow. Specify tasks that participants can perform
following a realistic workflow rather than asking participants to perform
isolated steps in a potential distorted or deconstructed manner.
• Reporting results by user group. Test results should be segregated accord-
ing to user group.

When Should You Ask Regulators to Review a Draft Test Plan?

It is a good idea to ask regulators to review a draft test plan if it is the first time
you are conducting a usability test of a medical device. It is also helpful to have
Introduction 15

regulators review and comment on your test plan if (1) regulators have been dissat-
isfied with previous test plans or (2) if the upcoming usability test has particularly
high stakes and having to repeat it to address regulatory concerns would create com-
mercial jeopardy. If you seek regulators’ feedback, be sure to allot ample time in the
project schedule for the review (ask regulators to estimate their response time) and to
revise and resubmit the test plan, if necessary.
16 Usability Testing of Medical Devices

Is Usability Testing of Medical Devices Required?

International standards bodies have made usability testing a de facto requirement.
As a result, usability testing has become standard operating procedure among man-
ufacturers that develop medical devices. Failing to conduct usability tests en route
to a final design invites regulators to reject a manufacturer’s application for clear-
ance to bring the device to market, citing insufficient use safety data.

At the time this was written (2010), usability testing of medical devices was not
explicitly required by any government. Let us just say that it is strongly recom-
mended, and medical device manufacturers create considerable exposure to regula-
tory roadblocks and liability claims if they do not conduct one or more usability tests
during the medical device development process.
For many years now, usability specialists, regulatory bodies and their particular
guidance documents, and industry standards have promoted usability testing as the
chief means to ensure that medical devices meet users’ needs and do not induce
dangerous user errors. Usability testing is not the only way to judge the interactive
qualities of a device, so current regulatory and guidance documents do not come
right out and state that manufacturers must conduct a usability test per se. But, there
is a virtual mandate—a standard of care, if you prefer—to conduct usability tests.
Moreover, it is hard to imagine alternative ways to assess specific interactive medical
device qualities without asking representative users to perform tasks using a given
device. It would be like assessing the battery life of a device without turning the
device on and seeing how long it stays on.
The FDA infers the need for usability testing, without using the term, in its revised
GMP regulation, released on October 7, 1996. The Code of Federal Regulations states:

Design validation shall ensure that devices conform to defined user needs and intended
uses, and shall include testing of production units under actual or simulated use
conditions.14

On its Web site, the FDA describes the human factors relevance of the design
validation section of the CFR:

Human factors relevance: Design validation should be used to demonstrate that the
potential for use error that can lead to patient injury has been minimized. The regula-
tion requires testing the device under actual or simulated use conditions. Realistic use
conditions, therefore, should be carried out by test participants who represent a range
of typical intended users in terms of their ability to acquire information from, manipu-
late and maintain the device and understand the accompanying labeling.15

The FDA provided further encouragement to manufacturers to conduct usability

tests as a means of design validation in its guidance document, Medical Device Use-
Safety: Incorporating Human Factors Engineering into Risk Management:

Validation establishes that the device meets the needs of the intended users. The pri-
mary need of medical device users is the ability to use the devices safely and effectively
Introduction 17

under the actual use conditions. Applying usability testing approaches can directly
validate a user interface design.
For the purpose of validation, it is particularly important to use a production ver-
sion of the device,* representative device users, and actual or simulated use environ-
ments and to address all aspects of intended use. If small-scale iterative testing of
interface components was done adequately as the device was developed, it might not
be necessary for validation efforts to be extensive at the end of the design process.
However, some degree of testing of the entire system under realistic conditions with
representative users is warranted. In the alarm volume example, determining whether
users with moderate hearing loss can hear the alarm well enough to allow them to use
the device safely and effectively is the essential component of validation of this user
interface requirement (p. 29).16

In addition, the FDA published Do It by Design in December 1996; that provided

detailed guidance on how to conduct a usability test. The document stated:

Microprocessing offers outstanding capabilities—ready data access, manipulation, com-

putation, speedy accomplishment of functions, and information storage. Technological
sophistication, however, can work to the user’s disadvantage if the software design is
done without a thorough understanding of the user. At a minimum, designers are advised
to utilize guidelines for human computer interface (HCI), do a thorough analysis, and
conduct usability testing during software development. A thorough knowledge of the
user population is necessary. . . . Testing for ease and accuracy of use is the only way to
ensure that users can safely and effectively operate, install, and maintain devices. By
means of iterative prototyping, individual concepts of design can be tested, refined, and
retested throughout the development process. This process culminates with full testing
of a model embodying all the user-interface characteristics for both hardware and soft-
ware of a fully functioning device.17

In 2001, the American National Standards Institute (ANSI) and the Association
for the Advancement of Medical Instrumentation (AAMI) released ANSI/AAMI
HE74:2001, Human Factors Design Process for Medical Devices. One of the pur-
poses of the document was to describe a human factors process that would address
the human factors-related guidance of the FDA. Soon after the release of the docu-
ment, the FDA formally recognized the standard, meaning that the agency believed
that the prescribed human factors methodologies, including usability testing, were
aligned with its expectations. The standard stated:

The systematic application of HFE [human factors engineering] design principles, rein-
forced by tests involving end users, is an effective means of identifying and resolving
[such] design flaws. . . . Usability tests using device mock-ups or simulations could iden-
tify the possibility of incorrect tubing connections resulting from uncommon physical fit
and appearance, unnecessarily complex input sequences, or ambiguous messages.18

In 2004, the International Electrotechnical Commission (IEC) published IEC

60601—1-6, Medical Electrical Equipment—Part 1–6: General Requirements for
Safety—Collateral Standard: Usability.19 This “collateral” standard included much
* Production-equivalent prototypes are actually most common and considered acceptable.
18 Usability Testing of Medical Devices

the same content found in ANSI/AAMI HE74:2001 as an informative annex and

applied to mechanical and electrical devices. In 2007, the IEC published IEC 62366:
2007, Medical Devices—Application of Usability Engineering to Medical Devices,
which applies to all medical devices. The document, which was adopted in 2008 by
the European Union as the governing human factors process guide,20 mentions usabil-
ity testing over 40 times and presents the case study of making minor modifications
to the user interface of a syringe pump, suggesting that the manufacturer should:

• “Conduct a usability test of an early prototype (computer simulation or

working model) to determine whether the prototype meets safety and
usability goals and to discover opportunities for design improvement”
• “Conduct a second, abbreviated usability test to validate the refined near-
final design”21

In short, the documents referenced above make usability testing a de facto require-
ment, if not an explicit law. Moreover, medical device manufacturers are practically
required to conduct usability tests as a matter of “due diligence” (see “Does Usability
Testing Offer Liability Protection?” in Chapter 3).
Introduction 19

Do You Have to Test Minor Design Changes?

From a regulatory standpoint, you need to test minor design changes if they might influ-
ence how users perform safety-related tasks. After all, a minor design change could trig-
ger a critical use error. Minor design changes that are largely invisible to users probably
do not warrant usability testing. That said, regulators might ask for a full-scale usability
test of even a slightly modified device if the device has never been tested.

Consider the hypothetical case of a manufacturer that has been selling an

approved medical device for the past five years and has just “refreshed” the design
to keep it competitive. The new design has a flat, LCD (liquid crystal digital) display
instead of a CRT (cathode ray tube) display, and a membrane keypad replaces a set
of mechanical keys. The device is 40% smaller thanks to the use of more compact
internal components, so it can now sit on a countertop rather than a dedicated cart.
The renovated software user interface is organized in the same manner as the origi-
nal device, but the text menu options are supplemented with icons, and previously
monochromatic content has been colorized. Users can select parameters of interest
and view customized trend graphs. Still, the device does pretty much the same thing
as its predecessor.
Does the new design require usability testing? In our view, the answer is definitely
“yes,” regardless of whether or not the original design underwent usability testing. It
is a matter of practicing due diligence. It is also likely that regulators would want to
review a summative usability test report prior to giving the device clearance.
We believe that usability testing is warranted because the design enhancements,
although arguably minor, are nontrivial from a user interaction standpoint. The
enhancements will change how users interact with the device and potentially affect
use safety, making any former safety studies (i.e., risk analyses) out of date. For
example, the new keypad might induce users to make more data entry errors (e.g.,
incorrect or double key presses), leading users to input the wrong number (e.g., 100
instead of 10). The LCD display might produce more glare, causing users to misread
critical parameter values. Users might struggle to interpret icons and read colored

How Many Participants Do You Need to Validate Minor Design Changes?

Assuming the predecessor device underwent extensive usability testing, it might be suf-
ficient to evaluate minor design changes with a relatively small participant sample. Let us
take the example of an infusion pump that, due to software changes, now issues a reminder
alarm every five minutes to notify users of any unresolved problems (i.e., ignored alarms).
In addition to the alarm, users can now view an “alarm history” screen that lists the active
pump alarms alongside possible causes and the amount of time for which the alarm has
been active. With the exception of these changes, the device is identical to the one vali-
dated with a 25-participant summative usability test last year. Rather than conduct a full
validation test, you can probably validate the new alarm and history screen with fewer
participants. We would be tempted to conduct 10–15 supplemental test sessions, but check
the adequacy of this number with the appropriate regulators. The key would be to link the
supplemental test results to the original test report, thereby explaining why the latest test
was tightly focused on a few new design elements.
20 Usability Testing of Medical Devices

text presented on different color backgrounds, perhaps selecting incorrect menu

options and delaying patient treatment. Users might misinterpret the trend graphs,
leading to a misdiagnosis and improper patient treatment. These kinds of problems,
which can arise when a manufacturer refreshes an aging design, can be quickly
detected during usability testing.
Truly minor user interface design changes might not warrant summative usability
testing, but only if the predecessor device had undergone rigorous usability testing.
The following is a sample of design changes that might not warrant further usability
testing because they are trivial or serve to improve usability with virtually no chance
of unintended consequences:

• Changing the outer casing color of the device from beige to light blue.
• Increasing the size of key labels by 15% to improve their legibility.
• Adding a softer grip to the device handle to improve its comfort.
• Adding a power switch guard.
• Using round versus square buttons on the screen.
• Installing a backup battery that enables the device to operate without inter-
ruption for up to two hours in the event of a power outage.

As suggested, if user interface design changes require a manufacturer to apply

for regulatory approval [e.g., 510(k) approval], the design changes probably war-
rant validation usability testing, especially if the predecessor device was not tested
because approvals at that time were not contingent on usability testing.
Introduction 21

How Do You Defend Usability Testing

Methods to Market Researchers?
Let us be positive minded and assume harmony among usability and market research
specialists. However, challenged to defend usability testing methods (particularly
running tests with relatively few participants), usability specialists should empha-
size the remarkable effectiveness and efficiency of their proven methods. Usability
testing is intended to reveal usability problems, not necessarily to determine their
likelihood of occurrence.

Market researchers and usability specialists should be—and often are—pro-

fessional allies. After all, they share the common goal of developing products that
fulfill customers’ needs and preferences. However, market researchers’ and usabil-
ity specialists’ differing approaches to achieving similar goals sometimes lead to
professional tensions. Perhaps the most common source of tension is choosing an
appropriate usability test sample size.
Market research, which might address factors ranging from device features to
price to serviceability, often involves hundreds of prospective customers. The large
sample size is typically driven by statistical power requirements and how market
researchers divide the potential user population into discrete segments. Moreover,
market researchers typically conduct research in multiple countries to obtain feed-
back from the largest target markets (e.g., United States, Germany, Japan).
In contrast, usability testing typically involves a few dozen test participants at
most and sometimes as few as five to eight. The small sample size sometimes draws
expressions of doubt and even scoffs from disbelieving market researchers, who con-
sider the results of small-sample tests to be unreliable. Therefore, usability special-
ists are sometimes put on the defensive, called on to explain why they are not taking
a more scientific approach to conducting their research. Here are some of our sup-
portive arguments:

• It is important for any organization to approach usability testing in the most

effective and efficient manner possible. Studies have proven that just a few
usability test sessions are likely to reveal most—and the most severe—user
interface design problems.22
• The primary usability test goal is to reveal usability problems, not necessar-
ily to determine their likelihood of occurrence. Therefore, comparatively
small numbers of test sessions are usually enough to identify the problems
you would be likely to have the time and resources to fix.
• Usability testing definitely obeys the law of diminishing returns. If the goal
is to identify usability problems, you will probably identify almost all of
them within the first dozen or so test sessions. Certainly, you might identify
more problems if you conducted another one or two dozen test sessions.
You might identify yet another usability problem during the 250th test ses-
sion. But, you can always postulate that there is a hidden problem that might
not reveal itself until the 1,000th or 10,000th test session. The key is to con-
duct enough test sessions to be confident that you have identified the major
22 Usability Testing of Medical Devices

and even moderate usability problems and then to modify the design and do
some more testing. As far as we are concerned, medical device manufac-
turers are better off conducting three 12-participant usability tests than one
36-participant formative usability test.
• Formative usability tests involving 5–12 test participants, for example,
match the guidance provided in authoritative textbooks on human factors
and usability testing.23
• Organizations such as the FDA and multiple human factors standards rec-
ognize that you can draw high-quality results from a formative usability
test with 5–8 participants and a summative usability test with as few as
15–20 test participants.24 Notably, these sample sizes refer to the number of
participants who should represent each distinct user group.
• If you find that even one or two of 12 test participants encounters a major
usability problem, it suggests that you should analyze the user interface design
to see if a design change is warranted. There is no point in asking whether the
finding is statistically significant with a high confidence level. To be practical,
because the usability problem appeared even once or twice, you should con-
sider changing the design because the use error is likely to occur many times
during hundreds and thousands of uses. Capable usability specialists should
be able to state confidently whether the problem will occur at a 10–20% rate,
for example, drawing on their judgment. That is what they are paid to do.

Usability specialists are usually disinclined to criticize market researchers for

conducting large studies and, in turn, do not seek criticism by market research spe-
cialists for conducting small studies. Each type of professional is applying their pro-
fessional standards in an intelligent and resource-conscious manner to serve their
clients (internal or external).

Can You Integrate Market Research into Usability Testing?

In theory, manufacturers should conduct market research and usability testing separately
and independently. However, you can include a few market research-type questions in the
posttest interview. For example, you could ask participants to comment on the viability of
the device concept and identify the advantages of the device over others already on the
market. If you choose to include such questions in the posttest interview, just be sure you
ask them after the questions about the usability and safety of the device, and that you do
not compromise any of the usability test goals.
Introduction 23

Notes
1. U.S. Food and Drug Administration (FDA). 2009. Is the product a medical device?
Retrieved from http://www.fda.gov/MedicalDevices/DeviceRegulationandGuidance/
Overview/ClassifyYourDevice/ucm051512.htm.
2. European Union. 1993. Council Directive 93/42/EEC of 14 June 1993 concerning
medical devices. Article 1: Definitions, scope. Retrieved from http://eur-lex.europa.eu/
LexUriServ/LexUriServ.do?uri=CELEX:31993L0042:EN:HTML. Note: Only European
Union legislation printed in the paper edition of the Official Journal of the European
Union is deemed authentic.
3. U.S. Food and Drug Administration (FDA). 2009. General and special controls.
Retrieved from http://www.fda.gov/MedicalDevices/DeviceRegulationandGuidance/
Overview/GeneralandSpecialControls/default.htm.
4. Ibid.
5. Kinnealey, E., Fishman, G., Sims, N., Cooper, J., and DeMonaco, H. 2003. Infusion
pumps with “drug libraries” at the point of care—A solution for safer drug delivery.
Retrieved from http://www.npsf.org/download/Kinnealey.pdf.
6. Design validation. Code of Federal Regulations, 21 CFR 820.30, Part 820, Subpart C,
Subsection G. Retrieved from http://www.accessdata.fda.gov/scripts/cdrh/cfdocs/cfcfr/
CFRSearch.cfm?FR=820.30.
7. Food and Drug Administration (FDA)/Center for Devices and Radiological Health
(CDRH). 1996. Do it by design: An introduction to human factors in medical devices.
Retrieved from http://www.fda.gov/downloads/MedicalDevices/DeviceRegulationand
Guidance/GuidanceDocuments/ucm095061.pdf.
8. Food and Drug Administration (FDA)/Center for Devices and Radiological Health
(CDRH). 2000. Medical device use-safety: Incorporating human factors engineering
into risk management. Retrieved from http://www.fda.gov/downloads/MedicalDevices/
DeviceRegulationandGuidance/GuidanceDocuments/ucm094461.pdf.
9. U.S. Food and Drug Administration (FDA) Web site. 2009. About FDA—What we do.
Retrieved from http://www.fda.gov/opacom/morechoices/mission.html
10. Medicines and Healthcare Products Regulatory Agency Web site. 2010. About us.
Retrieved from http://www.mhra.gov.uk/Aboutus/index.htm.
11. Pharmaceuticals and Medical Devices Agency, Japan, Web site. 2009. Message from
chief executive. Retrieved from http://www.pmda.go.jp/english/about/message.html.
12. Pharmaceuticals and Medical Devices Agency, Japan, Web site. 2010. Our Philosophy.
Retrieved from http://www.pmda.go.jp/english/about/philosophy.html.
13. Federal Institute for Drugs and Medical Devices Web site. 2007. About us. Retrieved from
http://www.BfArM.de/cln_030/nn_424928/EN/BfArM/bfarm-node-en.html_nnn=true.
14. Design validation. Code of Federal Regulations, 21 CFR 820.30, Part 820, Subpart C,
Subsection G. Retrieved from http://www.accessdata.fda.gov/scripts/cdrh/cfdocs/cfcfr/
CFRSearch.cfm?FR=820.30.
15. Food and Drug Administration (FDA). 2009. Human factors implications of the new
GMP rule overall requirements of the new quality system regulation. Retrieved January
24, 2010, from http://www.fda.gov/MedicalDevices/DeviceRegulationandGuidance/
PostmarketRequirements/HumanFactors/ucm119215.htm.
16. FDA/CDRH, 2000.
17. FDA/CDRH, 1996, pp. 19, 36.
18. Association for the Advancement of Medical Instrumentation (AAMI). 2001. ANSI/
AAMI HE74: 2001 Human factors design process for medical devices. Arlington, VA:
Association for the Advancement of Medical Instrumentation, page 2.
24 Usability Testing of Medical Devices

19. These regulations have been incorporated into IEC 62366, which applies to a broader set
of medical devices than the original standard. International Electrotechnical Commission
(IEC). 2007. IEC 63266:2007, Medical devices—Application of usability engineering to
medical devices. Geneva, Switzerland: International Electrotechnical Commission.
20. European Commission. 2009. Summary list of titles and references harmonised stan-
dards relating to in vitro diagnostic medical devices. Retrieved from http://ec.europa.eu/
enterprise/policies/european-standards/documents/harmonised-standards-legislation/
list-references/iv-diagnostic-medical-devices/.
21. International Electrotechnical Commission, IEC 63266:2007, p. 45.
22. Virzi, R. A. (1992). Refining the test phase of usability evaluation: How many subjects
is enough? Human Factors 34: 457–468.
23. Dumas, J. S., and Redish, J. C. 1999. A practical guide to usability testing (rev. ed.).
Portland, OR: Intellect.
24. Association for Advancement of Medical Instrumentation (AAMI). 2009. ANSI/AAMI
HE75:2009: Human factors engineering—Design of medical devices. Arlington, VA:
Association for Advancement of Medical Instrumentation, Annex A.
2 Risk Management
and Usability Testing

25
26 Usability Testing of Medical Devices

What Is the Relationship between Usability

Testing and Risk Management?
Usability testing may be considered part of an overall risk management scheme.
Testing helps determine if using a device poses risks that should be reduced or
eliminated before the device goes to market. Accordingly, a summative usability test
should be focused on tasks posing the greatest risk according to preceding analyses
and formative usability tests.

Risk management is a process that medical device developers go through to iden-

tify and then minimize the risks associated with using a medical device in speci-
fied scenarios. The people involved in the process (e.g., risk managers) identify the
fundamental hazards (e.g., a short circuit) of a device and potentially harmful events
associated with using it (e.g., erroneously plugging a sensor into an alternating cur-
rent [AC] power supply), estimate the level of risk based on the likelihood and sever-
ity of a hazardous event, and take action to mitigate the unacceptable risks. Possible
mitigations include software and hardware user interface design changes, warning
labels, instructions, and training.
In principle, the risk management process reduces risk to an acceptable minimum
without necessarily eliminating it. Accordingly, medical devices often have what are
termed residual risks: the lingering possibility that the device could cause personal
injury and damage.
Regulatory bodies such as the Food and Drug Administration (FDA) encour-
age manufacturers to conduct summative (i.e., validation) usability tests to judge
the effectiveness of user interface-related mitigations. By extension, regulators want
manufacturers to see if users commit any dangerous use errors while performing a
comprehensive set of tasks with the device.
So, usability testing and risk management are inexorably linked. As explained
in “Why Focus on Potentially Dangerous Tasks?” in Chapter 11, test planners need
to review risk analysis documents to determine the most appropriate set of tasks to
include in a summative usability test. Test planners might take the same approach to
selecting formative usability test tasks if they want to get a head start on producing
a valid design.
Ideally, designers will find a way to eliminate a device hazard altogether, driv-
ing the associated risk to zero. For example, they might eliminate a sharp edge on
the device that could cause a laceration or program an infusion pump to calculate
a proper infusion rate rather than requiring the user to perform the calculation,
which could open the door to a math error. In other cases, manufacturers might not
be able to eliminate the hazard but rather implement a safeguard. For example, a
laser treatment device might require users to perform two sequential actions to fire
the laser. This type of mitigation, which does not truly eliminate the potential to
accidentally fire the laser, would warrant validation through summative usability
testing. Specifically, you would direct test participants to simulate firing the laser
and confirm that they understood the consequences of their actions and that no
inadvertent firings occurred. Such validation efforts might seem perfunctory. You
might assume that a safeguard serves its purpose by virtue of its existence. But, it
Risk Management and Usability Testing 27

is worth verifying the effectiveness of a safeguard because unpredictable and coun-

terintuitive things can happen when people interact with medical devices. Also,
mitigations implemented in response to previously identified usability issues might
introduce unexpected new hazards.
Published by the FDA, Medical Device Use-Safety: Incorporating Human
Factors Engineering into Risk Management discusses the relationship of usability
testing to risk management in greater depth.1
Discovering Diverse Content Through
Random Scribd Documents
The Project Gutenberg eBook of William again
 This ebook is for the use of anyone anywhere in the United
States and most other parts of the world at no cost and with
almost no restrictions whatsoever. You may copy it, give it away
or re-use it under the terms of the Project Gutenberg License
included with this ebook or online at www.gutenberg.org. If you
are not located in the United States, you will have to check the
laws of the country where you are located before using this
eBook.

Title: William again

Author: Richmal Crompton

Illustrator: Thomas Henry

Release date: June 11, 2021 [eBook #65590]

Language: English

Credits: deaurider, Alan and the Online Distributed Proofreading

Team at https://www.pgdp.net

*** START OF THE PROJECT GUTENBERG EBOOK WILLIAM AGAIN

***
WILLIAM AGAIN
"GOOD MORNING, LITTLE BOY," SAID THE
VISITOR

"UMPH!" REPLIED WILLIAM.

"WHAT'S YOUR NAME, DEAR?" SHE ASKED.

"PETER," SAID WILLIAM.

WILLIAM AGAIN

BY

RICHMAL CROMPTON

ILLUSTRATED BY
 THOMAS HENRY

LONDON
GEORGE NEWNES, LIMITED
SOUTHAMPTON ST., STRAND, W.C.

First Published - - - - - June 1923

Reprinted - - - - - - - - - September 1923
Reprinted - - - - - - - - - December 1923
Reprinted - - - - - - - - - February 1924
Reprinted - - - - - - - - - July 1924
Reprinted - - - - - - - - - November 1924
Reprinted - - - - - - - - - January 1925
Reprinted - - - - - - - - - July 1925
Reprinted - - - - - - - - - November 1925
Reprinted - - - - - - - - - February 1926
Reprinted - - - - - - - - - August 1926
Reprinted - - - - - - - - - December 1926
Reprinted - - - - - - - - - July 1927
Reprinted - - - - - - - - - January 1928
Reprinted - - - - - - - - - November 1928
Reprinted - - - - - - - - - October 1929
Reprinted - - - - - - - - - October 1930
Reprinted - - - - - - - - - October 1931
 Reprinted - - - -- - - - - October 1932
Reprinted - - - -- - - - - November 1933
Reprinted - - - -- - - - - December 1934
Reprinted - - - -- - - - - August 1936
Reprinted - - - -- - - - - October 1937
Reprinted - - - -- - - - - May 1939
Reprinted - - - -- - - - - February 1940
Printed in Great Britain by
Wyman & Sons, Limited, London, Fakenham and Reading.
 CONTENTS
CHAPTER PAGE
I. What Delayed the Great Man 11
II. The Cure 29
III. That Boy 44
IV. William the Reformer 58
V. Not Much 74
VI. William and the White Cat 92
VII. William's Secret Society 108
VIII. The Native Protégé 133
IX. Just William's Luck 148
X. The Great Detective 166
XI. The Circus 181
XII. William Sells the Twins 197
XIII. William's Helping Hand 214
XIV. William gets Wrecked 233
TO
TOMMY
 CHAPTER I
WHAT DELAYED THE GREAT MAN
William, taking his character as a whole, was not of the artistic genre. He
had none of the shrinking sensitiveness and delicate imaginativeness of the
true artist. But the fact remains that this summer he was impelled by some
inner prompting to write a play.
The idea had been growing in his mind for some time. He had seen plays
acted by the village amateur dramatic society which was famous more for a
touching reliance on the prompter than for any real histrionic talent.
William had considered them perfect. He had decided, after their last
performance, to go on the stage. But none of his friends could inform him
of the preliminary steps necessary for getting on the stage. It is true that
the man in the boot-shop, whose second cousin was a scene-shifter in a
provincial music-hall, had promised to use his influence, but when William
was told the next week that the second cousin had been dismissed for
appearing in a state of undeniable intoxication and insisting on
accompanying the heroine on to the stage, he felt that all hopes from that
direction must be abandoned. It was then that he had the brilliant idea. He
would write a play himself and act in that.
William had great confidence in his own powers. He had no doubts
whatever of his ability to write a play and act in it. If he couldn't go on the
stage he'd go on a stage. Surely no one could object to that. All he'd want
would be some paper and ink and a few clothes. Surely his family—bent as
they always were on clouding his moments of purest happiness—couldn't
object to that?
"Jus' ink an' paper an' a few ole clothes," he said wistfully to his mother.
She eyed him with a mistrust that was less the result of a suspicious nature
than of eleven years' experience of her younger son.
"Won't pencil do?" she said.
"Pencil!" he said scornfully. "Did—did Shakespeare or—or the man wot
wrote 'The Red Gang'—well, did they write in pencil?"
Mrs. Brown, having no knowledge of the subject, shifted her point of
attack.
"What sort of clothes will you want?" she said.
"Oh—jus' clothes," said William vaguely.
"Yes, but what sort?"
"How can I tell," said William irritably, "till I've wrote the play?"

* * * * * *
William's family long remembered the silence and peace that marked the
next few afternoons. During them, William, outstretched upon the floor of
the summer-house, wrote his play with liberal application of ink over his
person and clothes and the surrounding woodwork. William was not of that
class of authors who neglect the needs of the body. After every few words
he took a deep draught from a bottle of Orange Ale that stood on his right
and a bite from an ink-coated apple on his left. He had laid in a store of
apples and sweets and chocolates under the seat of the summer-house for
his term of authorship. Every now and then he raised a hand to his
frowning brow in thought, leaving upon it yet another imprint of his ink-
sodden fingers.
"Where is he?" said his father in hushed wonder at the unwonted peace.
"He's in the summer-house writing a play," said his wife.
"I hope it's a nice long one," said her husband.

* * * * * *
William had assembled his caste and assigned them their parts. Little Molly
Carter was to be the heroine, Ginger the hero, Henry the hero's friend,
Douglas a crowd of outlaws, William himself was to be the villain, stage-
manager and prompter. He handed them their parts with a lofty frown. The
parts were in a grimy exercise book.
"It's all wrote out," he said. "You jus' learn it where it says your names.
Molly's Lady Elsabina——"
"Elsabina isn't a name I've ever heard," said that lady pertly.
"I didn't say it was, did I?" said William coldly. "I shu'n't be surprised if
there was lots of names you'd never heard of. An' Ginger is Sir Rufus
Archibald Green an' Henry is the Hon. Lord Leopold, an' I'm Carlo Rupino, a
villain. All you've gotter do is to learn your parts an' Wednesday morning
we'll go through it jus' to practise it, an' Wednesday afternoon we'll do it."
"We can't three learn out of one book," said the leading lady, who was
inclined to make objections.
"Yes, you can," said William. "You can take turns sitting in the middle."
Lady Elsabina sniffed.
"And such writing!" she said scornfully.
"Well, I don't count on my fingers," said William, returning scorn for scorn,
"not so's everyone can see me, at any rate."
At which public allusion to her arithmetical powers, Lady Elsabina took
refuge in another sniff, followed by a haughty silence.

* * * * * *
The rehearsal was not an unqualified success. The heroine, as is the way of
heroines, got out of bed the wrong side. After a stirring domestic scene,
during which she bit her nurse and flung a basin of bread and milk upon
the floor, she arrived tearful and indignant and half an hour late at the
rehearsal.
"Can't you come a bit later?" said the stage-manager bitterly.
"If you're going to be nasty to me," returned the heroine stormily, "I'm
going back home."
"All right," muttered the stage-manager, cowed, like most stage-managers,
by the threatening of tears.
The first item on the agenda was the question of the wardrobe. William had
received an unpleasant surprise which considerably lowered his faith in
human nature generally. On paying a quiet and entirely informal visit to his
sister's bedroom in her absence, to collect some articles of festive female
attire for his heroine, he had found every drawer, and even the wardrobe,
locked. His sister had kept herself informed of the date of the performance,
and had taken measures accordingly. He had collected only a crochet-
edged towel, one of the short lace curtains from the window, and a drawn-
thread work toilet-cover. Otherwise his search was barren. Passing through
the kitchen, however, he found one of her silk petticoats on a clothes-horse
and added it to his plunder. He found various other articles in other parts of
the house. The dressing up took place in an outhouse that had once been a
stable at the back of William's house. The heroine's dress consisted of
Ethel's silk petticoat with holes cut for the arms. The lace curtain formed an
effective head-dress, and the toilet-cover pinned on to the end of the
petticoat made a handsome train.
The effect was completed by the crochet-edged towel pinned round her
waist. Sir Rufus Archibald Green, swathed in an Indian embroidered table-
cover, with a black satin cushion pinned on to his chest, a tea-cosy on his
head, and an umbrella in his hand, looked a princely hero. The Hon. Lord
Leopold wore the dining-room table-cloth and the morning-room waste-
paper basket with a feather, forcibly wrested from the cock's tail by William,
protruding jauntily from the middle. Douglas, as the crowd, was simply
attired in William's father's top hat and a mackintosh.
William had quietly abstracted the top hat as soon as he heard definitely
that his father would not be present at the performance. William's father
was to preside at a political meeting in the village hall, which was to be
addressed by a Great Man from the Cabinet, who was coming down from
London specially for the occasion.
"Vast as are the attractions of any enterprise promoted by you, William," he
had said, politely at breakfast, "duty calls me elsewhere."
William, while murmuring perfunctory sorrow at these tidings, hastily ran
over in his mind various articles of his father's attire that could therefore be
safely utilised. The robing of William himself as the villain had cost him
much care and thought. He had finally decided upon the drawing-room rug
pinned across his shoulder and a fern-pot upon his head. It was a black
china fern-pot and rather large, but it rested upon William's ears, and gave
him a commanding and sinister appearance. He also carried an umbrella.
These preparations took longer than the caste had foreseen, and, when
finally large moustaches had been corked upon the hero's, villain's and
crowd's lips, the lunch-bell sounded from the hall.
"Jus' all finished in time!" said William the optimist.
"Yes, but wot about the rehearsal," said the crowd gloomily, "wot about
that?"
"Well, you've had the book to learn the stuff," said William, "that's enough,
isn't it? I don't s'pose real acting people bother with rehearsals. It's quite
easy. You jus' learn your stuff an' then say it. It's silly wasting time over
rehearsals."
"Have you learnt wot you say, William Brown?" said the heroine shrilly.
"I know wot I say," said William loftily, "I don't need to learn!"
"William!" called a stern sisterly voice from the house, "mother says come
and get ready for lunch."
William merely ejected his tongue in the direction of the voice and made no
answer.
"We'd better be taking off the things," he said, "so's to be in time for this
afternoon. Haf-past two it begins, then we can have a nice long go at it.
Put all the things away careful behind that box so's bothering ole people
can't get at them an' make a fuss."
"William, where are you?" called the voice impatiently.
The tone goaded William into reply.
"I'm somewhere where you can't find me," he called.
"You're in the stable," said the voice triumphantly.
"Seems as if folks simply couldn't leave me alone," said William wistfully, as
he removed his fern-pot and fur rug and walked with slow dignity into the
house.
"Wash yourself first, William," said the obnoxious voice.
"I am washed," returned William coldly, as he entered the dining-room,
forgetting the presence of a smudgy, corked moustache upon lips and
cheeks.

* * * * * *
It was an unfortunate afternoon as far as the prospects of a large audience
were concerned. Most of the adults of the place were going to listen to the
Great Man. Most of the juveniles were going to watch a football match.
Moreover, the caste, with the instincts of the very young, had shrouded the
enterprise so deeply in mystery in order to enjoy the sensation of
superiority, that they had omitted to mention either the exact nature of the
enterprise or the time at which it would take place.
On the side-gate was pinned a notice:

THIS WAY TO THE

BLOODY HAND ➔

In the stable was a row of old chairs all turned out of the house at various
times because of broken backs and legs. As a matter of fact, the caste
were little concerned with the audience. The great point was that they
were going to act a play—they scarcely cared whether anyone watched it
or not. Upon a broken chair in the middle sat a small child, attracted by the
notice. Her chair had only lost one leg, so, by sitting well on to one side,
she managed to maintain an upright position on it. At a stern demand for
money from William, she had shyly slipped a halfpenny into the fern-pot,
which served the double purpose of head-gear and pay-desk. She now sat
—an enthralled spectator—while the caste dressed and argued before her.
Outside down the road came the Great Man. He had come by an earlier
train by mistake and was walking slowly towards the village hall, intensely
bored by the prospect of the afternoon. He stopped suddenly, arrested by a
notice on a side gate:

THIS WAY TO THE

BLOODY HAND ➔

He took out his watch. Half an hour to spare. He hesitated a moment, then
walked firmly towards the Bloody Hand. Inside an outhouse a group of
curiously-dressed children stared at him unsmilingly. One of them, who was
dressed in a rug and a fern-pot, addressed him with a stern frown.
 INSIDE AN OUTHOUSE A GROUP OF CURIOUSLY-DRESSED
CHILDREN STARED AT HIM UNSMILINGLY.

"We're jus' going to begin," he said, "sit down."

The Great Man sat down obediently and promptly collapsed upon the floor.
"You shu'n't have sat on a chair with two legs gone," said William
impatiently. "You've broke it altogether now. You can manage all right if
you try one with only one gone. We're jus' going to begin."
The Great Man picked up himself and his hat and sat down carefully upon
the farthermost edge of a three-legged chair.
William, holding the mangled remains of an exercise book in his hand,
strode forward.
"'The Bloody Hand,' by William Brown," he announced in a resonant voice.
"Well, an' wot about us?" said the heroine shrilly.
"You didn't write it, did you?" said William. "I'm only saying who wrote it."
"Well, aren't you going to say who axe it?" she said pugnaciously.
"No, I'm not!" said the stage-manager firmly. "You jus' say the one wot
wrote it. You don't go on saying all them wot axe it."
"Well, I'm not going to be in it, then," she said. "I'm going home."
William decided to be a woman-hater for the rest of his life.
"All right," he capitulated, "'f you're going to be so disagreeable—jus' like a
girl"—he strode forward again and raised his voice, "'The Bloody Hand,'
wrote, every bit of it, by William Brown—acted by Molly Carter an Ginger
an' Douglas an' Henry—they jus' learnt wot William Brown wrote. Now, if
you'll be quiet a minute," he went on to his silent audience, "we'll begin.
You begin," he said to the damsel in the lace curtain.
She advanced. The rest of them stood in a corner and watched.
"She's on," William announced to the audience. "We're off. Go on!" he
repeated to her.
"I'm jus' going to," she replied irritably, "soon as you stop talking." Then,
changing her voice to one of shrill artificiality, "Ho! Where am I? Lorst in a
dreadful forest——"
"It's meant to be a forest," explained the author to the audience.
"I wish you'd stop keep on saying things," said the heroine. "I forget where
I am. Lorst in a dreadful forest. What shall I do? Ah, me! Crumbs! Who is
this who yawns upon my sight?"
"Dawns!" corrected the prompter.
"A fierce villain," went on the heroine, ignoring him, "methinks. I shouldn't
be surprised if it wasn't Carlo Rupino of the Bloody Hand. Oh Lor! what
shall I do? Ah me! He draws nearer."
"It is him," prompted William.
"I was jus' going to say that, if you wouldn't keep on interrupting. It is him.
I was jus' going to say it. Ah me! what shall I do? Whither shall I flee?
Nowhere. Gadzooks! He draws nearer."
"I come on now," explained William to the audience, holding on to his
plant-pot with one hand to steady it. "I'm him." He advanced threateningly
upon the maiden. "Aha!" he sneered. "Gadzooks! doest thou happen to
know who I am?"
"I am lorst in the dreadful forest," she replied. "Ah me! What shall I do?"
"I am Carlo Rupino of the Bloody Hand. Go on, faint!" he urged in an
undertone.
"'F you think I'm going to faint on this dirty ole floor," she replied, "I'm jus'
not. You should have brushed it up a bit 'f you wanted me to faint on it."
"You don't know how to," he jeered.
"I do! I can! I can faint beautifully on our drawing-room carpet. I'm jus' not
going to faint on a dirty ole stable floor an' I'm not going to be in your
nasty ole play 'f you're not going to be nice to me."
"All right, then, don't be. You jus' take off my sister's petticoat, an' our lace
curtain an' don't be in it, if you don't want to be."
"Well, I jus' won't, 'f you're going on like this at me."
"Well, 'f you keep on talkin' not out of the play who's to know when you're
talkin' play an' when you're jus' talkin' yourself?"
"Anyone with any sense could——"
"Oh, get on with it," said the hero off the scenes. "You'll never get to where
I come in, if you're going on like this all day. Pretend she's fainted and go
on from there."
"All right," said the villain obligingly. "Aha! I hast thee in my power. I wilt
hang thee ere dawn dawns from my remote mountain lair." The toilet-cover
train caught on a nail and the petticoat tore with an echoing sound. "That's
right," he went on, "go on messin' up my sister's things, so's she'll never be
able to wear them again."
"'F you're going to keep on being nasty to me," said the heroine again, "I'm
going straight back home an' I'm not going to be in your ole play."
"Well, anyway," said William, with a mental determination that his next play
should contain no heroines, "now we go off and they come on."
The hero and his friend advanced.
"Alas!" said Sir Rufus Archibald Green, "I see no trace of her. What canst
have happened to her? I hope she hast not met yon horrible ole villain,
Carlo Rupino, of the Bloody Hand. Seest thou any footmarks of her, the
Hon. Lord Leopold?"
The Hon. Lord Leopold examined the stable floor.
"Lookin' for footmarks," explained the stage-manager to the audience.
"Ah me! None!" said the Hon. Lord Leopold. Then, looking more closely.
"Crikey! Yes!" he said. "I seest footmarks. 'Tis hers and Carlo Rupino's. I
knowest their boots."
"Ah me!" said the hero. "What cattastrop is here? Gadzooks! Let us follow
to his remote mountain lair. I will kill him dead and cut out his foul black
heart and put an end to his foul black life."
He waved Mrs. Brown's best umbrella threateningly as he spoke. "Now they
come off," explained William, "an' we come on. Here's the gallows."
He carried forward a small reading stand, taken from his father's study,
then advanced holding the hand of the fair Elsabina. The crowd in his top
hat and mackintosh stood in attendance.
"Aha!" said Carlo Rupino to his victim, "I hast thee in my power, thou ole
girl! I am now going to hang thee from yon lofty gallows! Go on!" he
addressed the crowd.
The crowd took off his top hat and uttered a feeble "Hurray!"
"You couldn't hang me from that old thing," remarked the heroine
scornfully.
"That's not in the play," said William.
"I know it isn't. I'm jus' saying that myself."
"Well, say wot's in the play."
At that point the chair, upon which the Great Man was with difficulty sitting,
collapsed suddenly, precipitating the Great Man among its fragments.
William turned upon him sternly.
"'F you're going to keep on making noises breaking chairs," he said, "how
d'you think we're going to get on?"
The Great Man raised himself from the débris with a murmured apology,
brushed himself as well as he could, and sat down quietly upon an adjacent
packing-case.
"Well, go on!" said William to the heroine.
"Something about 'Oh, mercy, spare me!' an' then I've forgot what comes
after that."
"Well, why didn't you learn it?"
"I can't read your nasty old writing—all blots an' things spilt on it."
"Well, you can't write a play at all, so you needn't go making remarks about
people's writing what can."
"Oh, go on!" said the egoistical hero off the stage. "Let's get to where I
come on."
William studied his exercise-book carefully.
"Here's wot you say," he said, "'Oh, mercy, spare me——'"
"I said that."
"Be quiet! 'Oh, mercy, spare me——'"
"I said that."
"Be quiet! 'Oh, mercy, spare me an' let me return to my dear ole mother
an' father an' the young gentleman wot I'm going to marry. His name is Sir
Rufus Archibald Green.' That's wot you say."
"Well, you've said it, so I needn't say it all over again."
"'F you think I'm going to say all your stuff for you——" began William.
Elsabina, bored with the question, pointed an accusing finger at the Great
Man.
"Look at him!" she said. "He's come in without paying any money."
Overcome by embarrassment, the Great Man hastily took out a case and
handed a ten-shilling note to William. A half-crown would have won
rapturous gratitude. A ten-shilling note was beyond their ken. The entire
caste gathered round it.
"It's paper money," said Douglas, impressed.
"I don't suppose it's real," said William gloomily. "Well, where're we got to?"
He turned quickly, and the fern-pot descended sharply, extinguishing his
head. He struggled with it without success.
"Can't anyone do anything?" said his muffled voice from inside the fern-pot.
"I can't go on acting like this—people can't see me. Well, isn't anyone going
to do anything?"
The caste pulled without success.
"I didn't say pull my head off," said the stern, sarcastic voice from inside
the pot, "I said pull the thing off!"
The Great Man arose from his packing-case and came to the rescue. Finally
William's face appeared. William put his hands to his head. "Any one'd think
you wanted to pull my nose an' ears off—the way you did it," he said. "Now
let's get on." He turned to the heroine. "'No, I will not spare thee. I hatest
thy mother and thy father and the young gentleman thou ist going to
marry. Thy mother, thy father, and the young gentleman thou ist going to
marry wilt see thy lifeless body dangling on my remote mountain lair ere
dawn dawns. Gadzooks!' Now go on! Scream!"
The heroine screamed.
The crowd took off his top hat and cheered.
"'I will keep thee in a deep, dark dungeon, with all sorts of rats an' things
crawling about till even, and then—and then——'" He consulted his
exercise-book, "'and then I'll'—I've forgot this bit, and I can't read wot
comes next——"
"Yah!" yelled the heroine in shrill triumph.
"Shut up!" retorted William. "Now, you come on," to the hero. "Let's do the
rest as quick as we can. I'm getting a bit tired of it. Let's go down to the
pond an' race boats when we've done."
"Golly! Yes—let's!" said the crowd enthusiastically.
"Girls won't be allowed," said William to Elsabina.
Elsabina elevated her small nose.
"'S if I wanted to sail boats!" she said scornfully.
* * * * * *
William's father entered the house hastily.
"Surely the meeting isn't over, dear?" said William's mother.
"He hasn't come," said Mr. Brown. "Everybody's waiting. We met the train,
but he wasn't on it. The station-master says that he came by an earlier one
and walked up, but no one can find him. He must have lost his way."
"William seems to have collected an old tramp in the stable," said Mrs.
Brown; "he may have seen him on the road."
"I'll go and see," said Mr. Brown.
In the stable a fight was going on between his son in a fur rug and his
son's friend in a table-cloth and a tea-cosy. Upon both faces were the
remains of corked moustaches. A broken fern-pot and a battered top hat
were on the floor. Another boy in a mackintosh and a little girl in a lace
curtain were watching.
 "THOU BEASTLY OLE ROBBER," DOUGLAS WAS SHOUTING,
"I WILL KILL THEE DEAD AND CUT OUT THY FOUL,
BLACK HEART."

"Thou beastly ole robber," Douglas was shouting, "I will kill thee dead and
cut out thy foul, black heart."
"Nay!" yelled his son. "I will hang thee from my mountain ere dawn dawns
and thy body shall dangle from the gallows——"
A wistful-looking old man on a packing-case was an absorbed spectator of
the proceedings. When he saw William's father he took out his watch with
a guilty start.
"Surely——" he said. "I'd no idea—Heavens!"
He picked up his hat and almost ran.

* * * * * *
The Great Man rose to address his audience.
"Ladies and gentlemen—I must begin by apologising for my late arrival," he
said with dignity. "I have been unavoidably delayed."
He tried not to meet William's father's eye as he made the statement.
 CHAPTER II
THE CURE
Breakfast was not William's favourite meal. With his father shut off from the
world by his paper, and his mother by her letters, one would have thought
that he would have enjoyed the clear field thus left for his activities. But
William liked an audience—even a hostile one consisting of his own family.
True, Robert and Ethel, his elder brother and sister, were there; but
Robert's great rule in life was to ignore William's existence. Robert would
have preferred not to have had a small freckled, snub-nosed brother. But as
Fate had given him such a brother, the next best thing was to pretend that
he did not exist. On the whole, William preferred to leave Robert alone.
And Ethel was awful at breakfast—quite capable of summoning the Head of
the Family from behind his Daily Telegraph when William essayed a little
gentle teasing. This morning William, surveying his family in silence in the
intervals of making a very hearty meal, came to the conclusion, not for the
first time, that they were hardly worthy of him: Ethel, thinking she was so
pretty in that stuck-up-looking dress, and grinning over that letter from that
soft girl. Robert talking about football and nobody listening to him, and
glaring at him (William) whenever he tried to tell him what nonsense he
was talking about it. No, it wasn't rounders he was thinking of—he knew
'bout football, thank you, he just did. His mother—suddenly his mother put
down her letter.
"Great-Aunt Jane's very ill," she said.
There was a sudden silence. Mr. Brown's face appeared above the Daily
Telegraph.
"Um?" he said.
"Great-Aunt Jane's very ill," said Mrs. Brown. "They don't seem to think
there's much chance of her getting better. They say——" She looked again
at the letter as if to make quite sure: "They say she wants to see William.
She's never seen him, you know."
There was a gasp of surprise.
Robert voiced the general sentiment.
"Good Lord!" he said, "fancy anyone wanting to see William!"
"When they're dying, too," said Ethel in equal horror. "One would think
they'd like to die in peace, anyway."
"It hardly seems fair," went on Robert, "to show William to anyone who's
not strong."
William glared balefully from one to the other.
"Children! Children!" murmured Mrs. Brown.
"How," said Mr. Brown, "are you going to get William over to Ireland?"
"I suppose," said Mrs. Brown, "that someone must take him."
"Good Lord! Who?"
"Yes, who?" echoed the rest of the family.
"I can't possibly leave the office for the next few weeks," said Mr. Brown
hastily.
"I simply couldn't face the crossing alone—much less with William," said
Ethel.
"I've got my finals coming off next year," said Robert. "I don't want to
waste any time. I'm working rather hard these vacs."
"No one," said his father politely, "would have noticed it."
"I can go alone, thank you," said William with icy dignity.

* * * * * *
In the end William and Mrs. Brown crossed to Ireland together.
"If William drops overboard," was Robert's parting shot, "don't worry."
The crossing was fairly eventful. William, hanging over the edge of the
steamer, overbalanced, and was rescued from a watery grave by one of the
crew who happened to be standing near and who caught him by his
trousers as the overbalancing occurred. William was far from grateful.
"Pullin' an' tuggin' at me," he said, "an' I was all right. I was only jus'
lookin' over the edge. I'd have got back all right."
But the member of the crew made life hideous to Mrs. Brown.
"You know, lady," he muttered, "when I saved yer little boy's life, I give
myself such a wrench. I can feel it in my innards now, as it were——"
Hastily she gave him ten shillings. Yet she could not stem the flow.
"I 'ope, lady," he would continue at intervals, "when that choild's growd to
be a man, you'll think sometoimes of the poor ole man wot saved 'is life at
the expense of 'is own innards, as you might say when 'e were a little 'un."
A speech like that always won half-a-crown. In the end Mrs. Brown spent
her time avoiding him and fleeing whenever she saw him coming along the
deck. When a meeting was inevitable she hastily gave him the largest coin
she could find before he could begin on his "innards."
Meanwhile a passenger had discovered William neatly balanced through a
porthole, and earned his undying hatred by hauling him in and depositing
him upside down on the floor.
"Seems to me," said William to his mother, "that all these folks have come
for is to stop other folks having a good time. What do you come on a boat
for if you can't look at the sea—that's all I want to know?"
A gale rose, and Mrs. Brown, pale and distraught, sat huddled up on deck.
William hovered round sympathetically.
"I got some chocolate creams in my other coat. Like some of them?"
"William, dear, don't bother to stay here. I'd just as soon you went away
and played."
"Oh, no," said William nobly. "I wun't leave you feelin' bad."
The boat gave a lurching heave. Mrs. Brown groaned.
"Think you goin' to be sick, mother?" said William with interest.
"I—I don't know.... Wouldn't you like to go over to the other side for a
change?"
William wandered away. Soon he returned, holding in his hands two
doughnuts—masses of yellowy, greasy-looking dough, bearing the impress
of William's grimy fingers.
"I've got us one each," said William cheerfully. "You must be awful hungry,
mother."
 SOON WILLIAM RETURNED, HOLDING IN HIS HANDS
TWO DOUGHNUTS.

Mrs. Brown gave one glance and turned towards the sea.

* * * * * *