y

TheFDAPerspectiveon

HumanFactorsinMedical

DeviceSoftwareDevelopment
MollyFolletteStory,PhD
FDA/CDRH/ODE
2012IQPCSoftwareDesignforMedicalDevicesEurope
Munich,Germany Februar 1,2012
2
Overview

Background

GuidanceforFDApremarketsubmissions

involvingmedicaldevicesoftware

GuidanceforFDApremarketsubmissions

involvinghumanfactorsdata

Humanfactors/usabilityvalidation
Human Factors Software HF/U Validation Background
3
DeviceUserInterface
User
Device
Information
Processing
Processing
Information
Perception
Input
Control
Actions
Output
INTERFACE INTERFACE
Human Factors Software HF/U Validation Background
4
DeviceUserInterface
ERROR
User
Device
Information
Processing
Processing
Information
Perception
Input
Control
Actions
Output
INTERFACE INTERFACE
E
R
R
O
R
E
R
R
O
R
Human Factors Software HF/U Validation Background
UseErrors
5
Source:

IEC62366:200710

andANSI/AAMIHE75:2009
Action
Unintended
Intended
Slip
Lapse
Mistake
Correct Use
Use Error
Attentional failure
Memory failure
Rule-based error
Knowledge-based
error
Ignorance-based
error
Abnormal Use
Human Factors Software HF/U Validation Background
CommonReasonsforUseErrors
6

Theuseenvironmenthasnegativeeffects

Thedemandsassociatedwithuseofthedevice

exceedtheuserscapabilities

Aspectsofdeviceuseareinconsistentwiththe

usersexpectationsorintuition

Thedeviceisusedinunexpectedways

Thedeviceisusedininappropriatebut

foreseeableways,forwhichadequatecontrols

werenotapplied
Human Factors Software HF/U Validation Background
7
CommonUserInterface(UI)Issues

UIcomplexitycausesuserconfusion,delayinuse,

orinabilitytousethedevice

UImakesitdifficultforusertocorrectdataentry

errorsormodifydevicesettingsinatimelyfashion

UIfalselycausestheusertobelieveacritical

situationexistswhenitdoesnot,orviceversa

UIdoesnotdrawattentiontodangerous

conditionsofdeviceoperationorpatientstatus

UIdoesnotpreventknown,likelydatainputerrors
Human Factors Software HF/U Validation Background
8
MedicalDeviceSoftware

PrimarystandardrecognizedbyFDA

GuidancedocumentsissuedbyFDA

GuidanceforFDApremarketsubmissions

involvingmedicaldevicesoftware
Human Factors Software HF/U Validation Background
9
SoftwareStandard
IEC62304:2006
Medicaldevicesoftware

Softwarelifecycleprocesses

SWdevelopment

SWmaintenance

SWriskmanagement

SWconfiguration

management

SWproblemresolution
Human Factors Software HF/U Validation Background
FDASWGuidance
10
GeneralPrinciplesof

SoftwareValidation
http://www.fda.gov/

MedicalDevices/

DeviceRegulationand

Guidance/

GuidanceDocuments/

ucm085281.htm

Note:issuedin2002
Human Factors Software HF/U Validation Background
FDASWGuidance
11
GuidancefortheContentof

PremarketSubmissionsfor

SoftwareContainedin

MedicalDevices
http://www.fda.gov/

MedicalDevices/

DeviceRegulationand

Guidance/

GuidanceDocuments/

ucm089543.htm

Note:issuedin2005
Human Factors Software HF/U Validation Background
12
FDASoftwareGuidance(1of7)
DeviceHazardAnalysis

Includeallforeseeablesoftwarerelatedhazards

Identificationofthehazard

Severityofthehazard

Cause(s)ofthehazard

Methodofcontrol(e.g.,hardware,software,alarm)

Correctivemeasures(e.g.,toeliminate,reduce,orwarn)

Verification

Validation
Human Factors Software HF/U Validation Background
13
FDASoftwareGuidance(2of7)
Softwarelevelofconcern

Estimate(intheabsenceofmitigations)oftheseverity

ofinjury

thatadevicefailureorlatentdesignflawcould

permitorinflict,eitherdirectlyorindirectly,ona

patientordeviceoperator:

Major:coulddirectlyresultindeathorseriousinjury

Moderate:coulddirectlyresultinminorinjury

Minor:unlikelytocauseanyinjury

DocumentationrecommendedforanFDA

submissiondependsonthelevelofconcern
Human Factors Software HF/U Validation Background
14
FDASoftwareGuidance(3of7)
Softwarerelateddocumentation:
Overview

Describethedesign

ofyourdevice

Describehowyourdesignwasimplemented

Demonstratehowthedevice,withyourdesign

implementation,wastested

Showthatyouidentifiedhazards

appropriatelyand

managedrisks

effectively

Providetraceability

tolinkthedesign,implementation,

testing,andriskmanagement
Human Factors Software HF/U Validation Background
15
FDASoftwareGuidance(4of7)
Softwarerelateddocumentation:
VerificationandValidation(V&V)

MINORlevelofconcern:

Softwarefunctionaltestplan

Pass/failcriteria

Testresults
Human Factors Software HF/U Validation Background
16
FDASoftwareGuidance(5of7)
Softwarerelateddocumentation:
VerificationandValidation(V&V)

MODERATElevelofconcern:
V&Vactivitiesattheunit,integration,andsystemlevel

Systemleveltestprotocol

Pass/failcriteria

Testresults
Human Factors Software HF/U Validation Background
17
FDASoftwareGuidance(6of7)
Softwarerelateddocumentation:
VerificationandValidation(V&V)

MAJORlevelofconcern:
V&Vactivitiesattheunit,integration,andsystemlevel

Unit,integrationandsystemleveltestprotocols

Pass/failcriteria

Testreport,summary,testresults
Human Factors Software HF/U Validation Background
18
FDASoftwareGuidance(7of7)
SoftwaredesignneedstoaddressHF

Weavehumanfactorsengineeringintoentiredesign

anddevelopmentprocess,includingdevicedesign

requirements,analyses,andtests

Considerdevicesafetyandusabilityissueswhen

developingflowcharts,statediagrams,prototyping

tools,andtestplans

Performtaskandfunctionanalyses,riskanalyses,

prototypetestsandreview,andfullusabilitytests

Includeparticipantsfromtheuserpopulation(s)
Human Factors Software HF/U Validation Background
19
MedicalDeviceHumanFactors

StandardsrecognizedbyFDA

GuidancedocumentsissuedbyFDA

GuidanceforFDApremarketsubmissions

involvinghumanfactorsdata
Human Factors Software HF/U Validation Background
20
HumanFactorsStandards(1of4)
AAMI/ANSIHE75:2009

Generalconsiderationsand

principles

Managingtheriskofuseerror

Usabilitytesting

Designelements

Controls

Software

Integratedsolutions

Mobilemedicaldevices

Homehealthcare
Human Factors Software HF/U Validation Background
21
HumanFactorsStandards(2of4)
ISO/IEC62366:2007
Medicaldevices

Application

ofusabilityengineeringto

medicaldevices

Usabilityengineeringprocess

Accompanyingdocument

Training
Human Factors Software HF/U Validation Background
22
HumanFactorsStandards(3of4)
ANSI/AAMI/ISO

14971:2007
Medicaldevices

Application

ofriskmanagementto

medicaldevices

Riskmanagement

Riskanalysis

Riskevaluation

Evaluationofoverallresidual

riskacceptability
Human Factors Software HF/U Validation Background
23
HumanFactorsStandards(4of4)
IEC6060118:2006
Medicalelectrical

equipment

Collateral

standard:alarmsystems

Alarmsystems

Alarmcondition

Generationofalarmsignals

Alarmpresets

Distributedalarmsystem

Etc.
Human Factors Software HF/U Validation Background
FDAHFGuidance
24
MedicalDeviceUseSafety:

IncorporatingHuman

FactorsEngineeringinto

RiskManagement
http://www.fda.gov/

MedicalDevices/

DeviceRegulationand

Guidance/

GuidanceDocuments/

ucm094460.htm

NOTE:issuedin2000
Human Factors Software HF/U Validation Background
t
FDAHFGuidance
25
ApplyingHF&UEtoOptimize

MedicalDeviceDesign
http://www.fda.gov/

MedicalDevices/

DeviceRegulationand

Guidance/

GuidanceDocuments/

ucm259748.htm

NOTE:issuedin2011 I is

notyetineffect

butit

reflectsFDACDRHscurrent

thinkingandapproachto

humanfactors
Human Factors Software HF/U Validation Background
26
2011Draft

HumanFactorsGuidance

Considerations:DeviceUsers,Use

EnvironmentsandUserInterfaces

PreliminaryAnalyses

ExploratoryHF/UsabilityEvaluations

HazardMitigationandControl

HumanFactors/UsabilityValidation
Human Factors Software HF/U Validation Background
HumanFactorsofDeviceUse
27
Human Factors Software HF/U Validation Background
28

Professionalornonprofessional

Jobtitleandresponsibilities

Knowledgeandexperiencelevels

Ageandfunctionalcapabilities

Physical,sensory/perceptual,cognitive/intellectual

Mentalandemotionalcondition
Users
Human Factors Software HF/U Validation Background
29

Clinicalenvironment

Hospital,clinic,etc.

Transitionalcareenvironment

Rehabilitation,assistedliving,longtermcare,etc.

Homeenvironment

House,mobilehome,townhouse,apartment,etc.

Communitysetting

Office,school,retail,outdoors,etc.

Mobileenvironment

Car,plane,train,bus,ambulance,medevac,etc.
UseEnvironment
Human Factors Software HF/U Validation Background
30

Examplesoftwareinteractiontasks

Dataentry(initial)

Datareview

Datarevision

Interactions(devicehardware)

Input

Knobs/dials,switches,buttons,keyboards,touchscreens,etc.

Output

Visual:displays(GUI),lights,controlsettings,etc.

Auditory:alerts/alarms,beeps,voice,motors,fans,etc.
UserInterface
Human Factors Software HF/U Validation Background
31
QualitySystemregulation:
21CFR820.30,DesignControls

Theneedforhumanfactorsisimplied:
c)

Designinput

includesneedsoftheuserandpatient
f)

Designverification

performancecriteriamet
g)

Designvalidation

devicesconformto

defineduser

needsandintendeduses

andshallinclude

testingof

productionunits

underactualorsimulateduse

conditions.

Designvalidationshallincludesoftware

validation

andriskanalysis.

[incl.userelatedrisks]
RegulatoryBasisforHFatFDA
Human Factors Software HF/U Validation Background
Humanfactors

engineering

processfor

medical

devices
32
Source:

ANSI/AAMI

HE75:2009
Human Factors Software HF/U Validation Background
33
PreliminaryAnalyses:Inquiries

Analyzeneedsofcurrentsystemusers

Whowillusethesystem?

Wherewilltheybeworking?

Whattaskswilltheyperform?

Analyzesystemuserinteractions

Howwilltheusersinteractwiththesystem?

Whatuseerrorsandfailuresmightoccur?

Howmighterrorsandfailuresbepreventedorthe

severityofanynegativeconsequencesbereduced?
Human Factors Software HF/U Validation Background
34
PreliminaryAnalyses:Methods(1of3)

Contextualinquiry
Userdemonstrates;researcherobserves,inquires

Interviewsandfocusgroups
Discussionsaretargeted;researcherisneutral

Functionandtaskanalyses
Breakdowndeviceuseintodiscretesteps
Identifyuserelatedhazardsassociatedwitheachstep

ofuse
Identifypotentialcausesandconsequencesofuser

encounteringeachhazard
Developriskmitigationstrategies,ifneeded
Human Factors Software HF/U Validation Background
35

Heuristicanalysis

Formallyevaluateuserinterfaceagainstwell

establisheddesignrulesorheuristicguidelines

Expertreview

Useclinicalandhumanfactorsexperts

Expertsprovidepersonalopinionsofusabilityand

safetyofuserinterface,basedonprofessional

knowledgeandexperience
PreliminaryAnalyses:Methods(2of3)
Human Factors Software HF/U Validation Background
36
PreliminaryAnalyses:Methods(3of3)
Twowaystodiscoveruserelatedhazards:
1.

Analyticaltechniques

Toidentifyuserelatedhazardsandrisks

Userelatedhazardsanduserbehaviorcanbedifficulttopredict

Tofocustheprocessandprioritizeresources
2.

Humanfactors/usabilityevaluations

Conductexploratory,handsontestingtodiscoverand

exploreunanticipated

hazards

SometimescalledUsabilityTesting

orUseTesting

orUser

Testing

orFormative

Evaluations
Human Factors Software HF/U Validation Background
37
ExploratoryHF/UStudies(1of2)

Donewhilethedeviceisunderdevelopment

Conductstudiesiterativelytooptimizethedevice

designandensurethehumanfactors/usability

validationtestingresultswillbesuccessful

Testingcaninvolvesimpleproductmockupsorearly

prototypes,orevenpartialdesigns(e.g.,components)

operatinginsimulatedusemodes

Testparticipantsshouldberepresentativeofthe

intendeduserpopulation(s)

Atearlystages,userelatedproblemscanbeaddressed

moreeasilyandlessexpensively
Human Factors Software HF/U Validation Background
ExploratoryHF/UStudies(2of2)
38

Conductseparatestudiesonlabelingand

training:

Assesstheclarityandeffectivenessofalllabeling(e.g.,

instructionsforuse,otherdocumentation,packaging)

Determinethelevelandnatureoftrainingthatwillbe

necessary(ifany)
Human Factors Software HF/U Validation Background
39
RiskMitigation

Developriskmitigationstrategiesasneeded:

Modifytheinterfacedesign,userinstructions,and/or

trainingtoaddresstheproblemsfound

Retesttoassesswhethermitigationstrategies:

Effectivelyreducedtheknownrisksand

Didnotintroduceanynewrisks

Residualrisk

isacceptableifitis:

Reasonablylimited,notcapableofeliminationorfurther

reduction,andoutweighedbythedevicesbenefits
Human Factors Software HF/U Validation Background
820.30(f)DesignVerification

820.30(g)DesignValidation
40
DesignVerification:

DidImaketheproductright?
DesignValidation:

DidImaketherightproduct?
Source:

KimberlyA.Trautman,FDA
Human Factors Software HF/U Validation Background
41
HumanFactors/UsabilityValidation

Demonstratesandprovidesevidencethata

medicaldevice,asdesigned,canbeusedsafely

andeffectively:

Bypeoplewhoarerepresentativeoftheintendedusers

Underexpecteduseconditions

Foressentialandcritical(highrisk)tasks

Includesbothobjectiveandsubjectivedata:

Useerrorsandfailuresareobservedandrecorded

Userfeedbackiscollectedafteruseregardingessential

andcriticaltaskerrors,failuresanddifficulties
Human Factors Software HF/U Validation Background
42
DeviceTestingConditions

Usefinalizeddevicedesignandlabeling

Identifyexpecteduseconditions

E.g.,lighting,sound,andactivity(distraction)levels

Allowrealisticdeviceuserinteractions

Provideparticipantswithdeviceastheywouldreceiveit

(e.g.,initsoriginalpackaging,withalldocumentation)

Allowparticipantstousethedeviceastheynormally

would(e.g.,withoutinterferencefromtestfacilitator)
Human Factors Software HF/U Validation Background
43
SelectionofTasksTested(1of2)

IncludeintheHF/Uvalidationstudyprotocol

andreportarationaleforthetasksyouchoose

toincludeinyourtesting

Basetaskselectiononresultsofpreliminaryanalyses

E.g.,taskanalysesandassessmentofuserelatedrisks

IncorporatefindingsofexploratoryHF/Ustudies

E.g.,tasksfoundtobeproblematicforusers
Human Factors Software HF/U Validation Background
44
SelectionofTasksTested(2of2)

Taskstestedinvalidationdonotnecessarily

includeeverythingintheinstructionsforuse

Includeessential

tasks

i.e.,tasksnecessaryfor

successfuluseofthedevice

Includesafetycriticaltasks

i.e.,tasksonwhichusers

couldmakeerrorsorcouldfailtocomplete,which

wouldhavenegativeclinicalimpact

Allwarningsandmostcautionstatementsinthedevice

labelingimplysafetycriticaltasks

Alltasksrequiringresponsestoalarmsaresafetycritical
Human Factors Software HF/U Validation Background
45
InstructionsandTraining

Participantinteractionwithinstructionsand

trainingshouldapproximatereality

Labelingusedindevicevalidationshouldbefinalversions

Trainingusedindevicevalidationshouldbecomparable

tothetrainingthatactualuserswillreceive
Human Factors Software HF/U Validation Background
46
ValidationTestData

Assessmentofdeviceuserinteractions

Userperformance

Throughobservation,automateddatacollection,etc.

Essentialandcriticaltasks

Userknowledge

Throughquestionnaireorinterview(wordedneutrally)

Essentialandcriticalknowledge

E.g.,warningsandcautions

Usersubjectivefeedback

Throughinterview,afteruserhascompletedalltesttasks

Overalluse,essential/criticaltasks,allperformancefailures
Human Factors Software HF/U Validation Background
Humanfactors

engineering

processfor

medical

devices
47
Source:

ANSI/AAMI

HE75:2009
Human Factors Software HF/U Validation Background
48
Advice:ConsultFDAEarly

Discussproductdevelopmentplanswith

FDAbeforeyourdesignisconsidered

final

(andchangeswouldbedifficult)

StaffinCDRH,CDERandCBERcanadvise

E.g.,onsoftwarelevelofconcern

FDAwillreviewhumanfactors/usabilitytesting

protocolsonrequest

Beforeimplementationisrecommended!
49
FDA/CDRHHumanFactorsProgram
http://www.fda.gov/

humanfactors

PremarketInfo

Design&Documentation

PostmarketInfo

Surveillance&Reporting

InfoforConsumers,

Patients,Caregivers

GeneralHumanFactors

InfoandResources
50
NewHFESAAMIWebSite
http://

www.medicaldevice

humanfactors.org

Resources

ConsultantDirectory

Organizations

Events
AcknowledgmentandQuestions
51
51
Contact:
Molly Follette Story: molly.story@fda.hhs.gov
FDA/HF web site: www/fda.gov/humanfactors
My thanks to:
Ron Kaye, FDA / CDRH / ODE / DAGID