EMOTIONAL

INTERACTION
 Overview
⊡ Emotions and the user experience

⊡ Expressive and annoying interface

□ how the ‘appearance’ of an interface can affect users

⊡ Models of emotion
□ Ortony et al (2005)

⊡ Automatic emotion recognition and emotional technologies

⊡ Persuasive technologies and behavioral change

□ how technologies can be designed to change people’s attitudes and behavior

⊡ Anthropomorphism
www.id-book.com 3
 Emotions And The
User Experience

⊡ HCI has traditionally been about designing efficient and effective

systems

⊡ Now more about how to design interactive systems that make

people respond in certain ways
□ e.g. to be happy, to be trusting, to learn, to be motivated

⊡ Emotional interaction is concerned with how we feel and react

when interacting with technologies

www.id-book.com 4
 Emotional
interaction
⊡ What makes us happy, sad, annoyed, anxious, frustrated, motivated,
delirious and so on
□ translating this into different aspects of the user experience

⊡ Why people become emotionally attached to certain products (e.g.

virtual pets)

⊡ Can social robots help reduce loneliness and improve wellbeing?

⊡ How to change human behavior through the use of emotive feedback

www.id-book.com 6
 Emotional
Design Model
⊡ Norman, Ortony and Revelle (2004) model of emotion

www.id-book.com 8
 Claims From
Model
⊡ Our emotional state changes how we think
□ when frightened or angry we focus narrowly and body
responds by tensing muscles and sweating

■ more likely to be less tolerant

□ when happy we are less focused and the body relaxes

■ more likely to overlook minor problems and be more

creative
www.id-book.com 9
 “
Do you feel more creative when
you are in a happy mood?

Do you get less work done when

you are feeling stressed?

www.id-book.com 10
 Expressive Interfaces

⊡ Provide reassuring feedback that can be both informative and fun

⊡ But can also be intrusive, causing people to get annoyed and

even angry

⊡ Color, icons, sounds, graphical elements and animations are used

to make the ‘look and feel’ of an interface appealing
□ conveys an emotional state

⊡ In turn this can affect the usability of an interface

□ people are prepared to put up with certain aspects of an interface (e.g. slow
download rate) if the end result is appealing and aesthetic

www.id-book.com 11
 Friendly
Interfaces
⊡ Microsoft pioneered friendly interfaces for technophobes -
‘At home with Bob’ software

⊡ 3D metaphors based on familiar places (e.g. living rooms)

⊡ Agents in the guise of pets (e.g. bunny, dog) were included

to talk to the user
□ Make users feel more at ease and comfortable

www.id-book.com 12
 Bob

www.id-book.com 13
 Clippy
⊡ Why was Clippy disliked
by so many?

⊡ Was it annoying,
distracting,
patronising or other?

⊡ What sort of user

liked Clippy?

www.id-book.com 14
 Frustrating
⊡ Many causes: Interfaces
□ When an application doesn’t work properly or crashes

□ When a system doesn’t do what the user wants it to do

□ When a user’s expectations are not met

□ When a system does not provide sufficient information to enable the

user to know what to do

□ When error messages pop up that are vague, obtuse or condemning

□ When the appearance of an interface is garish, noisy, gimmicky or

patronizing

www.id-book.com
□ When a system requires users to carry out too many steps to15
 Gimmicks
⊡ Amusing to the designer but not the user, e.g.

□ Clicking on a link to a website only to discover

that it is still ‘under construction’

www.id-book.com 16
 Error Messages

“
“The application Word
Wonder has unexpectedly
quit due to a type 2 error.”

www.id-book.com 17
 Error Messages

Why not instead:

“the application has expectedly
quit due to poor coding in the
operating system”

www.id-book.com 18
 Error Messages

⊡ Shneiderman’s guidelines for error messages

include:
■ avoid using terms like FATAL, INVALID, BAD
■ Audio warnings
■ Avoid UPPERCASE and long code numbers
■ Messages should be precise rather than vague
■ Provide context-sensitive help

www.id-book.com 19
 Website Error
Messages

www.id-book.com 20
 More Helpful Error Message

“The requested page /helpme

is not available on the web
server.
“ If you followed a link or bookmark to
get to this page, please let us know, so
that we can fix the problem. Please
include the URL of the referring page
as well as the URL of the missing
page.
Otherwise check that you have
typed the address of the web
page correctly. The Web site you seek
Cannot be located, but
Countless more exist.”

www.id-book.com 21
 Should computers
say they’re sorry?

⊡ Reeves and Naas (1996) argue that computers

should be made to apologize

⊡ Should emulate human etiquette

⊡ Would users be as forgiving of computers saying

sorry as people are of each other when saying
sorry?

www.id-book.com 22
 Should computers
say they’re sorry?

⊡ How sincere would they think the computer was

being? For example, after a system crash:
□ “I’m really sorry I crashed. I’ll try not to do it again”

⊡ How else should computers communicate with

users?

www.id-book.com 23
 Detecting Emotions And
Emotional Technology

⊡ Sensing technologies used to measure GSR, facial expressions,

gestures, body movement

⊡ Aim is to predict user’s emotions and aspects of their behavior –

⊡ E.g. what is someone most likely to buy online when feeling sad,
bored or happy

www.id-book.com 24
 Facial Coding

⊡ Measures a user’s emotions as they interact with a computer or

tablet

⊡ Analyses images captured by a webcam of their face

⊡ Uses this to gauge how engaged the user is when looking at

movies, online shopping sites and ads

⊡ 6 core expressions - sadness, happiness, disgust, fear, surprise

and anger

www.id-book.com 25
www.id-book.com 26
 How to use the
emotional data?

⊡ If user screws up their face when an ad pops up -> feel disgust

⊡ If start smiling -> they are feeling happy

⊡ Website can adapt its ad, movie storyline or content to match

user’s emotional state
⊡ Eye-tracking, finger pulse, speech and words/phrases also
analysed when tweeting or posting to Facebook

www.id-book.com 27
 Persuasive
Technologies And
Behavioral Change

⊡ Interactive computing systems deliberately designed to

change people’s attitudes and behaviors (Fogg, 2003)

⊡ A diversity of techniques now used to change what they do or

think
□ Pop-up ads, warning messages, reminders, prompts, personalized messages,
recommendations, Amazon 1-click

□ Commonly referred to as nudging

www.id-book.com 29
 Nintendo’s
Pocket Pikachu
⊡ Changing bad habits and improving well being
□ Designed to motivate children to be more physically
active on a regular basis

□ owner of the digital pet that ‘lives’ in the device is

required to walk, run, or jump

□ If owner does not exercise the virtual pet becomes

angry and refuses to play anymore

www.id-book.com 30
 How effective?

⊡ Is the use of novel forms of interactive technologies (e.g., the

combination of sensors and dynamically updated information)
that monitor, nag, or send personalized messages
intermittently to a person more effective at changing a
person’s behavior than non-interactive methods, such as the
placement of warning signs, labels, or ads in prominent
positions?

www.id-book.com 31
 Which is most
effective?

www.id-book.com 32
www.id-book.com 33
www.id-book.com 34
 Tracking Devices

⊡ Mobile apps designed to help people monitor and

change their behaviour (e.g. fitness, sleeping, weight)

⊡ Can compare with online leader boards and charts, to

show how they have done in relation to their peers and
friends

⊡ Also apps that encourage reflection that in turn increase

well-being and happiness

www.id-book.com 35
 Energy reduction

www.id-book.com 36
 The Tidy Street
Project
⊡ large-scale visualization of the street’s electricity
usage
□ stencilled display on the road surface using chalk

□ provided real-time feedback that all could see change each

day

□ reduced electricity consumption by 15%

www.id-book.com 37
 Anthropomorphism

⊡ Attributing human-like qualities to inanimate objects

(e.g. cars, computers)

⊡ Well known phenomenon in advertising

□ Dancing butter, drinks, breakfast cereals

⊡ Much exploited in human-computer interaction

□ Make user experience more enjoyable, more
motivating, make people feel at ease, reduce anxiety

www.id-book.com 39
 Which do you prefer?

1. As a welcome message

⊡ “Hello Chris! Nice to see you again. Welcome

back. Now what were we doing last time? Oh
yes, exercise 5. Let’s start again.”

⊡ “User 24, commence exercise 5.”

www.id-book.com 40
 Which do you prefer?

2. Feedback when get something wrong

1. “Now Chris, that’s not right. You can do better than

that. Try again.”
2. “Incorrect. Try again.”

Is there a difference as to what you prefer depending

on type of message? Why?

www.id-book.com 41
 Evidence To Support
Anthropomorphism

⊡ Reeves and Naas (1996) found that computers

that flatter and praise users in education software
programs -> positive impact on them

“Your question makes an important and useful

distinction. Great job!”
⊡ Students were more willing to continue with
exercises with this kind of feedback

www.id-book.com 42
 Criticism Of
Anthropomorphism
⊡ Deceptive, make people feel anxious, inferior or stupid

⊡ People tend not to like screen characters that wave their fingers at the user
and say:
□ Now Chris, that’s not right. You can do better than that.Try again.”

⊡ Many prefer the more impersonal:

□ “Incorrect. Try again.”

⊡ Studies have shown that personalized feedback is considered to be less

honest and makes users feel less responsible for their actions (e.g. Quintanar,
1982)

www.id-book.com 43
 Virtual Characters

⊡ Appearing on our screens in the form of:

□ Sales agents, characters in videogames, learning
companions, wizards, pets, newsreaders

⊡ Provides a persona that is welcoming, has

personality and makes user feel involved
with them

www.id-book.com 44
 Disadvantages

⊡ Can lead people into false sense of belief,

enticing them to confide personal secrets with
chatterbots

⊡ Annoying and frustrating

□ e.g. Clippy

⊡ May not be trustworthy

□ virtual shop assistants?

www.id-book.com 45
 Virtual Agents

⊡ What do the virtual agents do?

⊡ Do they elicit an emotional response in you?
⊡ Do you trust them?
⊡ What is the style of interaction?
⊡ What facial expression do they have?
⊡ Are they believable, pushy, helpful?
⊡ Would it be different if they were male? If so, how?

www.id-book.com 46
 What makes a virtual
agent believable?

⊡ Believability refers to the extent to which users

come to believe an agent’s intentions and
personality
⊡ Appearance is very important
□ Are simple cartoon-like characters or more realistic characters, resembling the
human form more believable?

⊡ Behaviour is very important

□ How an agent moves, gestures and refers to objects on the screen

□ Exaggeration of facial expressions and gestures to show underlying emotions

(c.f. animation industry)

www.id-book.com 47
 Robot-like or cuddly?

⊡ Which do you prefer and why?

www.id-book.com 48
 Implications
⊡ Should we create products that adapt according
to people’s different emotional states?
□ When people are feeling angry should an
interface be more attentive and informative
than when they are happy?

www.id-book.com 49
 “
designers “can get away with more”
for products intended to be used
during leisure time than those
designed for serious tasks
Norman

www.id-book.com 50
 Summary
⊡ Emotional aspects of interaction design concerned with how to facilitate
certain states (e.g. pleasure) or avoid reactions (e.g. frustration)
⊡ Well-designed interfaces can elicit good feelings in people
⊡ Aesthetically pleasing interfaces can be a pleasure to use
⊡ Expressive interfaces can provide reassuring feedback to users
⊡ Badly designed interfaces make people frustrated, annoyed, or angry
⊡ Emotional technologies can be designed to persuade people to change
their behaviors or attitudes
⊡ Anthropomorphism is the attribution of human qualities to objects
⊡ Virtual agents and robot pets have been developed to make people feel
motivated, reassured, and in a good mood

www.id-book.com 51
 Assignment
This assignment requires you to write a critique of the persuasive impact of a virtual
agent b y considering what it would take for a virtual agent to be believable,
trustworthy, and convincing.
⊡ Look at a website that has a virtual assistant, e.g. Anna at Ikea.com, Jenn at Alaskaair.com
and answer the following:
⊡ What does the virtual agent do?
⊡ What type of agent is it?
⊡ Does it elicit an emotional response from you? If so, what kind?
⊡ What kind of personality does it have?
⊡ How is this expressed?
⊡ What kinds of behavior does it exhibit?
⊡ What are its facial expressions like?
⊡ What is its appearance like? Is it realistic or cartoon-like?
⊡ Where does it appear on the screen?

www.id-book.com 52
 Assignment
⊡ How does it communicate with the user (text or speech)?
⊡ Is the level of discourse patronizing or at the right level?
⊡ Is the agent helpful in guiding the user towards making a purchase or finding out something?
⊡ Is it too pushy?
⊡ What gender is it? Do you think this makes a difference?
⊡ Would you trust the agent to the extent that you would be happy to buy a product from it or
follow its guidance? If not, why not?
⊡ What else would it take to make the agent persuasive?

www.id-book.com 53
 Assignment
b. Next, look at an equivalent website that does not include an agent but is based on a
conceptual model of browsing, e.g. Amazon.com. How does it compare with the agent-
based site you have just looked at?
⊡ Is it easy to find information?
⊡ What kind of mechanism does the site use to make recommendations and guide
the user in making a purchase or
⊡ finding out information?
⊡ Is any kind of personalization used at the interface to make the user feel
welcome or special?
⊡ Would the site be improved by having an agent? Explain your reasons either
way.

c. Finally, discuss which site you would trust most and give your reasons for this.

www.id-book.com 54