The Interaction

Human Machine Interaction

-Dix, Finlay, Abowd, Beatle-

(Lecture Notes)

Ngo Thi Duyen

(duyennt@vnu.edu.vn)
Faculty of Information Technology
Vietnam National University, Hanoi
 Overview
 Interaction models
 Translation between user and system
 Ergonomics
 Physical characteristics of interaction
 Interaction styles
 The nature of user/system dialog
 Context
 Social, organizational, motivational

23年 3月 22日 The Interaction Page 2

 What is interaction?

 Communication
 User  system

 Number of different ways

 Batch input
 user provide all information to the computer at once
 Let the machine perform the task
  doesn’t support many tasks
 Highly interactive input devices & paradigms and the application
of VR
 User constantly provide instruction and receiving feedback

23年 3月 22日 The Interaction Page 3

 3.2 Models of interaction

 Terms of interaction

 Noman model

 Interactive framework

23年 3月 22日 The Interaction Page 4

3.2 Models of interaction

Terms of interaction
 Domain defines an area of expertise and knowledge in some real-world
activity
 Consists of concepts that highlight the important aspects
 E.g., graphic design (geometric shapes, a drawing surface, drawing utensil)
 Tasks are operations to manipulate the concepts of the domain
 E.g., construction of a specific geometric shape with particular attributes on the drawing
surface
 A goal is a desired output of a performed task
 E.g., to produce a solid red triangle centered on the canvas
 Intention is a specific action required to meet the goal
 User & system described by means of a language
 System’s language referred to as a core language (describe computational
attributes of the domain)
 User’s language referred to as a task language (desc. Psychological attributes)

23年 3月 22日 The Interaction Page 5

3.2 Models of interaction

Noman’s model

 The most influential in HCI because of its closeness to our intuitive

understanding of interaction between human & system
 Seven stages
 User establishes the goal
Goal
 Formulates intention
 Specifies actions at interface Execution Evaluation
 Executes action System
 Perceive system state
 Interprets system state Execution-Evaluation
 Evaluates system state w.r.t. goal cycle

 Concentrates on users’ view of the interface

 Doesn’t deal with the system’s communication through the interface

23年 3月 22日 The Interaction Page 6

3.2 Models of interaction

Noman’s model (cont’d)

 Why some interfaces cause problems to users?

 Gulfs of execution
 Difference between the user’s formulation of the actions to reach the
goal and the actions allowed by the system
 Interface should aim to reduce this gulf

 Gulfs of evaluation
 Distance between the physical presentation of the system state and the
expectation of the user

23年 3月 22日 The Interaction Page 7

3.2 Models of interaction

Human errors

 Slip
 Understand the system and goal
 Correct formulation of action
 Incorrect action
 Mistake
 May not even have right goal
 Fix things?
 Slip: better interface design
 Mistake: better understanding of the system

23年 3月 22日 The Interaction Page 8

3.2 Models of interaction

Abowd & Beale framework

 Extension of Norman’s model, called interaction framework

 Has 4 components: System, User, Input, Output
 Each has its own language
 Input & output together form the Interface

 4 steps in the interactive cycle : translation from one component to another

 User begins with the formulation of a goal and a task to achieve that goal
 manipulate the machine via the input
 Task is articulated within the input language
 The input language is translated into the core language as operations to be
performed by the system
 System transforms itself as described by the operations execution phase is
complete

23年 3月 22日 The Interaction Page 9

3.2 Models of interaction

Abowd & Beale framework (cont’d)

 System must now communicates to the user

 User observe the Output and assess the results of interaction relative to the
original goal
 End evaluation phase of interactive cycle

 Four translations involved in the interaction

 Articulation, performance, presentation, observation
trình diễn quan sát
(presentation) Ra (observation)

Hệ thống Người dùng

(ngôn ngữ nhân) (ngôn ngữ nhiệm vụ)

Vào
truyền đạt
(articulation)
23年 3月 22日 The Interaction Page 10
 3.3. Framework & HCI

 Framework
 A basis for discussing issues that relate to the interaction

 Field of ergonomics
 Address the issues on the user side of the interface…covering input,
output & user’s context
 Dialog design and interface styles placed along the input branch of
the framework
 Addressing both articulation & performance
 Presentation & screen design relate to the output branch of framework

 Entire framework can be placed within a social & organizational

context that also affects interaction

23年 3月 22日 The Interaction Page 11

 3.4. Ergonomics

 Study of the physical characteristics of interaction

 Also known as human factors

 But can also be used to mean much of HCI

 Ergonomics good at defining standards and guidelines

for constraining the way we design certain aspects of
systems

23年 3月 22日 The Interaction Page 12

 3.4. Ergonomics: examples

 Arrangement of control & displays

 E.g., controls grouped according to function or frequency of use, or
sequentially
 Surrounding environment
 E.g., seating arrangements adaptable to cope with all sizes of user
 Health issues
 E.g., physical position, environmental conditions (temperature,
humidity), lighting, noise
 Use of color
 E.g., use of red for warning, green for okay, awareness of color-
blindness, etc.
 Culturally determined

23年 3月 22日 The Interaction Page 13

 3.5. Interaction styles

 Interaction can be seen as a dialog between the computer & the

user

 Common interaction styles

 Common line interface
 Menus
 Natural language
 Question/answer and query dialogue
 Form-fills and speadsheets
 WIMP (window, icon, menu, pointing device)
 Point and click
 3D interfaces

23年 3月 22日 The Interaction Page 14

3.5. Interaction styles

Command line interface

 Way of expressing instructions to the computer directly

 Function keys, single characters, short abbreviations, whole words,
or a combination commands

 Offers direct access to system functionality

 Command names/abbreviations should be meaningful
 Suitable for repetitive tasks
 Better for expert users than novices
 Commands must be remembered

 Typical example: Unix system

23年 3月 22日 The Interaction Page 15

3.5. Interaction styles

Menus

 Set of options displayed on the screen

 Options visible
 Less recall, easier to use
 Rely on recognition, so names should be meaningful

 Selection by
 numbers, letters, arrow keys, mouse
 Combination (e.g., mouse plus accelerometers)

 Often options hierarchically grouped

 Sensible grouping is needed

 Restricted form of full WIMP system

23年 3月 22日 The Interaction Page 16

3.5. Interaction styles

Natural language

 The most attractive means of communicating with computers

 Natural language understanding, both speech & written input

 The subject of much interest and research
 But, ambiguity of natural language makes it very difficult for a machine
to understand
 E.g., the boy hit the dog with the stick

 Solutions to building systems

 To understand a restricted subsets of a language
 Provide sufficient information to disambiguate terms

 This can be called NATURAL language?

23年 3月 22日 The Interaction Page 17

3.5. Interaction styles

Question/answer & query dialog

 Question/answer interfaces
 User is asked a series of question (usually yes/no responses,
multiple choice
 Suitable to novice users, but restricted functionality
 Often used in information systems

 Query languages
 Used to retrieve information from database
 Requires understanding of database structure and language
syntax  require some expertise

23年 3月 22日 The Interaction Page 18

3.5. Interaction styles

Form-fills

 Primarily used for data entry or data retrieval

 Screen like paper form with slots to fill in
 These interfaces require
 Correction facilities, as users may change their mind or make a mistake
 Easy to learn and use for novice users

23年 3月 22日 The Interaction Page 19

3.5. Interaction styles

Spreadsheets

 A sophisticated variation of form filling

 Comprise a grid of cells, each of which contains a value or a formula

 Formula can involve values of other cells

 E.g., sum of all cells in this column
 User can enter and alter values and formulae in any order
 The system maintain consistency amongst the values displayed

 An attractive medium of interaction

 User is free to manipulate values at will
 The distinction between input & output is blurred  make interface more
flexible & natural

23年 3月 22日 The Interaction Page 20

3.5. Interaction styles

WIMP Interface

 Windows
 Icons
 Menus
 Pointers
 or Window, Icons, Mice, Pull-down menus

 Default style for majority of interactive computer systems

 Esp. PCs, desktop machine

23年 3月 22日 The Interaction Page 21

3.5. Interaction styles

Point & click interfaces

 Used in
 Multimedia
 Web browsers
 Hypertext

 Just click something

 Icons, text links or location of map

 Minimal typing

23年 3月 22日 The Interaction Page 22

3.5. Interaction styles

3D interfaces
 There is an increasing use of 3D effects in user interfaces
 E.g., virtual reality

 Simplest technique is where ordinary WIMP elements, buttons, scroll bars,

etc. are given 3D appearance using shading
 Used to highlight active areas
 Makes indiscriminate use of sculptural effect, on every text area, border
and menu
  all sense of differentiation is lost

 3D workshop
 Give more extra space
 In a more natural way than iconizing
windows

23年 3月 22日 The Interaction Page 23

 3.6. Elements of the WIMP interface

 Windows
 Icons
 Menus
 Pointers

 Buttons, toolbars, palettes, dialog boxes

23年 3月 22日 The Interaction Page 24

3.6. Elements of the WIMP interface

Windows
 Areas of the screen that behave as if they were independent terminals
 Can contain text or graphics
 Can be moved or resized

 More than one window can be on screen at once

 Allowing separate tasks to be visible at the same time
 Can overlap and obscure each other (cascaded), can be laid out next to another
(titled)

 Scrollbars
 Allow the user to move the contents of the window up & down
 or from side to side

 Title bars attached to the top of the window

 Describe the name of the window

23年 3月 22日 The Interaction Page 25

3.6. Elements of the WIMP interface

Icons

 Small picture or image

 Represent some object in the interface

 Windows can be shrunk to icons (iconifying the window)

 many windows can be available on the screen at the same time, ready to
be expanded to their full size by clicking on the icon

 Icons can take many forms

 Realistic representations of the objects that they stand for
 Highly stylized
 Arbitrary symbols

23年 3月 22日 The Interaction Page 26

3.6. Elements of the WIMP interface

Pointers

 Important component
 WIMP style relies on pointing & selecting thing such as icons

 Mouse and other alternatives such as joystick, trackballs

 Provide such tasks

 Different shapes of cursor are often used to distinguish modes

 Like icons, being graphical images

 Make sure the image has an obvious hot-spot
(location to which it points)

23年 3月 22日 The Interaction Page 27

3.6. Elements of the WIMP interface

Menus
 Present a choice of operations or services that can be performed by
the system at a given time

 Required option selected with pointer

 Are inefficient when there are many items

 Cascading menus are utilized
 In which item selection opens up another menu adjacent to the item

23年 3月 22日 The Interaction Page 28

3.6. Elements of the WIMP interface

Kind of menus
 Main menus
 Always visible to the user as a menu bar
 Often placed at the top of the screen or window
 Pop-up menu
 Can be hidden and upon request it will pop up on the screen
 Often used to present context-sensitive option
 E.g., allowing one to examine properties of a particular on-screen object
 Also used to access more global actions when the mouse is depressed over the screen
background
 Pull-down menus
 Are dragged down from the title at top of the screen by moving the mouse pointer into
the title bar area and pressing the button
 Fall-down menus
 Automatically appears when the mouse pointer enters the title bar, without the user
having press the button

23年 3月 22日 The Interaction Page 29

3.6. Elements of the WIMP interface

Buttons
 Individual and isolated regions within a display that can be selected
to invoke specific operations

 Special kinds
 Radio buttons: set of mutually exclusive choices
 Check boxes: set of non-exclusive choices

23年 3月 22日 The Interaction Page 30

3.6. Elements of the WIMP interface

Toolbars

 Collection of small buttons, each with icons

 Offering commonly used functions

 Functions of toolbar is similar to a menu bar

 But as the icons are smaller than the text  more functions can
be simultaneously displayed

 Often customizable
 Choose which toolbars to see
 Choose what options are on it

23年 3月 22日 The Interaction Page 31

3.6. Elements of the WIMP interface

Palettes
 Sometimes user unaware of the current mode

 A mechanism for making the set of possible modes and the active
mode visible to the user

 Usually a collection of icons that are reminiscent of the purpose of

the various modes
 E.g., in drawing package, icons indicate the pixel color or pattern that is
used to fill in the objects

 User can create palettes menus or toolbars

 In case of toolbars, he can drag the toolbar away from its normal
position & place it somewhere on the screen

23年 3月 22日 The Interaction Page 32

3.6. Elements of the WIMP interface

Dialogue boxes

 information windows that pop up to inform of an

important event or request information

 e.g., when saving a file, a dialogue box is displayed to

allow the user to specify the filename and location, one

the file is saved, the box disappears

23年 3月 22日 The Interaction Page 33

 3.7. Interactivity

 When looking at the interface

 It’s easy to focus on visually distinct parts (buttons, menus, text

areas)

 What about FEEL?

23年 3月 22日 The Interaction Page 34

3.7. Interactivity

Speech-driven interfaces
 Interactivity is the defining feature of the interactive system

 Speech-driven interface
 Rapidly improving, still inaccurate

 E.g., in an airline reservation system, as long as the system can

reliably recognize “yes” and “no”
 It can reflect back its understanding of what you said and seek
confirmation
 “you want a ticket from Ho Chi Minh City to Hanoi”

23年 3月 22日 The Interaction Page 35

3.7. Interactivity

Look and feel

 Interactivity is crucial in determining in the “feel” of an WIMP
environment

 WIMP systems have the same elements

 Windows, icons, menus, pointers, buttons, etc.
 But precise behavior of these elements differ within a single
environment & between environments
 E.g., different behavior of pull-down & fall-down menus

 Appearance + behavior = Look & feel

23年 3月 22日 The Interaction Page 36

3.7. Interactivity ha

Initiative
 who has the initiative?
 Old computer systems: computer
 WIMP interface: user

 WIMP exceptions
 Pre-emptive parts of the interface because of a problem it needs
information to continue
 E.g., Modal dialog boxes
 Come and will NOT go away
 Good for errors
 But use with care

23年 3月 22日 The Interaction Page 37

3.7. Interactivity ha

Error & repair

 Interactivity is critical in dealing with errors
 Cannot avoid errors
 But we can put them right
 By making sure user or system is able to tell when errors occur
 if users can detect errors, they can correct them

M: Hello, this is the Go Faster booking system. What would you like?
U: I want to flight from Ho Chi Minh City to Ha Noi
M: You want a ticket from Ho Chi Minh City to Hanam
U: No
M: Sorry, please confirm one at a time. Do you want to flight from Ho Chi
Minh City?
U: Yes
…
23年 3月 22日 The Interaction Page 38
 3.8. The context of the interaction
 In reality, users wok within a wide social and environmental context
 may influence the activity and motivation of the users

 Presence of other people  affect user’s performance in any task

 In case of peers, competition increase performance
 Desire to impress management and superiors
 But in case of the acquisition of new skills  inhibit performance owing to fear of
failure

 To perform well, users must be motivated

 Source of motivations: fear, allegiance, ambition, self-satisfaction

 User may loose motivation

 Introduced the system that doesn’t match the actual requirement
  Cause frustration

23年 3月 22日 The Interaction Page 39

 3.9. Experience, engagement, fun

 Usability in HCI?
 Effective, efficient, satisfaction

 It’s not sufficient that people CAN use a system, they

must WANT to use it

 Designing experience
 Physical engagement
 Managing value

23年 3月 22日 The Interaction Page 40

3.9. Experience, engagement, fun

Experience?

 Experience is a difficult thing to pin down

 More difficult to design it
 Psychology of experience
 Looking at climbing a rock face, can understand the feeling of total engagement
 When there is a balance between anxiety & boredom (Called flow)
flow

 If you do something you know, you can do it, it is not engaging

 do it while thinking of something else  become bored

 If you are half way up a rock, afraid !

 Flow comes when you’re teetering at the edge of your abilities, stretching
yourself to or a little beyond your limits

23年 3月 22日 The Interaction Page 41

3.9. Experience, engagement, fun

Designing experience

 Design virtual Christmas crackers

 Real cracker
 Cheap & cheerful
 Contents: bad joke, plastic toy, paper hat
 Pull & bang
 Virtual cracker
 Cheap & cheerful
 Contents: bad joke, web toy , cut-out mask
 Click & bang

23年 3月 22日 The Interaction Page 42

3.9. Experience, engagement, fun

Designing experience

 Mask
 A picture of a face with a
hat on it  not much
exciting !
 Instead, the link to a web
page provide a picture of
a mask that you can print
and, cut out, and wear
  change experience

23年 3月 22日 The Interaction Page 43

3.9. Experience, engagement, fun

Physical design

 Many constraints
 Ergonomic: cannot physically push button if they’re too small or close
 Physical
 size or nature of devices may force certain positions or styles of control
 high-voltage switches cannot be as small as low-voltage ones
 Legal & safety
 cooker controls far enough from the pans that don’t burn yourself, but high
enough to prevent children turning them on
 Context & environment
 easy to clean
 Aesthetic
 must look good
 Economic
 not cost too much

23年 3月 22日 The Interaction Page 44

3.9. Experience, engagement, fun

Design trade-offs

 Constraints are contradictory, need trade-offs

 Within categories
 E.g., safety: cooker controls
 Font panel: safer for adult
 Rear panel: safer for child

 Between categories
 E.g., ergonomics vs. physical- miniDisk remote
 Ergonomics: controls need to be bigger
 Physical: no room
 Solution: multifunction controls & reduced functionality

23年 3月 22日 The Interaction Page 45

3.9. Experience, engagement, fun

Managing value
 People use something
 ONLY IF it has perceived value
 AND value exceeds cost

 But Note
 Exceptions (e.g., habit)
 Value not necessarily personal gain or money

23年 3月 22日 The Interaction Page 46

3.9. Experience, engagement, fun

Judging value
 Value
 Helps us get our work done
 Fun
 Good for others

 Cost
 Download time
 Money
 Learning effort

23年 3月 22日 The Interaction Page 47

3.9. Experience, engagement, fun

Discounted future
 In economics, measure “net present value” used to calculate what a
future gain is worth today
 E.g., money invested today $1000 is worth the same as $2000 in three
years’ time

 We must understand people’s value systems

 and must be able to offer gains sooner and later
 or at least produce a very good demonstration of potential future gain so
that they have a perceived current value

23年 3月 22日 The Interaction Page 48

 General lessons

 If you want someone to do/use something

 Make it easy for them
 Understand their values

23年 3月 22日 The Interaction Page 49