CHAPTER 4: INTERFACES AND Lecturer

Name: Suhaila binti Khalip

INTERACTIONS Email: suhaila@msu.edu.my

TOPICS
1. Paradigms
2. Interface types
3. Which interface?

2
TOPIC OUTCOMES
• Provide an overview of the many different kinds of interfaces
• Highlight the main design and research issues for each of the
interfaces
• Discuss the difference between graphical and natural user
interfaces
• Consider which interface is best for given application or activity

TOPIC 1: INTRODUCTION
• How to enable people to access and interact with information in
their work, social, and everyday lives
• Designing user experiences for people using interfaces that are
part of the environment with no controlling devices
• What form to provide contextually-relevant information to people
at appropriate times and places
• Ensuring that information, that is passed around via interconnected
displays, devices, and objects, is secure and trustworthy

4
TOPIC 2: INTERFACE TYPES
• Consider which interface is best for a given application or activity
• Many, many kinds now

1980s interfaces 1990s interfaces 2000s interfaces

• Command • Advanced graphical • Mobile

• WIMP/GUI (multimedia, virtual • Multimodal
reality, information • Shareable
visualization)
• Tangible
• Web Speech (voice) • Augmented and
• Pen, gesture, and mixed reality
touch • Wearable
• Appliance • Robotic

COMMAND-BASED
• Commands such as abbreviations (e.g. ls) typed in at the prompt to
which the system responds (e.g. listing current files)
• Some are hard wired at keyboard, others can be assigned to keys
• Efficient, precise, and fast
• Large overhead to learning set of commands

6
SECOND LIFE COMMAND-BASED INTERFACE
FOR VISUALLY IMPAIRED USERS
• Second Life command based interface for visually impaired users.

RESEARCH AND DESIGN ISSUES

• Form, name types and structure are key research questions
• Consistency is most important design principle
• e.g. always use first letter of command

• Command interfaces popular for web scripting

8
WIMP AND GUI
• Xerox Star first WIMP -> rise to GUIs
• Windows
 could be scrolled, stretched, overlapped, opened, closed, and moved around the screen using the mouse

• Icons
 represented applications, objects, commands, and tools that were opened when clicked on

• Menus
 offering lists of options that could be scrolled through and selected

• Pointing device
 a mouse controlling the cursor as a point of entry to the windows, menus, and icons on the screen

GUIS
• Same basic building blocks as WIMPs but more varied
 Colour, 3D, sound, animation,
 Many types of menus, icons, windows

• New graphical elements, e.g.

 toolbars, docks, rollovers

• Challenge now is to design GUIs that are best suited for tablet,
smartphone and smartwatch interfaces

10
WINDOWS
• Windows were invented to overcome physical constraints of a
computer display
 enable more information to be viewed and tasks to be performed

• Scroll bars within windows also enable more information to be

viewed
• Multiple windows can make it difficult to find desired one
 listing, iconising, shrinking are techniques that help

11

EXAMPLE
• The boxy look of the first generation of GUIs. The window presents
several check boxes, notes boxes, and options as square buttons

12
APPLE’S SHRINKING WINDOWS

13

SAFARI PANORAMA WINDOW VIEW

• A window management technique provided in Safari

14
SELECTING A COUNTRY FROM A
SCROLLING WINDOW
• Screenshot of
Camino
browser

15

IS THIS METHOD ANY BETTER?

• An excerpt of the listing of countries in alphabetical order from
interflora.co.uk

16
RESEARCH AND DESIGN ISSUES
• Window management
 enables users to move fluidly between different windows (and monitors)

• How to switch attention between windows without getting distracted

• Design principles of spacing, grouping, and simplicity should be
used

17

MENUS
• A number of menu interface styles
 flat lists, drop-down, pop-up, contextual, and expanding ones, e.g., scrolling and cascading

• Flat menus
 good at displaying a small number of options at the same time and where the size of the display is small, e.g.
iPods
 but have to nest the lists of options within each other, requiring several steps to get to the list with the desired
option
 moving through previous screens can be tedious

18
EXPANDING MENUS
• Enables more options to be shown on a single screen than is possible
with a single flat menu
• More flexible navigation, allowing for selection of options to be
done in the same window
• Most popular are cascading ones
 primary, secondary and even tertiary menus
 downside is that they require precise mouse control
 can result in overshooting or selecting wrong options

19

CASCADING MENU

20
CONTEXTUAL MENUS
• Provide access to often-used commands that make sense in the
context of a current task
• Appear when the user presses the Control key while clicking on an
interface element
 e.g., clicking on a photo in a website together with holding down the Control key results in options ‘open it in a
new window,’ ‘save it,’ or ‘copy it’

• Helps overcome some of the navigation problems associated with

cascading menus

21

WINDOWS JUMP LIST MENU

• Windows jump list

22
RESEARCH AND DESIGN ISSUES
• What are best names/labels/phrases to use?
• Placement in list is critical
 Quit and save need to be far apart

• Choice of menu to use determined by application and type of

system
 flat menus are best for displaying a small number of options at one time
 expanding menus are good for showing a large number of options

23

ICON DESIGN
• Icons are assumed to be easier to learn and remember than
commands
• Can be designed to be compact and variably positioned on a
screen
• Now pervasive in every interface
 e.g. represent desktop objects, tools (e.g. paintbrush), applications (e.g. web browser), and operations (e.g. cut,
paste, next, accept, change)

24
ICONS
• Since the Xerox Star days icons have changed in their look and
feel:
 black and white -> color, shadowing, photorealistic images, 3D rendering, and animation

• Many designed to be very detailed and animated making them

both visually attractive and informative
• GUIs now highly inviting, emotionally appealing, and feel alive

25

ICON FORMS
• The mapping between the representation and underlying referent
can be:
 similar (e.g., a picture of a file to represent the object file)
 analogical (e.g., a picture of a pair of scissors to represent ‘cut)
 arbitrary (e.g., the use of an X to represent ‘delete’)

• Most effective icons are similar ones

• Many operations are actions making it more difficult to represent
them
 use a combination of objects and symbols that capture the salient part of an action

26
EARLY ICONS
• Poor icon set from early 1990s. What do you think they mean and
why are they so bad?

27

NEWER ICONS
• Constructing genres of Aqua icons used for the Mac. The top row of
icons have been designed for user applications and the bottom row
for utility applications

28
SIMPLE FLAT 2D ICONS
• Flat 2D icons designed
for smartphones apps

29

ACTIVITY
• Sketch simple icons to represent the following operations to appear
on a digital camera screen:
• Turn image 90 degrees sideways
• Auto-enhance the image
• Fix red-eye
• Crop the image

• Show them to someone else and see if they can understand what
each represents

30
BASIC EDIT ICONS ON IPHONE
• Which is which?
• Are they easy to understand
• Are they distinguishable?
• What representation forms are used?
• How do yours compare?

31

RESEARCH AND DESIGN ISSUES

• There is a wealth of resources now so do not have to draw or invent
new icons from scratch
• guidelines, style guides, icon builders, libraries

• Text labels can be used alongside icons to help identification for

small icon sets
• For large icon sets (e.g. photo editing or word processing) use
rollovers

32
MULTIMEDIA
• Combines different media within a single interface with various
forms of interactivity
 graphics, text, video, sound, and animations

• Users click on links in an image or text

-> another part of the program
-> an animation or a video clip is played
->can return to where they were or move on to another place

33

BIOBLAST MULTIMEDIA LEARNING

ENVIRONMENT
• Screen
dump from
the
multimedia
environment
BioBLAST

34
PROS AND CONS
• Facilitates rapid access to multiple representations of information
• Can provide better ways of presenting information than can any
media alone
• Can enable easier learning, better understanding, more
engagement, and more pleasure
• Can encourage users to explore different parts of a game or story
• Tendency to play video clips and animations, while skimming
through accompanying text or diagrams

35

RESEARCH AND DESIGN ISSUES

• How to design multimedia to help users explore, keep track of, and
integrate the multiple representations
 provide hands-on interactivities and simulations that the user has to complete to solve a task
 Use ‘dynalinking,’ where information depicted in one window explicitly changes in relation to what happens in
another (Scaife and Rogers, 1996).

• Several guidelines that recommend how to combine multiple media

for different kinds of task

36
VIRTUAL REALITY
• Computer-generated graphical simulations providing:
 “the illusion of participation in a synthetic environment rather than external observation of such an environment”
(Gigante, 1993)

• Provide new kinds of experience, enabling users to interact with

objects and navigate in 3D space
• Create highly engaging user experiences

37

PROS AND CONS

• Can have a higher level of fidelity with objects they represent
compared to multimedia
• Induces a sense of presence where someone is totally engrossed by
the experience
• “a state of consciousness, the (psychological) sense of being in the virtual
environment” (Slater and Wilbur, 1999)

• Provides different viewpoints: 1st and 3rd person

• Head-mounted displays are uncomfortable to wear, and can cause
motion sickness and disorientation

38
RESEARCH AND DESIGN ISSUES
• Much research on how to design safe and realistic VRs to facilitate
training
 e.g. flying simulators
 help people overcome phobias (e.g. spiders, talking in public)

• Design issues
 how best to navigate through them (e.g. first versus third person)
 how to control interactions and movements (e.g. use of head and body movements)
 how best to interact with information (e.g. use of keypads, pointing, joystick buttons);
 level of realism to aim for to engender a sense of presence

39

WHICH IS THE MOST ENGAGING GAME OF

SNAKE?
• Two screenshots from the game Snake

40
INFORMATION VISUALIZATION AND
DASHBOARDS
• Computer-generated interactive graphics of complex data
• Amplify human cognition, enabling users to see patterns, trends, and
anomalies in the visualization (Card et al, 1999)
• Aim is to enhance discovery, decision-making, and explanation of
phenomena
• Techniques include:
 3D interactive maps that can be zoomed in and out of and which present data via webs, trees, clusters,
scatterplot diagrams, and interconnected nodes

41

DASHBOARDS
• Show screenshots of data updated over periods of time - to be
read at a glance
• Usually not interactive - slices of data that depict current state of a
system or process
• Need to provide digestible and legible information for users
 design its spatial layout so intuitive to read when first looking at it
 should also direct a user’s attention to anomalies or unexpected deviations

42
WHICH DASHBOARD IS BEST?
• Screenshots from two dashboards (a) British Airway frequent flier
club that shows how a member has flown since joining them and (b)
London City that provides information feeds. Which is the easier to
red and most informative?

43

WHICH DASHBOARD IS BEST?

44
RESEARCH AND DESIGN ISSUES
• Whether to use animation and/or interactivity
• What form of coding to use, e.g. color or text labels
• Whether to use a 2D or 3D representational format
• What forms of navigation, e.g. zooming or panning,
• What kinds and how much additional information to provide, e.g.
rollovers or tables of text
• What navigational metaphor to use

45

WEB
• Early websites were largely text-based, providing hyperlinks
• Concern was with how best to structure information to enable users
to navigate and access it easily and quickly
• Nowadays, more emphasis on making pages distinctive, striking,
and pleasurable
• Need to think of how to design information for multi-platforms -
keyboard or touch?
 e.g. smartphones, tablets, PCs

46
USABILITY VERSUS ATTRACTIVE?
• Vanilla or multi-flavor design?
 Ease of finding something versus aesthetic and enjoyable experience

• Web designers are:

 “thinking great literature”

• Users read the web like a:

 “billboard going by at 60 miles an hour” (Krug, 2000)

• Need to determine how to brand a web page to catch and keep

‘eyeballs’

47

IN YOUR FACE ADS

• Web advertising is often intrusive and pervasive
• Flashing, aggressive, persistent, annoying
• Often need to be ‘actioned’ to get rid of
• What is the alternative?

48
RESEARCH AND DESIGN ISSUES
• Need to consider how best to design, present, and structure
information and system behavior
• But also content and navigation are central
• Veen’s (2001) design principles
(1)Where am I?
(2)Where can I go?
(3) What’s here?

49

ACTIVITY
• Look at the Nike.com website
• What kind of website is it?
• How does it contravene the design principles outlined by Veen?
• Does it matter?
• What kind of user experience is it providing for?
• What was your experience of engaging with it?

50
CONSUMER ELECTRONICS AND
APPLIANCES
• Everyday devices in home, public place, or car
 e.g. washing machines, remotes, photocopiers, printers and navigation systems)

• And personal devices

 e.g. MP3 player, digital clock and digital camera

• Used for short periods

 e.g. putting the washing on, watching a program, buying a ticket, changing the time, taking a snapshot

• Need to be usable with minimal, if any, learning

51

A TOASTER
• A typical toaster with basic physical controls

52
RESEARCH AND DESIGN ISSUES
• Need to design as transient interfaces with short interactions
• Simple interfaces
• Consider trade-off between soft and hard controls
 e.g. buttons or keys, dials or scrolling

53

MOBILE
• Handheld devices intended to be used while on the move
• Have become pervasive, increasingly used in all aspects of
everyday and working life
• Apps running on mobiles have greatly expanded, e.g.
 used in restaurants to take orders
 car rentals to check in car returns
 supermarkets for checking stock
 in the streets for multi-user gaming
 in education to support life-long learning

54
THE ADVENT OF THE IPHONE APP
• A whole new user experience that was designed primarily for
people to enjoy
 many apps not designed for any need, want or use but purely for idle moments to have some fun
 e.g. iBeer developed by magician Steve Sheraton
 ingenious use of the accelerometer that is inside the phone

55

IBEER APP
• The iBeer smartphone app

56
QR CODES AND CELL PHONES
• QR code appearing on
a magazine page

57

MOBILE CHALLENGES
• Smaller screens, small number of physical keys and restricted
number of controls
• Innovative physical designs including:
 roller wheels, rocker dials, up/down ‘lips’ on the face of phones, 2-way and 4-way directional keypads,
softkeys, silk-screened buttons

• Usability and preference varies

 depends on the dexterity and commitment of the user

• Smartphones overcome mobile physical constraints through using

multi-touch displays

58
RESEARCH AND DESIGN ISSUES
• Mobile interfaces can be tricky and cumbersome to use for those
with poor manual dexterity or ‘fat’ fingers
• Key concern is hit area
 area on the phone display that the user touches to make something happen, such as a key, an icon, a button or
an app
 space needs to be big enough for fat fingers to accurately press
 if too small the user may accidentally press the wrong key

59

SPEECH
• Where a person talks with a system that has a spoken language
application, e.g. timetable, travel planner
• Used most for inquiring about very specific information, e.g. flight
times or to perform a transaction, e.g. buy a ticket
• Also used by people with disabilities
 e.g. speech recognition word processors, page scanners, web readers, home control systems

60
HAVE SPEECH INTERFACES COME OF AGE?

61

GET ME A HUMAN OPERATOR!

• Most popular use of speech interfaces currently is for call routing
• Caller-led speech where users state their needs in their own words
 e.g. “I’m having problems with my voice mail”

• Idea is they are automatically forwarded to the appropriate

service
• What is your experience of speech systems?

62
FORMAT
• Directed dialogs are where the system is in control of the
conversation
• Ask specific questions and require specific responses
• More flexible systems allow the user to take the initiative:
 e.g. “I’d like to go to Paris next Monday for two weeks.”

• More chance of error, since caller might assume that the system is
like a human
• Guided prompts can help callers back on track
 e.g. “Sorry I did not get all that. Did you say you wanted to fly next Monday?”

63

RESEARCH AND DESIGN ISSUES

• How to design systems that can keep conversation on track
 help people navigate efficiently through a menu system
 enable them to easily recover from errors
 guide those who are vague or ambiguous in their requests for information or services

• Type of voice actor (e.g. male, female, neutral, or dialect)

 do people prefer to listen to and are more patient with a female or male voice, a northern or southern accent?

64
PEN
• Enable people to write, draw, select, and move objects at an
interface using lightpens or styluses
 capitalize on the well-honed drawing skills developed from childhood

• Digital pens, e.g. Anoto, use a combination of ordinary ink pen with
digital camera that digitally records everything written with the pen
on special paper

65

PROS AND CONS

• Allows users to quickly and easily annotate existing documents
• Can be difficult to see options on the screen because a user’s hand
can occlude part of it when writing
• Can have lag and feel clunky

66
TOUCH
• Touch screens, such as walk-up kiosks, detect the presence and
location of a person’s touch on the display
• Multi-touch support a range of more dynamic finger tip actions, e.g.
swiping, flicking, pinching, pushing and tapping
• Now used for many kinds of displays, such as Smartphones, iPods,
tablets and tabletops

67

RESEARCH AND DESIGN ISSUES

• More fluid and direct styles of interaction involving freehand and
pen-based gestures
• Core design concerns include whether size, orientation, and shape
of touch displays effect collaboration
 Much faster to scroll through wheels, carousels and bars of thumbnail images or lists of options by finger flicking
 More cumbersome, error-prone and slower to type using a virtual keyboard on a touch display than using a
physical keyboard

68
RESEARCH AND DESIGN ISSUES
Will finger-flicking, swiping,
stroking and touching a screen
result in new ways of consuming,
reading, creating and searching
digital content?
The Swype interface developed
mobile touch displays

69

AIR-BASED GESTURES

• Uses camera recognition, sensor and computer vision techniques

 can recognize people’s body, arm and hand gestures in a room
 systems include Kinect

• Movements are mapped onto a variety of gaming motions, such as

swinging, bowling, hitting and punching
• Players represented on the screen as avatars doing same actions

70
HOME ENTERTAINMENT
Universal appeal
 young children, grandparents,
professional gamers, technophobes

Touchless gesturing in the

operating theatre

71

GESTURES IN THE OPERATING THEATRE

• A touchless system that
recognizes gestures
• surgeons can interact with and
manipulate MRI or CT images
 e.g. two-handed gestures for zooming and panning

• The MusicJacket prototype with

embedded actuators that nudge
the player

72
RESEARCH AND DESIGN ISSUES
• How does computer recognize and delineate user’s gestures?
 Deictic and hand waving

• Does holding a control device feel more intuitive than controller free
gestures?
 For gaming, exercising, dancing

73

HAPTIC
• Tactile feedback
 applying vibration and forces to a person’s body, using actuators that are embedded in their clothing or a
device they are carrying, such as a smartphone

• Can enrich user experience or nudge them to correct error

• Can also be used to simulate the sense of touch between remote
people who want to communicate

74
REALTIME VIBROTACTILE FEEDBACK
• Provides nudges when
playing incorrectly
• Uses motion capture
• Nudges are vibrations
on arms and hands

75

RESEARCH AND DESIGN ISSUES

• Where best to place actuators on body
• Whether to use single or sequence of ‘touches’
• When to buzz and how intense
• How does the wearer feel it in different contexts?
• What kind of new smartphones/smart-watches apps can use
vibrotactile creatively?
 e.g. slow tapping to feel like water dropping that is meant to indicate it is about to rain and heavy tapping to
indicate a thunderstorm is looming

76
MULTI-MODAL
• Meant to provide enriched and complex user experiences
 multiplying how information is experienced and detected using different modalities, i.e. touch, sight, sound,
speech
 support more flexible, efficient, and expressive means of human–computer interaction
 Most common is speech and vision

77

RESEARCH AND DESIGN ISSUES

• Need to recognize and analyse speech, gesture, and eye gaze
• What is gained from combining different input and outputs
• Is talking and gesturing, as humans do with other humans, a natural
way of interacting with a computer?

78
SHAREABLE
• Shareable interfaces are designed for more than one person to use
 provide multiple inputs and sometimes allow simultaneous input by co-located groups
 large wall displays where people use their own pens or gestures
 interactive tabletops where small groups interact with information using their fingertips
 e.g. DiamondTouch, Smart Table and Surface

79

A SMARTBOARD
• A SmartBoard in use during a meeting

80
DIAMONDTOUCH TABLETOP
• Mitsubishi’s interactive table top interface, where collocated user
can interact simultaneously with digital content using their fingertips

81

ADVANTAGES
• Provide a large interactional space that can support flexible group
working
• Can be used by multiple users
 Can point to and touch information being displayed
 Simultaneously view the interactions and have same shared point of reference as others

• Can support more equitable participation compared with groups

using single PC

82
RESEARCH AND DESIGN ISSUES
• More fluid and direct styles of interaction involving freehand and
pen-based gestures
• Core design concerns include whether size, orientation, and shape
of the display have an effect on collaboration
• Horizontal surfaces compared with vertical ones support more turn-
taking and collaborative working in co-located groups
• Providing larger-sized tabletops does not improve group working
but encourages more division of labor

83

TANGIBLE
• Type of sensor-based interaction, where physical objects, e.g.,
bricks, are coupled with digital representations
• When a person manipulates the physical object/s it causes a digital
effect to occur, e.g. an animation
• Digital effects can take place in a number of media and places or
can be embedded in the physical object

84
EXAMPLES
• Chromarium cubes
 when turned over digital animations of color are mixed on an adjacent wall
 faciliates creativity and collaborative exploration

• Flow Blocks
 depict changing numbers and lights embedded in the blocks
 vary depending on how they are connected together

• Urp
 physical models of buildings moved around on tabletop
 used in combination with tokens for wind and shadows -> digital shadows surrounding them to change over time

85

BENEFITS
• Can be held in both hands and combined and manipulated in ways
not possible using other interfaces
 allows for more than one person to explore the interface together
 objects can be placed on top of each other, beside each other, and inside each other
 encourages different ways of representing and exploring a problem space

• People are able to see and understand situations differently

 can lead to greater insight, learning, and problem-solving than with other kinds of interfaces
 can facilitate creativity and reflection

86
VOXBOX
• A tangible system
that gathers
opinions at events
through playful
and engaging
interaction
(Goldsteijn et al,
2015)
• VoxBox – front
and back of the
tangible machine
questionnaire

87

RESEARCH AND DESIGN ISSUES

• Develop new conceptual frameworks that identify novel and
specific features
• The kind of coupling to use between the physical action and digital
effect
 If it is to support learning then an explicit mapping between action and effect is critical
 If it is for entertainment then can be better to design it to be more implicit and unexpected

• What kind of physical artifact to use

 Bricks, cubes, and other component sets are most commonly used because of flexibility and simplicity
 Stickies and cardboard tokens can also be used for placing material onto a surface

88
AUGMENTED AND MIXED REALITY
• Augmented reality - virtual representations are superimposed on
physical devices and objects
• Mixed reality - views of the real world are combined with views of
a virtual environment
• Many applications including medicine, games, flying, and everyday
exploring

89

EXAMPLES
• In medicine
 virtual objects, e.g. X-rays and scans, are overlaid on part of a patient’s body
 aid the physician’s understanding of what is being examined or operated

• In air traffic control

 dynamic information about aircraft overlaid on a video screen showing the real planes, etc. landing, taking off,
and taxiing
 Helps identify planes difficult to make out

90
AN AUGMENTED MAP
• An augmented map showing the flooded areas at high water level
overlaid on the paper map. The handheld device is used to interact
with entitles referenced on the map.

91

TOP GEAR JAMES MAY IN AR

• Appears as a 3D character to act as personal tour guide at Science
Museum
• James May appearing in 3D Augmented Reality

92
RESEARCH AND DESIGN ISSUES
• What kind of digital augmentation?
 When and where in physical environment?
 Needs to stand out but not distract from ongoing task
 Need to be able to align with real world objects

• What kind of device?

 Smartphone, head up display or other?

93

WEARABLES
• First developments were head- and eyewear-mounted cameras that
enabled user to record what was seen and to access digital
information
• Since, jewellery, head-mounted caps, smart fabrics, glasses, shoes,
and jackets have all been used
 provide the user with a means of interacting with digital information while on the move

• Applications include automatic diaries, tour guides, cycle indicators

and fashion clothing

94
GOOGLE GLASS: SHORT-LIVED
• What were the
pros and cons?
• Google Glass

95

RESEARCH AND DESIGN ISSUES

• Comfort
 needs to be light, small, not get in the way, fashionable, and preferably hidden in the clothing

• Hygiene
 is it possible to wash or clean the clothing once worn?

• Ease of wear
 how easy is it to remove the electronic gadgetry and replace it?

• Usability
 how does the user control the devices that are embedded in the clothing?

96
ROBOTS AND DRONES
• Four types of robot
 remote robots used in hazardous settings
 domestic robots helping around the house
 pet robots as human companions
 sociable robots that work collaboratively with humans, and communicate and socialize with them – as if they
were our peers

97

ADVANTAGES
• Pet robots are assumed to have therapeutic qualities, helping to
reduce stress and loneliness
• Remote robots can be controlled to investigate bombs and other
dangerous materials

98
DRONES
• Unmanned aircraft that are controlled remotely and used in a
number of contexts
 e.g. entertainment, such as carrying drinks and food to people at festivals and parties;
 agricultural applications, such as flying them over vineyards and fields to collect data that is useful to farmers
 helping to track poachers in wildlife parks in Africa

• Can fly low and stream photos to a ground station, where images
can be stitched together into maps
• Can be used to determine the health of a crop or when it is the best
time to harvest the crop

99

DRONE IN VINEYARD
• A drone used to survey the state of a vineyard

100
RESEARCH AND DESIGN ISSUES
• How do humans react to physical robots designed to exhibit
behaviors (e.g. making facial expressions) compared with virtual
ones?
• Should robots be designed to be human-like or look like and
behave like robots that serve a clearly defined purpose?
• Should the interaction be designed to enable people to interact
with the robot as if it was another human being or more human-
computer-like (e.g. pressing buttons to issue commands)?
• Is it acceptable to use unmanned drones to take a series of images
or videos of fields, towns, and private property without permission
or people knowing what is happening?

101

BRAIN-COMPUTER INTERFACES
• Brain–computer interfaces (BCI) provide a communication pathway
between a person’s brain waves and an external device, such as a
cursor on a screen
• Person is trained to concentrate on the task, e.g. moving the cursor
• BCIs work through detecting changes in the neural functioning in the
brain
• BCIs apps:
 Games
 enable people who are paralysed to control robots

102
BRAINBALL GAME
• The Brainball game using a brain-computer interface

103

TOPIC 4: NATURAL USER INTERFACES?

• A NUI is one that enables us to interact with a computer in the same
ways we interact with the physical world, through using our voice,
hands and bodies.
• Instead of using a keyboard and a mouse, a natural user interface
allows us to speak to machines, strokes their surface, gesture at them
in the air, dance on mats that detect our feet movements, smile at
them to get a reaction and so on.
• The naturalness refers to the way they exploit the everyday skills
we have learned such as talking, writing, gesturing, walking and
picking up objects.

104
NUI
• Instead of having to remember which function keys to press to open
a file, a NUI means a person only has to raise their arm or say
‘open’.
• NUIs will not replace GUIs as the new face of interaction design.
• Eg : using gestures and whole body movement has proven to be
highly enable as form of input for man computer games and
physical exercise such as those that have been developed for the
Wii and Kinect systems.

105

TOPIC 3: WHICH INTERFACE?

• Is multimedia better than tangible interfaces for learning?
• Is speech as effective as a command-based interface?
• Is a multimodal interface more effective than a monomodal
interface?
• Will wearable interfaces be better than mobile interfaces for
helping people find information in foreign cities?
• Are virtual environments the ultimate interface for playing games?
• Will shareable interfaces be better at supporting communication
and collaboration compared with using networked desktop PCs?

106
WHICH INTERFACE?
• Will depend on task, users, context, cost, robustness, etc.
• Mobile platforms taking over from PCs
• Speech interfaces also being used much more for a variety of
commercial services
• Appliance and vehicle interfaces becoming more important
• Shareable and tangible interfaces entering our homes, schools,
public places, and workplaces

107

SUMMARY
• Many innovative interfaces have emerged post the WIMP/GUI era,
including speech, wearable, mobile, brain and tangible
• Raises many design and research questions to decide which to use
 e.g. how best to represent information to the user so they can carry out ongoing activity or task

• New interfaces that are context-aware or monitor raise ethical

issues concerned with what data is being collected and what it is
used for

108
KEY POINTS
• Many interfaces have emerged post the WIMP/GUI era, including
speech, wearable, mobile and tangible
• A range of design and research questions need to be considered
when deciding which interface to use and what feature sto include.
• So called natural user interfaces mat not be as natural as graphical
user interfaces – it depends on the task, user and context.
• An important concern that underlies the design of any kind of
interface is how information is represented to the user so that they
can make sense of it with respect ot heir ongoing activity.

109