CS374 Human-Computer Interaction

Computer & Interaction

Monvorath Phongpaibul Prapaporn Rattanatamrong

Input/Output Devices
There are many equivalent devices - Plus compatible, can simulate others Some are more flexible/accurate for users/tasks - Equivalence may be only from the point of view of the software Many trade-offs in devices - Should be chose based on task, users, economics Device types - Input Devices many physical devices can provide similar logical functions - Output Devices

Input Devices
Logical Input Devices
Locator (Indicate a position via mouse or arrow key) Pick (Select a displayed entity, e.g. selection button) Valuator (Supply a numerical value) Keyboard (Input a string or function key input) Button (Choose menu selection item)

Physical Input Devices

Voice (speech recognition) Keys Button (used with many pointing devices, multi-press is common) Keyboard (styles next) Programmable function keys (hardware, software, or template overlays Cursor keys (e.g. hjkl, or arrow keys)

Keyboards
Sholes (QWERTY)
Standard Layout

Dvorak
Layout minimized finger movement

Alphabetic
Aids search for novice users, but no much else

Chords
For example, courtroom recorders

Point of sale
Customized keyboards/cash registers

Standard vs. Dvorak

Traditional (standard) keyboard folklore: 1) 2) design avoided mechanical key jamming the letters T-Y-P-E-W-R-IT-E-R are all on the first row for ease to salespersons

Dvorak keyboard layout was designed to minimize finger movement (note the location of the vowels: AE-I-O-U, and note Q & Z in the corners

One-Handed Keyboard
(for Mobile or Disabled Workers)
www.frogpad.com

Input Devices (concluded)

Joy Stick (2D and 3D) Thumb wheels Track ball (low space needs) Mouse (multi-button) Rat (foot mouse) Digitized tablet (puck in rink, exact input with cross-hairs on mouse) Pen and tablet (natural input) Light pen (synchronized with scan lines) Touch screen Touch pad (overlay templates) 3D devices (data glove, other devices for moving in space) Gesture/movement sensitive (projection; you select items by moving Scanners, including text-to-speech Eye trackers Stylus on PDAs Trackpoint on laptops Have I forgotten any?

See eye tracker products at http://www.naturalpoint.com/

Alphagrip.com

Input device example

Input device example 2 http://www.naturalinteraction.com/

Input device example 3

Input device example 4

http://www.emsense.com/

Input device example 5 Dance Your Work Away: Exploring Step User Interfaces, by Brian Meyers et al from
Microsoft Research

Input device example 6

NTT's Tangible-3D prototype gives feeling to on-screen imagery

Based around an improved version of the company's original 3D display, this prototype system relies on a sophisticated array of cameras and an actuator-stuffed glove that can allow the wearer to "feel the image" that shows up on the LCD.
http://www.engadget.com/2007/06/21/ntts-tangible3d-prototype-gives-feeling-to-on-screen-imagery/

Input device example 7

Pentop
Fly Pen educational toy, also knows as the Fly Pentop Computer

Input device example 8

Mini Pen Computer

http://boldt.us/things/computers/pen-pc/?g2_jsWarning=true

Visual Keyboard

3D Hologram Display

Input device example 9

Ohio Subway implements first drive-thru touchscreen kiosk

Please don't try this with other kiosks

http://www.kioskmarketplace.com/article.php?id=15926

Environments

Combination of the above in immersive environments Will discuss more with respect to USC IMSC & ICT

Speech Recognition
Discrete Word Recognition See translator products at 98% accuracy claimed http://www.speechgear.com/ Limited vocabulary Speaker dependent training Voice recognition, menu driven systems, e.g. United flight status line Commercially viable when you need hands free (e.g. baggage handling, medicine, jet maintenance, inspectors, inventory, etc.) Continuous Speech Recognition Becoming more common-place Research improving technology Speech Generation Commercially viable, good quality, supplements screen display, phone systems, physically challenged applications

Output Devices

Visual output Indicator light Monitor/displays of varying sizes, from hand-held (PDA, cell phone), laptop LCD, monitors, large-screen display, projection systems including heads-up displays, hardcopy
Auditory output Voice generation Tone generation

Ready to Wear
(Ref: IEEE Spectrum, Oct. 2000)

Heads-Up Situation Display

http://www.microvision.com/nomad/

Painting
http://web.media.mit.edu/~kimiko/iobrush/iobrush_m peg_medium.mpg

Gesturing

What's Next: Nice Gesture

The next interface for information-laden displays could be at the end of your arm.
Raytheon and g-speak are developing "gesturetechnology" systems, which use infrared cameras and reflective gloves rather than a mouse or trackball to manipulate video, text, and other information. Showcased in the 2002 sci-fi film "Minority Report," gesture technology could be commercially available within five years. In the 2002 movie Minority Report, Tom Cruise plays a detective who conducts investigations by standing in front of a chalkboard-sized display and using special gloves to sort through videos, photos, and text. It may seem like science fiction, but it also looks like a viable way to get a grip literally on more information than you could make sense of on a PC screen.
http://proav.pubdyn.com/2005_July/July2071200512 1700PM.htm http://www.minorityreportdvd.com/

Gesturing 2

Minority Report (Movie) and Matrix (Movie) The innovative use of

direct manipulation for interacting with virtual objects as if they are real objects
Thanks Sonali Joshi, Harishkumar Narayanan, Badri Rajan

Gesturing (concluded)
Researchers from MIT have developed a computer interface that enables a user to manipulate virtual shapes projected onto a screen using gestures Gestures such as pointing and spoken commands, Commands like "make a red cube in the middle of the screen." Standing in front of cameras mounted above the screen, a user can create a virtual cube, rotate it, and change its color and size. In one experiment, the researchers found that their gesture recognition system had an error rate of 6 to 17 percent with some gestures, but a zero error rate when the gesture was coupled with a corresponding spoken command.

Source: Demirdjian, D., T. Ko, and T. Darrell. 2005. Untethered gesture acquisition and recognition for virtual world manipulation. Virtual Reality.
http://www.technologyreview.com/articles/05/10/issue/ftl_info.2.asp

Northrop Grumman & Applied Minds, Inc. TouchTable

http://www.ms.northropgrumman.com/touchtable/ http://www.touchtable.com/

Mitsubishi has an analogous product

recently demod in multiple user game environments at the International Workshop on Pervasive Gaming Applications (PerGames)
http://www.merl.com/projects/DiamondTouch/DTflier.pdf

TouchTable example
http://seattlepi.nwsource.com/audio/bishop/tabletop.wmv

Also see: http://mblog.lib.umich.edu/~rdivecha/archives/2006/02/the_world_of_sm.html

Microsoft Table Surface Computer

Microsoft Corp. unveiled a coffee-table-shaped "surface computer" Wednesday in a major step towards co-founder Bill Gates' view of a future where the mouse and keyboard are replaced by more natural interaction using voice, pen and touch. Microsoft Surface, which has a 30-inch display under a hard-plastic tabletop, allows people to touch and move objects on screen for everything from digital finger painting and jigsaw puzzles to ordering off a virtual menu in a restaurant. http://www.microsoft.com/surface/

Interactive Workspace Project

Designing for multi-person multi-device environments

- Interaction devices and methods - Design theories and approach - Primary focus on explicit interaction, technical work - Experimenting with Interactive Room (iRoom)
with Terri Winograd (Stanford), Pat Hanrahan, Armando Fox, Jan Borchers, Maureen Stone, ++

What is an Interactive Room? An augmented dedicated space

e.g. conference room can be used with teleconnected spaces e.g. brainstorming meeting, design review

Task-oriented scenarios Multi-device, multi-user, multi-application

Easy to (re)configure Movement of information across multiple devices, and from personal to/from multi-user devices Integration & augmentation of legacy apps

Stanford iRoom

http://iwork.stanford.edu/

http://imsc.usc.edu/

USC VSoE IMSC conducts research in many technologies relevant to this course: 3D Modeling, 3D Stereospopic, Audio, Augmented Reality/Virtual Reality, Data Management, Haptics, Interactive Education, Music Processing, Panoramic Video, Software Architectures, Speech, Streaming, Tracking, User Centered Sciences, Video Processing, Wireless

The Interaction

Interaction Framework
O Ergonomics

Dialogue

Ergonomics
Study of the physical characteristics of the interaction Focus on user performance and how interface enhances or detracts from this This course will consider a few of the issues addressed by ergonomics as an introduction
Arrangement of controls and displays Physical environment Health issues Use of color

Dialog Interaction Style

Interaction is a dialogue between computer and the user The choice of interface style can have a profound effect on the nature of this dialogue Interaction Style
Command line interface Menus Natural Language Question / answer and query dialogue Etc

- Question -