Human Computer Interaction

Chapter 2
INTRODUCTION
When we interact with computers, a process of information transfer happens from the user to the
computer and from the computer to the user. By interacting with the computer we are trying to achieve
a process of information transfer. Devices needed for interaction are :

• Typical computer interface

• Largely variants of keyboard for text entry
• Mouse for positioning
• Screen for displaying output
• Virtual Reality and 3D Interaction (for interacting deeper into the electronic world)
• Printers and Scanners.

We then consider the computer itself, its processor and memory devices and the networks that link
them together. We note how the technology drives and empowers the interface. The details of
computer processing should largely be irrelevant to the end-user, but the interface designer needs to be
aware of the limitations of storage capacity and computational power.

A typical Computer System

There is the computer ‘box’ itself, a
keyboard, a mouse and a color
screen. The screen layout is shown
alongside it. There are variants on
these basic devices. Some of this
variation is driven by different
hardware configurations: desktop
use, laptop computers, PDAs. The
diversity of devices reflects the fact
that there are many different types
of data that may have to be entered
into and obtained from a system,
and there are also many different
types of user, each with their own
unique requirements.
Levels of interaction – batch processing
In the early days of computing, information was entered into the computer in a large mass – batch data
entry. There was minimal interaction with the machine: the user would simply dump a pile of punched
cards onto a reader, press the start button, and then return a few hours later. With batch processing the
interactions take place over hours or days. It is easy to assume that faster means better, but some of the
paper-based technology discussed in Section suggests that sometimes slower paced interaction may be
better. Application Examples:

• printing pay checks

• entering the results from a questionnaire.

Richer interaction – everywhere, everywhen

Internet fridge, mobile phone, video camera, Microwave, washing machine, WAP phones and
smartcards, web cams etc

TEXT ENTRY DEVICES

The most obvious means of text entry is the plain keyboard, but there are several variations on this:
different keyboard layouts, ‘chord’ keyboards that use combinations of fingers to enter letters, and
phone key pads. Handwriting and speech recognition offer more radical alternatives.

The alphanumeric keyboard

The keyboard is still one of the most common input devices in use today. It is used for entering textual
data and commands. The vast majority of keyboards have a standardized layout, and are known by the
first six letters of the top row of alphabetical keys, QWERTY. There are alternative designs which have
some advantages over the QWERTY layout, but these have not been able to overcome the vast
technological inertia of the QWERTY keyboard.

These alternatives are of two forms: 26 key layouts and chord keyboards.

A 26 key layout rearranges the order of the alphabetic keys, putting the most commonly used letters
under the strongest fingers, or adopting simpler practices.

The QWERTY keyboard

The layout of the digits and letters on a QWERTY keyboard is fixed but non-alphanumeric keys vary
between keyboards. For example, there is a difference between key assignments on British and
American keyboards. There can also be variation in the placement of brackets, backslashes and
suchlike. In addition different national keyboards include accented letters and the traditional French
layout places the main letters in different locations – the top line starts AZERTY.

The QWERTY keyboard layout isn't optimal for typing but persists due to its historical use in mechanical
typewriters. Despite no longer needing to address mechanical issues, it remains dominant because
retraining typists and replacing existing keyboards would be costly and disruptive. The widespread use
of this layout has made change unlikely due to technological inertia.

How keyboards work Keyboards work by sending a character code to the

computer when a key is pressed, either through a wired or wireless connection. The "feel" of keys
varies, with some requiring a hard press, while others are featherlight. Keyboards on most notebooks
are "half-travel," saving space but offering less tactile feedback. Touch-sensitive keyboards, often used
in shop tills, require little pressure, are easy to clean, but lack feel for typists.

Due to RSI concerns, ergonomic keyboards with custom layouts, single-handed designs, or sloped keys
may see a resurgence, offering relief for users with injuries or disabilities.

Alphabetic keyboards, though seemingly easier for beginners, aren't faster than QWERTY, even for
novices. DVORAK keyboards, optimized for speed and fatigue reduction, assign letters differently,
favoring alternating keystrokes between hands

Ease of learning – alphabetic keyboard

The alphabetic keyboard, with letters arranged in order, might seem easier for untrained typists, but
studies show it offers no speed advantage, even for trained typists or novices. It’s used in some small
devices, likely because it appears simpler than QWERTY, and discourages touch-typing on small
keyboards, reducing complaints about usability.

Ergonomics of use – DVORAK keyboard and split designs

The DVORAK keyboard:

- Rearranges keys to increase typing speed compared to QWERTY.

- 56% of keystrokes are made with the right hand, favoring right-handed users.
- Designed to alternate between hands, reducing fatigue and improving speed.
- 70% of keystrokes are made on the home row, minimizing finger movement.
- Improves typing speed by 10-15% and reduces fatigue due to its ergonomic design, which
minimizes hand movement.

Ergonomic keyboards (Split Designs):

- Designed to reduce strain and prevent Repetitive Strain Injury (RSI).

- Split and curved designs bring keys within easier reach for comfort.
- These ergonomic keyboards have not widely replaced standard models in the market.

Chord keyboards
Definition : Chord keyboards use only four or five keys to produce letters by pressing multiple keys
simultaneously.

Example : The Microwriter reflects letter shapes in its keypress patterns.

Advantages :

• Compact Design : Much smaller than conventional keyboards, making them easier to use in
tight spaces.
• Short Learning Curve : Users can learn the basics in a few hours.
• Fast Typing Speeds : Competent users can achieve high typing speeds.
• One-Handed Use : Suitable for situations where only one hand is available.
Limitations :

• Lack of Familiarity : Not widely accepted as mainstream text entry due to social resistance.
• Niche Applications : Commonly used by courtroom stenographers and may find use in the
wearables market.

Design Focus: Numeric Keypads

Alphanumeric keyboards (as the name suggests) include numbers as well as letters. In the QWERTY
layout these are in a line across the top of the keyboard, but in most larger keyboards there is also a
separate number pad to allow faster entry of digits. Numeric keypads are found on alphanumeric
keyboards, calculators, telephones, and ATMs.

• Layouts: There are two layouts:

o Calculator Style: '123' at the bottom (used in computer keyboards).

o Telephone Style: '123' at the top (used in ATMs).

• Practical Example: A user may forget their PIN and instinctively type on a calculator, only to
discover that ATMs use the telephone layout, leading to confusion.

Phone Pad and T9 Entry

Phone keypads are important for texting but only have numbers, not a full keyboard. To type letters,
users must press keys multiple times. For example, the '3' key corresponds to 'd,' 'e,' and 'f':

• Key Mapping:

o 1 – space, comma, etc.

o 2 – a, b, c

o 3 – d, e, f

o 4 – g, h, i

o 5 – j, k, l

o 6 – m, n, o

o 7 – p, q, r, s

o 8 – t, u, v

o 9 – w, x, y, z

o 0 – +, &, etc.
 Most phones have two modes: one for entering numbers (like phone numbers) and another for
typing letters (like in texts). Some phones help with accented letters and capital letters, often
capitalizing the first letter of sentences automatically.

Typing can be slow, but experienced users develop shortcuts and often type quickly with one hand
and just their thumb. Some phones also have tiny keyboards that can be plugged in or folded out for
easier typing.

The T9 system helps by guessing words based on fewer keypresses. For instance, typing '3926753'
might give you the word 'example.' If there’s confusion, like with '26,' the phone will show choices
like 'am' or 'an.'

Handwriting Recognition
Handwriting recognition lets users write like they do on paper, making it an easy way to enter text.
However, the technology isn't perfect; it can be inaccurate and struggles with different handwriting
styles. Recognizing handwriting relies on understanding how letters are drawn, which makes it
easier to read on a screen than from paper.

Speed is also a challenge, as most people write at about 25 words per minute, which is slower than
typing. Handwriting recognition can be useful in situations where keyboards are difficult to use.

Pen-based systems that use handwriting recognition are common in mobile devices for note-taking
and organizing. These systems are often small and easy to use, unlike tiny keyboards that can be
hard to work with. Writing by hand also allows for quick sketches.

When designing these systems, we can explore different ways to interact, like using gestures instead
of typing commands. This shift from keyboards opens up new possibilities for how we can use
technology.

Signature Authentication

Handwriting recognition is challenging due to the vast differences in individual handwriting.

However, these differences can be beneficial for signature authentication, where the goal is to verify
the user rather than read the signature. Analyzing stroke patterns is key, as signatures may look
similar but have unique details in their formation. For example, a forger can mimic the appearance
of a signature but typically can't replicate the specific stroke patterns.

Speech Recognition
Speech recognition is a promising but limited way to enter text. While it's great to talk to machines,
the current success rate of over 97% still means there are errors—about one mistake for every six
words, which isn't good enough. This accuracy usually only applies to a small set of command words,
and understanding natural language adds even more challenges.

Each person's speech is different, so the system must be trained for each user. Things like strong
accents, colds, and background noise can make recognition harder. In an office, if many people
 talked to their machines at once, it would create a lot of noise and make it hard to keep things
private.

Despite these issues, speech technology is useful in specific areas, like phone information systems,
helping disabled users, and hands-free situations, especially in the military. It can either replace
keyboards or be part of systems designed to use its strengths while reducing its problems. Overall,
speech technology is succeeding where typing isn't practical or possible.

POSITIONING, POINTING AND DRAWING

Pointing devices are essential for interacting with modern computing systems, letting users select and
manipulate items on the screen. They are widely used in areas like computer-aided design (CAD) for
positioning and drawing tasks. While the mouse remains the most common device for desktop
computers, it's being challenged by alternatives like touchpads as laptops and handheld devices grow in
popularity. Drawing with a mouse differs from using a pencil, but many pointing devices support
freehand drawing.

The mouse
Basic Function :

• A mouse is a small, palm-sized device used to control the cursor on a computer screen. It
typically contains a weighted ball inside.

Working Mechanism :

• As the mouse is moved over a surface, the ball rolls.

• Rollers inside the mouse detect this movement and adjust potentiometers to measure the ball's
horizontal and vertical motion.

• This information is sent to the computer, which moves the cursor on the screen based on the
ball's relative motion.

Connection :

• The movement information is sent to the computer via a wire or wirelessly, moving the cursor
on the screen.

Buttons :

• Mice usually have 1-3 buttons to select or initiate actions.

• Multi-button mice assign specific functions to each button, while single-button mice use
different click combinations.
Movement and Control :

• The mouse is an indirect input device, meaning horizontal movements on the desk are
mapped to the vertical movements on the screen.
• It provides information only on relative motion, allowing you to lift and reposition the
mouse without affecting the cursor

Advantages and Disadvantages:

• It requires little physical effort and is not tiring to use.

• However, it can be less intuitive for beginners due to the indirect nature of the hand-eye
coordination.

History :

• Invented by Douglas C. Engelbart in 1964, the early mouse used two wheels to transmit x–y
coordinates and was made of wood.
• The original design allowed for pure vertical or horizontal motion by tilting the device,
making it easier to control on large screens.

Footmouse:
1. Footmouse:

• A foot-operated device that moves the cursor based on foot pressure, similar to a
joystick. It allows hands to remain free for the keyboard.

• Rare and not widely accepted.

2. Use in Other Fields:

• Foot pedals are common in music (pianos, guitars, drums) and machinery (cars, cranes,
sewing machines), showing they can be effective in principle.

3. Limitations:

• Practicality: Foot pedals under desks can be accidentally pressed, and using them with
laptops would be awkward.

• Control: Pedals are mostly used for simple, one-dimensional control, which limits their
application in general computing.

In short, while foot controls work in other fields, they haven't found much use in computers due to
practicality and limited control capabilities.

Touchpad
1. What is a Touchpad?
 • A touch-sensitive tablet usually 2–3 inches in size, first popularized in Apple Powerbook
laptops and now used in many notebooks.

• Can also be used as a desktop mouse replacement.

2. How it Works:

• Operated by stroking a finger over its surface, similar to a trackball.

• It may take multiple strokes to move the cursor across the screen.

3. Acceleration Setting:

• Movement is controlled by an acceleration feature: slow finger movement results in

small cursor movement, while fast movement moves the cursor quickly across the
screen.

• This feature improves usability and makes the touchpad more responsive.

4. Feel and Usability:

• The feel is different from other input devices, but users quickly adapt.

• Fine adjustments to settings improve the overall experience.

In short, touchpads are small, touch-sensitive devices with adjustable settings for smooth cursor
movement, widely used in laptops.

Trackball and thumbwheel

Trackball :

• Works like an upside-down mouse.

• The user rotates a ball on top, and the motion controls the cursor.
• It's stationary, requiring no extra desk space.
• Motion detection is similar to that of a mechanical mouse.
• Requires separate buttons for selection.
• Commonly used by people with **Repetitive Strain Injury (RSI)** and in **gaming** due to its
responsive control.
• Hard to use for long movements or drawing.
• Available in different sizes, with larger trackballs offering more control.

Thumbwheel :

• Uses two dials at right angles to control cursor movement, allowing horizontal and vertical
motion only.
• **Useful for tasks needing precision in straight lines**, like **CAD design**.
• It's slower and not ideal for freehand drawing or curved movement.
• Commonly included on standard mice as **scroll wheels**, which allow faster and more
intuitive document scrolling.
• Reduces strain by avoiding the need to hold down the mouse button while dragging.

Both devices are indirect input tools, with their appropriateness depending on the task. Trackballs
are favored for gaming and ergonomic uses, while thumbwheels excel in tasks requiring precision in
straight-line movements.

Joystick and keyboard nipple

1. Joystick:

o An indirect input device used for cursor movement.

o Two types:

▪ Absolute joystick: The position of the stick directly matches the position of the
cursor on the screen. Moving the stick changes the cursor’s location.

▪ Isometric joystick (velocity-controlled): The pressure applied to the stick

controls the speed of the cursor. The stick returns to its centered position when
released.

o Buttons are usually located on the stick (top or front).

o Common in gaming due to robustness, affordability, and resemblance to aircraft

controls.

2. Keyboard Nipple:

o A small isometric joystick found on laptops, often called a nipple.

o Functions similarly to the isometric joystick: pressure controls cursor speed, and the
nipple controls the rate of movement across the screen.

o Though harder for beginners, adjusting speed settings can improve usability.

Both devices function through indirect control, with movements or pressure translating to screen
cursor manipulation.

Joystick and keyboard nipple

3. Joystick:

o An indirect input device used for cursor movement.

o Two types:

▪ Absolute joystick: The position of the stick directly matches the position of the
cursor on the screen. Moving the stick changes the cursor’s location.
 ▪ Isometric joystick (velocity-controlled): The pressure applied to the stick
controls the speed of the cursor. The stick returns to its centered position when
released.

o Buttons are usually located on the stick (top or front).

o Common in gaming due to robustness, affordability, and resemblance to aircraft

controls.

4. Keyboard Nipple:

o A small isometric joystick found on laptops, often called a nipple.

o Functions similarly to the isometric joystick: pressure controls cursor speed, and the
nipple controls the rate of movement across the screen.

o Though harder for beginners, adjusting speed settings can improve usability.

Both devices function through indirect control, with movements or pressure translating to screen
cursor manipulation.

Touch-sensitive screens (touchscreens)

Touchscreens let users interact directly with the display using a finger or stylus. They detect
touch through methods like interrupting light beams, sensing capacitance changes, or using
ultrasonic reflections, making them direct input devices with no need for mapping.

Touchscreens are fast, intuitive, and ideal for menu selection, durable in harsh environments, and
require no extra hardware. However, they can leave smudges, lack precision for small selections
or drawing, and cause arm fatigue with constant use. An optimal angle for comfort is around 15
degrees upward from horizontal.

Working Mechanism: Touchscreens detect input in several ways:

• Interrupting light beams: When a finger or stylus blocks the beams, the system registers the
touch location.

• Capacitance changes: A grid of conductive material overlays the screen, and the finger's
presence alters the capacitance at that point, identifying the touch position.

• Ultrasonic reflections: Ultrasonic waves bounce off the screen, and when disrupted by a touch,
the system locates the input based on the change in reflection.

This input is directly translated into actions on the display, providing immediate feedback.

Stylus and light pen

Styluses and light pens enhance precision when interacting with touch-sensitive screens. A
stylus is a pen-like tool that allows users to point and draw directly on the screen. It is popular
in PDAs and some laptops. Unlike fingers, styluses do not leave greasy marks, providing a
cleaner input method.

The light pen, an older technology, connects to the screen via a cable. It detects bursts of light
emitted by the screen’s phosphors during display scans. This capability allows the light pen to
address individual pixels, resulting in greater accuracy compared to touchscreens.
Working Mechanism:

• Stylus: The stylus interacts with the touch-sensitive surface, using either resistive or capacitive
technology to register touch. When the stylus touches the screen, it sends precise location data
to the device, allowing for accurate selection and drawing.

• Light Pen: The light pen detects the light emitted from the screen during its refresh cycle. As the
screen scans from left to right, the light pen registers the timing of the light burst, enabling it to
pinpoint the exact pixel being targeted.

Both devices are suitable for fine selection and detailed drawing but can become tiring to use on upright
displays. They are also challenging to manage alongside a keyboard.

Both the stylus and light pen are considered direct input devices, as they establish an immediate
relationship between the user’s actions and the selected item on the screen. In contrast, indirect input
devices like mice and trackballs require mapping movements on a desk to cursor movements on the
display.

Considerations:

Pointing with these devices can obscure parts of the display, complicating tasks that require quick,
detailed selections or movements. Therefore, user interface designs should account for hand
placement, such as positioning menus at the bottom of the screen. Additionally, offering alternative
layouts for left-handed and right-handed users enhances usability.

Digitizing tablet
A digitizing tablet is a specialized device primarily used for freehand drawing or as a mouse
substitute. It works by detecting the position of a stylus or puck (a mouse-like device) on a
tablet surface, which can be done in various ways:
Resistive Tablets : These detect point contact between two conductive sheets, allowing
operation with any object like a pen or finger.
Magnetic Tablets : Use a small coil in the pen to detect current pulses within a magnetic field.
Capacitive and Electrostatic Tablets : These detect position through changes in capacitance or
electrostatic fields.
Sonic Tablets : Emit ultrasonic pulses from a pen that are detected by microphones,
triangulating the position, and can also offer 3D input.
 Digitizing tablets provide high resolution and smooth cursor control, detecting both relative
and absolute motion. They can also recognize handwriting for text input. However, they
need a lot of desk space and can feel awkward if not placed well next to the keyboard.

Eyegaze
• Control interface by eyegaze direction, e.g look at a menu to select it.
• Uses laser beam reflected off retina – a very low power laser
• Mainly used for evaluation (ch x)
• Potential for hands free control
• High accuracy requires headset
• Cheaper and lower accuracy devices available – sit under the screen like a small webcam

Cursor keys and discrete positioning

Cursor keys

• Four keys (up, down , left and right) on keyboard

• Very, very cheap, but slow.
• Useful for not much more than basic motion for text editing tasks
• No standardized layout but inverted T most common

Discrete Positioning Controls

In phones , TV controls etc

- Cursor pads or mini-joysticks

- Discrete left-right,up-down
- Mainly for menu selection

DISPLAY DEVICES
The most obvious means of text entry is the plain keyboard, but there are several variations on this:
different keyboard layouts, ‘chord’ keyboards that use combinations of fingers to enter letters, and
phone key pads. Handwriting and speech recognition offer more radical alternatives.

Bitmap displays – resolution and color

Screen is a vast number of coloured dots

Resolution and Coloured Depth

Resolution used (inconsistently) for
 • Number of pixels on screen (width x height) . e.g SVGA 1024 x728, PDA perhaps 240 x 400
• Density of pixels (in pixels or dots per inch - dip) . typically bw 72 and 96 dpi

Aspect Ratio

• Ration bw width and height

• 4:3 for most screens , 16: 9 for wide-screen TV

Coloured Depth

• How many different colours for each pixel

• Black/white or greys only
• 256 from a pallete
• 8 bits each for reg/ green/ blue = millions of colours

Anti – Aliasing
Jaggies

Diagonal lines that have discontinuities due to horizontal raster scan process

Anti – Aliasing

Softens edges by using shades of line colour

Also used for text

Technologies
Cathode ray tube
Stream of electrons emitted from electron gun, focused and directed by magnetic fields , hit phosphor-
coated screen which glows

Used in TV and computer monitors

Health hazards of CRT displays

• X-rays which are largely absorbed by the screen (but not at the rear!)
• ultraviolet and infrared radiation from phosphors in insignificant levels
 • radio frequency emissions, plus ultrasound (approximately 16 kHz)
• electrostatic field which leaks out through the tube to the user. The intensity is dependent on
distance and humidity. This can cause rashes in the user
• electromagnetic fields (50 Hz to 0.5 MHz) which create induction currents in conductive
materials, including the human body. Two types of effects are attributed to this: in the visual
system, a high incidence of cataracts in visual display unit (VDU) operators, and concern over
reproductive disorders (miscarriages and birth defects).

Health hints

• not sitting too close to the screen

• not using very small fonts
• not looking at the screen for a long time without a break
• working in well-lit surroundings
• not placing the screen directly in front of a bright window.

Liquid crystal display

Liquid Crystal Displays (LCDs) are thin, lightweight screens commonly used in personal organizers,
laptops, and now even desktop computers. Found in PDAs , even in home TVs. Also used in dedicated
displays : digital watches, mobile phones, HiFi controls

• Compact and Efficient : LCDs are smaller, lighter, and consume less power compared to
traditional CRTs, making them ideal for portable devices like laptops and notebooks.
• No Radiation : LCDs do not emit harmful radiation, unlike CRTs, making them safer for
prolonged use.
• Matrix Addressable : They allow individual pixels to be accessed without scanning, improving
efficiency.
• Working Mechanism :
1. A thin layer of liquid crystal is sandwiched between two glass plates.
2. The top plate is transparent and polarized, while the bottom plate is reflective.
3. External light passes through the polarized top plate, goes through the liquid crystal, and
reflects off the bottom plate, making the pixel appear white.
4. When voltage is applied, the liquid crystal twists, changing the light’s polarization, preventing it
from passing back through the top plate. This makes the pixel appear black.
5. The screen refreshes at standard rates, but the slow response time of the crystal reduces flicker,
lessening eye strain. (slides – N.B. light reflected not emitted = less eye strain)
• Widely Adopted : LCDs have revolutionized portable computing, enabling smaller, lighter
computers and providing better screen quality in personal DVD players and even home TVs.
• Color Technology : Once limited to black and white or grayscale, LCDs have advanced to include
color displays, replacing CRT monitors in most cases.

Special displays
Gas Plasma Displays : Commonly used in large screens, these displays utilize plasma technology, which
differs from standard LCDs or CRTs.

Random Scan Display (Vector Display) :

• Draws lines directly instead of scanning the entire screen sequentially.

• Updates the screen at least 30 Hz to reduce flicker.
• Avoids jaggies, producing smoother lines at any angle.
• Supports higher resolutions, up to 4096 × 4096 pixels.
• Color is achieved through beam penetration technology, but generally of lower quality.
• Still prone to eyestrain, and is more expensive than raster displays, so it's used mainly in niche
applications.

Direct View Storage Tube :

• Used extensively in analog storage oscilloscopes.

• Similar to random scan CRT but uses flood guns to maintain a flicker-free stable display.
• The screen image can be incrementally updated but cannot be selectively erased—redrawing
the image requires erasing the whole screen.
• High resolution, around 4096 × 3120 pixels, but suffers from low contrast, low brightness, and
difficulty displaying color.

Large displays
• Used for meetings, letcures etc
• Technology
▪ Plasma – usually wide screen
▪ Video walls – lots of small screens together
▪ Projected – RGB lights or LCD projector
▪ Hand/body obscures screen
▪ May be solved by 2 projectors + clever software
▪ Back projected
▪ Frosted glass + projector behind

Situated Displays
• Display in public places
• Large or small
• Very public or for small group
• Display only
▪ For information relevant to location
• Or interactive
▪ Use stylus , touch sensitive screem
• In all cases … the location matters
▪ Meaning of information or interaction is related to the location.
Hermes: a situated display
• Small displays beside office doors
• Handwritten notes left using stylus
• Office owner reads notes using web interface

Digital paper
A new, flexible display technology that retains content even without power, mimicking the function of
paper.

Technologies :

• Tiny Spheres : The surface is covered with tiny spheres that are black on one side and white on
the other. Embedded electronics rotate the spheres to display black or white. Once rotated, the
spheres remain in position when the signal is removed.
• Tiny Tubes : Small reflective spheres in liquid-filled tubes can be moved to the surface to display
white or away to show black. Similar to the sphere method, the position is retained after the
signal is removed.

Applications : Initially expected to be used for large banners that require reprogramming or slow
animation. As the technology improves, programmable paper could offer changeable "soft" printouts or
even books with re-downloadable content.

DEVICES FOR VIRTUAL REALITY AND 3D

INTERACTION (positioning in 3D space, moving and grasping, seeing 3D (helmets and caves))

Positioning in 3D space
6 Degrees of Freedom

• In VR, users need to move and orient themselves in a virtual 3D space, which requires six
degrees of freedom: three for position (up/down, forward/backward, left/right) and three
for orientation (pitch, yaw, and roll).
• Unlike traditional 2D inputs, VR devices like the 3D mouse enable direct manipulation and
rotation of virtual objects.

Cockpit and Virtual Controls :

• Steering wheels, knobs and dials- just like real
• Aircraft-style controls are used to navigate virtual environments. These are common in
arcade games and simulators, but they demand skill, similar to piloting.
 • In PC games, virtual controls (buttons, levers) are manipulated using 2D devices like a
mouse, introducing two levels of indirection, which users adapt to rapidly.

3D Mouse :
• A 3D mouse allows movement and rotation in all six degrees of freedom. Sensors like
magnetic coils, ultrasound, or mechanical joints track position and orientation.
• Users may experience fatigue from holding the device in the air, and in some systems,
actions like putting the mouse down are treated as interactions.

Dataglove : (fibre optic used to detect finger position)

• The dataglove detects finger and thumb joint angles using fiber optics. It provides multi-
dimensional input by measuring wrist rotation and 3D positional data via ultrasound
sensors.
• It’s highly expressive but expensive, with potential applications in gesture recognition, sign
language, and for assisting disabled people.

Virtual Reality Helmets : (detect head motion and possible eye gaze)
• These helmets display 3D visuals to each eye and track the user's head movements, enabling
users to see different parts of the scene as they turn their heads.
• In some systems, head movement controls navigation and interaction with objects, similar to
eyegaze systems.

Whole-Body Tracking : (accelerometers strapped to limbs or reflective dots and video processing)
• Advanced VR systems can track full-body movements for an immersive experience. Examples
include motorbike games that track leaning and even trampolines that respond to body weight
shifts to control movement.
• Full-body tracking can use image-processing techniques, where cameras track white markers on
joints, allowing precise joint mapping for virtual interactions.

3D displays
The immersive nature of virtual reality requires not only advanced input devices but also sophisticated
output systems. Standard desktop VR utilizes computer screens where 3D effects are created using
techniques like shadows, occlusion, and perspective to simulate depth. While effective, desktop VR lacks
true depth perception as it's displayed on a flat screen.

Desktop VR

• Ordinary screen, mouse or keyboard control

• Perspective and motion give 3D effect
Stereoscopic Vision and 3D Viewing (use stereoscopic vision, VR helmets, screen plus
shuttered specs etc)

In the real world, our perception of depth is largely based on stereoscopic vision, where each eye sees
slightly different images due to the distance between them, allowing our brain to assess depth. In virtual
reality, this stereoscopic effect can be replicated using various devices. Some VR systems use goggles
with two screens, where each eye receives a separate image from a slightly different angle, mimicking
the natural stereoscopic vision.

Alternatively, special spectacles may be used in conjunction with a monitor. These glasses can
synchronize with the monitor’s frame rate, allowing each eye to see alternating images, or use polarized
lenses to achieve the 3D effect, much like the classic red-green 3D glasses. Although such systems offer
improved quality compared to traditional methods, they can be cumbersome and still have limitations
like narrow viewing angles.

Challenges of Focus and Eye Strain

VR systems also face the challenge of addressing focus cues. In reality, each eye adjusts its focus based
on the distance of objects, but in VR, images are typically presented with fixed focal depth. This can lead
to eye strain as the brain is confused by the mismatch between what the eyes are seeing and their
actual focus.

Slides – VR headsets – small TV screen for each eye, slightly different angles, 3D effect

Motion Sickness in VR
Another issue in VR displays is motion sickness, which arises from a lag between head movement and
the corresponding change in the scene. If this delay exceeds 100 milliseconds, users can feel disoriented,
similar to seasickness, as their eyes and inner ear give conflicting signals to the brain.

• Time delay
o Move head …lag…display moves
o Conflict – head movement vs eyes
• Depth perception
o Headset gives different stereo distance
o But all focused in same plane
o Conflict – eye angle vs focus
• Conflicting cues => Sickness
o Helps motivate improvement in technology

Simulators and VR Caves

To counteract some of these issues, many systems use simulators or **VR caves**. These systems
envelop the user in large displays that project the virtual environment around them.

• Scenes projected on walls

• Realistic environment
• Hydraulic rams
 • Real controls
• Other people

PHYSICAL CONTROLS, SENSORS AND

SPECIAL DEVICES (special displays and gauges , sound, touch, feel, smell, physical
controls, environmental and bio sensing)

Special displays
Analog Representations:

• Dials, Gauges, Lights: Used to represent numerical values or system states in complex systems.

• Example: Show system values like pressure or temperature in control systems.

Digital Displays (LED Indicators):

• Small LCD screens, LED lights etc

• Flashing LEDs: Found on some computers to indicate processor status or activity.

• Working Mechanism: LEDs blink based on the system's operation, signaling various states like
power, processing, or errors.

Head-Up Display (HUD):

• Purpose: Used primarily in aircraft to project essential information directly into the pilot's field
of view.

• Information Displayed: Tactical data, navigational information, and aircraft status.

• Working Mechanism:

o Used in aircraft to project key data (like navigation and status) in front of the pilot’s
eyes.

o How it Works: Transparent screens display the info so the pilot can see both the data
and the outside view without looking down at instruments.

Additional Info: Only critical information is shown on the HUD to prevent clutter, while less important
data is presented on cockpit gauges for periodic monitoring.

Sound output
• Auditory Signals: Sounds like beeps, clanks, or clicks are used to provide feedback in systems,
often alongside visual displays.
 • Feedback: Sound feedback, like keyboard clicks or telephone keypad tones, assures users that
an action has been performed successfully.

• Improving Interaction: Auditory feedback enhances user experience by reducing uncertainty,

like knowing a doorbell rang after pressing it.

• Efficiency: Auditory feedback can speed up performance by providing instant confirmation of

actions, improving overall system interaction.

Touch, feel and smell

Haptic Devices :

• These provide tactile feedback (force, resistance, texture) to enhance physical interaction
with systems.
• Used in virtual environments, such as surgical simulators, and real-world applications like
electronic braille displays.
• Example: Force feedback in game controllers or steering wheels that vibrate to simulate in-
game events.

Tactile Feedback :

• Quick, unconscious reactions (like adjusting steering after hitting a bump) are driven by
tactile feedback.
• This feedback is processed by the lower brain for fast response without conscious effort.

Challenges with Texture :

• Texture perception depends on detecting small changes on the skin, which is easier when
moving fingers over surfaces.
• Technology to simulate realistic texture is still developing.

Smell in Systems :

• Smell strongly triggers memory and enhances immersive experiences in exhibits and games.
• Dynamic smell generation is difficult due to our many scent receptors.
• Devices release scent capsules on demand, but haven't reached mass markets yet.

Potential Uses of Smell :

• Smell can enhance ambient displays and boost awareness in immersive settings like VR or
exhibits.

BMW iDrive
• For controlling menus
• Feel small bumps for each item
• Makes it easier to select options by feel
 • Uses haptic technology from Immersion Corp

Physical controls
Different Physical Controls:

Devices like microwaves, washing machines, and MiniDisc players use unique controls designed for their
specific functions. A microwave has a flat, easy-to-clean plastic panel, ideal for a kitchen environment,
while washing machines use larger dials and switches that also display settings. MiniDisc players use
small buttons with multifunction controls for compactness.

Haptic Feedback and Resistance:

Physical controls, like buttons or dials, provide feedback to users through touch. Proper resistance in
these controls helps users feel that they've successfully pressed a button or adjusted a setting. For
devices with smooth panels (e.g., microwaves), auditory feedback like beeps replaces tactile feedback.

Importance of Texture:

Texture differences in materials aid in usability. For example, touchpads are smooth for easy finger
movement, while textured buttons help prevent slipping. Raised dots on keyboards (like home keys) or
phone keypads assist in identifying key positions without needing to look.

Environment and bio-sensing

-Sensors in Everyday Life: We encounter sensors in many places, like washbasins with infrared sensors
that activate water flow or car doors that trigger courtesy lights. These sensors control various functions
in our environment, from automatic doors to energy-saving lights.

Types of Sensors: Sensors can detect a wide range of things, such as movement (infrared), location
(GPS), temperature, and even weight. Audio and video analysis can identify individuals or activities.

Bio-sensing: Sensors can also measure physiological signs like body temperature or blink rate, helping
systems respond to unconscious reactions. For instance, in speech-based games, pauses and tone shifts
can reveal a user’s emotional state for better interactions.

Everyday Tech and Fiction: Technologies like iris scanners, which seem futuristic, are already available
in everyday devices, narrowing the gap between sci-fi concepts and reality.

The Smart-Its project

The Smart-Its project simplifies building systems with sensors, eliminating the need for advanced
electronics skills. It offers a collection of components and an architecture to easily integrate new
sensors. Basic Smart-It boards come with microprocessors and wireless connectivity. Various modules,
like sensor boards for temperature and light or power controllers, can be plugged into these boards,
streamlining sensor integration for different projects.
PAPER: PRINTING AND SCANNING (print technology, fonts,
page description, WYSIWYG, Scanning, OCR)

Printing
Ease of Printing : Computer systems have made it easier to produce paper documents, allowing multiple
copies for refinement.

Older Printers : Older printers like line, golf-ball, and daisy-wheel printers required changing the
printhead to switch typefaces or sizes, which was cumbersome. Daisy-wheel printers were once the only
affordable option for high-quality output but are now rare.

Dot-Based Printing : Modern printers (dot-matrix, inkjet, and laser) use a dot-based system to form
images. Resolution, measured in dots per inch (dpi), affects image clarity.

Types of Printers :

• Dot-Matrix Printers : (use inked ribbon like a (typewritter)).Line of pins that can strike the
ribbon dotting the ribbon . Have a resolution of 80–120 dpi and are slow, producing about 200
characters per second (cps) for draft mode and 50 cps for letter quality.
• Inkjet Printers and bubble jet printers : Tiny blobs of ink sent from print head to paper. Offer
resolutions of 300–600 dpi and can produce photo-quality prints. They typically print 4–8 pages
per minute.
• Laser Printers : Provide the highest resolution (600–2400 dpi) and are faster, also printing 4–8
pages per minute. Like photocopier : dots of electrostatic charge deposited on drum, which
picks up toner (black powder form of ink ) rolled onto paper which is then fixed with heat.

Speed : Inkjet and laser printers print around 4–8 pages per minute, while dot-matrix printers print
slower, often rated in characters per second.

Cost of Consumables : Inkjet and laser printers have recurring costs for consumables like cartridges,
toner, and high-grade paper. Finding suitable recycled paper for laser printers can be challenging.

Printing in the Workplace

Shop tills

• Dot matrix
• Same print head used for several paper rolls
• May also print cheques

Thermal printers

• Special heat sensitive paper

• Paper heated by pins makes a dot
• Poor quality but simple and low maintenance
• Used in some fax machines
Fonts and page description languages
Fonts : Particular style of text

Size of a font measured in points (1 point about 1/72’’) vaguely related to its height

Font Shape: Fonts are identified by names like Times Roman, Courier, Helvetica.

• Fixed-pitch fonts (e.g., Courier): Each character has the same width.

• Variable-pitch fonts (e.g., Times Roman, Gill Sans): Characters have different widths (e.g., 'm' is
wider than 'i').

Serif and Sans-Serif:

• Serif fonts: Have small cross-lines at the ends of strokes (imitating carved stone).

• Sans-serif fonts: Have clean, square-ended strokes.

Readability of Text
• Lowercase – easy to read shape of words
• Uppercase – better for individual letters and non-words e.g flight numbers : BA793 VS ba793
• Serif fonts – helps your eye on long lines of printed text. But san serif often better on screen

Page Description Languages

• Pages very complex – different fonts, bitmaps lines, digitized photos etc
• Can convert it all into a bitmap and send to the printer but often huge
• Alternative use a page description language
o Sends a description of the page can be sent,
o Instructions for curves, lines, text in different styles etc
o Like a programming language for printing
• PostScript is the most common

Screen and page

-WYSIWYG Concept: WYSIWYG (What You See Is What You Get) ensures on-screen appearance matches
printed output, such as centered text.

Screen vs. Paper:

• Screen resolution is typically 72 dpi , while laser printers exceed 600 dpi.
• Screens use an RGB color model ; printers use CMYK . Size and aspect ratios differ, with A4
pages measuring **297 × 210 mm**.

Design Challenges : Designers must balance resembling printed pages and optimizing for screen
strengths, which can lead to font alignment issues.
Font Issues : Display discrepancies occur due to different widths when printed and shown on screens.

Unified Representation Systems: Some systems use the same font descriptions for both mediums (e.g.,
PostScript in Next OS), which shifts problems rather than solving them.

Graphic Design Differences : Graphic design approaches differ between paper and screens, requiring
adjustments for clarity and consistency in parallel formats.

Scanners and optical character recognition

Scanners convert images into electronic formats, starting with a bitmap and potentially using OCR to
extract text. They can scan printed pages, photographs, or drawings. The two main types are:

Flat-Bed Scanners : Pages are placed on a glass plate for scanning. Some can handle multiple sheets
automatically.

Hand-Held Scanners : Users manually move the scanner over the image, which requires stitching
together multiple strips of the bitmap.

Scanners operate by shining light on the page and recording the reflected intensity and color, often at
resolutions between **600 and 2400 dpi**. Color scanners can output up to **256 levels** of gray or
RGB colors. Monochrome images can be created by thresholding grayscale images.

Scanners are vital in desktop publishing, allowing for electronic cut-and-paste operations. They enable
complex image manipulations like color correction and photomontage effects. Special film scanners can
digitize photographic negatives and slides.

Optical Character Recognition allows computers to "read" printed text, a challenging task due to
diverse fonts and sizes. Modern OCR software can accurately interpret text, preserving formatting like
superscripts and italics.

In electronic publishing, scanned images are optimized for screen viewing, often requiring consideration
of various graphic formats and color discrepancies between different monitors. Digital cameras have
gained popularity, capturing images directly to memory rather than film.

Paper Based Interaction

• Paper usually regarded as output only
• Can be input too – OCR, Scanning etc
• Xerox PaperWorks
o Glyphs – small patterns of /\\//\\\
▪ Used to identify forms etc
▪ Used with scanner and fax to control applications
• More recently
o Papers micro printed – like watermarks
▪ Identify which sheet and where you are
o Special pen can read locations
▪ Know where they are writing