Multimedia

Principles

Chapter 13

Multimedia Architecture

TMH Chapter - 13 1
Multimedia Architecture
User Interface
ƒ The user interface is the interactive input and output of the
computer as it is perceived and operated on by users.

ƒ Multimedia user interfaces are computer interfaces that

communicate with users using multiple media, sometimes using
multiple modes such as written text together with spoken
language.

ƒ In a textual interface the user is restricted to enter key

combinations in a single line on the screen, called the command
line, and press ENTER to see the result.

TMH Chapter - 13 2
Multimedia Architecture
User Interface
ƒ Using the mouse as the main input device – have greatly
simplified human computer interaction. A graphical user interface
allows the user to interact with the computer in a much more
flexible manner than textual or command-line interface

ƒ The GUI consists of graphical widgets such as windows, menus,

radio buttons, checkboxes and icons and employs a pointing
device such as mouse, trackball, lightpen or touchscreen) in
addition to a keyboard.

ƒ The user can interact with both textual and graphical objects
throughout the screen by clicking, moving, dragging over them
using the pointing device.

TMH Chapter - 13 3
Multimedia Architecture
User Interface
ƒ The main emphasis in the design of multimedia user interfaces is
multimedia presentation. The general design issues are :

· To determine the information content to be communicated.

· To choose the proper media for representing the content.
· To coordinate the use of different media within a presentation.
· To provide interactive exploration of the information presented.

TMH Chapter - 13 4
Multimedia Architecture
User Interface
ƒ User-friendliness is the main property of a good user interface. What
this user-friendliness means and how the property is achieved is not
always clear. The design of a user-friendly graphical interface requires
consideration of many conditions :

· Application instructions must be easy to learn.

· A context sensitive help function using hypermedia techniques should
be available.
· The user should be able to easily remember the application instruction
rules.
· The visual perceptions like color, shape, layout etc. should be
aesthetic.
· The interface should be easily implementable without increasing the
cost too much

TMH Chapter - 13 5
Multimedia Architecture
User Interface
ƒ A widget is a graphical interface element that a computer user
interacts with like a button, check-box, radio-button, icon, text-
box, menu, window, scrollbars, toolbar etc.

ƒ The widget toolkit are sets of widget elements for GUI, often
implemented as a library. There can be several types ranging
from low-level widget toolkits integrated within the operating
system to high-level toolkits available as separate application
packages.

ƒ The X Window System is a standard toolkit to build GUI

capabities on Unix based systems. It provides basic capabilities
like working with windows and interacting via the mouse pointer.

TMH Chapter - 13 6
Multimedia Architecture
Windows MM Support
ƒ An application programming interface (API) is a set of definitions
for determining how one set of software (usually low-level)
communicates with another set of software (usually high-level).

ƒ One of the primary purposes of an API is to provide a set of

commonly used functions, for example to draw common
graphical objects like windows on the screen.

ƒ APIs themselves are abstract, software that provides access to

API is called the implementation of the API, usually in the form of
a software development kit (SDK).

TMH Chapter - 13 7
Multimedia Architecture
Windows MM Support
ƒ Windows API is a set of application programming interfaces
available on the Microsoft Windows platform.

ƒ A software development kit (SDK) provides tools and

documentation for developers to create software using the
Windows API. It is essentially a set of low-level functions for
implementing GUI in Windows.

ƒ Win16 was the first 16-bit version of these APIs, while Win32 is
the 32-bit version. The API consists of C and C++ functions
implemented in dynamic linked libraries (DLLs) mainly in the OS
core files : KERNEL32.DLL, USER32.DLL and GDI32.DLL.
TMH Chapter - 13 8
Multimedia Architecture
Windows MM Support
ƒ A library is a set of sub-programs used to develop software. Sub-
programs in a library are not independent programs, they are
designed to perform specific tasks.

ƒ Nowadays typical application software contain a lot of such sub-

programs to performing definite functions. This provides a way of
using existing functions into new programs without writing
everything from scratch.

ƒ DirectX and OpenGL are two graphic libraries used to develop

graphic applications. The process of making a library accessible
to a new program is called library linking.
TMH Chapter - 13 9
Multimedia Architecture
Windows MM Support
ƒ Library linking can be either static or dynamic. Static linking embeds a
library directly into an executable program at compile time by a linker.

ƒ Dynamic linking involves creating pointers to the library from within the
program by a loader. At runtime the actual library is loaded into memory
and executed.

ƒ Microsoft’s DirectX is a collection of APIs designed to overcome these

kinds of limitations, and promote high-performance multimedia
application development. DirectX APIs provide real-time low-level
access to peripherals specifically used in intensive audio/video
applications. The DirectX SDK is available for free from Microsoft.

TMH Chapter - 13 10
Multimedia Architecture
Windows MM Support
ƒ OpenGL (Open Graphics Library) is a set of cross-platform APIs
for developing 2D and 3D graphics.

ƒ It includes about 250 functions for drawing complex three

dimensional scenes from primitives. It is very popular in the video
games industry and competes with Direct3D API.

ƒ Apart from game development OpenGL is also used in CAD and

virtual reality applications. Effective implementations of the
OpenGL routines are generally provided by the display device
manufacturers and rely heavily on the hardware.

TMH Chapter - 13 11
Multimedia Architecture
Windows MM Support
ƒ The Component Object Model (COM) is a technology developed by
Microsoft for cross-software communication.

ƒ Its predecessor was Object Linking and Embedding (OLE) and its
successor is planned to the .NET framework. OLE was Microsofts
earliest object-based technology built on top of Dynamic Data Exchange
(DDE) designed for compound documents.

ƒ In recent times COM technology has been superseded by the .NET

framework. There exists a limited backward capability between COM
and .NET : COM objects may be used in .NET by implementing Runtime
Callable Wrapper (RCW) while .NET objects may be used in COM
environments by a COM Callable Wrapper (CCW).

TMH Chapter - 13 12
Multimedia Architecture
Windows MM Support
ƒ The .NET framework from Microsoft is a software development
platform focused on Rapid Application Development (RAD)
especially of Internet and intranet applications, through
introduction of new tools and functionalities to the API.

ƒ It provides a new object oriented environment to develop

applications for both Windows and the Web, often referred to as
Web Services.

ƒ A collection of development tools is specifically developed for

use with the .NET environment, the main among them being
Visual Studio .NET.
TMH Chapter - 13 13
Multimedia Architecture
Hardware Support
ƒ The Universal Serial Bus (USB) was designed to replace older computer
serial and parallel I/O buses, to provide a faster, more user-friendly
interconnection method.

ƒ Computer keyboards, mice, modems, CD-ROM drives, printers,

scanners, digital cameras etc. can all be connected in the USB.

ƒ USB provides low to moderate speed interconnection; its speed of 12

Mbps is much faster than the 115 Kbps speed of serial ports.

ƒ USB can support 127 devices in a plug and play fashion; moreover USB
devices can be hot-swapped without powering down the system.

TMH Chapter - 13 14
Multimedia Architecture
Hardware Support
ƒ The Small Computer System Interface (SCSI) is mostly used for
connecting hard disks and tape drives but can also be used for
connecting a wide range of other devices like scanners, CD
drives and DVD drives.

ƒ It was initially developed to provide faster disk access than the

existing IDE drives and was mostly used for high-end server
systems, although being more costly than its IDE counterparts.

ƒ SCSI requires a host controller to be installed on the computer

usually on the PCI bus, while the SCSI bus itself starts from the
controller and can connect a number of different types of
devices.

TMH Chapter - 13 15
Multimedia Architecture
Hardware Support
ƒ Each SCSI device, including the host adapter, must be
configured to have an unique SCSI ID on the bus.

ƒ The early SCSI standards supported 8 devices on a single bus

with the ID “0” usually allotted to the controller while the IDs “1”
through “7” being allotted to the devices. The other free end of
the SCSI bus was terminated by a terminator.

ƒ SCSI-1 - The original standard that was derived from SASI and
adopted by ANSI in 1986. It featured an 8-bit bus with parity,
running asynchronously at 3.5 MB/s or 5 MB/s in synchronous
mode.
TMH Chapter - 13 16
Multimedia Architecture
Hardware Support
ƒ SCSI-2 - Introduced in 1989 this standard aimed to improve the
data transfer rate of SCSI-1. It doubled the maximum data
transfer rate from 5 MB/s to 10 MB/s and this variant was called
Fast-SCSI.

ƒ This standard also doubled the bus width from 8-bit to 16-bit
leading to another variant called Wide-SCSI. Wide-SCSI also
doubled the total number of devices that could be connected on
a single bus from 8 to 16.

ƒ Combining these variants Fast & Wide SCSI could now support
20 MB/s.
TMH Chapter - 13 17
Multimedia Architecture
Hardware Support
ƒ SCSI-3 - Introduced in 1992 this standard again doubled the bus
speed to 20 MB/s for 8-bit version (also called narrow-SCSI) and
40 MB/s for the Fast and Wide version.

ƒ Ultra-2 SCSI - Introduced in 1997 it included a differential mode

of operation which allowed a maximum cable length of 12 meters
with much greater noise immunity. Additionally the data transfer
rate was increased to 80 MB/s.

ƒ Ultra-3 SCSI - Introduced in 1999, here the data transfer rate

was again doubled to 160 MB/s and offered CRC error checking
features.
TMH Chapter - 13 18
Multimedia Architecture
Hardware Support
ƒ MMX (Matrix Math Extension) is an instruction set developed by
Intel for their Pentium MMX processors designed to accelerate
graphics, audio and video signal processing.

ƒ It consists of a set of operations for efficiently handling large

quantities of data in parallel, referred to by the term SIMD (Single
Instruction Multiple Data).

ƒ An example of an SIMD instruction is where the same value is

being added to a large number of data say, changing the pixel
values of an image by the same amount. An MMX processor
handles such operation in blocks instead of individual pixel
values thereby leading to higher efficiency.

TMH Chapter - 13 19
Multimedia Architecture
Hardware Support
ƒ The IEEE 1394 (also called FireWire) bus is a universal, platform
independent digital interface that can connect digital devices
such as computers, audio and video multimedia products,
printers, scanners, video cameras etc.

ƒ For example a digital video camera can be connected to a

personal computer for transferring audio/video data.

ƒ The IEEE 1394 cable is thin (like a phone cable) and uses
copper wires. There are two separately shielded twisted pair
transmission lines for signaling, two power conductors and a
shield.
TMH Chapter - 13 20
Multimedia Architecture
Hardware Support
ƒ The conductor is the same on both ends of the cable and is small
with side-locking tab restraints.

ƒ Like a SCSI cable it can be used as a point-to-point connection

or use daisy-chaining scheme. However unlike SCSI, no device
addressing or termination is needed.

ƒ Upto 63 devices can be connected together before a bridge is

required.

ƒ IEEE 1394 is hot pluggable so that users can add or remove

devices from a powered bus
TMH Chapter - 13 21
Multimedia Architecture
Distributed MM Applications
ƒ Distributed computing refers to a computing paradigm in which the
various computing tasks being carried out are distributed to multiple
geographically dispersed processors.

ƒ Similar to client-server computing, distributed computing relies heavily

on data networks for communication between distributed resources.

ƒ One of the main motivations in moving from a single-site computing to

distributed computing is to avoid the possibility of a single-point failure
which should bring the entire system to a halt.

ƒ In a true distributed system not only the hardware but the software also
remains dispersed. For example a database application may be
subdivided into geographically dispersed components, but this should
be transparent to the user to whom it should seem like a single entity.
TMH Chapter - 13 22
Multimedia Architecture
Distributed MM Applications
ƒ Some examples of distributed multimedia applications are :

· Companies can communicate product information to customers

in format and location they prefer.
· In advertising, creative designers can share ideas for video
commercials with colleagues and customers through audio/video
enhanced multimedia documents.
· Medical professionals can collaborate by sharing diagnostic
images or video-taped procedures across the world.
· Education through distance learning saves on travel cost and
increase productivity by allowing office workers to gain
immediate access to materials rather than wait for scheduled
classes.
TMH Chapter - 13 23
Multimedia Architecture
Distributed MM Applications
ƒ The basic architecture of a distributed multimedia system would
involve :

· A creation subsystem where the content is created.

· A storage subsystem where the created content is stored.
· A distribution subsystem using which the content may be
distributed to multiple users.
· A management subsystem where the entire operations are
managed.

TMH Chapter - 13 24
Multimedia Architecture
Distributed MM Applications
ƒ The concept of videophony is straightforward : telephone with
motion video. Its main objective is to show the facial expressions
of the communicating persons rather than environmental details
and hence the camera is focused for a “head and shoulders”
view.

ƒ A medium resolution and a frame rate of 6 to 12 fps is

acceptable. Usually an ISDN or leased line is used to transmit
the audio-video data.

TMH Chapter - 13 25
Multimedia Architecture
Distributed MM Applications
ƒ Unlike video-phony, videoconferencing involves several
individuals or several groups of individuals engaged in dialog.

ƒ The objective is not to hold a single bilateral conversation but to

support a meeting between more than two remote participants.

ƒ Participants may gather in an office or meeting place and the

camera has to cope with “several heads and shoulders”
scenario.

ƒ There may be one fixed camera whereby the whole group is

captured by the camera, or a moving camera which captures the
view of the current speaker.
TMH Chapter - 13 26
Multimedia Architecture
Distributed MM Applications
ƒ Paper documents may be captured by a second vertical camera
called the document camera, or electronic documents may be
transmitted prior to the session.

ƒ ISDN and leased line connections are also used for

videoconferencing applications.

ƒ Since these applications occur in real-time, audio-video

CODECs play a crucial part in transmitting audio-video
information over networks, usually over 128 Kbps channels
(16Kbps for audio and 112 Kbps for video). CODECs such as
H.261 and H.263 are usually used for the purpose
TMH Chapter - 13 27
Multimedia Architecture
Distributed MM Applications
ƒ Video on demand (VoD) covers a range of applications whereby users
can request access to video servers of still and moving pictures on an
individual basis.

ƒ The definition does not specify where the video server should be located
so VoD can be a service offered within an organization as well as a
public service.

ƒ The definition also does not specify the level of interaction that the users
are expected to perform with the video server. VoD usually do not imply
any special kind of video.

ƒ Sometimes specific applications like distribution of movies is explicitly

called movie on demand.

TMH Chapter - 13 28
Multimedia Architecture
Distributed MM Applications
ƒ Movie-on-demand (MoD) is a public VoD service where the accessed
object is a stored movie. The objective of the MoD is to replace and
improve on two earlier services by combining them into one :

· Pay-per-View (PPV) by offering a much wider choice of movies.

· Videotape rental service by eliminating the fetch/return cycle and
manipulation of physical tapes.

ƒ However one drawback of MoD systems might be a lack of interactivity

options. Earlier systems offered either no interactivity or coarse controls
like pause and play.

ƒ Ideally all VCR-like functions should be available during the movie

delivery, called a fully interactive MoD.

TMH Chapter - 13 29
Multimedia Architecture
Distributed MM Applications
ƒ Voice over IP (VoIP) applications have originated from the
concept of Internet telephony which makes it possible to have a
telephone conversion over the Internet or dedicated network
where the voice data is carried via IP packets instead of voice
transmission lines.

ƒ This incorporates packet switching instead of circuit switching for

conventional telephone lines. Its advantages include lower cost
per call and lower infrastructural costs.

ƒ The protocols used to carry voice data over IP networks are

called VoIP protocols. One of the most widely used protocols is
the H.323 which is an ITU-T recommendation.

TMH Chapter - 13 30
Multimedia Architecture
Distributed MM Applications
ƒ Real Time Transport Protocol (RTP) is a protocol providing
network transport functions suitable for applications transmitting
real time data such as audio, video and animation.

ƒ It was developed by the Audio Video Transport Working Group of

the IETF and published in 1996.

ƒ RTP defines a standardized packet format for delivering audio

and video over the Internet in real time.

TMH Chapter - 13 31
Multimedia Architecture
Distributed MM Applications
ƒ RTP consists of a data and a control part. The latter is called
RTCP (Real Time Control Protocol).

ƒ The data part of RTP is a thin protocol providing support for

applications with real-time properties such as continuous media
(e.g., audio and video), including timing reconstruction, loss
detection, security and content identification.

ƒ RTCP provides support for real-time conferencing of groups of

any size within an internet. This support includes source
identification and support for gateways like audio and video
bridges as well as multicast-to-unicast translators.
TMH Chapter - 13 32
Multimedia Architecture
Distributed MM Applications
ƒ Resource Reservation Protocol (RSVP) is a protocol that
transfers reservations and keeps a state at the intermediate
nodes.

ƒ The RSVP protocol is used by a host to request specific qualities

of service from the network for particular application data
streams or flows.

ƒ RSVP requests will generally result in resources being reserved

in each node along the data path.

TMH Chapter - 13 33
Multimedia Architecture
Distributed MM Applications
ƒ Real Time Streaming Protocol (RTSP) is a client server multimedia
presentation control protocol designed to address the needs for efficient
delivery of streamed multimedia over IP networks.

ƒ Streaming of data involves a number of components. Multimedia

content must be stored in specific file formats that supports streaming
like Apple QuickTime or Microsoft Active Streaming Format (ASF),
usually on dedicated servers.

ƒ The servers in turn stream the contents using protocols like RTSP to the
clients.

ƒ On the client the streamed content is played back using compatible

players like Apple’s QuickTime Movie Player or RealNetwork’s
RealPlayer.
TMH Chapter - 13 34