Multimedia - IT451

Lecture 2

BY…
DR. NOURA SEMARY
Lecture 2: Multimedia Data
Basics
Multimedia Definition
Multimedia systems/applications have to deal with the
◦ Generation of data,
◦ Manipulation of data,
◦ Storage of data,
◦ Presentation of data, and
◦ Communication of information/data
Multimedia Data
All data must be in the form of digital information.
The data may be in a variety of formats:
◦ text,
◦ graphics,
◦ images,
◦ audio,
◦ video.
Synchronization
A majority of this data is large and the different media may need synchronisation:
◦ The data will usually have temporal relationships as an integral property.
Static and Continuous Media
Static or Discrete Media :
◦ Some media is time independent:
◦ Normal data, text, single images, graphics are examples.

Continuous Media :
◦ Time dependent Media:
◦ Video, animation and audio are examples.
Analog and Digital Signal Conversion
The world we sense is full of analog signals:
◦ Electrical sensors convert the medium they sense into electrical signals
◦ E.g. transducers, thermocouples: temperature sensor, microphones: acoustic sensor Cameras (Still
and Video): light sensor.
◦ (usually) continuous

◦ Analog signals (e.g. Sound and Light) Analog: continuous signals must be
converted or digitized for computer processing.

◦ Digital: discrete digital signals that computer can readily deal with.
Analog-to-Digital Converter (ADC)
Special hardware devices : Analog-to-Digital converters. E.g. Audio:

Take analog signals from analog sensor (e.g. microphone) and digitally sample data (More details
later)
Analog-to-Digital Converter (ADC)
An ADC converts a continuous-time and continuous-
amplitude analog signal to a discrete-time and discrete-
amplitude digital signal.
The conversion involves quantization of the input, so it necessarily
introduces a small amount of error or noise. 
ADC does the conversion periodically, sampling the input, limiting
the allowable bandwidth of the input signal.
Analog-to-Digital Converter (ADC)
The performance of an ADC is primarily characterized by its bandwidth and
signal-to-noise ratio (SNR).
rized by its bandwidth and signal-to-noise ratio (SNR). The bandwidth of an ADC
is characterized primarily by its sampling rate.
The SNR of an ADC is often summarized in terms of its effective number of bits
(ENOB), the number of bits of each measure it returns that are on average not
noise.
An ideal ADC has an ENOB equal to its resolution.
Sampling rate
An analog signal is continuous in time and it is necessary to convert this to a
flow of digital values.
It is therefore required to define the rate at which new digital values are
sampled from the analog signal.
Nyquist–Shannon sampling theorem:
When sampling rate is greater than twice the bandwidth (highest frequency) of the signal,
reconstruction is possible.
Resolution
The resolution of the converter indicates the number of different, ie discrete, values it can
produce over the allowed range of analog input values.
Thus a particular resolution determines the magnitude of the quantization error and therefore
determines the maximum possible signal-to-noise ratio for an ideal ADC without the use of
oversampling.
The input samples are usually stored electronically in binary form within the ADC, so the
resolution is usually expressed as the audio bit depth.
In consequence, the number of discrete values available is usually a power of two.
For example, an ADC with a resolution of 8 bits can encode an analog input to one in 256
different levels (28 = 256). The values can represent the ranges from 0 to 255 (i.e. as
unsigned integers) or from −128 to 127 (i.e. as signed integer), depending on the application.
Digital-to-Analog Converter (DAC)
Playback – a converse operation to Analog-to-Digital
Takes digital signal, possible after modification by computer (e.g. volume change, equalisation)
Outputs an analog signal that may be played by analog output device (e.g. loudspeaker, RGB
monitor/display)
Analog-to-Digital-to-Analog Pipeline (1)
Begins at the conversion from the analog input and ends at the conversion from the output of
the processing system to the analog output as shown:
Analog-to-Digital-to-Analog Pipeline (2)
Anti-aliasing filters (major part of Analog Conditioning) are needed at the input to remove
frequencies above the sampling limit that would result in aliasing. More later The anti-aliasing
filter at the output removes the aliases that result from the sampling (see sampling theorem).
After the anti-aliasing filter, the analog/digital converter (ADC) quantizes the continuous input
into discrete levels.
After digital processing, the output of the system is given to a digital/analog converter (DAC)
which converts the discrete levels into continuous voltages or currents.
This output must also be filtered with a low pass filter to remove the aliases from the sampling.
Subsequent processing can include further filtering, mixing, or other operations. However, these
will not be discussed further in this course.
Multimedia Data: Input and format
How to capture and store each Media format?
Note that text and graphics (and some images) are mainly generated directly by
computer/device (e.g. drawing/painting programs) and do not require digitizing:

They are generated directly in some (usually binary) format.

•Printed text and some handwritten text can be scanned via Optical Character Recognition
•Handwritten text could also be digitized by electronic pen sensing
•Printed imagery/graphics can be flatbed scanned directly to image formats.
Text and Static Data
Source: keyboard, speech input, optical character recognition, data stored on disk.

Stored and input character by character:

• Storage: 1 byte per character (text or format character), e.g. ASCII; more bytes for Unicode.
• For other forms of data (e.g. Spreadsheet files). May store as text (with formatting, e.g. CSV – Comma-
Separated Values) or may use binary encoding.
Text and Static Data (cont.)
Formatted Text: Raw text or formatted text e.g HTML, Rich Text Format (RTF),
Word or a program language source (Java, Python, MATLAB etc.)
Data Not temporal — BUT may have natural implied sequence e.g. HTML
format sequence, Sequence of Java program statements.
Size Not significant w.r.t. other Multimedia data formats. 4
Compression: convenient to bundle files for archiving and transmission of
larger files. E.g. Zip, RAR, 7-zip. General purpose compression programs may not
work well for other media types: audio, image, video etc.
Graphics
Format: constructed by the composition of primitive objects such as lines, polygons, circles,
curves and arcs.
Input: Graphics are usually generated by a graphics editor program (e.g. Illustrator, Freehand) or
automatically by a program (e.g. Postscript).
Graphics (cont.)
Graphics input devices: keyboard (for text and cursor control), mouse, trackball
or graphics tablet.
Graphics are usually selectable and editable or revisable (unlike images).
Graphics files usually store the primitive assembly
Do not take up a very high storage overhead.
Graphics (cont.)
Graphics standards : OpenGL - Open Graphics Library, a standard specification defining a cross-
language, cross-platform API for writing applications that produce 2D/3D graphics.
Animation: can be generated via a sequence of slightly changed graphics 2D animation: e.g.
Flash — Key frame interpolation: tweening: motion & shape
Graphics (cont.)
Animation (cont.) 3D animation: e.g. Maya. Change of shape/texture/position, lighting, camera
Graphics animation is compact – suitable for network transmission (e.g. Flash).
Images
Still pictures which (uncompressed) are represented as a bitmap (a grid of pixels).
Images (cont.)
Input: scanned for photographs or pictures using a digital scanner or from a digital camera.
Input: May also be generated by programs like graphics or animation programs.
Analog sources will require digitizing.
Stored at 1 bit per pixel (Black and White), 8 Bits per pixel (Grey Scale, Color Map) or 24 Bits per
pixel (True Color)
Size: a 512x512 Grey scale image takes up 1/4 MB, a 512x512 24 bit image takes 3/4 MB with
no compression.
This overhead soon increases with image size — modern high digital camera 10+ Megapixels ≈
29MB uncompressed!
Compression is commonly applied.
Can usually only edit individual or groups of pixels in an image editing application, e.g.
photoshop.
Audio
Audio signals are continuous analog signals. Input: microphones and then digitized and stored
CD Quality Audio requires 16-bit sampling at 44.1 KHz: Even higher audiophile rates (e.g. 24-bit,
96 KHz)
1 Minute of Mono CD quality (uncompressed) audio = 5 MB. Stereo CD quality (uncompressed)
audio = 10 MB.
Usually compressed (E.g. MP3, AAC, Flac, Ogg Vorbis)
Video
Input: Analog Video is usually captured by a video camera and then digitized, although digital
video cameras now essentially perform both tasks. There are a variety of video (analog and
digital) formats (more later) Raw video can be regarded as being a series of single images. There
are typically 25, 30 or 50 frames per second.