Multimedia Concept & Topics

• Multimedia Concept
• Multimedia Computing
• Multimedia Classification
• Multimedia Topics
• Multimedia Driving Forces
• Multimedia Applications

• Course Outline

Information & Communication Department

Feb. 20, 2023 1
What is Humain Multimedia

2
 What is Informatics Multimedia

• Multi: more than one

• Medium (singular): middle, intermediary, mean
• Media (plural): means for conveying information
– Media in the press, newspaper, radio and TV context -mass media
– Media in communications: cables, satellite, network–transmission media
– Media in computer storage: floppy, CD, DVD, HD, USB – storage media -
Media in Human–computer interaction (HCI) context: text, image, audio,
video, CG – interaction media

3
Analog Systems vs Digital Systems

4
Digital Representation of Multimedia
Contents

5
Digital Representation of Multimedia
Contents

6
EX: Digital Representation of Speech /
Audio Signal

7
 Multimedia Concept & Topics

• Multimedia: refers to various information forms text, image,

audio, video, graphics, and animation in a variety of
application environments
• Multimedia … : product, application, technology, platform,
board, device, network computer, system, classroom, school,
…
• Word “multimedia” is widely used to mean many different
things

8
 What is Multimedia in terms of
Computing
• Computing: Computer-based technologies and applications →
What computers? →Various forms of computers/devices!
• computing terms, four fundamental multimedia attributes:
– Digitized: All media including audio/video are represented in
digital format
– Distributed: The information conveyed is remote, either pre-
produced and stored or produced in realtime, distributed over
networks
– Interactive: It is possible to affect the information received, and
send your own information, in a non-trivial way beyond start, stop,
fast forward
– Integrated: The media are treated in a uniform way, presented in
an orchestrated way, but are possible to manipulate independently 9
 Definition of Multimedia:

Computer-based techniques of text, images, audio, video,

graphics, animation, and any other medium where every
type of information can be represented, processed, stored,
transmitted, produced and presented digitally.

This course focuses on

Audio and Vision

10
 Benefits of Multimedia

• Some authors claim that humans get their information in the

following way:
– more than 80 % by sight - of which 20 % is remembered
– 11 % by hearing - of which 30 % is remembered
– 3.5 % by smell
– 1.5 % by touch and taste.

... Where 50 % of what is both seen and heard is remembered

... Further 80 % of what is seen, heard and done, is remembered
That is, multiple, media, and interactive
should be a good
thing 11
 A Classification of Multimedia
• Text - ASCII/Unicode, HTML, Postscript, PDF
• Audio – Sound, music, speech, structured audio (e.g. MIDI-
Musical Instrument Digital Interface)
• Still Image - Facsimile, photo, scanned image
• Video (Moving Images) – Movie, a sequence of pictures
• Graphics – Computer-produced image
• Animation – A sequence of graphics images
• Discrete Media (DM, Static): text, image, graphics
• Continuous Media (CM, Dynamic): audio, video, animation
• Captured vs Synthesized media
• Standalone vs Networked media
That is, multimedia is multidiscipline of;
these classes in different 12
 System Implications of Multimedia
Multimedia imposes new requirements on all parts of the system
architecture:
•Representation - digitization and coding (compressing)
•Storage - database, larger volumes and new access patterns
•Processing - OS, scheduling, indexing, searching
•Understanding - speech/object recognition, content analysis
•Production - more complex authoring and user interface
software
•Presentation - user perception, user friendly in HCI (Human
Computer Interface)
•Protection – media encryption, copyright, privacy
•Distribution - media delivery and broadcast
•Communication - media transmission over network/internet,
session control 13
 Why is Multimedia Important ?

• Digital audio/video is revolutionizing music, film, game, and

video & audio industries
• Convergence of computers, telecommunication, radio, and
TV
Caused by technology and competition
Dramatic changes in products and infrastructure
• New application potential
Huge potential markets
Improving our lives (learning, entertainment, and work)
• Interesting technical issues

Multimedia has become hot and been emerged

in CS/IT since 1985 14
Forces Driving the Multimedia Revolution

• Evolution of communication and data networks: Increasing

availability of bandwidth on demand in the office, home,
road.... Thanks to high-speed data modems, cable modems,
hybrid fiber-coax systems, xDSL, wireless
• Ubiquitous access to network. Via local-area networks
(LAN), wireline and wireless networks, Internet, world wide
web, → “anywhere, anytime”.
• Fast processor and large capacity storage devices, including
3-D hardware. Moore’s law: computation and memory
capacity of chips doubles every 18 months or so.
15
Forces Driving the Multimedia Revolution
(Cont…)
• New algorithms and data structures. Compression
techniques, graphics, computer vision, speech
understanding...
• Smart terminals such as digital phones, screen phones,
multimedia PC’s, web-TV, personal digital assistants, etc.,
accessing and interacting the network with wired and
wireless connections.
• And of foremost importance, the digitization of virtually any
device: cameras, video capture, and playback devices,
handwriting terminals, sound capture, etc., together with
plug-and-play standards; and the digitization of
text/audio/video documents and libraries that allows better
communications, storage, and fast access and browsing. 16
 Technological Aspects
• Techniques for compressing and coding the various media:
models, algorithms, forms, standards, etc.
• Communications aspects: downloading and streaming
techniques, synchronization, layering of signals, issues
involved in the definition of QoS (quality of service.)
• Techniques for accessing multimedia signals by providing
tools that match user to the machine: “natural” spoken
language queries, media conversion tools and multimodal
user interface (speech recognition, lip reading, face tracking,
OCR,..), agents that monitor the multimedia sessions and
provide assistance in all phases of access and utilization.
• Techniques for organizing, storing and retrieving
multimedia, for searching and browsing individual
multimedia documents and libraries. 17
 Are Multimedia Applications Hard?

• Large size of multimedia objects

– Speech: 8000 samples/s – 8 Kbytes/s
– CD audio: 44,100 samples/sec, 2 bytes/sample,
– stereo audio – 176 Kbytes/s
– NTSC video: 30 frames/s, 640x480 pixels, 3 bytes/pixel – 30
Mbytes/s (too big, 2-8 Mbits/s if compressed)
– More storage required
– More main memory
– 10-30 GB secondary storage
– TB’s of tertiary storage
• Real-time performance requirements
18
 Are Multimedia Applications Hard?
(Cont…)
• Higher data rates – Fast I/O subsystem (SCSI-Small
Computer System Interface , fiber channel, HIPPI- High
Performance Parallel Interface )
– E.g., Ultra SCSI2 – 80 Mbytes/s – High speed backplane (PCI
or faster)
– Faster network (1-25Mbs per video stream)
– 1-4 Gbits/s network
• Smart terminals such as digital phones, screen phones,
multimedia PC’s, web-TV, personal digital assistants, etc.,
accessing and interacting the network with wired and
wireless connections.
• Hardware CODEC, modified CPU (?), and modified frame
buffer/graphics subsystem
Essentially, new hardware and software 19
Further, audio/image/video “content” processing
 Examples of Multimedia Applications
Examples of Multimedia Applications

• Residential services –  Science and technology –

Video-On-Demand – Virtual environment –
Video phone, A/V Scientific visualization,
conferencing – Home prototyping
shopping  Entertainment – Games
• Business services – – Interactive TV – Post
Corporate education – production of movie and
E-business music
• Education – Digital  Medicine, Web
libraries – Distance applications, etc.
learning

20
 General Overview of a Multimedia
System
Acquisition – Generation
Processing Processing
Manipulation
Integration Storage of Data Transmission
• TV Cameras Tap, CD-Rom, Disk farms, Network
Servers
• Microphones • ADSL
Coding / •
• Digital Live Data HFC
Compression • Wireless
Cameras
• LAN
• Scanners • MAN
• WAN
• Pc
Analysis Indexing of
data

• Mixers Other
Local/Remote
• Synthesizer Network
Synchronization
For Multimedia
and Presentation
System
User Site
TV, VCR, DVD, PC, CD-RW, 21
Printer
 General Overview of a Multimedia
System
√ √ √
√

√ √
√

√
√
22
Audio/Video Broadcast over the Internet

23
Digital Library

24
 Teaching Plan

Part I: Multimedia Fundamentals and Coding Techniques Lesson

1. Multimedia Concept and Topics Lesson
2.Audio Fundamentals Lesson
3.Audio Coding and Standard Lesson
4.Image/Video Fundamentals Lesson
5.Image/Video Coding: JPEG and H.26x Lesson
6.MPEG Coding Standards Lesson
7.Review of Advanced MM Coding

Part II: Multimedia Technologies and Applications

25