Multimedia in embedded Linux

systems

Multimedia
in embedded
Linux systems
Free Electrons

1
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http
 Rights to copy
© Copyright 2004-2009, Free Electrons
feedback@free-electrons.com

Document sources, updates and

translations:
http://free-electrons.com/docs/multimed
ia

Attribution – ShareAlike 3.0 Corrections, suggestions, contributions

You are free and translations are welcome!
to copy, distribute, display, and perform the work
to make derivative works Latest update: May 6, 2020
to make commercial use of the work
Under the following conditions
Attribution. You must give the original author credit.
Share Alike. If you alter, transform, or build upon this
work, you may distribute the resulting work only
under a license identical to this one.
For any reuse or distribution, you must make clear to
others the license terms of this work.
Any of these conditions can be waived if you get
permission from the copyright holder.
Your fair use and other rights are in no way affected by the
above.
License text: 2
http://creativecommons.org/licenses/by-sa/3.0/legalcode
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http
 Scope of this training

Multimedia in embedded Linux systems

This training targets the development of
multimedia-capable embedded Linux systems.
Though it can be useful to playing or creating
multimedia content on GNU/Linux desktops, it is
not meant to cover everything about this topic.
Audio in embedded Linux systems is already
covered in a dedicated training:
http://free-electrons.com/training/audio
Linux 2.6
This training only targets new systems based on
the Linux 2.6 kernel. This way, you leverage the
most advanced technology and don't learn about
something getting obsolete. 3
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http
 Contents

Introduction Kernel subsystems

Glossary Video4Linux API
DVB API
Multimedia libraries
SDL Free Software Video
DirectFB Video codecs and file for
mats
Multimedia distributions

4
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http
 Quick Glossary (1)

Alpha channel:
A additional value in image pixels, describing
their opacity (1: opaque, 0: transparent)
Blitting: from BitBLT "Bit Block Transfer"
Copying image data (e.g. copying a surface on
another), applying image combination
operations.
Keying:
When compositing 2 image frames together,
removing parts of one according to a key (like
its color or brightness), to reveal the
corresponding part in the other.

5
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http
 Quick Glossary (2)

VBI: Vertical Blanking Interval

Interval during which the transmission of a video
signal is suspended to let the electron gun go back
to the first screen line.
Used to transmit extra data, such as teletext.
See http://wikipedia.org for details!

6
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http
 Multimedia in embedded Linux
systems

Multimedia libraries
SDL

7
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http
 SDL

Simple DirectMedia Layer

http://libsdl.org/
License: LGPL
Cross-platform multimedia library
Originally created for writing games, by Sam
Lantinga, for Loki Entertainment Software,
specializing in porting games to GNU/Linux.
Provides low level access to audio, keyboard,
mouse,
joystick, 3D hardware via OpenGL, and 2D video
framebuffer.
Native C API
Bindings available for most programming and 8
scripting
Free Electrons. languages
Kernel, drivers and embedded Linux development, consulting, training and support. http
 SDL screenshots

QEMU: a CPU and system emulator Pig: a demo arcade game. 7000 lines.

9
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http
 SDL supported operating systems

Officially supports: Linux, Windows, Windows CE,

BeOS, MacOS, MacOS X, FreeBSD, NetBSD,
OpenBSD, BSD/OS, Solaris, IRIX, and QNX.
Code with no official support: AmigaOS,
Dreamcast, Atari, AIX, OSF/Tru64, RISC OS,
SymbianOS and OS/2.
Makes it very easy to create platform independent
applications!

10
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http
 SDL capabilities (1)

Video
Video mode setting at any depth (8-bpp or greater).
Automatic detection of the closest available mode.
Optional conversion if the video mode is not supported
by the hardware.
Writing directly to a linear graphics framebuffer.
Creating surfaces with colorkey or alpha blending
attributes.
Hardware accelerated blit and fill operations are used if
supported by the hardware (using MMX acceleration in
x86).
OpenGL support.
Facility to load BMP images.
11
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http
 SDL capabilities (2)

Events
Application visibility changes
Keyboard input
Mouse input
User-requested quit
Each event can be enabled or disabled with
SDL_EventState().
Events passed through a user-specified filter
function before being posted to the internal event
queue.
Thread-safe event queue.
12
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http
 SDL capabilities (3)

Audio
Audio playback of 8-bit and 16-bit audio, mono or
stereo.
Optional conversion if the format is not
supported by the hardware.
Audio run independently in a separate thread,
filled via a user callback mechanism.
CD-ROM audio
Complete CD audio control API

13
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http
 SDL capabilities (4)

Threads Endian independence

Simple thread Detect the endianism
creation API. of the current system.
Simple binary Routines for fast
semaphores swapping of data
for synchronization. values.
Timers Read and write data
of a specified
Measure elapsed
endianism.
milliseconds.
Wait for n
milliseconds.
Set a single periodic
14
timer with 10ms
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http
 SDL implementation on GNU/Linux

Video display. Either uses

X11: taking advantage of XFree86 DGA extensions
and new MTRR acceleration for fullscreen display.
Or DirectFB (see the next section)
Uses the OSS API for sound. Fine for Linux systems
with ALSA, thanks to OSS emulation.
Threads are implemented using either the clone()
system call and SysV IPC, or glibc-2.1 pthreads.

15
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http
 SDL in embedded systems

Should work on any platform supporting Linux.

Even runs on lightweight devices without X11.
A framebuffer driver is sufficient (using the
DirectFB API).
Useful to develop your application on the host in
parallel with target system development.
Facilities to support the target endianism.

16
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http
 Additional SDL libraries

SDL_image: http://libsdl.org/projects/SDL_image/
Image loading library supporting JPEG, PNG, GIF,
BMP...
SDL_ttf: http://libsdl.org/projects/SDL_ttf/
TrueType font support library
SDL_mixer: http://libsdl.org/projects/SDL_mixer/
Sample multi-channel audio mixer library. Supports
Ogg, MP3, MIDI.
SDL_gfx: http://freshmeat.net/projects/sdl_gfx/
SDL graphics drawing primitives and other support
functions.
SAgl: http://freshmeat.net/projects/sagl/
Small Anti-aliased graphics library. 17
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http
 Other free SDL libraries

Listed on Glyph Keeper

http://libsdl.org/libraries.p TrueType font rendering
hp library.
Lots of libraries to ease Fast Events
the development of A fast thread based
games, in particular replacement for SDL
game engines! event routines.
Many graphics, GUI and Kyra - A simple and
OpenGL libraries. powerful Sprite engine
written in C++.

18
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http
 Free SDL applications (1)

A list is available on http://libsdl.org/applications.php

Very active development.
Lots of games, of course!
A significant number of system emulators.
Example: QEMU (http://qemu.org).
Some multimedia visualization plugins (e.g. for xmms).
Open Picture Space 2
http://membres.lycos.fr/mavati/openpicturespace2/
Shows pictures in a directory through thumbnails.

19
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http
 Free SDL applications (2)

MPEG Menu System

Version 2
http://mms.sunsite.dk/
Menu system for easy
movie, audio and
image playback.
Electronic TV program
guide.
Supports MP3, OGG,
DivX, XviD, VCD,
SVCD, DVD, JPEG,
PNG, and GIF
Controlled using a
20
remote control or
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http
 Getting started with SDL - Hello
World
#include "SDL.h" /* close button clicked */
case SDL_QUIT:
int main( int argc, char* argv[] ) gameover = 1;
{ break;
/* initialize SDL */
SDL_Init(SDL_INIT_VIDEO); /* handle the keyboard */
case SDL_KEYDOWN:
/* set the title bar */ switch (event.key.keysym.sym) {
SDL_WM_SetCaption("SDL Test", "SDL Test"); case SDLK_ESCAPE:
case SDLK_q:
/* create window */ gameover = 1;
SDL_Surface* screen = SDL_SetVideoMode(640, 480, 0, 0); break;
}
/* load bitmap to temp surface */ break;
SDL_Surface* temp = SDL_LoadBMP("hello.bmp"); }
}
/* convert bitmap to display format */
SDL_Surface* bg = SDL_DisplayFormat(temp); /* draw the background */
SDL_BlitSurface(bg, NULL, screen, NULL);
/* free the temp surface */
SDL_FreeSurface(temp); /* update the screen */
SDL_UpdateRect(screen, 0, 0, 0, 0);
SDL_Event event; }
int gameover = 0;
/* free the background surface */
/* message pump */ SDL_FreeSurface(bg);
while (!gameover)
{ /* cleanup SDL */
/* look for an event */ SDL_Quit();
if (SDL_PollEvent(&event)) {
/* an event was found */ return 0;
switch (event.type) { }

Thanks to Tony (http://tonyandpaige.com) for the

example!
21
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http
 Getting started with SDL (2)

Compiling
Easy to do with sdl-config:
gcc `sdl-config --cflags --libs` \ hello.c
-o hello
More examples
Plenty of code examples available on
http://www.libsdl.org/demos.php (check the
licenses).

22
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http
 Useful reading about SDL

2 nice introductions to the different parts of

the API
through simple examples.
SDL Introduction, by Sam Lantinga
http://www.libsdl.org/intro.php
SDL Library Documentation
http://www.libsdl.org/cgi/docwiki.cgi/
3 nice SDL tutorials with simple examples
http://andrew.textux.com/Articles.html

23
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http
 Multimedia in embedded Linux
systems

Multimedia libraries
DirectFB

24
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http
 DirectFB

http://www.directfb.org/
License: LGPL
User space library developed for the special
requirements of embedded devices. Small, powerful,
flexible and easy to use.
Hardware accelerated graphics support. Software
fallbacks for operation not supported by the
underlying hardware.
Input device handling and abstraction.
Also supports SDL (when /dev/fb0 is not available).
Windowing system with support for translucent
windows.
Multiple layer support.
25
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http
 DirectFB screenshot

Demo of the new

TextureTriangles()
method using existing
windows as textures
(updated in real time)

26
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http
 DirectFB in embedded systems

Extra value in embedded systems

No need to run a X server using disk space and CPU.
Lightweight: very few graphics libraries needed.
Your interfaces can be very simple. They don't have
to have the look and feel of desktop applications.
Portability: you can develop your application on the
PC host while the target is still under development.

27
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http
 DirectFB graphical capabilities

Rectangle filling/drawing. Blending with an alpha

factor
Triangle filling/drawing.
(alpha modulation).
Line drawing.
Nine source and
Flat shaded triangles. destination
blend functions.
Simple blitting.
Porter/Duff image
Stretched blitting.
composition rules support.
Textured triangles
Pre-multiplied alpha
(perspective correct).
supported.
Blending with an alpha-
Colorized blitting
channel
(color modulation).
(per pixel alpha).
Source color keying.
Destination color keying.
28
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http
 DirectFB operation

User
DirectFB app. 1 DirectFB app. 2 DirectFB app. 3 space

Disabled when DirectFB library

DirectFB is used Chipset driver
mmap mapping
I/O registers
Kernel
space
Framebuffer driver

Hardware

Timing and mode

Framebuffer Accelerator
registers

29
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http
 DirectFB implementation

Writing to the framebuffer: directly writes to the

memory mapped graphics device video RAM.
Accessing hardware acceleration controls: directly
writes to the memory mapped graphics device I/O
ports.

30
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http
 Input device support

Standard keyboards.
Serial and PS/2 mice.
Joysticks.
Devices using the Linux input layer (many!).
Infrared remote controls (using lirc).
Some touchscreens.

31
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http
 DirectFB image loading

The library includes image providers, which allow to

load images directly into DirectFB surfaces:
JPEG (using libjpeg).
PNG (using libpng2).
GIF (built-in).
Various other image formats (using imlib2).

32
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http
 DirectFB video playback

Includes different video providers,

for rendering in DirectFB surfaces:
V4L (Video4Linux) and V4L2 (/dev/video).
mpeg1/2 (using libmpeg3).
AVI (using avifile).
MOV (using OpenQuicktime).
Flash (using libflash).

33
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http
 DirectFB font rendering

Supports anti-aliased text,

and supports font loading through 2 providers:
DirectFB bitmap font.
TrueType (using FreeType2).

34
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http
 DirectFB API (1)

Native API available in C.

13 interfaces or providers: structures with function
pointers. Example:
Methods of IDirectFBVideoProvider

Retrieving information Get Capabi l i t i es Retrieve information about the video provider's capabilities.

Get Sur f aceDescr i pt i on Get a surface description that best matches the video
contained in the file.

Playback Pl ayTo Play the video rendering it into the specified rectangle of the
destination surface.

St op Stop rendering into the destination surface.

Media Control SeekTo Seeks to a position within the stream.

Get Pos Gets current position within the stream.

Get Lengt h Gets the length of the stream.

Color Adjustment Get Col or Adj ust ment Gets the current video color settings.

Set Col or Adj ust ment Adjusts the video colors.

35
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http
 DirectFB API (2)

More than 120 types: data structures. Example:

DFBTriangle
int x1; X coordinate of first edge
int y1; Y coordinate of first edge
int x2; X coordinate of second edge
int y2; Y coordinate of second edge
int x3; X coordinate of third edge
int y3; Y coordinate of third edge
DFBScreenEncoderTVStandards
DSETV_UNKNOWN Unknown standard
DSETV_PAL PAL
DSETV_NTSC NTSC
DSETV_SECAM SECAM

36
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http
 DirectFB API (3)

9 definitions: macros to simplify code. Examples:

DFB_COLOR_EQUAL (x,y)
Macro to compare two colors.
DFB_DISPLAYLAYER_IDS_MAX
Maximum number of layer ids.
DFB_DISPLAYLAYER_IDS_ADD (ids,id)
Adds the id to the bitmask of layer ids.

Full API documentation available on

http://directfb.org/docs/DirectFB_Reference/

37
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http
 DirectFB API (4)

Only a few global functions

DirectFBInit(argc, argv)
DirectFBCreate(IDirectFB **dfb);

Then, through the IDirectFB interface, it is possible to

work on surfaces, input devices, display layers,
screens, media files, etc.
For example, the dfb->CreateSurface() method will
return a IDirectFBSurface interface pointer
The IdirectFBSurface then contains methods to
manipulate the surface: flipping, blitting, drawing,
etc.

38
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http
 DirectFB Hello World

DirectFBInit(&argc, &argv);
DirectFBCreate(&dfb);

dfb->SetCooperativeLevel(dfb, DFSCL_FULLSCREEN);

memset(&dsc, 0, sizeof(DFBSurfaceDescription));
dsc.flags = DSDESC_CAPS;
dsc.caps = DSCAPS_PRIMARY;
dfb->CreateSurface(dfb, &dsc, &primary);

dfb->CreateImageProvider(dfb, "test.png", &provider);

provider->RenderTo(provider, primary, NULL);
provider->Release(provider);

sleep(5);

primary->Release(primary);
dfb->Release(dfb);

39
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http
 Compiling a DirectFB program

That's very easy to do thanks to pkg-config:

gcc `pkg-config --cflags directfb`
-o test test.c
`pkg-config --libs directfb`
pkg-config is used to query needed C library
headers and shared objects.

40
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http
 Fusion

Makes it possible for multiple DirectFB

applications
to run simultaneously.
Layer used by DirectFB applications to
communicate.
Previously implemented completely in user
space using semaphores and message queues..
Now critical parts of Fusion are implemented in
kernel space (“Fusion Kernel Device”), which
provides better performance and stability.
Supports both Linux 2.4 and 2.6.
Linux kernel patch available in the Core
downloads page 41
(linux-fusion
Free Electrons. Kernel, drivers and module).
embedded Linux development, consulting, training and support. http
 A few DirectFB applications

ByzantineOS, GeexBox Applications with DirectFB

Home entertainment back-ends
DBFSee gdk-directfb
Image viewer and video DirectFB GDK for GTK
player
MPlayer
DFBTerm DirectFB back-end since
Terminal application 2001.
XDirectFB SDL
Rootless X Server using Also has a DirectFB back-
DirectFB windows. Useful end!
to run any X windows
More on http://directfb.org!
app.

42
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http
 Useful reading about DirectFB

DirectFB overview
http://www.directfb.org/docs/DirectFB_overview
_V0.2.pdf

A nice and synthetic article!

43
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http
 Multimedia in embedded Linux
systems

Kernel subsystems
Video4linux

44
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http
 V4L: video4linux subsystem

Provides support for audio/video capture and overlay

devices
and FM radio
V4L: Original video capture/overlay API of the Linux
kernel. Appeared in the 2.1.x development cycle of
the Linux kernel.
V4L2: Second generation of the video4linux API
fixing a number of design bugs. Integrated into the
standard kernel in 2.5.x. Not supported yet by all
drivers.
Main web pages:
Video for Linux resources:
http://www.exploits.org/v4l 45
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http
 v4l original API

API: Documentation/video4linux/API.html
in the kernel sources (even in Linux 2.6).
Device files
/dev/video (minor 0-63): Video Capture
Interface
Of course, can read video input from several of
these devices:
/dev/video0, /dev/video1...
/dev/radio (minor 64-127): AM/FM Radio
Devices
/dev/vtx (minor 192-223): Teletext Interface
Chips
/dev/vbi (minor 224-239): Raw VBI Data 46
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http
 v4l2 new API

Also on Documentation/video4linux/API.html
Linux 2.4 no longer supported by V4L2.
Linux 2.6 ships the latest V4L2 versions
Development CVS snapshots available on
http://dl.bytesex.org/cvs-snapshots/.
Offers backward compatibility with V4L:
true for V4L applications, not for V4L drivers.

47
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http
 V4L2 API specification

New API specification:

http://v4l2spec.bytesex.org/spec/
By Michael H. Schimek and Bill Dirks
A very exhaustive document (Oct 2007: 280
pages!)
Specifies all details about the interface a driver
should implement. Lots of code examples.
Provides details about application programming
too.
A lot of technical background provided too!
Read this like a reference book!

48
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http
 A few V4L2 features

Devices: Uses the same device files, but it's up to

the system administrator to assign minor numbers.
Multiple opens: may support concurrent access to
/dev/ files, for example to control brightness or
color (panel application) while another application is
playing the video.
Standard controls: the API defines hardware
independent device controls (brightness, color
saturation, etc), and a way to enumerate available
controls. Applications can be driver independent.

49
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http
 Communicating with ioctl (1)

A few details about the ioctl() system call are

needed to understand how a video4linux client
application can interact with a video4linux driver.
Prototype:
#include <sys/ioctl.h>
int ioctl(int fd, int request, void *argp);
fd: file descriptor (the opened file)
request: an ioctl number, supported by the driver
argp: argument string or pointer to a value or
structure returned by the driver.

50
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http
 Communicating with ioctl (2)

ioctl is used to interact with the driver

handling the opened device file, without having
to read or write data.
To access a particular setting of the driver
Example: getting the serial number of the
underlying hardware.
To set a particular driver setting
Example: setting the speed and flow control
of a serial port.
To have the driver perform a special
operation
Example: formatting a floppy disk.
51
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http
 A few V4L2 ioctl number examples

VIDIOC_QUERYCAP VIDIOC_ENUMSTD
Querying drivers capabilities Query supported video standards
VIDIOC_ENUMINPUT (PAL, NTSC...)
Querying the number, VIDIOC_QUERYCTRL
types and names of Query available controls
video connectors. VIDIOC_S_FMT
VIDIOC_G_FREQUENCY Setting the image format
Getting the tuner (including colorspace
frequency info)
VIDIOC_S_FREQUENCY VIDIOC_REQBUFS
Setting the tuner Asking the driver to
frequency allocate a buffer for
memory mapped I/O

52
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http
 Concurrent access to /dev/video0

App1 (fd1) App2 (fd2)

read()
write()
ioctl() ioctl()

/dev/video0 Identifying App1

Driver and App2 by
different file
descriptors, the
driver can send
Hardware them different data.

53
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http
 I/O modes

Several I/O modes can be proposed by the driver

read() and write()
Standard way of accessing data. Basic but can still be efficient (can use
DMA internally). But can't carry meta information like frame counters
or timestamps.
Streaming I/O (memory mapping)
Sharing pointers to memory mapped buffers (mmap), allocated by the
driver. No data copying. Buffers can't be swapped out to disk.
Streaming I/O (user pointers)
Sharing pointers to application allocated memory buffers. Buffers can
reside in shared or virtual memory (can be swapped out).

54
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http
 video4linux user tools

V4L
Lots of tools still seem to support only V4L.
Fortunately, the V4L2 high-level driver comes with a
backward compatibility layer which translates V4L ioctl
commands to V4L2 ones
Listed on http://www.exploits.org/v4l
V4L2
MPlayer (plugin)
DirectFB
Supports V4L2 devices as video providers for the display
More listed on
http://www.thedirks.org/v4l2/peopleprojects.htm

55
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http
 xawtv toolset (1)

A set of V4L2 compatible tools for watching TV,

recording video, browsing teletext...
http://linux.bytesex.org/xawtv/
X11 applications Command line / console
xawtv tools
Watching TV
fbtv
motv Framebuffer TV
Motif based TV GUI
application
mtt
streamer
Text or Motif teletext
Recording movies
browser
pia v4lctl
Simple movie player Setting V4L2
parameters

56
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http
 xawtv toolset (2)

Console / command line Fonts

tools http://linux.bytesex.org/xawtv/t
vfonts.html
scantv
Channel scanner led-fixed
Big font used by xawtv
radio for on-screen display.
Simple, curses based
radio application. ets-teletext
Teletext font. Block
webcam graphic characters.
Captures images and
uploads them to a ets-caption
Font for closed captions
Web-Server
(audio subtitles and
alevtd scene description).
http server for
videotext pages 57
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http
 Writing to screens

Video is best when you can watch it!

Need to choose an existing application displaying
video
Or create your own using a given multimedia capable
library:
DirectFB, SDL...
2 ways of accessing the graphics:
X server or framebuffer based
(mainly DirectFB or Qtopia Core)
Anyway, you most probably won't have to access
the display by yourself, but through the low level
library.
Supporting video codecs in a custom application.
Best to rely on gstreamer: 58
http://gstreamer.freedesktop.org/
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http
 Multimedia in embedded Linux
systems

Kernel subsystems
Digital Video Broadcasting

59
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http
 Digital Video Broadcasting

DVB: Digital Video Broadcasting

DVB Project: http://www.dvb.org/
Since 1993, industry-led consortium of broadcasters,
manufacturers, network operators, software
developers, regulatory bodies and others interested
in standards for the delivery of any digital television
and data services.
Committed to open standards
Standard specifications supposed to be available
free of charge (registration required) from ETSI
(European Telecommunication Standards Institute)
at http://www.etsi.org/.
Based on MPEG2 transport streams.

60
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http
 LinuxTV project

http://linuxtv.org/
LinuxTV is a vendor independent, non profit Linux
project that works on a standardized Linux DVB API
since 2000.
Version 3 of the Linux DVB API is included in Linux
2.6.
Very popular on PC systems, mostly in Europe and in
Australia.
Used by lots of Free Software projects and
commercial Set Top Boxes on different hardware
platforms.

61
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http
 Multimedia in embedded Linux
systems

Free Software Video

Video codecs and file formats

62
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http
 RealVideo

From RealNetworks
Free Software decoders
http://realnetworks.com/
available (mplayer)
Proprietary format
Supported by some
Originally used for low mobile phones.
bitrate streaming
Mainly used for
Supports streaming streaming, used by a
through the Real-time significant number of
Transport Protocol (RTP) on-line media.
http://en.wikipedia.org/wiki/Rea Only proprietary
lVideo
encoders. RealNetworks
encoder free of charge
only for personal use.

63
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http
 WMV

Windows Media Video However, more and more on-

line streams available in this
Proprietary video codec: format. Customers are likely to
Microsoft's non-standard expect support for this format
MPEG4 implementation. in a hardware player.
Often comes with WMA for Lack of Free Software players
Audio. (except libavcodec) and
Packed into AVI or encoders.
Advanced Streaming Format Relies on patented algorithms.
(ASF) container.
Licenses may apply to
Supports DRM encryption. encoding, decoding or even
Now many hardware players songs, though MS is still very
available. tolerant so far (to achieve
domination).
Not many files owned by
users
64
(prefer DivX or XviD).
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http
 QuickTime

From Apple Computer Video often encoded with

http://apple.com/quicktim the Sorenson proprietary
e/ codec.
Multimedia technology Few users have QuickTime
supporting several digital videos
video formats, sound, and will ask for QuickTime
text, animation, music, support.
and virtual reality
No Free Software encoders
panoramic images.
Sorenson patent license
Open file format would be needed anyway in
1998: Approved by ISO as some countries.
the basis for the MPEG4
Just 1 Free Software
standard
decoder (FFmpeg's
MPEG4 compatibility libavcodec).
65
since 2002.
More on consulting, training and support. http
Free Electrons. Kernel, drivers and embedded Linux development,
 DivX

From DivXNetworks
http://divxnetworks.com/
DivX 3.11: hacked version DivX 4: MPEG 4 layer 2
of the Microsoft MPEG 4 compliant
layer 2 codec, which made
Free Software players
the codec popular.
available.
Able to shrink a 5-6 GB
Supported by most
DVD video to a file fitting
hardware video players too.
in a CDROM (700 MB) with
minimum quality loss. Still burdened by MPEG 4
patents. In some countries,
DivX 4 and later: clean
license needed to
room implementation from
implement or use a
DivXNetworks, to avoid
software or hardware
issues with Microsoft
player.
copyright. 66
See
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http
 XviD

From XviD.org
http://xvid.org/
Free Software codec, Many hardware players.
MPEG 4 based, competing Many Free Software
with DivX. players.
Started from DivXNetworks Still burdened by MPEG 4
OpenDivX, after access patents. In some
to its sources got closed. countries, license needed
License: GNU GPL. to implement or use a
software or hardware
player.
See
http://en.wikipedia.org/wik
i/XviD
67
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http
 Ogg Theora

From Xiph.org Vorbis or Flac: sound layer

http://theora.org/ Theora library available
Ogg:container for under a BSD-style license.
multimedia streams. Theora already supported
Theora:video codec by various software
competing with MPEG4, players.
RealVideo and WMV. Theora library still in beta
Derived from On2 stage, but getting close to
Technologies VP3 codec. a stable release. Anyway,
On2 irrevocably granted the bitstream format is
to anyone a royalty-free frozen and will be
license to use all its VP3 supported by all future
patents. versions.
Very rare hardware players 68
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http
 Multimedia in embedded Linux
systems

Multimedia distributions

69
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http
 GeexBox

http://geexbox.org/
MPlayer based full featured home multimedia system:
Multimedia (DVD, DivX, CD, Ogg, MP3) and photo gallery.
Available through a live CD: boot your computer
with it, eject the CD, and load whatever media
you want to read.
Size: 6 MB.
Great for learning about a embedded
Linux media player!
Graphics: DirectFB.
Supported platforms: i386 and ppc.
Actively maintained.

70
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http
 MythTV

http://mythtv.org/
Set of Free Software applications to turn a PC into
a digital video recorder / digital multimedia
home entertainment system. Based on the QT library.
Requires TV tuner cards supported
by the video4linux(2) kernel drivers.
Lots of great features: commercial skipping,
access to free TV listings, DVD burning, web
browser, RSS news reader, IP telephony...
Can be tried through the KnoppMyth live CD:
http://en.wikipedia.org/wiki/Mythtv.
See http://en.wikipedia.org/wiki/Mythtv for details.

71
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http
 FreeVo

http://freevo.sourceforge.net/
Another PC based digital multimedia
home entertainment system / personal
video recorder.
Supposed to be easy to install
for new users.
Based on the SDL library.

72
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http