How are images represented

in the computer?
Digital Image
Representation

Digital image fundamentals 2

◼ A signal is a function depending on some ◼ A scalar function may be sufficient to describe a

variables with physical meaning. monochromatic (單色的) image, while vector
◼ Signals can be functions are to represent, for example, color
❑ one-dimensional (e.g., dependent on time), images consisting of three component colors.
❑ two-dimensional (e.g., images dependent on two co-
ordinates in a plane),
❑ three-dimensional (e.g., describing an object in space),
❑ or higher-dimensional.

Digital image fundamentals 3 Digital image fundamentals 4

Image formation CCD (Charged-Coupled Device)(電荷耦
合器) cameras
◼ There are two parts to the image formation
process: ◼ Tiny solid state cells convert light energy into
electrical charge.
❑ The geometry of image formation, which determines
where in the image plane the projection of a point in ◼ The image plane acts as a digital memory that can be
the scene will be located. read row by row by a computer.

❑ The physics of light, which determines the brightness

of a point in the image plane as a function of
illumination and surface properties.

Digital image fundamentals 5 Digital image fundamentals 6

◼ The image can be modeled by a continuous

function of two or three variables;
◼ arguments are co-ordinates x, y in a plane, while
if images change in time a third variable t might
be added.
◼ The image function values correspond to the
brightness at image points.
𝑓(𝑥, 𝑦)

Digital image fundamentals 7 Digital image fundamentals 8

◼ The image on the human eye retina or on a TV ◼ The real world which surrounds us is intrinsically
camera sensor is intrinsically (本質上) 2D. 3D.
◼ We shall call such a 2D image bearing ◼ The 2D intensity image is the result of a
information about brightness points an intensity perspective projection (透視投影) of the 3D
image scene.

Digital image fundamentals 9 Digital image fundamentals 10

Dimensionality Reduction Machine (3D to 2D)

◼ When 3D objects are mapped into the camera 3D world 2D image

plane by perspective projection a lot of
information disappears as such a transformation is
not one-to-one.
◼ Recognizing or reconstructing objects in a 3D
scene from one image is an ill-posed problem.
Point of observation

Digital image fundamentals 11 Digital image fundamentals 12

Projection can be tricky… Projection can be tricky…

Digital image fundamentals 13 Digital image fundamentals 14

Projection can be tricky… Projection can be tricky…

Digital image fundamentals 15 Digital image fundamentals 16

Projective Geometry Projective Geometry

What is lost? What is lost?

Who is taller?
◼ Length ◼ Length
Parallel?
◼ Angles

Which is closer? Perpendicular?

Digital image fundamentals 17 Digital image fundamentals 18

Projective Geometry Vanishing points and lines

◼ Parallel lines in the
What is preserved? world intersect in the
◼ Straight lines are image at a “vanishing
still straight point”

Digital image fundamentals 19 Digital image fundamentals 20

Vanishing points and lines Vanishing points and lines
◼ Parallel lines intersect at a point Vertical vanishing
◼ Sets of parallel lines on the same plane form a vanishing line point
◼ Not all lines that intersect are parallel (at infinity)

Vanishing
line

Vanishing
Vanishing Point
o oVanishing Point
Line

Vanishing
Vanishing
point
point

Digital image fundamentals 21 Digital image fundamentals 22

Vanishing points and lines

Digital image fundamentals 23 Digital image fundamentals 24

Projection: world coordinates→image
coordinates
◼ Recovering information lost by perspective
X 
Optical
Center
. P = Y 
 
projection is only one, mainly geometric, problem
of computer vision
(u0, v0)
 Z 

. f
. Z Y

v
Camera
.u
Center
(tx, ty, tz)
u 
p= 
v 

Digital image fundamentals 25 Digital image fundamentals 26

◼ The second problem is how to understand image ❑ object surface reflectance properties (given by the
brightness. surface material, microstructure and marking),
❑ illumination properties,
◼ The only information available in an intensity
image is brightness of the appropriate pixel, ❑ and object surface orientation with respect to a viewer
and light source.
which is dependent on a number of independent
factors such as

Digital image fundamentals 27 Digital image fundamentals 28

◼ Some scientific and technical disciplines work ◼ Many basic and useful methods used in digital
with 2D images directly; for example, image analysis do not depend on whether the
❑ an image of the flat specimen viewed by a microscope object was originally 2D or 3D
with transparent illumination,
❑ a character drawn on a sheet of paper,
❑ the image of a fingerprint, etc.

Digital image fundamentals 29 Digital image fundamentals 30

◼ Image processing often deals with static images, ◼ Computerized image processing uses digital
in which time t is constant. image functions which are usually represented by
◼ A monochromatic （單色）static image is matrices, so co-ordinates are integer numbers.
represented by a continuous image function f(x,y) ◼ The range of image function values is also limited;
whose arguments are two co-ordinates in the by convention, in monochromatic images the
plane lowest value corresponds to black and the highest
to white.
◼ Brightness values bounded by these limits are
gray levels.

Digital image fundamentals 31 Digital image fundamentals 32

Image digitization

◼ Sampling means measuring the value of an ◼ An image captured by a sensor is expressed as a

image at a finite number of points. continuous function f(x,y) of two co-ordinates in
◼ Quantization is the representation of the the plane.
measured value at the sampled point by an ◼ Image digitization means that the function f(x,y)
integer. is sampled into a matrix with M rows and N
columns.

Digital image fundamentals 33 Digital image fundamentals 34

◼ The image quantization assigns to each ◼ The finer the sampling (i.e., the larger M and N)
continuous sample an integer value. and quantitation (the larger K) the better the
◼ The continuous range of the image function f(x,y) approximation of the continuous image function
is split into K intervals. f(x,y).

◼ 128*128
◼ 256*256
◼ 1024*1024

Digital image fundamentals 35 Digital image fundamentals 36

Sampling Image digitization (cont’d)

Digital image fundamentals 37 Digital image fundamentals 38

Image quantization (example)

256 gray levels (8bits/pixel) 32 gray levels (5bits/pixel) 16 gray levels (4bits/pixel) 8 gray levels (3bits/pixel) 4 gray levels (2bits/pixel) 2 gray levels (1bits/pixel)
𝑓(𝑥, 𝑦)=256 𝑓(𝑥, 𝑦)=32 𝑓(𝑥, 𝑦)=16

Digital image fundamentals 39 Digital image fundamentals 40

Image sampling (example)
original image (x=y=256) sampled by a factor of 2 (x=y=128)

◼ N: # of rows, M: # of columns, Q: # of gray

levels
❑ N = 2 n , M = 2 m , Q = 2 q (q is the # of
bits/pixel)
sampled by a factor of 4(x=y=64) sampled by a factor of 8(x=y=32) ❑ Storage requirements: N*M*Q (e.g., N=M=1024, q=8,
1MB)

Digital image fundamentals 41 Digital image fundamentals 42

Image coordinate system

f (0,0) f (0,1) ... f (0, M − 1)

f (1,0) f (1,1) ... f (1, M − 1)
... ... ... ...
f ( N − 1,0) f ( N − 1,1) ... f ( N − 1, M − 1)

Digital image fundamentals 43 Digital image fundamentals 44

Image file formats

◼ Many image formats adhere to the simple model

shown below (line by line, no breaks between
lines).
◼ The header contains at least the width and height
of the image.
◼ Most headers begin with a signature or “magic
number” - a short sequence of bytes for
identifying the file format.

Digital image fundamentals 45 Digital image fundamentals 46

Common image file formats ◼ TIFF (Tagged Image File Format): it is used on
all popular platforms and is often the format used
by scanners.
◼ GIF (Graphic Interchange Format): has been used
❑ Supports multiple images with 1 to 24 bits of color per
to encode a huge number of images in the internet
pixel.
and databases.
❑ Options are available for either lossy or lossless
❑ Relatively easy to work with compression
❑ Cannot be used for high-precision （高精度）color,
since only 8-bits are used to encode color
◼ Normally 256 color values available
◼ 16-color is another option
❑ Non-lossy compression is available

Digital image fundamentals 47 Digital image fundamentals 48

◼ JPEG (Joint Photographic Experts Group): is a ◼ MPEG (Motion Picture Experts Group): a
more recent standard from the Joint stream-oriented encoding scheme for video,
Photographic Experts Group, the major purpose audio, text, and graphics.
was to provide for practical compression of high- ❑ MPEG-1: suitable for multimedia for popular personal
computers, but are too low for high-quality TV.
quality color still images.
❑ MPEG-2: handle high definition TV rates.
❑ Allows for real-time hardware for coding and decoding
❑ High compression: a flexible but complex lossy coding
scheme is used which often can compression a high
quality image 20:1 without noticeable degradation.
❑ The compression works well when the image has large
regions of nearly constant color.
❑ Is not designed for video.

Digital image fundamentals 49 Digital image fundamentals 50