DATA SCIENCE PROGRAM

DS617

Image Processing

Lecture 2 – Part I
 Digital Image Fundamentals

(1,1) x

(X-1,Y-1) (X,Y-1) (X+1,Y-1)

y
(X-1,Y) (X,Y) (X+1,Y)

(X-1,Y+1) (X,Y+1) (X+1,Y+1)

Conventional Indexing Method

 Some Basic Relationships Between Pixels
 Neighborhood relation is useful for analyzing regions.

4-neighborhood relation
considers only the vertical
(X,Y-1)
and horizontal neighbors

(X-1,Y) p (X+1,Y) 4-neighbors of pixel p at (X,Y):

(X-1,Y)
(X+1,Y)
N4(p) =
(X,Y-1)
(X,Y+1) (X,Y+1)

Note: q N4(p) implies p N4(q)

 Some Basic Relationships Between Pixels-2
 Diagonal-neighborhood relation considers only diagonal neighbors.

(X-1,Y-1) (X+1,Y-1)
D-neighbors of pixel p at (X,Y):
(X-1,Y-1)
(X+1,Y-1)
p ND(p) =
(X-1,Y+1)
(X+1,Y+1)

(X-1,Y+1) (X+1,Y+1)

Note: q ND(p) implies p ND(q)

 Some Basic Relationships Between Pixels-3
 8-neighborhood relation considers all the neighboring pixels.

8-neighbors of pixel p at (X,Y):

(X-1,Y-1) (X,Y-1) (X+1,Y-1) (X-1,Y-1)
(X,Y-1)
(X+1,Y-1)
(X-1,Y)
(X-1,Y) p (X+1,Y) N8(p) =
(X+1,Y)
(X-1,Y+1)
(X,Y+1)
(X-1,Y+1) (X,Y+1) (X+1,Y+1) (X+1,Y+1)

Note: q N8(p) implies p N8(q)

 Some Basic Relationships Between Pixels-4
 Adjacency is adapted from neighborhood relation.

 Two pixels are adjacent if they have the same intensity or color
and they are neighbors of one another.

 Let V be the set of intensity values used to define adjacency (i.e., in a

binary image, V ={1}), so pixels p and q with values from V have:

4-adjacency: p and q are 4-adjacent if q N4(p).

8-adjacency: p and q are 8-adjacent if q  N8(p).

m-adjacency (mixed-adjacency): p and q are m-adjacent if:

q  N4(p) or
q ND(p) and N4(p) ∩N4(q) =  (has no pixels whose values from V).
Some Basic Relationships Between Pixels-5
 Some Basic Relationships Between Pixels-6
 m-adjacency is a modification of 8-adjacency, so it is introduced to
eliminate the ambiguities that often arise when 8-adjacency is used.
8-path m-path
p p p

q q q
8-adjacency from p to q m-adjacency from p
results in some ambiguity to q solves ambiguity

 Two image subsets S1 and S2 are

adjacent if some pixel in S1 is
adjacent to some pixel in S2 .
S1 S2
Some Basic Relationships Between Pixels-7
 Some Basic Relationships Between Pixels-8
 A digital path (curve) of length n from pixel p at coordinates (x,y) to
pixel q at coordinates (s,t) is a sequence of distinct pixels at
coordinates: (x0,y0), (x1,y1), (x2,y2),…, (xn,yn)
such that: (x0,y0) = (x,y) and (xn,yn) = (s,t)
and (xi,yi) is adjacent to (xi-1,yi-1), i = 1,…, n-1

 If (x0,y0) = (xn,yn), then path is a closed path.

q
p

 We can define type of path: 4-path, 8-path or m-path depending on

type of adjacency specified.
 Some Basic Relationships Between Pixels-9
 Example: According to V = {2,3,5}, find p and q pixels 4‐, 8‐ and
m‐path?
 Some Basic Relationships Between Pixels-10
 Connectivity is based on adjacency and is very useful for establishing
object boundary and defining image region.

 Let S represent a subset of pixels in an image, two pixels p and q in S

are said to be connected if there exists a path between them
consisting entirely of pixels in S.
 Some Basic Relationships Between Pixels-11
 For any pixel p in S, the set of pixels that are connected to p in S is
called a connected component.

 If there is only one connected component in S, then S is called a

connected set.
 Some Basic Relationships Between Pixels-12
 Let R be a subset of pixels in an image, R is called a region of the
image if R is a connected set.

 The boundary (also called border or contour) of a region R is the

set of pixels in the region that have one or more neighbors that are
not in R.

 If R happens to be an entire image, then its boundary is defined as

the set of pixels in the first and last rows and columns of the
image.

 There is a key difference between boundary and edge concepts,

where boundary of a finite region forms a closed path and is thus a
“global” while edge is a “local” concept that is based on a measure
of intensity-level discontinuity at a point.
 Some Basic Relationships Between Pixels-13
 Boundary Detection Vs Edge Detection
 Each coin in the image below has only one boundary surrounds it
but multiple edges inside it.
 Some Basic Relationships Between Pixels-14
 For pixel p, q, and z with coordinates (x,y), (s,t) and (u,v) respectively,
D is a distance function or metric if:
1. D(p,q) ≥ 0 D(p,q) = 0 if and only if p = q
2. D(p,q) = D(q,p)
3. D(p,z) ≤ D(p,q) + D(q,z)

 Euclidean distance (De) between p and q is defined as:

De ( p, q)  ( x  s)2  ( y  t )2

 The pixels having an Euclidean

distance from (x,y) less than or
equal to some value r form a disk
of radius r centered at (x,y).
 Some Basic Relationships Between Pixels-15
 D4-distance (city-block distance) between p and q is defined as:

D4 ( p, q)  x  s  y  t

2
2 1 2
2 1 0 1 2
2 1 2
2
The pixels with D4 (p) = 1 are 4-neighbors of p

 The pixels having a D4-distance from (x,y) less than or equal to some
value r form a diamond centered at (x,y).
 Some Basic Relationships Between Pixels-16
 D8-distance (chessboard distance) between p and q is defined as:

D8 ( p, q)  max( x  s , y  t )

2 2 2 2 2
2 1 1 1 2
2 1 0 1 2
2 1 1 1 2
2 2 2 2 2
The pixels with D8 (p) = 1 are 8-neighbors of p

 The pixels having a D8-distance from (x,y) less than or equal to some
value r form a square centered at (x,y).
 Some Basic Relationships Between Pixels-17
 Dm-distance between p and q is defined as the shortest m-path
between them.

 This distance will depend on the values of the pixels along the shortest
m-path, as well as the values of their neighbors, other than previously
defined three distances (De , D4 and D8) which depends only on the
coordinates of the pixels.

 Example: Suppose that V={1}, find the Dm-distance between p and p4

 Image Mathematics
 Although images can be viewed equivalently as matrices, but most of
the arithmetic operations performed on images are array-based.

 An array operation involving one or more images is carried out on a

pixel-by-pixel basis, so it is for this reason that a clear distinction
must be made between array and matrix operation.

 Array product of two 2x2 images A and B:

 Matrix product of two 2x2 images A and B:

 Linear Vs Non-Linear Operations
 Let H be a general operator producing an output image, g(m,n), for a
given image, f(m,n): H[ f(m,n) ] = g(m,n)

 H is said to be a linear operator if:

H[ ai fi(m,n) + aj fj (m,n) ] = ai H[ fi(m,n) ] + aj H[ fj(m,n) ]
= ai gi(m,n) + aj gj(m,n)

Where ai and aj, are arbitrary constants, and fi(x,y) and fj(x,y) are
arbitrary images of the same size.

 The above equations indicate that the output of a linear operation

due to the sum of two inputs is the same as performing the operation
on the inputs individually and then summing the results (additivity
property). In addition, the output of a linear operation to a constant
times an input is the same as the output of the operation due to the
original input multiplied by that constant (homogeneity property).
 Linear Vs Non-Linear Operations-2
 For example, the max operator whose function is to find the
maximum value of the pixels in an image, is nonlinear.

 The simplest way to prove that max operator is nonlinear, is to find an

example that fails the test in the previous equation.

 Consider the following two images:

If suppose that a1=1 and a2 = -1, then the equation right-hand side:

and the equation left-hand side is:

 Linear Vs Non-Linear Operations-3
 Linear operations are exceptionally important because they are
based on a large body of theoretical and practical results that are
applicable to image processing.

 Nonlinear operators are not nearly as well understood, so their

scope of application is more limited. However, we will encounter
several nonlinear image processing operations whose performance
far exceeds what is achievable by their linear counterparts
 Image Enhancement
 The objective of enhancement is to process an image so that the
result is more suitable than the original image for a specific
application.

 Enhancement methods are very much problem oriented, for

example, a method that is quite useful for enhancing X-ray images
may not necessarily be the best approach for enhancing pictures of
Mars transmitted by a space probe.

 Also the visual evaluation of enhanced image quality is a highly

subjective process because the viewer is the ultimate judge of how
well a specific method works.
 Image Enhancement-2
 Enhancement methods fall into two broad categories:
 Spatial domain methods: are based on direct manipulation of the
image pixels.

 Frequency domain methods: are based on modifying the Fourier

transform of the image.

 Enhancement methods based on various combinations of methods

from these two categories are not usual.
 Spatial Image Enhancement Methods
 Enhancement methods in spatial domain can be reduced to the form:
g(x,y) = T[ f(x,y) ]
f(x,y) is the input image, g(x,y) is the output enhanced image and T is an
enhancement operator or function defined over a point, some local
neighbourhood mask centered around a point, or the image globally.

 Spatial enhancement methods are divided Into:

(a) Point (Single-Pixel) Methods: The neighbourhood is simply
of size 1x1 (the pixel itself).

(b) Mask (Local) Methods: The neighbourhood is any rectangular group

of pixels of any size centred around the pixel of interest.

(c) Geometric (Global) Methods: The neighbourhood is simply the

whole image.
Spatial Image Enhancement Methods-2

g(x,y)

g(x,y)
 Point (Single-Pixel) Methods
 T is pixel-wise enhancement operator or function.

f(x,y) g(x,y)

 Point (Single-Pixel) methods are divided into:

 Arithmetic Operations
 Set Operations
 Logical Operations
 Intensity Transformation Methods (Mathematical & Arbitrary)
 Histogram Based Methods
 Contrast Stretching
 Lookup Table
 Equalization
 Adaptive Equalization
 Matching (Specification)
 Arithmetic Operations
 They are performed between the corresponding pixels of two images
IA , IB of the same size, MxN, or between an image IA and a constant
value C:
Ioutput= IA + IB
Ioutput= IA + C
Ioutput= IA - IB
Ioutput= IA - C
Ioutput= IA * IB
Ioutput= IA * C
Ioutput= IA ÷ IB
Ioutput= IA ÷ C

 Image Addition
The value of an individual pixel location (m,n) in the output
enhancement image is mapped as follows:
Ioutput(m,n) = IA(m,n) + IB(m,n)
Ioutput(m,n) = IA(m,n) + C
 Arithmetic Operations-2
 Adding a value to each image pixel can be used to achieve the
contrast adjustment, because adding a positive constant, C, to each
pixel value increases (lightens) the pixel value.
 Arithmetic Operations-3
 Adding images together produces a composite image of both input
images. This can be used to produce blending effects using weighted
addition.
 Arithmetic Operations-4
 A clear example of using image addition operation is in calculating
the average image of a scene for noise reduction as enhancement
technique in the astronomy field.

 The corrupted noisy image must be formed by the addition of noise,

where the assumption is that at every pair of coordinates (m,n), the
noise is uncorrelated and has zero average value.

 Imaging under very low light levels frequently causes sensor noise to
render single images virtually useless for analysis.

 So by observing the same scene over long periods of time and

averaging K different noisy images of it, the noise is reduced as K
increases to some limit.

 In practice, the averaged noisy images must be aligned.

Arithmetic Operations-5
 Arithmetic Operations-6
 Image Subtraction: Subtracting a value from each image pixel can
also be used to achieve the contrast adjustment, because subtracting
a positive constant, C, from each pixel value decreases the pixel value
(darken) and, hence, its brightness.
 Arithmetic Operations-7
 Image Subtraction: Subtracting one image from another shows the
difference between images.

 If subtracting two images in a video sequence then getting a

difference image, assuming a static camera, which shows the
movement or changes that have occurred in the scene video
sequence between the video sequence frames.
 Arithmetic Operations-8
 A clear example of using image subtraction operation is in the
medical imaging for enhancement of differences between images,
where the mask, h(x,y), is an X-ray image of a patient’s anatomical
region captured by an intensified TV camera (instead of traditional
X-ray film) located opposite an X-ray source.

 The procedure consists of injecting an X-ray contrast medium into

the patient’s bloodstream, taking a series of images called live
images, f(x,y), of the same anatomical region as the mask, h(x,y),
and then subtracting the mask from the series of incoming live
images after injection of the contrast medium, g(x,y) = f(x,y) – h(x,y).

 The net effect of subtracting the mask, h(x,y), from each sample live
image, f(x,y), is that the areas that are different appear in the
output image, g(x,y), as enhanced detail.
Arithmetic Operations-9
 Arithmetic Operations-10
 Image Multiplication: Multiplying each image pixel by a value can
also be used to achieve the contrast adjustment, because multiplying
a positive constant, C, by each pixel value increases the pixel value
(e.g., increasing contrast by 50% = multiplication by 1.5).
 Arithmetic Operations-11
 A clear example of using image multiplication operation is in
masking, also called region of interest (ROI) extraction.

 A given input image is multiplied by a mask image that has 1s only in

the region(s) that correspond to the ROI(s) and 0s elsewhere.
 Arithmetic Operations-12
 Image Division: Dividing each image pixel by a value can also be used
to achieve the contrast adjustment, because dividing each pixel value
by a positive constant, C, decreases the pixel value (e.g., reducing
contrast to 25% = division by 4).
 Arithmetic Operations-13

 A clear example of using image division operation is in image shading

correction. Suppose that an imaging sensor produces images, g(x,y),
that can be modeled as the product of a “perfect image”, denoted by
f(x,y), times a shading function, h(x,y), that is: g(x, y) = f(x, y) . h(x, y).

 If h(x,y) is known, we can obtain f(x,y) by dividing g(x,y) by h(x,y).

 If h(x,y) is unknown, but access to the imaging sensor is possible, we

can obtain an approximation to the shading function by imaging the
target of constant intensity.
Arithmetic Operations-14
 Arithmetic Operations-15
 Multiplication and division can be seen as extension to addition and
subtraction respectively.

 Division can be used for image differencing, as dividing an image by

another gives a result of 1 where the image pixel values are identical
and a value not equal to 1 where differences occur.

 However, image differencing using subtraction is computationally

more efficient

 Multiplying different images together or dividing them by one

another is not a common operation in image processing.

 For three-channel RGB images, the image arithmetic operation is

generally performed separately for each color channel.
 Arithmetic Operations-16
 Combining two or more image arithmetic operations for better
contrast enhancement of the image (e.g., division followed by
addition for better edges visibility)
Arithmetic Operations-17
Arithmetic Operations-18
 Arithmetic Operations-19
 Most images are displayed using 8 bits (even 24-bit color images
consist of three separate 8-bit channels). Thus, we expect image
values to be in the range from 0 to 255.

 When images are saved in a standard format, such as TIFF or JPEG,

conversion to this range is automatic depending on the system used.

 But after arithmetic operations, the difference of two 8-bit images

can range from -255 to +255, and the values of a sum image can
range from 0 to 510.

 Given an image, f, an approach that guarantees that the full range of

an arithmetic operation between images is “captured” into a fixed
number of bits is as follows:
 Arithmetic Operations-20
(1) Calculate fm = f – min(f)
which creates an image whose minimum value is 0.

(2) Calculate fs = K [ fm / max(fm) ]

which creates a scaled image whose values are in the range [0, K].
for 8-bit image, setting K=255 gives scaled image whose intensities
span from 0 to 255

 This approach can be used for all arithmetic operations, except when
performing division, an extra requirement that is a very small
number, ϵ (e.g., ϵ = 0.001), should be added to pixels of the divisor
image to avoid division by 0.
 Set Operations
 The sets are regions representing objects and backgrounds in images.

 The main set operations are: Union, Intersection, Complement and

Difference.
 Set Operations-2
 The Union and intersection operations on two images are defined as
the max and min of corresponding pixel pairs, respectively:
IAᴜB(m,n) = max( IA (m,n) , IB (m,n) )
IA∩B(m,n) = min( IA (m,n) , IB (m,n) )

 The complement operation is defined on an image as the pairwise

differences between the maximum possible value in the given image
representation and the value of every pixel in it, which produces the
photographic negative of it:
Ic(m,n) = MAX – I(m,n)
Where the MAX is 1 for binary image and 255 for intensity image.

 The difference operation on two images is defined as the pixels that

belong to the minuend image and, at the same time, they do not
belong to the subtrahend image:
IA-B(m,n) = IA (m,n) ∩ IBc (m,n)
 Set Operations-3
 Image Complement: Suppose that we have an intensity image and
we want to form the negative of that image, so if the elements of
that image are represented by a set A, then its negative image of the
same size can be obtained by applying the set complement operation
Set Operations-4
 Set Operations-5
 Interesting special effects can be obtained by complementing only
part of the image (solarization). For example by taking the
complement of pixels with gray values of 128 or less, and leaving
other pixels untouched. Or by taking the complement of pixels with
gray values of 128 or greater, and leave other pixels untouched.
Set Operations-6
 Logical Operations
 They are: OR, AND, NOT and XOR.

 It is common to map the OR, AND, NOT and XOR to the set union,
intersection, complement and difference operations respectively.

 When dealing with binary images, these operations are applied on

images’ regions which clearly can be irregular and of different sizes.

 This is as opposed to dealing with intensity images, which are array

operations and thus require images whose spatial dimensions are
the same. So, intensity logical operations involve complete images.

 Any other logic operation can be implemented by using only the

three basic functions: AND, OR and NOT. For example, the XOR
operation is implemented using AND and NOT.

 Logical operations are used extensively in image morphology.

Logical Operations-2
Logical Operations-3