Image processing

3-2020-? Lecture 2
Image Restoration
Image Restoration
• Image restoration methods are used to improve the appearance of an
image by applying a restoration process that uses a mathematical model
for image degradation.
• The degradation may be due to:
– Blurring caused by motion(Camera shake) or atmospheric
disturbance.
– Geometric distortion caused by imperfect lenses.
– Superimposed interference patterns caused by mechanical systems.
– Noise from electronic sources(Shot noise and quantization).

• It is assumed that the degradation model is known or

can be estimated.
• The idea is to model the degradation process and then
apply the inverse process to restore the original image.
• Let us look at an over-simplified example. Assume that during the
modelling stage, it is determined that the image is corrupted by additive
noise defined as follows:
d(r,c) = I(r,c) + n
d(r,c) : Degraded image
I(r,c) : Original image
n : Constant Noise

• Based on this model, the degraded image could be restored back to

original image by applying the inverse of the degradation process.
– This is done by subtracting n from each of the pixel in d(r,c).
– I(r,c) = d(r,c) - n
System Model
System Model
• The degradation process model consists of two parts:
– The degradation function.
– The noise function.
• The general model in spatial domain:
– d(r,c) = h(r,c) * I(r,c) + n(r,c)
– d(r,c) = degraded image, h(r,c) = degradation function
– I(r,c) = original image ,n(r,c) = additive noise function

• Because convolution in the spatial domain is equivalent to multiplication

in the frequency domain, the frequency domain model is:
– D(u,v) = H(u,v)I(u,v) + N(u,v)
– D(u,v) : Fourier transform of degraded image ,
– H(u,v):Fourier transform of degradation function
– I(u,v) : Fourier transform of the original image
– N(u,v) : Fourier transform of the additive noise function.
Noise
Noise
• At every step in the process, there are fluctuations caused by natural
phenomena that add a random value to the exact brightness value for a
given pixel.
• In typical images, the noise can be modeled using either a Gaussian
(“normal”), uniform, or salt-and-pepper (“impulse”) distribution.
• The shape of the distribution of these noise types can be modeled as a
histogram.

Some noise models are

:graphically depicted as follows
Different Types
Different Types of
of Noise
Noise

here

Gaussian Noise Rayleigh Noise Erlang(Gamma Noise)

Different Types
Different Types of
of Noise
Noise

Exponential Noise Uniform Noise Impulse Noise

 Image with Gaussian noise added
Original Image

Image with uniform noise added Image with salt-and-pepper noise added
Gaussian Noise
Gaussian Noise
•Noise (image) can be classified according the distribution of the values of
pixels (of the noise image) or its (normalized) histogram, It is
characterized by two parameters, (mean) and σ2(variance), by
( g  )2
• Gaussian distribution:  P(g)
HISTOGRAMGaussian  P ( g ) 
1
e 2 2
2 2
– g = gray level
– µ = mean (average)
– σ = standard deviation (σ2 = variance)
It is occurring from electronic noise in image
acquisition system.

About 70% of all the values fall within the range

from one standard deviation(σ) below the mean(μ)
to one above, and about 95% fall within two
standard deviations.
Uniform Noise
Uniform Noise It is
usefu
used
to ge l because
of no n
 1
 if a  g  b ise di erate any it can b
p( g )   b  a stribu other e
– tion.
P; type
 0 otherwise
–The mean and variance of this density are given by
(b  a) 2
  (a  b) / 2 and   2

12

Rayleigh distribution
Rayleigh distribution
( g a )2 P(g)

2 ( g a )  2
HISTOGRAM
Rayleigh
 P(g )   e 0 . 607


mean a 
4

 ( 4 )
var iance  4

Radar range and velocity images


a  g
2
 Impulse Noise
Impulse Noise

P(g)
 Pa for g  a

– p ( g )   Pb for g  b
ep p e r
 0 otherwise
n d- p
Salt-a


– If either Pa or Pb is zero, the

impulse noise is called unipolar
– a and b usually are extreme
values because impulse corruption
is usually large compared with the
strength of the image signal
– It is the only type of noise that g
can be distinguished from others
visually. It is typically caused by
– In the salt-pepper model there malfunctioning pixel elements
are only two possible values(a,b) , in the camera sensors, faulty
and the probability of each is memory locations, or timing
typically less than 0.1. errors in the digitization
– The Value of pepper noise is 0
process .
and 255 for salt noise.
 Negative Exponential
Negative Exponential Noise
Noise P(g)
g

e 
HISTOGRAM  P(g )  
Negative Exponential

2
var iance  
Laser based images

Gamma Distribution(Erlang):
Gamma Distribution(Erlang): g

g

g 1 a
HISTOGRAM  P(g )  e
Gamma (  1 )! a

var iance  a 2
Noise
Noise
• There are various approaches to determine the type of noise that has
corrupted as image.
• Ideally, find an image that contains only noise, and then use its
histogram for the noise model.
– generates a noisy image, try to take picture of a blank wall.

• If we cannot find "noise-only" images or access to the actual system,

we may start from the sample of degraded image.
• A portion of image where we know what to expect in the histogram is
selected (a constant-value image or a well-defined line).
• We can then subtract the known values from the histogram and what
is left is our noise distribution.

• then compare this noise model to the ones available and select the best
match.
• In order to develop a valid model with any of the above approaches,
many sample images need to be evaluated.
 Noise Removal using Spatial Filters
Noise Removal using Spatial Filters
• Spatial filters can be used to remove various types of noise in digital
images.
• These spatial filters typically operate on small neighborhood, between
3x3 to 11x11.
• Use the degradation model defined before, assume h(r,c) causes no
degradation.
• Therefore, corruption on the image is only caused by additive noise,
n(r,c).
- d(r,c) = I(r,c) + n(r,c)
• There are two primary categories of spatial filters for noise removal.
-Order filters: arrange the pixels from smallest to largest and select
the “correct” value.
-Mean filters: calculate the average value.
• The mean filters work best with gaussian or uniform noise.
• The order filters work best with salt-and-pepper, negative exponential,
or Rayleigh noise.
• The mean filters are essentially low pass filters:
– They tend to blur the edges or details.
Spatial Filtering Process
Spatial Filtering Process

Origin x

*
Original Image Filter (w)
Simple 3*3 Pixels
e Filter 3*3
Neighbourhood

eprocessed = o*e +
k*a + l*b + m*c +
y n*d + p*f +
Image f (x, y)
q*g + r*h + s*i
Smoothing Spatial Filtering
Smoothing Spatial Filtering

Origin x
104 100 108 1
/9 1
/9 1
/9

99 106 98
95 90 85
* 1

1
/9

/9
1

1
/9

/9
1

1
/9

/9

/9 1100
1
104 /9 1108
/9
Original Image Filter
Simple 3*3 /9 106
1
99 1
/9 198
/9
Smoothing 3*3 Pixels
Neighbourhood /9 190
1
95 /9 185
/9 Filter
e = 1/9*106 +
1
/9*104 + 1/9*100 + 1/9*108 +
1
/9*99 + 1/9*98 +
y Image f (x, y)
1
/9*95 + 1/9*90 + 1/9*85
= 98.3333
Order Filters
Order Filters
• The order filters are nonlinear filters:
– The results are sometimes unpredictable.
• In general, there is a tradeoff between preservation of image detail
and noise elimination.
• In practical applications, a good approach is to use an adaptive filter (a
filter that can adapt itself to the underlying pixel values).
• Order filters are based on a specific type of image statistics called
order statistics.
– Order statistics is a technique that arranges all the pixels in
sequential order, based on gray-level value.
– The placement of the value within this ordered set is referred to as
the rank.
Order Filters
Order Filters
• Given an NxN window, the pixel values can be ordered from smallest to
largest as follows:
– I1 I2  I3..... IN2
– Where {I1,I2,I3,.....,IN2} are the gray-level values of the subset of
pixels in the image, that are in the NxN window.
• Different types of order filters select different values from the
ordered pixel list.
• Median filter:
– Select the middle pixel value from the ordered set.
– Used to remove salt-and-pepper noise.
• Maximum filter:
– Select the highest pixel value from the ordered set.
– Remove pepper-type noise.
Median Filter: Works
Median Filter: Works

A median filter is good for removing impulse, isolated noise

Salt noise
Pepper noise
Median

Moving
Degraded Window
image
Sorted
Salt noise Array
Pepper noise
Filter output

Normally, impulse noise has high magnitude and is

solated. When we sort pixels in the moving window,
.noise pixels are usually at the ends of the array

.Therefore, it’s rare that the noise pixel will be a median value
Order Filters
Order Filters
• Minimum filter:
– Select the lowest pixel value from the ordered set.
– Remove salt-type noise.
• As the size of the window gets bigger, the more information loss occurs.
– With windows larger than about 5x5, the image acquires an artificial,
“painted”, effect.

Salt noise Probability Mask 3x3 Mask 5x5 Mask 9x9

=.04
Order Filters
Order Filters
pepper noise
Probability = .04

Mask 3x3 Mask 5x5

Mask 9x9

• Order filters can also be defined to select a specific pixel rank within the
ordered set.
– For example, we may find the second highest value is the better choice than
the maximum value for certain pepper noise.
– This type of ordered selection is application specific.
• Minimum filter tend to darken the image and maximum filter tend to brighten
the image.
Order Filters
Order Filters
• Midpoint filter:
– Average of the maximum and minimum within the window.
– Useful for removing gaussian and uniform noise.
I1I 2
= Midpoint 2
N

Gaussian noise Uniform noise Midpoint filter

Midpoint filter
Variance = 300 Variance = 300 Mask size = 3
Mask size = 3 mean = 0
mean = 0
Order Filters
Order Filters
• Alpha-trimmed mean filter:
– The average of the pixel values within the window, but with some
endpoint-ranked values excluded. 2
1 N T
 I
= Alpha-trimmed mean N22T i
T1
i

– T is the number of pixels excluded at each end of the ordered set

• The alpha-trimmed mean filter ranges from a mean to median filter,

depending on the value selected for the T parameter.
– If T = 0,  mean filter.
– If T = (N2 – 1) / 2,  median filter.
• The alpha-trimmed mean filter is useful for images containing multiple
types of noise.
– Example: Gaussian + salt-and-pepper.
Order Filters
Order Filters
Gaussian noise
Variance=200, mean=0
Salt-and-pepper noise
probability = 0.02
Result of alpha-
trimmed
mean filter
Mask size = 3
Trim size = 0

Result of alpha-trimmed mean filter

Mask size = 3
Trim size = 1

Result of alpha-trimmed mean filter

Mask size = 3
Trim size = 4
Mean Filters
Mean Filters
The mean filters function by finding some form of an average within the
NxN window.
The most basic of these filters is the arithmetic mean filter.
-This filter mitigates the noise effect, but at the same time tend to
blur the image.
-The blurring effect is not desirable, and therefore other mean
filters are designed to minimize this loss of detail information.

• Arithmetic mean filter:

– Find the arithmetic average of the pixel values in the window.
1
= Arithmetic Mean 2
 d(r,c)
N (r,c)w

– Smooth out local variations in an image.

– Tend to blur the image.
– Works best with Gaussian and uniform noise.
Mean Filters
Mean Filters
Gaussian noise
Variance=300, mean = 0

Result of arithmetic mean filter

Mask size = 3

Result of arithmetic mean filter

Mask size = 5
Result of arithmetic mean filter
Mask size = 9
Mean Filters
Mean Filters
Image with gamma noise
Variance=300, mean = 0
Result of arithmetic mean filter
Mask size = 3

Result of arithmetic mean filter

Result of arithmetic mean filter Mask size = 9
Mask size = 5
Mean Filters
Mean Filters
• Contra-harmonic mean filter: 
R1
d(r, c)
(r,c)w
= Contra-Harmonic Mean
d(r,
(r,c)w
c)R

– Works for salt OR pepper noise, depending on the filter

order R.
– Negative R  Eliminate salt-type noise.
– Positive R  Eliminate pepper-type noise.
Mean Filters
Mean Filters
Salt noise
Probability =.04

contra-harmonic filter
Mask size=3; order= 0

Result of contra-harmonic filter Result of contra-harmonic filter

Mask size=3; order=-1 Mask size=3; order=-5
Mean Filters
Mean Filters

pepper noise
Probability =.04
Result of contra-harmonic filter
Mask size = 3; order = 0

Result of contra harmonic filter

Mask size = 3; order = +1

Result of contra harmonic filter

Mask size = 3; order = +5
Mean Filters
Mean Filters
• Geometric mean filter:
1
= Geometric Mean  d
(r,c
) w
(r,c) N2

– Works best with Gaussian noise.

– Retains detail better than arithmetic mean filter.
– Ineffective in the presence of pepper noise (if very low values
present in the window, the equation will return a very small number).

Gaussian noise Result of geometric filter

Variance = 300, mean = 0 Mask size = 3
Mean Filters
Mean Filters

Image with pepper noise

Result of geometric filter
Probability = .04
Mask size = 3

Image with salt noise Result of geometric filter

Probability=.04 Mask size = 3
Mean Filters
Mean Filters
• Harmonic mean filter:
N2 Image with pepper noise
= Harmonic Mean 
1 Probability=.04
(r,c)wd(r,c)

– Works with Gaussian noise.

– Retains detail better than arithmetic mean filter.
– Works well with pepper noise.

Result of harmonic filter Image with salt noise Result of harmonic filter
Mask size = 3 Probability=.04 Mask size = 3
Mean Filters
Mean Filters
• Yp mean filter: 1

= Yp Mean   P
d(r,c)P


(r,c)w N
2

– Remove salt noise for negative values of P.
– Remove pepper noise for positive values of P.
Adaptive Filters
Adaptive Filters
• An adaptive filter alters its basic behavior as the image is processed.
– It may act like a mean filter on some parts of the image and a
median filter on other parts of the image.
• The typical character used to determine the filter behavior are the
local image characteristics.
– Measured by local gray-level statistics.
Adaptive Filters
Adaptive Filters
• The minimum mean-square error (MMSE) filter is a good example
of an adaptive filter.
d
(
r,
c)n
d
(
r,
c)
m(
2

r
,c
)
= MMSE 2 l
 l
– σn2 = noise variance.
– σl2 = local variance (in the window).
– ml = local mean (average in the window).

• MMSE filter exhibits varying behavior based on local image statistics:

– No noise  variance = 0  equation returns original image.
– Regions with fairly constant value (no edge/details)  noise
variance == local variance  equation reduces to mean filter.
– Regions with high details (edges)  local variance >> noise variance
 equation returns values close to original image.
Adaptive Filters
Adaptive Filters
• In general, MMSE filter modifies the image based on the noise to local
variance ratio.
– High ratio implies the existence of noise in the window, and
therefore the filter returns primarily the local average to reduce
the noise.
– Low ratio implies high local detail, therefore the filter returns more
of the original unfiltered image to preserve the detail.
• By being able to adapt itself to the local image statistics, the MMSE
filter can preserve the details while at the same time remove the noise.
• MMSE filter works best with Gaussian or uniform noise, and can
perform better compared to the other filters discussed before.
Adaptive Filters
Adaptive Filters
Image with Gaussian noise Result of MMSE
Original Image Variance=300, mean = 0 Mask size = 3

Result of MMSE Result of MMSE Result of MMSE

Mask size = 5 Mask size = 7 Mask size = 9
Correlation & Convolution
Correlation & Convolution
•The filtering we have been talking about so far is referred to as
correlation with the filter itself referred to as the correlation
kernel
•Convolution is a similar operation, with just one subtle difference
(the mask is flipped)
a b c r s t eprocessed = v*e +
z*a + y*b + x*c +
d
f
e
g
e
h
* u
x
v
y
w
z
w*d + u*e +
t*f + s*g + r*h
Original Image Filter
Pixels
•For symmetric filters it makes no difference
•But an important difference is that convolution is associative (if F
and G are filters F*(G*Image)=(F*G)*Image)

k k 3 2 1 9 8 7
g [i , j ]    h [ u ,v ]  f [ i  u , j  v ] 7 8 9
u  k v k 6 5 4 6 5 4 4 5 6
9 8 7 3 2 1 1 2 3
Imnoise
Add noise to Image
Mean
J = imnoise(I,type)
J = imnoise(I,type,parameters)
J = imnoise(I,'gaussian',m,v) variance
J = imnoise(I,'localvar',V)
J = imnoise(I,'localvar',image_intensity,var)
J = imnoise(I,'poisson')J = imnoise(I,'salt & pepper',d)
J = imnoise(I,'speckle',v)
I = imread('eight.tif');
;J = imnoise(I,'salt & pepper',0.02)
figure, imshow(I)
figure, imshow(J)
Imfilter
N-D filtering of multidimensional images
B = imfilter(A,H)
B = imfilter(A, H, option1, option2,...)

B = imfilter(A,H) filters the multidimensional

array A with the multidimensional filter H.
;originalRGB = imread('peppers.png')
imshow(originalRGB)
;h = fspecial('motion', 50, 45)
;filteredRGB = imfilter(originalRGB, h)
figure, imshow(filteredRGB)
The end