CSE 483: Computer Vision

Morphological Image Processing

= On Shapes

Prof. Mahmoud Khalil

Spring 2022
1
Morphological Image Processing

2
a b c
FIGURE 9.1
(a) A set, (b) its
reflection, and
(c)
c) its translat
by z. z2
B̂
B
z1

(B) z

Morphological Image Processing

3
Morphological operators
• Change the shape of the foreground regions via
intersection/union operations between a scanning
structuring element and binary image.
• Useful to clean up result from thresholding.

• Basic operators are:

• Dilation
• Erosion

4
Dilation
• Dilation expands the connected sets of a binary image.
• It can be used for
• growing features

• filling holes and gaps

5
Ahmed Elgammal, Rutgers
Erosion
• Erosion shrinks the connected sets of a binary image.
• It can be used for
• shrinking features

• Removing bridges, branches and small protrusions

6
Kristen Grauman, UT-Austin
Structuring elements
• Masks of varying shapes and sizes used to perform
morphology, for example:

This is 3x3, 4 0s; 5 1s

• Scan mask across foreground pixels to transform the

binary image.
Structuring elements 1, 2, and 4, dilate/erode from vertical
and horizontal sides equally, but no. 3 only applies itself
horizontally only.
The shape of the structuring element is a hyper-parameter
7
Ahmed Elgammal, Rutgers
9.2.2 Dilation

With A and B as sets in Z2, the dilation of A by B, denoted A { B, is defined as

A { B = E z ƒ (B
N ) ¨ A Z F
z (9.2-3)

This equation is based on reflecting B about its origin, and shifting this reflection
by z (see Fig. 9.1). The dilation of A by B then is the set of all displacements,
z, such that BN and A overlap by at least one element. Based on this inter-
pretation, Eq. (9.2-3) can be written equivalently as

A { B = E z ƒ [(B)
N ¨ A] 8 A F
z (9.2-4)
Example for Dilation (1D)

Input image
1 0 0 0 1 1 1 0 1 1

Structuring Element 1 1 1
g ( x)  f ( x)  SE

Output Image 1

8
Adapted from T. Moeslund
Example for Dilation
Input image
1 0 0 0 1 1 1 0 1 1

Structuring Element 1 1 1

Output Image 1 1 0

10
Example for Dilation
Input image
1 0 0 0 1 1 1 0 1 1

Structuring Element 1 1 1

Output Image 1 1 0 1 1 1 1 1 1

Note that the object gets bigger and holes are filled.
>> help imdilate
16
Salamander notice: The grow
directions will be in this diag.
direction (either in top-right
or bottom-left) only!
In general, if the cross-sectional-area
of the entire thing never surpasses '1'
in both directions for at least one row
and column, then it will be restricted
to growing in one direction.
 extreme case:

When solving morphological

analysis questions like this,
must mention the new
dimensions in terms of
whatever given symbols
(like the d in here)
 Reminder: Global solutions seldom
give good practical results. Example
here, horizontal gaps need horizontal
dilation, vertical dilation in them gives
Dilation unnecessary dilation vertically.

Compared with other

methods, this is way faster.
19
9.2.1 Erosion
With A and B as sets in Z 2, the erosion of A by B, denoted A | B, is defined as

A | B = 5z ƒ (B)z 8 A6 (9.2-1)
Or, their intersection = (B)z
In words, this equation indicates that the erosion of A by B is the set of all
points z such that B, translated by z, is contained in A. In the following discus-
sion, set B is assumed to be a structuring element. Equation (9.2-1) is the
mathematical formulation of the example in Fig. 9.3(e), discussed at the end of
the last section. Because the statement that B has to be contained in A is
equivalent to B not sharing any common elements with the background, we
can express erosion in the following equivalent form:

A | B = 5z ƒ (B)z ¨ Ac = 6 (9.2-2)
Example for Erosion (1D)

Input image
1 0 0 0 1 1 1 0 1 1

1 1 1
_
- SE
Structuring Element
g ( x )  f ( x )O

Output Image 0

20
Example for Erosion
Input image
1 0 0 0 1 1 1 0 1 1

Structuring Element 1 1 1

Output Image 0 0 0 0 0 1

25
 If the structuring element is just "1" in the
center, this is trivial, output is identity.

Example for Erosion

Input image
1 0 0 0 1 1 1 0 1 1

Structuring Element 1 1 1

Output Image 0 0 0 0 0 1 0 0 0 1 We assumed a

padding of 1s. In
general, mention
your assumption!

Note that the object gets smaller We can sometimes assume that padding
is of 0s instead, so really, mention it.
>> help imerode 29
extreme case where erosion stops onwards
 Remember: If given a problem with numbers, your answer
must use these numbers and be expressed in numbers too.

Morphological Image Processing

What if I want 9x9 elements only? Doable in multiple steps... First, remove everything less than 9 with the
same technique we did to get the 15x15 ones, then subtract the 15x15s one from the result, done.
 Notice how the circular parts don't
really matter for the dilation step

Problem 9.6

33
Opening
• Erode, then dilate
• Remove small objects, keep original shape

Before opening After opening

34
Kristen Grauman, UT-Austin
Closing
• Dilate, then erode
• Fill holes, but keep original shape

Before closing After closing

35
Kristen Grauman, UT-Austin
 we want to close this:

Open and Close

 Opening and Closing are
not linear, so order matters

Open and Close

37
Hit or Miss

Basically, first you match all what fits the

structuring element's 1s. Next, you want
to match all what fits its 0s, but since
you can't match 0s, you instead get the
complement of the original image and
the structuring element, and now you
can match the 0s. Then you take the
intersection of the two, i.e., the parts
where both the 0s and 1s matched.
39
 Want to use W-X as the complement structure
element to match the 0s of the X for hit or miss.

If we didn't do that border, we'd

have a structuring element of all
0s... that makes no sense, does it?

Hit or Miss

Ac  (W
X)

Ac  (W
X)

40

(A  X)  (Ac  [W
X])
So let's re-visit this. Initially all you knew was opening and closing. Now that you know about
hit-or-miss, don't you think it's more suitable? To be fair, the two methods are very similar.
 If you use a structuring element that does
not have the ones in the diagonals, it may
work out here... but not always trustable.
Boundary Extraction

41
 An alternative method to extracting boundaries
would be to dilate the original image and remove
the image from it. Is it better? Is it worse? Neither.
Depends on where you want your border, if you
Boundary Extraction want it surrounding the image, do dilation process.
If you want it to be the actual boundary part of the
image, then do the erosion process.

42
 The starting point of region fill
must be defined manually.
Then in every step, dilate then subtract (intersect),
repeatedly, until you reach a step that ends up
being no different than the previous one.

Similar to boundary extraction

but the opposite way, the
filter must have its corners in
0s, 1s may lead to filling
getting out of the border. This
actually can happen here
(look at the top row in A)

Region Filling 43
Region Filling
 The algorithm here follows the same
concept used in region filling

Extraction of Connected Components

Let's do our usual test of
checking other S.E.s and
how they work out.

First, how about a 5x5 one

instead of the current 3x3?
Well, that one will lead to
connecting parts that may
actually have 1-pixel gaps.

Okay, what about using 5

1s instead of 9? This will
skip diagonal connections.

45
Extraction of Connected Components

46
 Problem 9.17

9.17 Refer to the image and structuring element shown. Sketch what the sets C, D,
E, and F would look like in the following sequence of operations: C = A | B;
D = C { B; E = D { B; and F = E | B. The initial set A consists of all the
image components shown in white, with the exception of the structuring ele-
ment B. Note that this sequence of operations is simply the opening of A by B,
followed by the closing of that opening by B. You may assume that B is just
large enough to enclose each of the noise components.
47
Target is to get the clean rectangle.
Problem 9.17
Ans

48
 (a) Write a for loop on the boundary of the image, whenever you hit a
white pixel, do the connected component process starting from it.
Note that one of these border particles may be overlapping with
another which itself wouldn't have been considered a border particle.
Let's assume that both are border particles to simplify the problem.
(c) Knowing they're all same-size, apply connected component on
Problem 9.36 everything and remove the particles of the same size (that have a high
number to avoid anomalies). Could instead do hit-or-miss with the size
of the particle. Same idea kinda, but more automatable, so better.
(b) Subtract (a) and (c) from the image hehe.

A preprocessing step in an application

of microscopy is concerned with the
issue of isolating individual round
particles from similar particles that
overlap in groups of two or more
particles. Assuming that all particles are
of the same size, propose a
morphological algorithm that produces
three images consisting respectively of
(a) Only of particles that have merged
with the boundary of the image.
(b) Only overlapping particles. 49

(c) Only nonoverlapping particles.