Filtering Techniques Applied To Streaming Video and Control Applications
Filtering Techniques Applied To Streaming Video and Control Applications
Abstract
A new efficient filters for real time image processing were presented in this paper.
The new approaches are based on vector median filter and digital paths concepts
[1,19,20]. Presented techniques utilize noise detection to alleviate filtering artifacts
and decrease numeric complexity [17].
The new optimized algorithms can be applied to real time control application,
such as navigation of mobile robots, and as filters for digital video enhancement.
The performance of the proposed filters is compared to the performance of com-
monly used filters, such as the vector median, under a variety of performance crite-
ria. It is shown that the proposed filters are better able to suppress impulsive and
Gaussian noise than the existing techniques. Also, they are robust to inaccuracies
in parameter settings.
1 Introduction
In this work general fuzzy filtering structure proposed in [10,8,7] will be used.
The general form of the fuzzy adaptive filters proposed in this work is defined
The relationship between the pixel under consideration (window center) and
each pixel in the window should be reflected in the decision how to define the
filter weights. In our case weights will be calculated using similarity functions
calculated over digital paths included in the processing window W .
and symmetric
2
In digital image processing three basic distance functions are usually applied.
If p = (p1 , p2 ) and q = (q1 , q2 ) denote two image points (p, q ∈ Z 2 ) then we
define the
• city-block distance
• chessboard distance
• Euclidean distance
h i1
ρE (p, q) = (p1 − q1 )2 + (p2 − q2 )2 2
(7)
Using the city-block and chessboard distances we are able to define the two
basic types of neighborhoods, 4-neighborhood N4 and 8-neighborhood N8
Using the distances between neighboring points, which are called prime dis-
tances [18], we are able to define a distance between any two image points by
following all admissible paths linking those points and then taking the mini-
mum over the total length of all possible paths which is the sum of the prime
distances between the nodes of the paths. In this way the distance between
two image points is the length of the path for which the sum of the prime
distances between the path nodes is minimal.
For the city-block distance the admissible paths consist of horizontal and ver-
tical moves only, whereas for the chessboard distance also the diagonal moves
are allowed. The prime distances for the two kinds of neighborhood are de-
clared to be one.
3
The distance transformation is often used to convert a binary image consisting
of objects (foreground) and background into a new image in which each pixel
has a value corresponding to the minimum topological distance from this pixel
to the background.
The minimal Euclidean distance between an image pixel and the background
is resource demanding to compute and therefore different kinds of approxima-
tions are used. As a result the approaches to achieve distance transform do not
adopt directly the definition of the minimum distance on Z 2 from the object
pixel to the background border pixels, since their computations are extremely
time consuming. Instead the distances are computed by calculating the prime
distances and propagating the minimal distances from the background to the
object’s point. This approach is clearly based on the concept of paths and as
a consequence the minimal distance from the pixel to the background is the
length of a path linking the pixel and the background which has the minimal
length. In the same way the distance between two image points is the length of
a path linking that points which is of minimal length. Of course the minimal
path and its length depends on the adopted metric and neighborhood system.
An increasing polygonal line P on the path Π is any polygonal line such that
P = {Π(λi )}ni=0 , a = λ0 < . . . < λn = b. The length of the polygonal line
P is considered to be the total sum of its constitutive line segments L(P ) =
Pn
i=1 ρ(Π(λi−1 ), Π(λi )) where ρ(x, y) is the distance between the points x and
y, when a specific metric is adopted. A path Π from x to y is called rectifiable,
if and only if L(P ), where P is an increasing polygonal line, is bounded. Its
upper bound is called the length of the path Π (Fig. 1b).
The geodesic distance ρS (x, y) between points x and y is the lower bound of
4
a) b)
Fig. 1. a) Continuous path Π leading from the x to y, and b) increasing polygonal
line on the path Π.
the length of all paths leading from x to y which are totally included in S. If
such paths do not exist, then the value of the geodesic distance is set to ∞.
In general ρS (x, y) ≥ ρ(x, y). However, if the set S is convex, meaning that
there are no points on the line between x and y that are not members of S,
the geodesic distance verifies ρS (x, y) = ρ(x, y).
The notion of the path can be extended to a lattice, which is a set of discrete
points in the plane, in our case the spatial locations of the image pixels. Let
a digital lattice H = (F, N ) be defined by F, which is the set of all points of
the plane (pixels of a color image) and a neighborhood relation N between
the lattice points [13]. In the case of the ranked-type non-linear filters the
processing window W forms a lattice where N is defined through the window
size.
Let us adopt the following notation, which will help us define the distance
functions defined over digital paths.
The starting point of a path will be denoted as p0 = (x0 , y0 ). Its neighbors will
be denoted as p1 = (xu1 , yv1 ), which means that the neighbors are the second
points of all digital paths originating at p0 . Then the third point of a digital
5
path starting at p0 will be p2 = (xu2 , yv2 ) and so on, till the path reaches in n
steps the ending point pn = (xun , yvn ).
The set of all possible digital paths contained in W joining two points x, y ∈ W
will be denoted as ΦW (x, y).
The connection cost over the digital path ΛW,n will be defined as a measure of
dissimilarity between color image pixels p0 , p1 , . . ., pn forming a specific path
linking p0 and pn [21,4].
If a path joining two distinct points x, y, such that F(x) = F(y) consists of
lattice points of the same values, then the connection cost should be zero,
otherwise ΛW,n > 0.
For two given points pi = (xui , yvi ) and pi−1 = (xui−1 , yvi−1 ), i = 1, 2, . . . , n,
which are in neighborhood relation, let the generalized distance between the
two points will be called connection cost
6
q -q q q q q q q q q q q
q @q q q q q q q q q q -
@q
I
@ I
@
q q q q q q q q q q q q
a)
q -q q q q q q q q q q q q q q q q q
q q q q q -q q q q q q @q q q @q q q q
6 6 I
@ I
@
q q q q q q q q q q q @
Rq q q? q q @q? q
6 @ 6 I
@
b)
Fig. 2. Geodesic paths of length: a) length of 2; b) length of 3, connecting two
neighboring points within a predefined window W of size 3 × 3, when the 8-neigh-
borhood system is applied.
n h i
ΛW,n {p0 , p1 , . . . , pn } = kF(p1 ) − F(pi−1 )k + ρW (pi , pi−1 )
X
(11)
i=1
Similarly to the gray-scale case, we will call the minimal connection cost
ΓW,n (x, y) of a path of length n linking two points x, y ∈ W , the n-geodesic
between x and y:
n o
ΓW,n (x, y) = min Λ (γ) , γ ∈ ΦW,n (12)
In this way the n-geodesic is defined as the path of length n, which gives
the minimal connection cost between two points linked by a digital path. If
we take the minimum of the connection cost generated by all possible paths
joining two points x and y ∈ W , then we get the generalized multichannel
geodesic distance between these points
n o n o
ΓW (x, y) = min ΓW,n (x, y) = min Λ (p) , p ∈ P W,n (x, y) , n ∈ N . (13)
n∈N
In general, two distinct pixel’s locations on the image lattice could be con-
nected by many paths. Moreover the number of possible geodesic paths of
certain length n connecting two distinct points depends on their locations,
length of the path and the neighborhood system used (Figs. 2 and 3).
7
q q q q q q q q q q q q q q q
q q q qs y q q q -q -qs y q q q q qs y q
6 6 6
q q -q -q q q q q q q q q q q q
6 6 6
q x sq q q q q x sq q q q q x sq -q -q q
q q q q q q q q q q q q q q q
q q q q q q q q q q q q q q q
q q q -qs y q q q q
qs y q q q q -qs y q
6 6 6
q q q q q q q q -q q q q -q q q
6 6 6
q x sq -q q q q x sq -q q q q x sq q q q
q q q q q q q q q q q q q q q
Fig. 3. There are six paths of length 4 connecting point x and y when the 4-neigh-
borhood system is used (W of size 3 × 3).
The aim of taking into account the points p2 , . . ., pn when calculating the
similarity between p0 and p1 is to explore not only the direct neighborhood of
p0 but also to use the information on the local image structure.
This can be done by acquiring the information on the local image features
investigating the connection costs of digital paths originating at p0 , passing p1
and then visiting successive points, till the path reaches length n. In this case
the similarity function takes the form:
µW,n (x, y) = µW,n (p0 , p1 ) = g ΛW,n {p0 , p1 , p∗2 , . . . , p∗n }
X
(14)
P {p0 ,p1 ,p∗2 ,...p∗n }
where P {p0 , p1 , p∗2 , . . . , p∗n } denotes the set of all paths originating at x = p0
crossing y = p1 and ending in pn which are totally included in W , ΛW,n {·}
is a dissimilarity value along a specific path and g(·) is a smooth function of
ΛW,n .
8
(1) g is a decreasing in (0; ∞],
(2) g is convex in (0; ∞],
(3) g (0) = 1,
(4) g (Λ) = 0 , when Λ → ∞ .
Several functions satisfying the above conditions have been proposed in the
literature [11,15,9,16]:
1
g2 (x) = , β2 ∈ (0; ∞), (16)
1 + β2 x
1
g3 (x) = , β3 ∈ (0; ∞), (17)
(1 + x)β3
2
g4 (x) = 1 − arctan(β4 x) , β4 ∈ (0; ∞), (18)
π
2
g5 (x) = , β5 ∈ (0; ∞), (19)
1 + eβ5 x
1
g6 (x) = , β6 ∈ (0; 1), (20)
1 + xβ 6
1 − β7 x if x < 1/β7 ,
g7 (x) = , β7 ∈ (0; ∞). (21)
0 if x ≥ 1/β7 ,
h i
µW,n (x, y) = µW,n (p0 , p1 ) = exp −β · ΛW,n {p0 , p1 , p∗2 , . . . , p∗n }
X
and then if F(x) = F(y), ΛW,n (x, y) = 0, µ(x, y) = 1, and for ||F(x)−F(y)|| →
∞ then µ → 0 [8].
9
p2 d21 ps 1 2 p1
F1 F2 F3 s s s s d1 - s
6P1 d11 6P2
s d11 s s s s s
F8 F0 F4
p0 p0
F7 F6 F5 s s s s s s
Fig. 4. Digital paths of length n = 2 connecting points x and y, where d11 , d21 , d12
and d22 are connection costs: d11 = kF0 − F2 k, d21 = kF2 − F3 k, d12 = kF0 − F4 k and
d22 = kF4 − F3 k.
Figure 5 illustrates the calculation of the similarity function between two
points connected by two paths of length n = 2. In this case
ΛW,2 1 2
1 (x, y) = d1 + d1 , ΛW,2 1 2
2 (x, y) = d2 + d2 , (24)
with d11 , d21 distances between neighboring points on the path P1 defined ac-
cording to (11), while d12 , d22 are similarly defined on P2 . The total similarity
value can be expressed as follows:
h i
µW,2 = exp −β · ΛW,2
1 + exp −β · ΛW,2
2 (25)
h i
exp −β · ΛW,n {p0 , p1 , p2 , . . . , pn }
P
P {p2 ,p3 ,...,pn }
ψ W,n (x, y) = ψ W,n (p0 , p1 ) =
exp [−β · ΛW,n {p0 , p∗1 , p∗2 , . . . , p∗n }]
P
P {p0 ,p∗1 ,p∗2 ,...,p∗n }
(26)
Assuming that the pixel x = p0 is the pixel under consideration, with F(y)
representing the pixel y = p1 the filter output F̂(x) is given as follows:
10
h i
exp −β · ΛW,n {p0 , p1 , p2 , . . . , pn }
P
X P {p0 ,p1 ,p2 ...,pn }
F̂ (p0 ) = · F (p1 ) (28)
exp [−β · ΛW,n {p0 , p∗1 , p∗2 , . . . , p∗n }]
P
p1
P {p0 ,p∗1 ,p∗2 ,...,p∗n }
¿¿¿¿¿¿¿¿¿¿¿¿¿¿¿¿¿¿¿¿¿¿¿¿¿¿¿¿¿¿¿¿¿¿¿¿¿¿¿¿¿¿¿¿¿¿¿¿¿¿¿¿¿¿¿¿¿¿¿¿¿¿¿¿¿¿¿¿¿¿¿¿
ω n o
µW,n (x, y) = µW,n (p0 , pn ) = g (ΛW,n
X
m (x, y) , (29)
m=1
where ω is the number of all paths connecting x and y, ΛW,n m (x, y) is a dissimi-
larity value along a specific path m from the set of all ω possible paths leading
from x to y and g(·) is a smooth function of ΛW,n m . By definition µ
W,n
(x, y)
returns a value evaluated over all possible routes linking the starting point x
to the endpoint y.
Several functions satisfying the above conditions have been proposed in the
literature [11,15,9,16]:
11
p2 d21 pr 1 2 p1
F1 F2 F3 r r r r d1 - r
1
6 P1 d1 6P2
1
r d1 r r r r r
F8 F0 F4
p0 p0
F7 F6 F5 r r r r r r
Fig. 5. Digital paths of length n = 2 connecting points x and y, where d11 , d21 , d12
and d22 are connection costs: d11 = kF0 − F2 k, d21 = kF2 − F3 k, d12 = kF0 − F4 k and
d22 = kF4 − F3 k.
ω h i
µW,n (x, y) = exp −β · ΛW,n
X
m (x, y) , (37)
m=1
and then if F(x) = F(y), ΛW,n (x, y) = 0, µ(x, y) = 1, and for ||F(x)−F(y)|| →
∞ then µ → 0 [8].
ΛW,2 1 2
1 (x, y) = d1 + d1 , ΛW,2 1 2
2 (x, y) = d2 + d2 , (39)
with d11 , d21 distances between neighboring points on the path P1 defined ac-
cording to (11), while d12 , d22 are similarly defined on P2 . The total similarity
value can be expressed as follows:
h i
µW,2 = exp −β · ΛW,2
1 + exp −β · ΛW,2
2 (40)
µW,n (x, y)
ψ W,n (x, y) = P W,n , (41)
µ (x, y)
y↔x
12
clearly seen from (41) that the normalized similarity function satisfies following
property:
ψ W,n (x, y) = 1 .
X
(42)
y↔x
Assuming that the pixel x is the pixel under consideration, with F(y) repre-
senting the pixel y included in the supporting element W which is connected
to F(x) via a digital path, the filter output F̂(x) is given as follows:
4 Filters Optimization
References
[1] J. Astola, P. Haavisto, and Y. Neuovo. Vector median filters. In IEEE Proc.,
volume 78, pages 678–689, 1990.
13
[7] K.N. Plataniotis, D. Androutsos, and A.N. Venetsanopoulos. Colour image
processing using fuzzy vector directional filters. In Proc. of the IEEE Workshop
on Nonlinear Signal/Image Processing, Greece, pages 535–538, 1995.
[8] K.N. Plataniotis, D. Androutsos, and A.N. Venetsanopoulos. Fuzzy adaptive
filters for multichannel image processing. Signal Processing Journal, 55(1):93–
106, 1996.
[9] K.N. Plataniotis, D. Androutsos, and A.N. Venetsanopoulos. Adaptive
fuzzy systems for multichannel signal processing. Proceedings of the IEEE,
87(9):1601–1622, 1999.
[10] K.N.
Plataniotis, D. Androutsos, S. Vinayagamoorthy, and A.N. Venetsanopoulos.
Color image processing using adaptive multichannel filters. IEEE Trans. on
Image Processing, 6(7):933–950, 1997.
[11] K.N. Plataniotis and A.N. Venetsanopoulos. Color Image Processing and
Applications. Springer Verlag, August 2000.
[12] F. Preteux and N. Merlet. New concepts in mathematical morphology: the
topographical distance function. In P. D. Gader and E. R. Dogherty, editors,
Proceedings of SPIE, volume 1568, pages 66–77, 1991.
[13] M. Schmitt. Lecture notes on geodesy and morphological measurements.
In Proceedings of the Summer School on Morphological Image and Signal
Processing, Zakopane, Poland, pages 36–91, 1995.
[14] F. Y. Shih and J. J. Liu. Size-invariant four-scan euclidean distance
transformation. Pattern Recognition, 31(11):1761–1766, 1998.
[15] B. Smolka, A. Chydzinski, K. Plataniotis, and A.N. Venetsanopoulos.
New filtering technique for the impulsive noise reduction in color images.
Mathematical Problems in Engineering, submitted for publication, 2002.
[16] B. Smolka, A. Chydzinski, K. Wojciechowski, K. Plataniotis, and A.N.
Venetsanopoulos. On the reduction of impulsive noise in multichannel image
processing. Optical Engineering, 40(6):902–908, 2001.
[17] B. Smolka, K.N. Plataniotis, A. Chydzinski, and M. Szczepanski. Self-adaptive
algorithm of impulsive noise reduction in color images. Pattern Recognition,
35(8):1771–1784, August 2002.
[18] V. Starovoitov. Towards a distance transform generalization. In G. Borgefors,
editor, Proceedings of the 9th Scandinavian Conference on Image Analysis,
Uppsala, pages 499–506, 1995.
[19] M. Szczepanski, B. Smolka, K. N. Plataniotis, and A. N. Venetsanopoulos.
On the geodesic paths approach to color image filtering. Signal Processing,
83(6):1309–1342, June 2003.
[20] M. Szczepański. Random Walk Theory Applied to Low Level Color Image
Processing. PhD thesis, department of Automatic, Electronics and Computer
Science, Silesian University of Technology, Gliwice, Poland, 2003.
14
[21] P.J. Toivanen. New geodesic distance transforms for gray scale images. Pattern
Recognition Letters, 17:437–450, 1996.
15