Corner Detection: 7.1 Points of Interest
Corner Detection: 7.1 Points of Interest
Corner Detection
∂I ∂I
Ix (u, v) = (u, v) and Iy (u, v) = (u, v). (7.1)
∂x ∂y
For each image position (u, v), we first calculate the three quantities
that constitute the elements of the local structure matrix M(u, v):1
! 2 " ! "
Ix Ix Iy AC
M= = . (7.5)
Ix Iy Iy2 CB
Next, each of the three scalar fields A(u, v), B(u, v), C(u, v) is indi-
vidually smoothed by convolution with a linear Gaussian filter H G,σ
(see Sec. 5.2.7),
! " ! "
A ∗ HσG C ∗ HσG Ā C̄
M̄ = = . (7.6)
C ∗ HσG B ∗ HσG C̄ B̄
From Eqn. (7.7) we see that the difference between the two eigenval-
ues of the local structure matrix is
% 2
λ1 − λ2 = 2 · 0.25 · trace(M̄) − det(M̄), (7.8)
C = (c1 , c2 , . . . , cN ),
Gradient filter (Alg. 7.1, line 3): Computing the first partial
derivative in the x and y directions with ) *
−0.5
hdx = −0.5 0 0.5 and hdy = hdx = 0 .
0.5
Blur filter (Alg. 7.1, line 10): Smoothing the individual components
of the structure matrix M with separable Gaussian filters
Hb = Hbx ∗ Hby with ⎡1⎤
⎢6⎥
⎢15⎥
1
1 ⎢ ⎥
· ⎢20⎥
hbx = · 1 6 15 20 15 6 1 and hby = hbx = ⎥.
64 64 ⎢⎢15⎥
⎣ ⎦
6
1
the false corners that tend to arise in densely packed groups around
true corners, all except the strongest corner in a specified vicinity
are eliminated. To accomplish this, the list C is traversed from the
front to the back, and the weaker corners toward the end of the list,
which lie in the surrounding neighborhood of a stronger corner, are
deleted.
The complete algorithm for the Harris detector is summarized
again in Alg. 7.1; the associated parameters are listed in Table 7.1.
7.2.4 Examples
Figure 7.1 uses a simple synthetic image to illustrate the most impor-
tant steps in corner detection using the Harris detector. The figure
shows the result of the gradient computation,
A C the three components
of the structure matrix M(u, v) = C B , and the values of the cor-
ner response function Q(u, v) for each image position (u, v). This
example was calculated with the standard settings as given in Table
7.1.
The second example (Fig. 7.2) illustrates the detection of corner
points in a grayscale representation of a natural scene. It demon-
strates how weak corners are eliminated in favor of the strongest
corner in a region.
150
1: HarrisCorners(I, α, tH , dmin ) 7.2 Harris Corner
Input: I, the source image; α, sensitivity parameter (typ. 0.05); Detector
tH , response threshold (typ. 20 000); dmin , minimum distance
between final corners. Returns a sequence of the strongest corners Alg. 7.1
Harris corner detector. This al-
detected in I. gorithm takes an intensity im-
age I and creates a sorted list
Step 1 – calculate the corner response function: of detected corner points. ∗ is
2: Ix ← (I ∗ hpx ) ∗ hdx horizontal prefilter and derivative the convolution operator used
for linear filter operations. De-
3: Iy ← (I ∗ hpy ) ∗ hdy vertical prefilter and derivative tails for the parameters Hp ,
Hdx , Hdy , Hb , α, and tH can
4: (M, N ) ← Size(I) be found in Table 7.1.
5: Create maps A, B, C, Q : M × N → R
6: for all image coordinates (u, v) do C
Compute the local structure matrix M = A
C B :
7: A(u, v) ← (Ix (u, v))2
8: B(u, v) ← (Iy (u, v))2
9: C(u, v) ← Ix (u, v) · Iy (u, v)
Blur the components of the local structure matrix (M̄):
10: Ā ← A ∗ Hb
11: B̄ ← B ∗ Hb
12: C̄ ← C ∗ Hb
13: for all image coordinates (u, v) do calc. corner response:
14: Q(u, v) ← Ā(u, v)· B̄(u, v)− C̄ 2 (u, v)−α·[Ā(u, v)+ B̄(u, v)]2
Step 2 – collect the corner points:
15: C←() start with an empty corner sequence
16: for all image coordinates (u, v) do
17: if Q(u, v) > tH ∧ IsLocalMax(Q, u, v) then
18: c ← u, v, Q(u, v)
create a new corner c
19: C ← C (c) add c to corner sequence C
20: Cclean ← CleanUpCorners(C, dmin )
21: return Cclean
22: IsLocalMax(Q, u, v) determine if Q(u, v) is a local maximum
23: N ← GetNeighbors(Q, u, v) se below
24: return Q(u, v) > max(N ) true or false
25: GetNeighbors(Q, u, v)
Returns the 8 neighboring values around Q(u, v).
26: N ← (Q(u+1, v), Q(u+1, v−1), Q(u, v−1), Q(u − 1, v−1),
Q(u−1, v), Q(u−1, v+1), Q(u, v+1), Q(u+1, v+1))
27: return N
28: CleanUpCorners(C, dmin )
29: Sort(C) sort C by desc. qi (strongest corners first)
30: Cclean ← ( ) empty “clean” corner sequence
31: while C is not empty do
32: c0 ← GetFirst(C) get the strongest corner from C
33: C ← Delete(c0 , C) the 1st element is removed from C
34: Cclean ← Cclean (c0 ) add c0 to Cclean
35: for all cj in C do
36: if Dist(c0 , cj ) < dmin then
37: C ← Delete(cj , C) remove element cj from C
38: return Cclean
151
7 Corner Detection
Fig. 7.1
Harris corner detector—
Example 1. Starting with the
original image I(u, v), the first
derivative is computed, and
then from it the components of
the structure matrix M(u, v),
with A(u, v) = Ix2 (u, v), B =
Iy2 (u, v), C = Ix (u, v) · Iy (u, v).
A(u, v) and B(u, v) represent,
respectively, the strength of
the horizontal and vertical
edges. In C(u, v), the values
are strongly positive (white) or
strongly negative (black) only
where the edges are strong in
I(u, v) A = Ix2 (u, v)
both directions (null values
are shown in gray). The cor-
ner response function, Q(u, v),
exhibits noticeable positive
peaks at the corner positions.
7.3 Implementation
Since the Harris detector algorithm is more complex than the al-
gorithms we presented earlier, in the following sections we explain
its implementation in greater detail. While reading the following
you may wish to refer to the complete source code for the class
HarrisCornerDetector, which is available online as part of the
imagingbook library.4
4
Package imagingbook.pub.corners.
152
7.3 Implementation
Fig. 7.2
Harris corner detector—
Example 2. A complete result
with the final corner points
marked (a). After selecting the
strongest corner points within
a 10-pixel radius, only 335 of
the original 615 candidate cor-
ners remain. Details before
(b, c) and after selection (d, e).
(a)
(b) (c)
(d) (e)
From the original 8-bit image (of type ByteProcessor), we first cre-
ate two copies, Ix and Iy, of type FloatProcessor:
5 FloatProcessor Ix = I.convertToFloatProcessor();
6 FloatProcessor Iy = I.convertToFloatProcessor();
Now the components A(u, v), B(u, v), C(u, v) of the structure matrix
M are calculated for all image positions (u, v):
11 A = ImageMath.sqr(Ix); // A(u, v) ← Ix2 (u, v)
12 B = ImageMath.sqr(Iy); // B(u, v) ← Iy2 (u, v)
13 C = ImageMath.mult(Ix, Iy); // C(u, v) ← Ix (u, v) · Iy (u, v)
14
First (in line 49), a new instance of ArrayList8 is created and as-
signed to the variable C. Then the CRF image Q is traversed, and
when a potential corner point is located, a new Corner is instan-
tiated (line 54) and added to C (line 55). The Boolean method
isLocalMax() (defined in class HarrisCornerDetector) determines
if the 2D function Q is a local maximum at the given position u, v:
61 boolean isLocalMax (FloatProcessor Q, int u, int v) {
62 if (u <= 0 || u >= M - 1 || v <= 0 || v >= N - 1) {
63 return false;
64 }
65 else {
66 float[] q = (float[]) Q.getPixels();
67 int i0 = (v - 1) * M + u;
68 int i1 = v * M + u;
69 int i2 = (v + 1) * M + u;
70 float q0 = q[i1];
71 return // check 8 neighbors of q0:
72 q0 >= q[i0 - 1] && q0 >= q[i0] && q0 >= q[i0 + 1] &&
73 q0 >= q[i1 - 1] && q0 >= q[i1 + 1]
&&
74 q0 >= q[i2 - 1] && q0 >= q[i2] && q0 >= q[i2 + 1] ;
75 }
76 }
Initially (in line 80) the corner list C is sorted by decreasing corner
strenth q by calling the static method sort().9 The sorted sequence
is then converted to an array (line 82) which is traversed from start
to end (line 84–95). For each selected corner (c0), all subsequent
corners (cj) with a distance dmin are deleted from the sequence (line
92). The “surviving” corners are then transferred to the final corner
sequence Cclean.
Note that the call c0.dist2(cj) in line 91 returns the squared
Euclidean distance between the corner points c0 and cj , that is, the
quantity d2 = (x0 −xj )2 + (y0 −yj )2 . Since the square of the distance
suffices for the comparison, we do not need to compute the actual
distance, and consequently we avoid calling the expensive square root
function. This is a common trick when comparing distances.
7.3.4 Summary
7.4 Exercises
Exercise 7.1. Adapt the draw() method in the class Corner (see
p. 155) so that the strength (q-value) of the corner points can also
be visualized. This could be done, for example, by manipulating
158
the size, color, or intensity of the markers drawn in relation to the 7.4 Exercises
strength of the corner.
Exercise 7.3. Explore how rotation and distortion of the image af-
fect the performance of the Harris corner detector. Based on your
experiments, is the operator truly isotropic?
159