See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/277648222

Object Tracking using Adaptive Template Matching

Article · February 2015

DOI: 10.5573/IEIESPC.2015.4.1.001

CITATIONS READS

8 741

4 authors, including:

Wisarut Chantara Ji-Hun Mun

Gwangju Institute of Science and Technology Gwangju Institute of Science and Technology
7 PUBLICATIONS   10 CITATIONS    9 PUBLICATIONS   10 CITATIONS   

SEE PROFILE SEE PROFILE

Yo-Sung Ho
Gwangju Institute of Science and Technology
476 PUBLICATIONS   3,898 CITATIONS   

SEE PROFILE

Some of the authors of this publication are also working on these related projects:

Asia-Pacific Signal and Information Processing Association, Annual Summit and Conference (APSIPA ASC) 2016 View project

3-dimensional TV View project

All content following this page was uploaded by Yo-Sung Ho on 15 January 2016.

The user has requested enhancement of the downloaded file.

IEIE Transactions on Smart Processing and Computing, vol. 4, no. 1, February 2015
http://dx.doi.org/10.5573/IEIESPC.2015.4.1.001 1

IEIE Transactions on Smart Processing and Computing

Object Tracking using Adaptive Template Matching

Wisarut Chantara, Ji-Hun Mun, Dong-Won Shin, and Yo-Sung Ho

Gwangju Institute of Science and Technology (GIST), 123 Cheomdan-gwagiro, Buk-gu, Gwangju 500-712, Korea
{wisarut, jhm, dongwonshin, hoyo}@gist.ac.kr

* Corresponding Author: Yo-Sung Ho

Received April 5, 2014; Revised June 13, 2014; Accepted November 19, 2014; Published February 28, 2015

* Regular Paper

Abstract: Template matching is used for many applications in image processing. One of the most
researched topics is object tracking. Normalized Cross Correlation (NCC) is the basic statistical
approach to match images. NCC is used for template matching or pattern recognition. A template
can be considered from a reference image, and an image from a scene can be considered as a source
image. The objective is to establish the correspondence between the reference and source images.
The matching gives a measure of the degree of similarity between the image and the template. A
problem with NCC is its high computational cost and occasional mismatching. To deal with this
problem, this paper presents an algorithm based on the Sum of Squared Difference (SSD) and an
adaptive template matching to enhance the quality of the template matching in object tracking. The
SSD provides low computational cost, while the adaptive template matching increases the accuracy
matching. The experimental results showed that the proposed algorithm is quite efficient for image
matching. The effectiveness of this method is demonstrated by several situations in the results
section.

Keywords: Template matching, Object tracking, Normalized cross correlation, Sum of squared difference,
Pattern recognition

1. Introduction
Nowadays, video surveillance systems are being
installed worldwide in many different sites, such as
airports, hospitals, banks, railway stations, and even at
homes. Surveillance cameras help a supervisor to oversee
many different areas in a single room and to quickly focus
on abnormal events taking place in the controlled space.
Intelligent video monitoring expresses a fairly large
research direction that is applied in different fields. On the
other hand, several question arise, such as how can the
intensity of the objects tracking with occlusion be Fig. 1. Shading real target by other objects.
improved. When an occlusion occurs, some objects are
partially or totally invisible. As shown in Fig. 1, this
phenomenon makes it difficult to localize the real object tracking consists of an estimation of the trajectory of
target accurately. moving objects in the sequence of frames generated from a
video [1].
Most tracking algorithms assume that the object motion
2. Object Tracking is smooth with no abrupt changes to simplify tracking [2].
For example, the traditional color histogram Mean Shift
Object tracking in real-time environment is a different (MS) algorithm only considers the object’s color statistical
task in different computer vision applications. Object information, and does not contain the object’s space
2 Chantara et al.: Object Tracking using Adaptive Template Matching

information, so when the object color is close to the

background color, or the object’s illumination is changed,
the traditional MS algorithm easily causes inaccurate
object tracking or it can be lost. Therefore, Mao et al. [3]
proposed an object tracking approach that integrated two
methods consisting of histogram-based template matching
method, and the mean shift procedure was used to estimate
the object location.
Tracking methods also are related to a traffic
surveillance systems. Choi et al. [4] proposed a vehicle (a) (b)
tracking scheme using template matching based on both
the scene and vehicle characteristics, including background
information, local position and size of a moving vehicle.
Alternative object tracking can be found in Refs. [1, 5, 6].
On the other hand, these basic tracking algorithms have
weak intensity when the other object occludes the real
source image. Automation of the computer object tracking
is a difficult task. Therefore, to solve these kind of
problems, this paper proposes the ‘template matching’
object tracking method. Before explaining the algorithm,
some brief concepts of ‘template matching’ are introduced. (c)

Fig. 2. (a) Template image, (b) Source image, (c)

3. Template Matching Window sliding and matching.

Template matching is the process of finding the (x, y)

location of a sub image, called a template, inside an image.

A number of methods for identical images can be used. (u, v) (u, v)
This section discusses the template matching application
for matching a small image, which is a part of a large
image. Once a number of corresponding templates are
(Static) (Moving)
found, their centers can be used as the corresponding
control points to determine the matching parameters.
Template matching involves comparing a given template
with windows of the same size in an image and identifying Template Image T Scene Image I

the window that is most similar to the template.

The basic template matching algorithm consists of
calculating at each position of the image under Fig. 3. Matching procedure.
examination a distortion function that measures the degree
of similarity between the template and image. The
minimum distortion or maximum correlation position is involved in the numerator of NCC. Wei and Lai [13]
then taken to locate the template into the examined image. proposed a fast pattern matching algorithm based on NCC
Typical distortion measures are the Sum of Absolute criterion by combining adaptive multilevel partition with
Differences (SAD) [7, 8] and the Sum of Squared the winner update scheme to achieve a very efficient
Differences (SSD) [9]. On the other hand, as far as search. Alternative similarity measures can be found in
template matching is concerned, the Normalized Cross Refs. [14-17].
Correlation (NCC) [10] is often adopted for similarity In Fig. 2, to identify a matching area, the method needs
measure. Essannoun et al. [11] proposed a fast frequency to compare a template image against a source image by
algorithm to speed up the process of SAD matching. They sliding it. By sliding the template, the process can measure
used an approach to approximate the SAD metric by a the similarity between the template image and a region in
cosine series, which could be expressed in correlation the source image. At each location, a metric is calculated
terms. Furthermore, Hel-Or and Hel-Or [12] proposed a so it represents how similar the patch is to that particular
fast template matching method based on accumulating the area of the scene image. The process then finds the
distortion on the Walsh-Hadamard domain in the order of maximum or the minimum location in the resulting image
the associated frequency using SSD. depending on the measurement.
The traditional NCC needs to compute the numerator
and denominator, which is a very time-consuming. Lewis 3.1 Matching Methods
[7] employed the sum table scheme to reduce the
computation in the denominator. Although the sum table Fig. 3 shows the matching procedure. Here, u and v
scheme could reduce the computation of the denominator represent a horizontal and vertical position in the kernel,
in NCC, it was essential to simplify the computation respectively, and x and y correspond to a position of the
IEIE Transactions on Smart Processing and Computing, vol. 4, no. 1, February 2015 3

kernel. The procedure can calculate a result pixel using matched within the source image, I(x,y), of size p×q,
various matching methods. Eqs. (1)-(4) show the matching where (p>m and q>n). For each pixel location (x,y) in the
methods. image, the SSD distance is calculated as follows:

1. Sum of Squared Difference (SSD) R ( x, y ) = ∑ (T (u, v) − I ( x + u, y + v) )

2
(5)
R ( x, y ) = ∑ (T (u, v) − I ( x + u, y + v) )
2
(1) u ,v
u ,v

The smaller the distance measured SSD at a particular

2. Normalized Sum of Squared Difference (NSSD) location, the more similar the local sub-image found is the
∑ (T (u, v) − I ( x + u, y + v) ) searched template. If the distance SSD is zero, the local
2

sub-image is identical to the template. The minimum

R ( x, y ) = u ,v
(2)
∑ T (u, v) ⋅ ∑ I ( x + u, y + v)
2 2 distance provides the location of the corresponding object
u ,v u ,v
(CL) in the source image. Other small distances and
locations (SLs), which are less than the threshold value,
can also be found. These values will be proposed in a next
3. Cross Correlation (CC)
subsection.
R ( x, y ) = ∑ (T (u , v) ⋅ I ( x + u, y + v) ) (3)
u ,v
4.2 The best object location
4. Normalized Cross Correlation (NCC) This subsection proposes a method for locating the
∑ (T (u, v) ⋅ I ( x + u, y + v) ) position of an interesting object in an image (OL). The aim
R ( x, y ) = u ,v
(4) of this process was to identify an appropriate coordinate
∑ T (u, v)2 ⋅ ∑ I ( x + u, y + v)2 that finds in the source image.
u ,v u ,v Assumption: The previous location of object (PL) is
stored in a memory buffer.
For SSD or NSSD, the process needs to find the Condition 1:
minimum value in the resulting image. Otherwise, if using If the CL is the nearest of PL then,
the CC or NCC, the process needs to find the maximum Set OL = CL, and assign Location flag (FL) = 1
value in the result.
This paper introduces object tracking using an adaptive Condition 2:
template matching technique. In this technique, SSD was If the SLs are the nearest of PL then,
used to measure the similarity between a template image Set OL = the nearest of SLs, and assign the FL = 0
and source image. The finest object location was selected
in the source image. Finally, the template image, which is Condition 3:
an adapted template image, was updated. If the CL doesn’t match in Condition 1 and 2 then,
Set OL = PL, and assign FL = 0
Fig. 4 shows a flowchart of a method for finding the
4. Proposed Methods best object location following the above-mentioned.

The objective of the proposed method is to reduce a 4.3 Adaptive Template Matching
computational cost and increase accurate matching. As a In this subsection, the FL parameter from section 4.2 is
similarity measure, the SSD is the most popular and considered to determine an appropriate template image.
widely used for several applications. NCC is more robust
against illumination changes than SSD; nevertheless, NCC Location flag = 0, it means that
is more time-consuming than SSD. ·The OL is the SLs or PL.
The following three key steps are involved in
·Assign the previous template with the current tem-
implementing of the proposed method.
plate.
·Detection of interesting object,
·Update the current template with the current object
·Tracking of object from frame to frame,
template.
·Updating of suitable template. Location flag = 1, it means that
·The OL is the CL from the SSD method.
4.1 Template Matching ·Assign the previous template with the current tem-
The SSD is a simple algorithm for measuring the plate
similarity between the template image (T) and sub-images ·Update the current template with the original tem-
in the source image (I). It works by taking the squared plate.
difference between each pixel in T and the corresponding The result image
pixel in the sub-images used for the comparison in I. These ·Green box is the PL position in the source image.
squared differences are summed to create a simple metric ·Red box is the OL position in the source image.
of similarity. Assume a 2-D m×n template, T(x,y) is The overall actions are described more detail in Fig. 5.
4 Chantara et al.: Object Tracking using Adaptive Template Matching

(a) (b)

Fig. 6. (a) Source image, (b) Template image.

Condition 1:

Fig. 4. Algorithm to find the best object location.

(a)

Fig. 5. Algorithm to update template.

(b) (c)
5. Experiment Results
Fig. 7. Condition 1 (a) SSD result, (b) Object location,
This section shows the advantage of the proposed (c) Update the current template with the original
method to full track an interested object. Experiments were template.
performed to examine the matching accuracy. A template
image of 27x67 pixels (Fig. 4(b)) was used to match in a section 4.2. An illumination of (a), (b), and (c) in each
source image the sequences of size of 352x288 pixels, as figure shows the result of the SSD method, the position of
shown in Fig. 4(a). involved object and the updated template respectively.
Figs. 7-9 illustrate each condition that is explained in To illustrate the performance of the proposed algorithm,
IEIE Transactions on Smart Processing and Computing, vol. 4, no. 1, February 2015 5

Condition 2:

(a) (b) (c)

Fig. 8. Condition 2 (a) SSD result with the SLs, (b) Object location shows the PL in a green box and the current
location in a red box, (c) Update the current template with the current object template.

Condition 3:

(a) (b) (c)

Fig. 9. Condition 3 (a) SSD result with the SLs, (b) Object location shows the current location with the PL in a green
box, (c) Update the current template with the current object template.

Fig. 10. Object’s positions on consecutive frame sequence.

the template matching algorithm like the NSSD and NCC image frames. The proposed method outperforms the two
were compared. The result is shown in Fig. 11. Fig. 10 conventional methods.
shows x and y object positions curves on a consecutive In addition, other experiment samples consisting of
6 Chantara et al.: Object Tracking using Adaptive Template Matching

F.1

F.10

F.35

F.64

F.177

F.250
(a) (b) (c) (d)

Fig. 11. Experiment result (a) NSSD, (b) NCC, (c) Ground truth [18], (d) Proposed method, the results are captured at
1st, 10th, 35th, 64th, 177th, and 250th frame respectively.
IEIE Transactions on Smart Processing and Computing, vol. 4, no. 1, February 2015 7

(a) (b)

Fig. 12. Result related to condition 3 (a) Template image, (b) Object tracking.

(a) (b)

Fig. 13. Result related to condition 2 (a) Template image, (b) Object tracking.

(a) (b)

Fig. 14. Result related to condition 1 (a) Template image, (b) Object tracking.

three source images and their templates with the proposed different sizes and different illuminations. Figs. 12-14
algorithm were implemented. These samples contain show the outcomes.
8 Chantara et al.: Object Tracking using Adaptive Template Matching

6. Conclusion [9] J. Shi and C.Tomisto, “Good feature to track,”

Proceedings of IEEE Computer Society Conference
This paper proposed an object tracking using an on Computer Vision Pattern Recognition, pp. 593-
adaptive template matching algorithm. The adaptive 600, Jun. 1994. Article (CrossRef Link)
template matching provides the proper object location in a [10] W. K. Pratt, “Correlation techniques of image
source image. The SSD uses a small number of operations registration,” IEEE Trans. On Aerospace and
for a similarity purpose. Therefore, the proposed method Electronic Systems, vol. AES-10, pp. 353-358, May
reduces the computational cost and increases the accuracy. 1974. Article (CrossRef Link)
Based on the experiment results, the proposed method is [11] F. Essannouni, R. Oulad Haj Thami, D. Aboutajdine,
more efficient than the conventional methods, such as and A. Salam, “Adjustable SAD matching algorithm
NSSD and NCC. Furthermore, a comparison of the using frequency domain” Journal of Real-Time
identification object in the source image confirmed that the Image Processing, vol. 1, no. 4, pp. 257-265, Jul.
proposed method outperforms the conventional methods. 2007. Article (CrossRef Link)
[12] Y. Hel-Or and H. Hel-Or, “Real-time pattern
matching using projection kernels,” IEEE Trans.
PAMI, vol. 27, no. 9, pp. 1430-1445, Sep. 2002.
Acknowledgement Article (CrossRef Link)
[13] S. Wei and S. Lai, “Fast template matching based on
This work was supported by the National Research normalized cross correlation with adaptive multilevel
Foundation of Korea (NRF) grant funded by the Korean winner update” IEEE Trans. Image processing, vol.
government (MSIP) (No. 2013-067321) 17, No. 11, pp. 2227-2235, Nov. 2008. Article
(CrossRef Link)
References [14] N.P. Papanikolopoulos, “Selection of Features and
Evaluation of Visual Measurements During Robotic
[1] N. Prabhakar, V. Vaithiyananthan, A.P. Sharma, A. Visual Servoing Tasks,” Journal of Intelligent and
Singh, and P. Singhal, “Object Tracking Using Frame Robotic System, vol.13 (3), pp. 279-304, Jul. 1995.
Differencing and Template Matching,” Research Article (CrossRef Link)
Journal of Applied Sciences, Engineering and [15] P. Anandan, “A computational framework and an
Technology, pp. 5497-5501, Dec. 2012. Article algorithm for the measurement of visual motion,”
(CrossRef Link) International Journal of Computer Vision, vol. 2 (3),
[2] A.Yilmaz, O.Javed, and M.Shah, “Object Tracking: A pp. 283-310, Jan. 1989. Article (CrossRef Link)
Survey,” ACM Computing Surveys, vol. 38(4), Article [16] A. Singh and P. Allen, “Image flow computation: an
13, pp.1–45, Dec. 2006. Article (CrossRef Link) estimation-theoretic framework and a unified
[3] D.Mao, Y.Y. Cao, J.H. Xu, and K. Li, “Object perspective,” Computer Vision Graphics and Image
tracking integrating template matching and mean Processing: Image Understanding, vol. 56 (2), pp.
shift algorithm,” Multimedia Technology (ICMT), 152-177, Sep. 1992. Article (CrossRef Link)
2011 International Conference on, pp. 3583-3586, [17] G. Hager and P. Belhumeur, “Real-time tracking of
Jul. 2011. Article (CrossRef Link) image regions with changes in geometry and
[4] J.H. Choi, K.H. Lee, K.C. Cha, J.S. Kwon, D.W. Kim, illumination,” Proceedings of IEEE Computer
and H.K. Song, “Vehicle Tracking using Template Society Conference on Computer Vision Pattern
Matching based on Feature Points,” Information Recognition, pp. 403-410, Jun. 1996. Article
Reuse and Integration, 2006 IEEE International (CrossRef Link)
Conference on, pp. 573-577, Sep. 2006. Article [18] Y. Wu, J. Lim, and M.H. Yang, “Online Object
(CrossRef Link) Tracking: A Benchmark,” Computer Vision and
[5] S. Sahani, G. Adhikari, and B. Das, “A fast template Pattern Recognition (CVPR), IEEE Conference on,
matching algorithm for aerial object tracking,” Image pp. 2411-2418, Jun. 2013. Article (CrossRef Link)
Information Processing (ICIIP), 2011 International
Conference on, pp. 1–6, Nov. 2011. Article
(CrossRef Link) Wisarut Chantara received the B.S.
[6] H.T. Nguyen, M. Worring, and R.V.D. Boomgaad, and M.S. degrees in Computer Engi-
“Occlusion robust adaptive template tracking,” neering from Prince of Songkla
Proceedings. Eighth IEEE International conference University (PSU), HatYai, Thailand in
on Computer Vision (ICCV), pp. 678-683, 2001. 2004 and 2008, respectively. From
Article (CrossRef Link) 2008 to 2013, he worked as a lecturer
[7] J. P. Lewis, “Fast template matching,” Vis. Inf., pp. at Department of Computer Engi-
120-123, 1995. Article (CrossRef Link) neering, PSU, Phuket Campus, Phuket,
[8] F. Alsaade and Y.M. Fouda, “Template Matching Thailand. He is currently a Ph.D. student in the School of
based on SAD and Pyramid,” International Journal Information and Communications at Gwangju Institute of
of Computer Science and Information Security Science and Technology (GIST), Korea. His research
(IJCSIS), vol. 10 no.4, pp. 11-16, Apr. 2012. Article interests are object tracking, computer vision, light field
(CrossRef Link) camera, digital signal, and image processing.
IEIE Transactions on Smart Processing and Computing, vol. 4, no. 1, February 2015 9

Ji-Hun Mun received his B.S. degree

in Electric and Electrical Engineering
from Chonbuk National University,
Korea, in 2013. He is currently a M.S.
student in the School of Information
and Communications at Gwangju
Institute of Science and Technology
(GIST), Korea. His research interests
include digital signal and image processing.

Dong-Won Shin received his B.S.

degree in Computer Engineering from
Kumoh National Institute of Tech-
nology, Korea, in 2013. He is currently
a M.S. student in the School of
Information and Communications at
Gwangju Institute of Science and
Technology (GIST), Korea. His
research interests include 3D image processing and depth
map generation.

Yo-Sung Ho received the B.S. and

M.S. degrees in Electronic Engineering
from Seoul National University, Seoul,
Korea, in 1981 and 1983, respectively,
and Ph.D. degree in electrical and
computer engineering from the
University of California, Santa Barbara,
in 1990. He joined the Electronics and
Telecommunications Research Institute (ETRI), Daejeon,
Korea, in 1983. From 1990 to 1993, he was with Philips
Laboratories, Briarcliff Manor, NY, where he was
involved in development of the advanced digital high-
definition television system. In 1993, he rejoined the
Technical Staff of ETRI and was involved in development
of the Korea direct broadcast satellite digital television and
high-definition television systems. Since 1995, he has been
with the Gwangju Institute of Science and Technology,
Gwangju, Korea, where he is currently a Professor in the
Department of Information and Communications. His
research interests include digital image and video coding,
image analysis and image restoration; advanced coding
techniques, digital video and audio broadcasting, 3-D
television, and realistic broadcasting.

Copyrights © 2015 The Institute of Electronics and Information Engineers

View publication stats