I.J.

Information Technology and Computer Science, 2016, 8, 13-22

Published Online August 2016 in MECS (http://www.mecs-press.org/)
DOI: 10.5815/ijitcs.2016.08.02

IP Camera Based Video Surveillance Using

Object’s Boundary Specification
Natalia Chaudhry
Department of Computer Sciences, Kinnaird College for Women Lahore, Pakistan
E-mail: nataliawr123@gmail.co m

Kh. M. Umar Suleman

Faculty of IT, University of Central Punjab, Pakistan
E-mail: umar.suleman@ucp.edu.pk

Abstract—The ability to detect and track object of allows tracking of registered objects. The process of
interest from sequence of frames is a crit ical and vital cropping out and saving object of interest from the given
problem of many vision systems developed as yet. This frame is referred to as registration of that object of interest.
paper presents a smart surveillance system that tracks The proposed surveillance system will allow mon itoring
objects of interest in a sequence of frames in their own of different registered objects under a certain boundary
defined respective boundaries. The objects of interest are that is defined and personalized for each object by the user
registered or saved within the system. We have proposed and it notifies the user about theft of object only if it is
a unique tracking algorith m using co mbination of SURF found missing for predefined duration in its own defined
feature matching, Kalman filtering and temp late matching boundary. This boundary is known as virtual fence for that
approach. Moreover, an efficient technique is proposed object. If the object is flipped then the registered image of
that is used to refine reg istered object image, ext ract the object will not match with the current image of the
object of interest and remove extraneous image area fro m object. Our video surveillance system caters this using an
it. The system will track reg istered objects in their efficient algorithm known as auto-registration. In it,
respective boundaries using real time v ideo generated registered object image is modified and updated with
through two IP cameras positioned in front of each other. object's current image to ensure correct object detection
for monitoring in future.
Index Terms—SURF feature matching, temp late Rest of the paper is organized as: Section 2 contains
matching, Kalman filtering, track, IP cameras. description of work done related to video surveillance.
Section 3 describes the architecture of the system. Section
4 contains the description of each module involved in a
I. INT RODUCT ION system. Section 5 contains experiments and results.
Section 6 concludes the paper. Section 7 contains future
Various network based video surveillance software are
recommendations.
proposed in industry. There are many benefits offered by
IP solutions. Beyond all such advantages, the use of coax
cables has been eliminated. Network technology allows IP
II. RELAT ED W ORK
cameras to be ad ministered and viewed, even though the
operators are located far away. Video surveillance system Various surveillance systems have been developed to
finds events of interest in various aspects. date. Vital task in each of these video surveillance systems
Many video surveillance systems have been introduced includes object detection and tracking. Some of the video
so far. One of them is NUUO surveillance system. It surveillance system projects are surveyed as follows. M.
allows setting up the software to detect when defined Sivarathinabala et al, [1] had proposed an intelligent video
objects are removed from areas being monitored by surveillance system. Hu man activity is recognized through
surveillance system. Apart fro m this, system for tracking a image classification using HMM model. The system
moving object, license plate recognition, face recognition generates notification alarm when it detects abnormal
and tracking, finding pedestrian in infrared videos, system activity through motion and event detection. H. Shengluan
for monitoring and alerting based on animal behavior and et al, [2] presented a tracking system that hinders the error
a system for detection of abnormal motion in video stream that arises due to occlusion. This is done through
have also been introduced. The ability to detect and track estimations and predictions in Kalman filtering approach.
object of interest fro m sequence of frames is a critical and It merely tracks the objects. Y. T. Hwang et al, [3] also
vital problem of many vision systems developed so far. presents a tracking system that is based on feature points.
The above mentioned systems notify the user merely on Object segmentation is eliminated to lessen the
detecting motion and other activity. Each object of interest complexity. Feature points are detected and then matched
in a scene has common tracking criteria. to achieve matching. This matching is followed b y
This paper introduces a video surveillance system that tracking. C. Tuscano et al, [4] worked on a project of

Copyright © 2016 MECS I.J. Information Technology and Computer Science, 2016, 8, 13-22
14 IP Camera Based Video Surveillance Using Object’s Boundary Specification

making a surveillance system that also notifies the user which it should be tracked.
about intrusion, records and sends this footage to the user
besides just detecting motion. Email and MMS are sent on
discovering abnormal activity. Chandana had similarly III. SYST EM OVERVIEW AND A RCHIT ECT URE
proposed a system that detects motion and tracks object in
Our v ideo surveillance system mon itors and controls
a live video. Background subtraction technique is used for
detecting motion [5]. A. Girgensohn et al, [6] proposed a the theft of objects that are marked and registered by the
user. User will be allowed to refine the registered object
mu lti camera indoor surveillance system named as
image to remove the irrelevant area fro m it for better
Dynamic Ob ject Tracking System. User will be notified
about remote events. Integration with different sensors is tracking results. This will help to avoid false notification
that may be due to irrelevant area existing in the registered
done and interface is provided that allows the user to
object's image. The system will track the registered object
follow a person across series of cameras. Detection is
done through technique of foreground segmentation. T. efficiently even if the object changes its position. It checks
if the object is stolen or not. In case of theft of registered
Yang et al, [7] introduced a multi-camera people tracking
object, system will notify and in form the user about it.
surveillance system that is robust. Approach based on
greedy search is applied that efficiently tracks mu ltiple The detailed working of main modules involved in our
system is discussed in the next section. Two IP cameras
people in the presence of occlusion. Moreover individual
are aligned in such a way that both of these face the front
trajectories are associated in series of cameras that are
adjacent to each other. The system proposed by X. Ke et of each other. Due to this set up, one of the IP cameras
captures the front side of object and other one captures its
al, [8] automatically checks the attendance of traffic
b ack s ide. Th is set up h elps to deal with o cclus io n
controllers based on detection scheme. Detection and
tracking scheme includes motion detection technique with problems up to some extent. If one camera captures a
scene of object being occluded due to some reasons (e.g.,
feature matching and color segmentation. This helps to
person standing in front of object and occluding its front
detect the object of interest accurately.
All of the above mentioned surveillance systems have side) then the other camera will capture its opposite side
which is not occluded. Hence system will be able to detect
same detection criteria for every object of interest present
the object in captu red frame fro m the second camera.
in a video frame. But the system proposed by us allows
the user to customize the detection and tracking criteria Objects of interest and their relevant virtual fences are
cropped and registered in database. Frames are grabbed
for each object according to his or her needs. This is done
using these two IP cameras. Both of these frames are
by allowing the user to personalize and select the
boundary (virtual fence) for each object of interest under processed to find if the object exists in its virtual fence or

Fig.1. System architecture

Copyright © 2016 MECS I.J. Information Technology and Computer Science, 2016, 8, 13-22
 IP Camera Based Video Surveillance Using Object’s Boundary Specification 15

not. This is accomplished by comparing these frames other than object of interest in the registered object image.
with the image of registered object that is registered To overcome this difficulty, this module allows the user to
within the system. Informat ion regarding v irtual fence is refine the image of registered object to crop out irrelevant
fetched fro m the database. If the registered object is not region. This will eventually improve tracking results.
found for some predefined period in both of the frames We have used the approach presented in the paper [9]
captured by these two IP cameras then the system for implementing this module with some proposed
generates alarm, sends email and SMS based on user modifications. Threshold value is automatically calculated.
selection. Firstly, pyramid of input image is calculated. Pyramid
down operation helps in calculating threshold value. Then,
Laplace filter is applied for spot detection. After finding
IV. M ODULES INVOLVED IN A SYST EM good spots and isolating them a threshold value is
calculated on the basis of histogram operations. Initial
Our video surveillance system helps the users to estimated value of threshold is calculated by taking
prevent theft and keep a track of different objects of
average of grayscale values in an image region. Final
interest in their own specified boundary. Refinement
value for threshold is calculated by summing up the
module includes cropping out the extraneous region from estimated value calculated and the value we get through
the image of registered object so that only intended
histogram operation. Pixel intensity of each pixel of an
registered object is included in the image. System will
image is compared with automatically calculated
count the number of objects present in the image that threshold value. If this intensity value is greater than the
makes it easy for the user to select and identify required
automatically calculated threshold value, the pixel is
registered object from these objects. Based on this user
labeled as white in an image otherwise it is set as black. [9]
selection, objects and region other than intended object is In this binary threshold image, some small regions may
cropped out and thus image will be refined. The most
appear in an image due to noise. To overcome this,
important module of the system is “Object Tracking”.
detected foreground objects that have less than 400 pixels
This will track and monitor registered objects in frames are removed. On this, morphological closing operation is
captured by IP camera and check whether each registered
applied. [10] User is allowed to increase or decrease the
object is in its virtual fence or not. If not, then through
final threshold value to get threshold image of his or her
notification module user of the system will be notified choice. Generally white pixels assimilate as foreground
about the theft. Notification module contains logic about
and black as background respectively. But due to
notification.
intensity variations in an image there may arrive a case in
The main modules involved in our video surveillance which foreground appears as black and background as
system are:
white. This is shown in Fig. 2 below. To cater this, we
A. Registration of objects and their virtual fence have devised a solution in which the system allows the
user to choose whether white pixels correspond to
System tracks the object and check if that object has foreground or black pixels corresponds to foreground.
crossed a certain distance. The distance may vary from Based on this input the system labels and numbers the
object to object as specified by the user. To handle the objects in an image. The user is allowed to give as input
distance property we need to create some virtual fence for the number or label of the required object that need to be
each object that specifies the boundary under which object registered and tracked afterwards. Using this input, the
should be tracked. On reg istration of objects that need to system crops out and refine an image on the basis of
be tracked, system also needs to register and save the height and width properties of labeled object. This is
boundary parameters specified by user for each object. shown in Fig.3.
Registration is done through cropping algorithm. User is
allowed to crop the desired object from frame along with
its virtual fence and save it within the system. This
process refers to as registration.
B. Registered objects Refinement
This module aims at improving the image of registered
object to provide better results of tracking. When the user
registers the object there may exists a possibility that the
image of that registered object also contains other objects
that does not intend to be tracked. Due to this negligence
of user, the system will may not be able to know about
absence of registered object due to existence of objects Fig.2. Binary images

Copyright © 2016 MECS I.J. Information Technology and Computer Science, 2016, 8, 13-22
16 IP Camera Based Video Surveillance Using Object’s Boundary Specification

Fig.3. Refinement of image

C. Object tracking image location that tells how well the match. Good match
indicates that the patch is found in source image. [12] If
After registration of objects, the next task is to track the
template matching technique fails to detect registered
object in their respective virtual fences to check whether
object then it means that object is not in its virtual fence
they are stolen or not. For this, we need a mechanism to
and appropriate measures should be applied. Fig. 12
detect and match the object to verify its existence. SURF
illustrates the flow diagram of this module.
feature matching approach contributes to this. [11] We
The tracking algorith m proposed by us does not involve
take current frame as query frame and image of registered
SURF feature matching and template matching
object and then apply SURF feature matching algorithm
approaches only. We have proposed a unique algorithm
on these two images. Using this, it can be figured out
for handling issues arising during tracking and increasing
whether the object is in the query frame or not.
efficiency of tracking. We need to redefine and adjust
There may arises a case when SURF feature matching
virtual fence at appropriate times during tracking. Th is is
fails to detect registered object in its virtual fence even if
discussed in detail in the next sections.
that object is present in its virtual fence. Reasons may be
There are two sub modules for this module which are
the poor video quality due to which weak feature points
discussed in detail below. These are:
will be detected thereby generating poor matches. So, to
verify whether the result generated by SURF feature
 Auto registration
matching technique is true some other technique is needed
that checks the availability of registered object in its  redefinition of virtual fence
virtual fence. If registered object is not found using SURF
feature matching approach, then template matching D. Auto registration
approach is applied to detect whether the registered object
is in its virtual fence or not. If template matching also After extracting out the detected image area it must be
indicate mismatch between registered object image and compared with registered object image already stored
query frame only then appropriate notification actions within the system to check if there is a significant amount
would be taken. So template matching is used as a of change or not. This will update already saved registered
verification approach for finding theft. It is applied in object image when it undergoes orientation change i.e.
tracking algorithm if and only if SURF feature matching rotated or flipped. This process is known as auto -
fails to detect or find object in its virtual fence. registration. Due to this the system will now detect and
Template matching approach finds area in an image that match the rotated or flipped registered object correctly as
has highest match. To find this area sliding window the already saved registered object's image is updated
approach is used. That is, we slide template image patch according to its current orientation. This ensures correct
on query image (source image). Patch is moved one pixel matching. This is shown in fig. 4.
left to right and up to down. A metric is calculated at each

Fig.4. Auto registration process

In fig. 4, object is shown flipped thereby indicating tracking and flipped object will be accurately detected.
orientation change. The image of registered object will be This must be handled efficiently because merely the
updated. Already registered object's image is updated with illu mination change or false detection can result in
the extracted one. Due to this, updated image, as shown significant amount of change between detected area image
extracted in Fig.4 above, will be referenced for further and registered object image. So, simple motion detection

Copyright © 2016 MECS I.J. Information Technology and Computer Science, 2016, 8, 13-22
 IP Camera Based Video Surveillance Using Object’s Boundary Specification 17

may not correctly handle this. After experimenting with detected area image. The match scores of these two
couple of images and scenarios we have devised a method operations are considered. This is shown in Fig. 5. In case
for handling this and named it as double cross correlation of our verification approach of template matching (if the
template matching technique. Double cross correlation SURF could not find the registered object), apart from
template matching is basically cross correlation template these two score matches, match score achieved by
matching technique applied two times. First, the detected matching registered object in its virtual fence is also
area image is used as first parameter for cross correlation considered. After thorough experimentation, we have
algorithm with registered object image as a second computed ranges that find whether the detected area needs
parameter. That means we slide detected area image over to be auto registered or not. Ranges with their
registered object image g iving a matching score as output. interpretation are shown in Fig. 6. "scr" is the difference
Second step is the inverse of first step in terms of inputs to of two match scores obtained by applying cross
cross correlation template matching algorithm. In it, correlation two times in double cross correlation algorithm.
registered object image is used as first parameter for cross "maxVal" is the match score achieved in template
correlation algorith m with detected area image as second matching approach by detecting registered object in its
parameter. That is, registered object image slides over virtual fence.

Fig.5. Double cross correlation template matching approach

Fig.6. Ranges with their interpretation

consecutive frames the registered object should have not

E. Redefinition of virtual fence
gone too far in virtual fence. So area with size calculated
For tracking the registered object image of large size in with respect to size of template image will be sufficient.
its virtual fence the tracking algorithm tends to run slow That is we need to redefine virtual fence. This redefined
due to this. Specially in case of template matching, the virtual fence will be under the boundary of originally
larger template image is slow to process. To avoid this one defined virtual fence by the user. At first, the point at
way is to find the registered object in an area around that which the registered object is detected is considered. That
point inside a virtual fence where the registered object is point’s x and y coordinates are recorded that serves as
last detected. The size of that area is calculated with centroid for the next area (redefined virtual fence) that
respect to size of registered object image. As in two needs to be redefined in original virtual fence for detection

Copyright © 2016 MECS I.J. Information Technology and Computer Science, 2016, 8, 13-22
18 IP Camera Based Video Surveillance Using Object’s Boundary Specification

purpose. Then this centroid is processed and rectangular height of registered object's image. This process is
virtual fence is created. Width and height of that illustrated in Fig. 7 and Fig. 8 below.
rectangular redefined virtual fence is double the width and

Fig.7. Mapping of previous detection

Fig.8. Redefined sub virtual fence creation

some frames then we may miss the points of detection that

will be used for redefining fence in the next frame. Due to
this the sub fence created will not be updated and object
will eventually move out of it. Although the object may
exists in originally defined fence but as the redefined
fence is not updated so object will not be tracked. This is
shown in Fig. 10 in which the redefined fence is shown
not updated. To handle this we have used Kalman filtering
optical flow analysis approach to serve the purpose. The
detailed working of the same is described in [13]. A small
history about x and y coordinates position of detected area
is maintained for each iteration. Kalman filter gives as
output the position coordinates of motion detected area.
We are using Kalman filter in our surveillance system as
an estimator of position of moved object in a frame. First
of all Kalman filter is applied that gives the position where
the motion is detected i.e. the position where the object
might be moved. Position at which the object is detected
in last frame is adjusted with this value. After this the
virtual fence is redefined using the procedure described
above. If the object is moved and is not detected in last
Fig.9. Redefinition of virtual fence frame then due to this adjustment the fence will be
updated according to the position parameters of object due
The sub virtual fence defined within the original virtual to its motion. So even if the object is not detected for
fence may go out of boundary of originally defined some reasons then the position parameters will not be
virtual fence during processing. So its existence must be missed and the fence will be redefined. Fig. 11 illustrates
guaranteed inside virtual fence only. the flow diagram of approach adopted for virtual fence
As the virtual fence is redefined the system will not be adjustment.
too slow in processing the large fences. But there may
arise an issue. If the object is moving and not detected in

Copyright © 2016 MECS I.J. Information Technology and Computer Science, 2016, 8, 13-22
 IP Camera Based Video Surveillance Using Object’s Boundary Specification 19

Fig.10. Issue in redefinition

Fig.11. Virtual fence adjustment

Copyright © 2016 MECS I.J. Information Technology and Computer Science, 2016, 8, 13-22
20 IP Camera Based Video Surveillance Using Object’s Boundary Specification

Fig.12. Object tracking

Scenario frame rate=Frames taken/time taken (2)

F. Notification of theft
This module involves informing the user about theft. Overall efficiency=∑ No. of frames satisfying the
When any registered object got stolen and it is not found scenario /∑ No. of frames taken*100 (3)
in some specified number of frames then the system
notifies the user about it. Three categories of notification Overall frame rate=∑ Frames taken/∑ time taken (4)
are catered by our system. These are:
Scenario erro r=No. of frames not satisfying the
 Generation of alarm  scenario/No. of frames taken*100 (5)
 Sending SMS 
 Sending Email with an attachment of footage with Overall error=∑No. of frames not satisfying the
theft scene.  scenario/∑No. of frames taken*100 (6)

User is allowed to select any of these categories and the B. Results categories and scenarios
system will do action accordingly.
T able 1. Possible results and their meanings

V. EXPERIMENT AL RESULT S
The algorith ms and approaches discussed in the above
section are imp lemented in MATLAB 2013a and tested
for three different objects using two IP cameras. Objects
include laptop, mobile and duplicate objects. Different
scenarios are derived and considered. The algorithm is
tested for each scenario.
A. Formulas used
Scenario efficiency=No. of frames satisfying the
scenario/No. of frames taken*100 (1)

Copyright © 2016 MECS I.J. Information Technology and Computer Science, 2016, 8, 13-22
 IP Camera Based Video Surveillance Using Object’s Boundary Specification 21

Overall frame rate = 517/563.2*100 = 0.917

T able 2. Scenarios
E. Results with mobile
Virtual fence width: 226.98
Virtual fence height: 110.98

T able 4. Results using mobile

C. Objects used in experiment

Fig.13. Laptop

Overall efficiency = 399/415*100 = 96.14%

Overall frame rate = 415/282.87*100 = 1.46
F. Results with duplicate objects
Fig.14. Mobile
Virtual fence width: 466
Virtual fence height: 271

T able 5. Results using duplicate objects

Fig.15. Duplicate objects

D. Results with laptop

Virtual fence width: 311
Virtual fence height: 133

T able 3. Results using laptop

Overall efficiency = 195/311*100 = 62.70%

Overall frame rate = 311/404.1*100 = 0.76 frames/s

The tracking efficiency achieved by using laptop and

mobile as a registered object is above 90% and acceptable.
For duplicate objects, efficiency of scenario in wh ich
object is not present in the virtual fence is 0. This is
because although the object intended to be tracked is
removed from virtual fence there exist the other duplicate
object. So the system detects and tracks that duplicate
Overall efficiency = 487/517*100 = 94.19% object present in virtual fence. If there is no restriction that

Copyright © 2016 MECS I.J. Information Technology and Computer Science, 2016, 8, 13-22
22 IP Camera Based Video Surveillance Using Object’s Boundary Specification

only the original intended object must be detected or [5] S. Chandana, "Real time video surveillance system using
tracked rather than its duplicate then this scenario motion detection," in India Conference, pp. 1-6, 2011.
efficiency would be 100%. For other scenarios the [6] A. Girgensohn, et al. "DOTS: support for effective video
surveillance," in Proceedings of the 15th international
efficiency is acceptable because after first iteration the
conference on M ultimedia. ACM , pp. 423-432, 2007.
system tracks the object in redefined. This means that the [7] T. Yang, et al. "Robust people detection and tracking in a
correct detection will depend on the distance between two multi-camera indoor visual surveillance system," in
objects and specifications of virtual fence. If the distance International Conference on M ultimedia & Expo, 2007.
between the two duplicate objects is large and the virtual [8] K. Xiang, et al. "Intelligent video surveillance for
fence is broad enough to be redefined then this will lead checking attendance of traffic controllers in level
towards correct detection. But the first detection will be crossing," in Journal of Shanghai Jiaotong University,
anonymous. The system will may detect the intended Vol. 19, pp. 41-49, 2014.
original object or it may detect the other duplicate one. [9] M . Shneier, "Using Pyramids to Define Local Thresholds
for Blob Detection," in Pattern Analysis and M achine
The system will detect that first detected object onwards
Intelligence, IEEE, Vol. 5, pp. 345 - 349, 1983.
in next frames after the first detection, until the other [10] A. Bishnoi, "Noise Removal with M orphological
duplicate object is placed next to this object. Operations Opening and Closing Using Erosion and
Dilation," in International Journal Of M odern Engineering
Research, Vol. 4, 2014.
VI. CONCLUSION [11] H. Bay, T. Tuytelaars and L.V. Gool, "SURF: Speeded
up robust features," in European Conference on
In this paper, a smart surveillance system is presented Computer Vision, Vol. 3951, pp. 404-417
that can track objects of interests in their respective virtual [12] Tutorials on Opencv template matching approaches and
fences. The previously proposed systems do not have implementation algorithms working. Available:
virtual fence specification property. Moreover, we have http://docs.opencv.org/doc/tutorials/imgproc/histograms/t
presented an efficient unique tracking algorithm that emplate_matching/template_matching.html
combines different approaches together to provide [13] C. Suliman, C. Cruceru, and F. M oldoveanu, "Kalman
filter based tracking in an video surveillance system," in
efficient results. We have integrated various algorithms to
Advances in Electrical and Computer Engineering, pp. 30-
avoid false alarms and do correct detections and tracking.
34, 2010.
In the end we presented the results and discussed some
implementation related issues and future directions. We
have provided our model for the development of video
surveillance system with virtual fence customization and Authors’ Profiles
would like to urge prospective researchers to explore this
field further as it needs more attention. Natalia Chaudhry is a student of M Phil in
Computer Science at University of Punjab,
Lahore. She has received a BS (Hons) in
VII. FUT URE RECOMMENDAT ION Computer Science from Kinnaird College
for Women University Lahore.
Further improvements in speed and efficiency of
system are needed. The algorithm can be extended to
handle tracking of mult iple registered objects
simultaneously. Occlusion must also be managed. Issues
in tracking that may arrive due to blurred objects and
objects with reflective surface must be appropriately Dr. Umar S uleman has done PhD in
handled. For duplicate objects, adjustments can be made Computer Science in the area of robot
learning from demonstration from LUM S.
to correctly detect object especially for the first time. He is presently working as Assistant
Professor at University of Central Punjab,
REFERENCES Lahore. He has received a BS(Hons) and
[1] M . Sivarathinabala and S. Abirami, "An Intelligent Video M .S(Computer Science) in Computer
Surveillance Framework for Remote M onitoring," in Science from FAST-NUCES, Lahore.
International Journal of Engineering Science and
Innovative Technology, Vol. 2, 2013.
[2] S. Huang and J. Hong, "M oving object tracking system
based on camshift and Kalman filter," in Consumer How to cite this paper: Natalia Chaudhry, Kh. M . Umar
Electronics, Communications and Networks, pp. 1423- Suleman, "IP Camera Based Video Surveillance Using Object's
1426, 2011. Boundary Specification", International Journal of Information
[3] Y.T. Hwang et al., "Feature Points Based Video Object Technology and Computer Science (IJITCS), Vol.8, No.8,
Tracking for Dynamic Scenes and Its FPGA System pp.13-22, 2016. DOI: 10.5815/ijitcs.2016.08.02
Prototyping," in Intelligent Information Hiding and
M ultimedia Signal Processing, pp. 325 - 328, 2014.
[4] C. Tuscano, et al. "Smart Web Cam M otion Detection
Surveillance System," in International Journal of M odern
Engineering Research, Vol. 3, pp -1169-1171, 2013.

Copyright © 2016 MECS I.J. Information Technology and Computer Science, 2016, 8, 13-22