Scribd
Upload
31 views11 pages

13-Tracking L13

The document discusses object tracking techniques including optical flow and the Kanade-Lucas-Tomasi tracker. Optical flow computes apparent motion between images by assuming brightness constancy, while KLT tracker extends this idea to track local image patches by minimizing brightness change. The document also covers concepts like single/multi-object tracking, online/offline tracking and computing optical flow using optimization of image gradients.

Uploaded by

hisisthesongoficeandfire
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
31 views11 pages

13-Tracking L13

The document discusses object tracking techniques including optical flow and the Kanade-Lucas-Tomasi tracker. Optical flow computes apparent motion between images by assuming brightness constancy, while KLT tracker extends this idea to track local image patches by minimizing brightness change. The document also covers concepts like single/multi-object tracking, online/offline tracking and computing optical flow using optimization of image gradients.

Uploaded by

hisisthesongoficeandfire
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 11

Tracking

Jayanta Mukhopadhyay
Dept. of Computer Science and Engg.
Object tracking
◼ Estimating an object trajectory in the image
plane as it moves.
◼ Temporal sequence of images or A video
◼ Different contexts
◼ Single / multi-object tracking
◼ 2D / 3D trajectory
◼ Point / Mass tracking
◼ Rigid / deformable body tracking
◼ Offline / Online tracking
A few examples
Optical flow
◼ The distribution of apparent velocities of movement
of brightness pattern in an image
◼ Computed by assuming the brightness remains
constant due to apparent motion of image point for
a moving object.
I(x+Δx,y+Δy,t+Δt)=I(x,y,t)+(dI/dx) Δx+(dI/dy) Δy+(dI/dt) Δt+ higher orders

(dI/dx) Δx+(dI/dy) Δy+(dI/dt) Δt=0 0


(dI/dx) (Δx/ Δt)+(dI/dy) (Δy/ Δt)+(dI/dt) =0

Ixvx+Iyvy+It=0
Assuming the velocity profile smooth,
minimize also
∇I. v=-It |∇2vx|+|∇2vy| or ||∇vx||2+||∇vy||2
Optical flow: Optimization
problem (Horn and Shunk, AI,1980)
To minimize:
E1=∑( (It+Ixvx+Iyvy)2+k.(|∇2vx|+|∇2vy|))
∇2v=d2v/dx2+ d2v/dy2
Another function:
E2=∑((It+Ixvx+Iyvy)2+ k.(||∇vx|| 2+||∇vy||2))
||∇v||2=(dv/dx)2+ (dv/dy)2
A constant
dE/dvx=0 and dE/dvy=0
From E2: Ix2vx+IxIyvy=k∇2vx-IxIt
IxIyvx + Iy2vy=k∇2vy-IyIt
Ix2vx+IxIyvy=k∇2vx-IxIt
IxIyvx + Iy2vy=k∇2vy-IyIt

Optical flow solution:


∇2v~c.(vm– v) Vm=avg(v)

A constant, usually taken as 3 in 2D.


Solving two equations,
(k+Ix2+Iy2)vx=(k+Iy2)vx,m-IxIyvy,m-IxIt
(k+Ix2+Iy2)vy= -IxIyvx,m+(k+Ix2) vy,m -IyIt
Need to solve a set of 2N simultaneous equations
where N is the number of points in the image.
vx(n+1)=vx,m(n) -Ix(Ix. vx,m(n)+Iyvy,m(n)+It)/ (k+Ix2+Iy2)
vy(n+1)=vy,m(n) -Iy(Ix. vx,m(n)+Iyvy,m(n)+It)/ (k+Ix2+Iy2)
Optical flow: Pros and Cons
◼ Pros
◼ No assumption on shape and motion
◼ Applicable for rigid body and deformable motion
◼ Cons
◼ Computes for every pixel, but does not model
whole body motion.
◼ Aggregation to be done as a post-processing.
◼ non-trivial
◼ Computationally intensive
◼ Susceptible to noise
Kanade-Lucas-Tomasi (KLT)
Tracker (Lucas & Kanade, IJCAI, 1981;
Tomasi and Kanade, TR (CMU), 1991)

◼ Similar principle of optical flow.


◼ Intensity remains constant in the direction
of movement. Taylor series
J(x) J(x)=I(x-d) = I(x)- ∇I.d + higher terms
I(x) I(x) - J(x)= h= ∇I.d
d To minimize: E=∑w(x)(I(x)-J(x)- ∇I.d)2 / ∑w (x)
x 𝛜 Nx
𝜕E/ 𝜕d =0 w’: Normalized
∑w’(h-∇I.d) ∇I =0
x 𝛜 Nx w’=w/ ∑w
KLT Tracking solution
∑w’(h-∇I.d) ∇I =0 ∑w’(h ∇I -∇I∇IT d) =0
x 𝛜 Nx x 𝛜 Nx

∇IT d Column vector

d= (∑ w’∇I∇IT)-1 ( ∑w’ h ∇I)


x 𝛜 Nx x 𝛜 Nx

2x2 matrix
KLT Tracker: Selection of
feature points
◼ Corners: Changes perceptible in any
direction in the neighborhood
◼ Similar principle of Harris Operator.
◼ Eigen values of ∑w’∇I∇IT at x should be
high.
◼ Minimum Eigen value to be greater than a
threshold
◼ Compute d at those points and
aggregate them appropriately
Object Tracking: Pipeline
of Processing
Video Tracker Meta
data

Feature Localization Meta Data


extraction Extraction

Target Track Trajectory


representation Management state

You might also like

pFad - Phonifier reborn

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy