C18 Machine Vision 1(MT 2015) Page 1

C18 Machine Vision 1

Bugs/queries to david.murray@eng.ox.ac.uk MT 2015
For answers, hints, corrections see the course page David Murray
at www.robots.ox.ac.uk/∼dwm/Courses/4CV

1. Lecture 1: The camera as a geometric device

The camera shown in Figure 1 has its x, y and z axes aligned with the world’s y , z and x axes
respectively. The world frame’s origin is at (0, −h, 4h) in the camera’s frame.

(a) By deriving a succession of Euclidean transformations, find the camera’s extrinsic camera
calibration matrix [R|t], such that
 
R t
XC = XW
0> 1

(b) The “ideal” image plane is placed at zC = 1. (Ideal means that the intrinsic camera matrix
K is just a 3 × 3 identity matrix.) Derive the image coordinates of the vanishing point of the
family of lines parallel to the following line, expressed parametrically as:

(XW , YW , ZW ) = (2 + 4t, 3 + 2t, 4 + 3t) .

(c) The intrinsic calibration matrix maps ideal image positions onto actual positions in pixels.
Explain, with the aid of diagrams, the physical meaning of the focal length f , the aspect ratio
γ, and the principal point (uo , vo ), and show why and where they appear in the calibration
matrix. (Assume the skew s is negligible.)
(d) The actual camera has f = 800 pixels, γ = 0.9, s = 0, and (uo , vo ) = (350, 250) pixels.
Derive the coordinates in the actual image plane of the vanishing point of part (b).

yC Image plane
xC

zC zW yW
Camera
h

xW
4h World

Figure 1
C18 Machine Vision 1(MT 2015) Page 2

2. Lecture 1: Camera calibration

(a) Starting with the projection equation x = PX = K[R|t]X (all up to scale), describe an algorithm
which uses the measured image positions (x, y )i of at least six points with known positions
in the world (X, Y, Z)i to recover the extrinsic and intrinsic calibration parameters.
(b) Download the matlab code calibration.m from the web page at the top of the sheet, and
annotate it with comments to confirm that it fits with your algorithm description in part (b).
(c) Set h = 1 in the geometry of Question (1), and determine the image positions that would be
obtained in a camera with the following intrinsic matrix when viewing the following points X.
        
1 0 2 0 0 0 0 1 1
K= 0 2 1
  X1−6 = 0   1   1   0   0  1 
0 0 1 0 0 1 1 1 1

(d) Use the code to recover the extrinsic and intrinsic calibration from the image and world point
data.

3. Lecture 2: Edges and Corners

(a) Modest radial distortion in a lens can be modelled by a single parameter κ. As shown in
Figure 2(a), an undistorted image point at radius r from the image centre is moved in the
√
radial direction to rd = r / 1 − 2κr 2 .
i) By considering the distortion of the straight line (x = k, y ) in Figure 2(b), determine
whether κ < 0 gives rise to barrel or pin cushion distortion.
ii) Determine the form of the inverse expression r = f (rd ) used to correct distortion.
iii) You have an edge detector and straight line fitter at your disposal, and can point the
camera at any scene you wish. Devise the skeleton of an algorithm to determine κ
automatically.
(b) Explain the statement matching sections of straight edge constrains only one component of
the 2D motion of the edge.
(c) Write short notes on the Harris corner detector.

r
rd

Image centre Image centre

Figure 2
C18 Machine Vision 1(MT 2015) Page 3

4. Lecture 3: Epipolar geometry

Figure 3 shows a scene point X imaged at points x and x0 in cameras C and C0 . The cameras have
projection matrices P = K[I|0] and P0 = K0 [R|t], respectively.

(a) i) Draw a labelled diagram and write short notes to explain the epipolar geometry arising
from the observation x in the first camera. Indicate entities such as the optic centre,
baseline, epipole, epipolar line, and epipolar plane.
ii) Assuming convergent cameras, explain how the epipolar lines corresponding to different
image points x appear in the second camera C0 .
(b) Derive the point at infinity Q along the ray through x, and determine its projection q0 in
the image of the second camera C0 . Also determine the epipole e0 , and hence show that a
homogeneous expression for the epipolar line in C0 is

l0 = (K0 t) × K0 RK−1 x .

(c) i) Use the result of part (b), and the general identity Ma × Mb = M−> (a × b), to derive a
compact expression for the Fundamental Matrix F.
ii) Show that x0 > Fx = 0.
(d) A single camera with K = I captures an image, and then translates along its optic axis before
capturing a second image, so that t = [0, 0, tz ]> .
i) Use x0 > Fx = 0 to derive an explicit relationship relating x 0 , y 0 , x, and y .
ii) Briefly relate your result to the expected epipolar geometry for this camera motion.

x
x

C
C
Figure 3
C18 Machine Vision 1(MT 2015) Page 4

5. Lecture 3: More epipolar geometry

(a) Figure 4 shows a pair of ideal cameras each with focal length unity whose principal axes meet
at a point. The y -axes of both cameras are parallel and point out of the page.
i) Assuming C has projection matrix P = [I|0], work out the projection matrix P0 for C’ and
hence evaluate the fundamental matrix (using the form derived in the previous question).
ii) Use the relationship l0 = Fx to compute the epipolar line in the right image corresponding
to the homogeneous point x = [1, 1, 1]> in the left image.

(b) i) Expressing the projection matrices of the two cameras as P = K[I|0] and P0 = K0 [R|t],
derive the 4-vectors which represent the cameras’ optical centres, and hence derive ex-
pressions for the epipoles e and e0 .
ii) Show that e and e0 are the right and left null-spaces respectively of the fundamental
matrix F.
iii) Using Matlab or otherwise, verify your results using configuration of part (a).

(c) A camera rotates abouts its optical centre and changes its intrinsics so that the camera
matrices before and after P = K[I|0] and P0 = K0 [R|0].
i) Show that the images are related by an homography x0 = Hx.
ii) Devise the skeleton of a practical algorithm to use this expression to rectify a pixelated
image. (You will need to consider where in the original unrectified image the corners of
each rectified pixel appear.)

x d
z
C
1
(one) 60o

x’ z’
1
C’
Figure 4
C18 Machine Vision 1(MT 2015) Page 5

6. Lecture 3: Even more epipolar geometry

(a) A single camera captures an image I, then rotates, translates, and changes its lens zoom
setting before capturing a second image I 0 . The projection matrices in the two positions are
P = K[I|0] and P0 = K0 [R|t], and a scene point is imaged at the corresponding points x and x0 .
i) Derive the Fundamental matrix — without using the notes or your solution to Q4!
ii) Derive the constraint between x0 and x.
(b) For a scene point [XT , 1]T the projection equations can be written with explicit scale factors
as

λx = KX and λ0 x0 = K0 (RX + t) .

When the changes between images I and I 0 consist of pure rotation and zoom only, show that
the corresponding points are related by an homography

x0 ∼ Hx

and determine an expression for H (up to scale).

(c) The camera’s motion and zoom change are again as in part (a), but the scene viewed is now
a planar surface n̂T X = d, where n̂ is the plane’s unit normal.
i) Using the projection equation for the first position show that

X t tn̂T K−1 x
= K−1 x and = .
λ λ d
λ0 x0
ii) Using that for the second position, develop an expression for and show that the
0
λ
corresponding points are again related by an homography x ∼ Jx and determine the
expression for J (up to scale).
(d) Show that if x0 = Hx, then H> F must be an antisymmetric matrix.
C18 Machine Vision 1(MT 2015) Page 6

7. Lecture 4: Stereo correspondence

(a) The ordering constraint is often used in stereo correspondence algorithms to disambiguate
point matches on corresponding epipolar lines. Sketch a configuration of surfaces and cameras
for which the ordering constraint is valid, and a configuration for which it is not valid.
(b) Prove that if normalized cross-correlation is used to measure the similarity of image regions
between two images (including adjusting the signal to have mean zero), then the measure is
invariant (unchanged) under the following transformation of the intensities

I(x, y ) ← αI(x, y ) + β

where α and β are scalars.

(c) Explain why using normalized cross-correlation is likely to be more important for stereo match-
ing than for matching between images in a monocular sequence.

8. Lecture 4: Triangulation

(a) Two parallel cameras, each with focal length f and otherwise ideal, are separated along the
x-axis by tx = 4 units. By drawing a plan view of the system (ie, one with the x and z
axes in the plane of the paper), and using very simple drawing construction, show that the
coordinates of the 3D points reconstructed for the correspondences: (i) [−1, 0] ↔ [1, 0] and
(ii) [0, 0] ↔ [0, 0] are (i) [−2, 0, 2f , 1]> and (ii) [0, 0, 1, 0]> .
(b) i) Consider two general cameras with projection matrices K[I|0] and K[R|t]. Derive a vector
expression for the position X of the 3D scene point given a correspondence x ↔ x0 ,
assuming no noise (Ie, when the backprojected rays do actually intersect.)
ii) Now specialize your expression for the camera geometry given by part (a). Using the
correspondences given in part (a), show that your expression in part(b)(i) gives the same
results for the scene point X as does the simple construction in part (a).
(c) For a parallel camera stereo configuration where each camera has focal length f the “hori-
zontal” disparity d is given by d = (x 0 − x) = f tx /Z. Use differentiation to show that the
error δZ in depth Z corresponding to an error δd in disparity is given by
 
δd
δZ = − Z2 .
f tx
By considering the nature of f , tx and δd, suggest an obvious consequence of this relationship.

9. For interest only: Back to Lecture 3, Computing the F matrix

Download the matlab code sevenpoint.m from the web page at the top of the sheet. Study and
run the code.

For answers, hints, etc

See the web page given at start of sheet.