Photogrammetry & Robotics Lab

Direct Solutions for Computing

Fundamental and Essential
Matrix

Cyrill Stachniss

The slides have been created by Cyrill Stachniss.

1
Motivation

Image courtesy: Collins 2

Topics of Today
Compute the
§ Fundamental matrix
given corresponding points
§ Essential matrix
given corresponding points
§ Rotation matrix and basis
given an essential matrix

3
 Computing the
Fundamental Matrix
Given Corresponding Points

4
Fundamental Matrix
§ The fundamental matrix is

§ It encodes the relative orientation

for two uncalibrated cameras
§ Coplanarity constraint through

5
Fundamental Matrix
The fundamental matrix can directly
be computed if we know the
§ calibration matrices
§ viewing direction of the cameras
§ baseline
§ or the projection matrices

How to compute F given ONLY

corresponding points in images?
6
Problem Formulation
§ Given: N corresponding points

§ Wanted: fundamental matrix

7
Fundamental Matrix
From Corresponding Points
§ For each point, we have the
coplanarity constraint

8
Fundamental Matrix
From Corresponding Points
§ For each point, we have the
coplanarity constraint

§ or

unknowns! 9
What is the Issue here?
§ In standard least squares problems,
we have a vector of unknowns
§ Here, the matrix elements of F are
the unknowns

Question:
How to turn the unknown matrix
elements into a vector of unknowns?

10
Linear Dependency
§ Linear function in the unknowns

11
Linear Dependency
§ Linear function in the unknowns

12
Linear Dependency
§ Linear function in the unknowns

13
Linear Dependency
§ Linear function in the unknowns

14
Linear Dependency
§ Linear function in the unknowns

15
Using the Kronecker Product
§ Linear function in the unknowns

(it holds in general: )

16
Linear System From All Points
§ We directly obtain a linear system if
we consider all N points

17
Solving the Linear System
§ Singular value decomposition solves

§ and thus provides a solution for

§ SVD: f is the right-singular vector

corresponding to a singular value of A
that is zero
18
How Many Points Are Needed?
§ The vector has 9 dimensions

19
How Many Points Are Needed?
§ The vector has 9 dimensions

§ Matrix has at most rank 8

§ We need 8 corresponding points

20
More Than 8 Points…
§ In reality: noisy measurements
§ With more than 8 points, the matrix A
will become regular (but should not!)
§ Use the singular vector of A that
corresponds to the smallest singular
value is the solution

21
Singular Vector
§ Use the singular vector of A that
corresponds to the smallest singular
value is the solution

22
8-Point Algorithm 1st Try

23
8-Point Algorithm 1st Try

singular vector of the

smallest singular value
Not necessarily a matrix of rank 2
(but F should have: rank(F)=2) 24
Enforcing Rank 2
§ We want to enforce a matrix with

§ should approximate our computed

matrix as close a possible

What to do?

25
Enforcing Rank 2
§ We want to enforce a matrix with

§ should approximate our computed

matrix as close a possible
§ Use a second SVD (this time of F)

with
and

26
8-Point Algorithm

27
8-Point Algorithm

28
Well-Conditioned Problem
§ Example image 12MPixel camera
4000
3000

§ Ill-conditioned, numerically instable

new
coordinate
system

29
Conditioning/Normalization
to Obtain a Well-Conditioned
Problem
§ Normalization of the point coordinates
substantially improves the stability
§ Transform the points so that the
center of mass of all points is at (0,0)
§ Scale the image so that the x and y
coordinated are within [-1,1]

30
Conditioning/Normalization
§ Define so that coordinates
are zero-centered and in [-1,1]
§ Determine fundamental matrix from
the transformed coordinates

§ Obtain essential matrix through

31
Singularity – Points on a Plane
§ If all corresponding points lie on a
plane, then
§ Numerically instable if points are close
to a plane

Images from the “Fundamental Matrix Song” Video by D. Wedge

32
Singularity – No Translation
§ The projection centers of both
cameras are identical:
§ This happens if the translation of the
camera is zero between both images

scene

cam1
cam2
33
Summary so far
§ Estimating the fundamental matrix
from N pairs of corresponding points
§ Direct solution of N>7 points based on
solving a homogenous linear system
(“8-Point Algorithm”)

34
 Computing the
Fundamental Matrix
Given 7 Corresponding Points

35
Direct Solution with 7 Points
§ We know that the fundamental matrix
has seven degrees of freedom
§ There exists a direct solution for 7 pts

The solution itself is more complex,

so just the idea should matter here

36
Direct Solution with 7 Points
§ We know that the fundamental matrix
has seven degrees of freedom
§ There exists a direct solution for 7 pts
§ Idea: 2-dimensional null space of A
§ Matrix must fulfill

vectors spanning
the null space

37
Direct Solution with 7 Points
§ We know that the fundamental matrix
has seven degrees of freedom
§ There exists a direct solution for 7 pts
§ Idea: 2-dimensional null space of A
§ Matrix must fulfill
§ We also know that the determinant of
the 3x3 matrix must be zero:
§ Can be combined to an equation of
degree 3 up to three solutions
38
Direct Solution with 7 Points
§ We know that the fundamental matrix
has seven degrees of freedom
§ There exists a direct solution for 7 pts
§ Exploit
§ 2-dimensional null space of A
§ Determinant of the 3x3 matrix must be
zero:
§ Can be combined to an equation of
degree 3 up to three solutions

39
Summary so far
§ Estimating the fundamental matrix
from N pairs of corresponding points
§ Direct solution of N>7 points based on
solving a homogenous linear system
(“8 point algorithm”)
§ Idea for a direct solution with 7 points
(up to 3 solutions)

40
Let’s Do the Same for the
Essential Matrix

41
Reminder: Essential Matrix
§ Essential matrix = “fundamental
matrix for calibrated cameras”

§ Often parameterized through

(general parameterization of dependent images)

§ Coplanarity constraint for calibrated

cameras
42
Essential Matrix
from 8+ Corresponding Points
§ For each point, we have the
coplanarity constraint

§ Note: Same equation as for the

fundamental matrix but for the
points in the camera c.s.
Remember:
43
Essential Matrix
from 8+ Corresponding Points
§ For each point, we have the
coplanarity constraint

§ or

44
As for the Fundamental Matrix…

build matrix A

solve Ae=0

build matrix E

Which constraints to consider? 45

Constraints
§ For the fundamental matrix, we
enforced the rank(F) constraint

§ For the essential matrix, both non-

zero singular values are identical

homogenous

More details: Förstner, Skript Photogrammetrie II, Sect 1.2.3 46

8-Point Algorithm for the
Essential Matrix

build matrix A

solve Ae=0

build matrix Ea

compute SVD of Ea

build matrix E from Ea

by imposing constraints
47
Conditioning/Normalization
to Obtain a Well-Conditioned
Problem (As Done Before)
§ As for the 8-Point algorithm for the
fundamental matrix, normalization of
the point coordinates is essential
§ Transform the points so that the
center of mass of all points is at (0,0)
§ Scale the image so that the x and y
coordinated are within [-1,1]

48
Conditioning/Normalization
§ Define so that coordinates
are zero-centered and in [-1,1]
§ Determine essential matrix from the
transformed coordinates

§ Obtain essential matrix through

49
Properties of the Essential Mat.
§ Homogenous
§ Singular: (determinant is zero)
§ Two identical non-zero singular values

§ As a result of the skew-sym. matrix:

50
5-Point Algorithm

51
5-Point Algorithm
§ Proposed by Nistér in 2003/2004
§ Standard solution today to obtaining
the direct solution
§ Solving a polynomial of degree 10
§ 10 possible solutions
§ Often used together RANSAC
§ RANSAC proposes correspondences
§ Evaluate all 5-point solutions based on
the other corresponding points

52
5-Point Algorithm
§ More details in the script by Förstner
“Photogrammetrie II”, Ch 1.2
§ Stewenius, Engels, Nistér: “Recent
Developments on Direct Relative
Orientation”, ISPRS 2006
§ Li and Hartley: “Five-Point Motion
Estimation Made Easy”

53
 Computing the
Orientation Parameters
Given the Essential Matrix

54
Compute Basis and Rotation
Given E
§ In short:

Question: Is there a unique solution?

55
The Solution We Want…

56
Multiple Solutions from Math…

We only know b up to a scalar.

So we can multiply it by -1…

57
Multiple Solutions from Math…

We only know b up to a scalar.

So we can multiply it by -1…

58
Multiple Solutions from Math…

We can also rotate the

(second) camera by PI

59
Multiple Solutions from Math…

Or do both…

60
Four Possible Solutions from
Math…

61
One Solution from Physics…

The only physically plausible

solution that that the point are
in front of both cameras.
62
 Algebraic Solution
for Obtaining the Basis and Rotation
Matrix Given the Essential Matrix

63
Solution by Hartley & Zisserman
§ We know that

rotation
matrices

64
Solution by Hartley & Zisserman
§ We know that

rotation
matrices

§ Define
skew-sym. mat rotation mat

§ So that
65
Solution by Hartley & Zisserman

66
Solution by Hartley & Zisserman

67
Solution by Hartley & Zisserman

68
Four Possibilities to Define Z, W

69
Yields Four Solutions

2 solutions for SB 2 solutions for R

4 solutions

70
Solution by Hartley & Zisserman
§ Compute the SVD of E:
§ Normalize by
§ Compute the four solutions

§ Test for which solutions all points are

in front of both cameras
§ Return the physically plausible
configuration

71
Summary (1)
§ Algorithms to compute the relative
orientation from image data
§ Allow us to estimate the camera
motion (except of the scale)
§ Direct solutions
§ F from N>7 points (“8-Point Algorithm”)
§ E from N>7 points (“8-Point Algorithm”)
§ E from N=5 points (“Nister’s 5-Point
Algorithm”)

72
Summary (2)
§ Direct solutions
§ Extracting from
§ Not statistically optimal
§ Often used in combination with
RANSAC for identifying in/outliers
§ Direct solutions & RANSAC serves as
initial guess for iterative solutions
§ Subsequent refinement using least
squares only based on inlier points
73
Literature
§ Förstner, Wrobel: Photogrammetric
Computer Vision, Ch. 12.3.1-12.3.3
§ Hartley: In Defence of the 8-point
Algorithm
§ Stewenius, Engels, Nistér: Recent
Developments on Direct Relative
Orientation, ISPRS 2006

74
Slide Information
§ The slides have been created by Cyrill Stachniss as part of the
photogrammetry and robotics courses.
§ I tried to acknowledge all people from whom I used images or
videos. In case I made a mistake or missed someone, please
let me know.
§ The photogrammetry material heavily relies on the very well
written lecture notes by Wolfgang Förstner and the
Photogrammetric Computer Vision book by Förstner & Wrobel.
§ Parts of the robotics material stems from the great
Probabilistic Robotics book by Thrun, Burgard and Fox.
§ If you are a university lecturer, feel free to use the course
material. If you adapt the course material, please make sure
that you keep the acknowledgements to others and please
acknowledge me as well. To satisfy my own curiosity, please
send me email notice if you use my slides.

Cyrill Stachniss, cyrill.stachniss@igg.uni-bonn.de, 2014 75