Chapter 24: Single-Source Shortest Paths

 Given: A single source vertex in a weighted, directed

graph.
 Want to compute a shortest path for each possible
destination.
 Similar to BFS.
 We will assume either
 no negative-weight edges, or
 no reachable negative-weight cycles.
 Algorithm will compute a shortest-path tree.
 Similar to BFS tree.

Jim Anderson Comp 122, Fall 2003 Single-source SPs -

 Outline
 General Lemmas and Theorems.
 CLRS now does this last. We’ll still do it first.
 Bellman-Ford algorithm.
 DAG algorithm.
 Dijkstra’s algorithm.
 We will skip Section 24.4.

Jim Anderson Comp 122, Fall 2003 Single-source SPs -

 General Results (Relaxation)
Lemma
Lemma24.1: 24.1:Let
Letpp==‹v ‹v11,,vv22,,…,
…,vvkk››be
beaaSP
SPfrom
fromvv11to
tovvkk.. Then,
Then,
ppijij==‹v
‹vi,i,vvi+1, …, v › is a SP from v to v , where 1  i  j  k.
i+1, …, vj j› is a SP from vi i to vj j, where 1  i  j  k.

So, we have the optimal-substructure property.

Bellman-Ford’s algorithm uses dynamic programming.
Dijkstra’s algorithm uses the greedy approach.
Let δ(u, v) = weight of SP from u to v.
p'
Corollary: Let p = SP from s to v, where p
Corollary: Let p = SP from s to v, where p = s= s   uuv.
v.Then,
Then,
δ(s,
δ(s,v)
v)==δ(s,
δ(s,u)
u) ++w(u,
w(u,v).
v).
Lemma
Lemma24.10:
24.10:Let
LetssV.
V. For
Forall
alledges
edges(u,v)
(u,v)E,
E,we
wehave
have
δ(s,
δ(s,v) δ(s,
v) δ(s,u)
u) ++w(u,v).
w(u,v).
Jim Anderson Comp 122, Fall 2003 Single-source SPs -
  Lemma 24.1 holds because one edge gives the
shortest path, so the other edges must give sums
that are at least as large.

Jim Anderson Comp 122, Fall 2003 Single-source SPs -

 Relaxation
Algorithms keep track of d[v], [v]. Initialized as follows:

Initialize(G,
Initialize(G,s)s)
for
foreach
eachvvV[G]
V[G]do
do
d[v]
d[v]:=
:=;;
[v]:=
[v] :=NIL
NIL
od;
od;
d[s]
d[s]:=
:=00
These values are changed when an edge (u, v) is relaxed:
Relax(u,
Relax(u,v,v,w)w)
ififd[v]
d[v]>>d[u]
d[u]++w(u,
w(u,v)
v)then
then
d[v]
d[v]:=
:=d[u]
d[u]++w(u,
w(u,v);
v);
[v]:=
[v] :=uu
fifi
Jim Anderson Comp 122, Fall 2003 Single-source SPs -
 Properties of Relaxation
 d[v], if not , is the length of some path from s
to v.
 d[v] either stays the same or decreases with time
 Therefore, if d[v] = (s, v) at any time, this
holds thereafter
 Note that d[v]  (s, v) always
 After i iterations of relaxing on all (u,v), if the
shortest path to v has i edges, then d[v] = (s, v).

Jim Anderson Comp 122, Fall 2003 Single-source SPs -

 Properties of Relaxation
Consider any algorithm in which d[v], and [v] are first initialized
by calling Initialize(G, s) [s is the source], and are only changed by
calling Relax. We have:
Lemma
Lemma24.11:
24.11:(
(v::
v::d[v]
d[v] (s,v))
(s, v))isisan
aninvariant.
invariant.
Implies d[v] doesn’t change once d[v] = (s, v).
Proof:
Initialize(G, s) establishes invariant. If call to Relax(u, v, w)
changes d[v], then it establishes:
d[v] = d[u] + w(u, v)
 (s, u) + w(u, v) , invariant holds before call.
 (s, v) , by Lemma 24.10.
Corollary
Corollary24.12:
24.12:IfIfthere
thereisisno
nopath
pathfrom
fromssto
tov,v,then
then
d[v]
d[v]==δ(s,
δ(s,v)
v)==
isisan
aninvariant.
invariant.
Jim Anderson Comp 122, Fall 2003 Single-source SPs -
  For lemma 24.11, note that initialization makes
the invariant true at the beginning.

Jim Anderson Comp 122, Fall 2003 Single-source SPs -

 More Properties
Lemma
Lemma24.13:
24.13:Immediately
Immediatelyafter
afterrelaxing
relaxingedge
edge(u,
(u,v)
v)by
bycalling
calling
Relax(u,
Relax(u,v,v,w),
w),we
wehave
haved[v] d[u]
d[v] d[u]++w(u,
w(u,v).
v).

p'
Lemma 24.14: Let p = SP from s to v, where
Lemma 24.14: Let p = SP from s to v, where p = s p = s   uuv.
v.
IfIfd[u]
d[u]==δ(s,
δ(s,u)
u)holds
holdsatatany
anytime
timeprior
priorto
tocalling
callingRelax(u,
Relax(u,v,v,w),
w),
then
thend[v]
d[v]==δ(s,
δ(s,v)
v) holds
holdsatatall
alltimes
timesafter
afterthe
thecall.
call.
Proof:
After the call we have:
d[v]  d[u] + w(u, v) , by Lemma 24.13.
= (s, u) + w(u, v) , d[u] = (s, u) holds.
= (s, v) , by corollary to Lemma 24.1.
By Lemma 24.11, d[v]  δ(s, v), so d[v] = δ(s, v).
Jim Anderson Comp 122, Fall 2003 Single-source SPs -
  Lemma 24.13 follows simply from the structure
of Relax.
 Lemma 24.14 shows that the shortest path will
be found one vertex at a time, if not faster.
Thus after a number of iterations of Relax equal
to V(G) - 1, all shortest paths will be found.

Jim Anderson Comp 122, Fall 2003 Single-source SPs - 1

  Bellman-Ford returns a compact representation
of the set of shortest paths from s to all other
vertices in the graph reachable from s. This is
contained in the predecessor subgraph.

Jim Anderson Comp 122, Fall 2003 Single-source SPs - 1

 Predecessor Subgraph
Lemma
Lemma24.16:
24.16:Assume
Assumegivengivengraph
graphGGhas hasno
nonegative-weight
negative-weightcycles
cycles
reachable
reachablefrom
froms.s. Let
LetGG==predecessor
predecessorsubgraph.
subgraph.GGisisalways
alwaysaa
tree
treewith
withroot
rootss(i.e.,
(i.e.,this
thisproperty
propertyisisan
aninvariant).
invariant).
Proof:
Two proof obligations:
(1) G is acyclic.
(2) There exists a unique path from source s to each vertex in V.

Proof of (1):
Suppose there exists a cycle c = ‹v0, v1, …, vk›, where v0 = vk.
We have [vi] = vi-1 for i = 1, 2, …, k.
Assume relaxation of (vk-1, vk) created the cycle.
We show cycle has a negative weight.
Note: Cycle must be reachable
Jim Anderson Compfrom s.2003(Why?)
122, Fall Single-source SPs - 1
 Proof of (1) (Continued)
Before call to Relax(vk-1, vk, w):
[vi] = vi-1 for i = 1, …, k–1.
Implies d[vi] was last updated by “d[vi] := d[vi-1] + w(vi-1, vi)”
for i = 1, …, k–1. [Because Relax updates .]
Implies d[vi]  d[vi-1] + w(vi-1, vi) for i = 1, …, k–1. [Lemma 24.13]

Because
k
[v
kk
] is changed by call, d[vk] > d[vk-1] + w(vk-1, vk). Thus,
 d[vi ]   (d[vi 1 ]  w(vi 1 , vi ))
i 1 i 1
k k
 d[vi  1 ]   w(v i  1 , v i )
i 1 i 1
k k k
Because  d[vi ]  d[v i  1 ],  w(v i 1 , v i )  0, i.e., neg. - weight cycle!
i 1 i 1 i 1
Jim Anderson Comp 122, Fall 2003 Single-source SPs - 1
 Comment on Proof

 d[vi]  d[vi-1] + w(vi-1, vi) for i = 1, …, k–1

because when Relax(vi-1 , vi , w) was called,
there was an equality, and d[vi-1] may have
gotten smaller by further calls to Relax.
 d[vk] > d[vk-1] + w(vk-1, vk) before the last call to
Relax because that last call changed d[v k].

Jim Anderson Comp 122, Fall 2003 Single-source SPs - 1

 Proof of (2)
Proof of (2):

( v: v  V:: ( path from s to v)) is an invariant.

So, for any v in V,  at least 1 path from s to v.

Show  1 path.
Assume 2 paths.
x

s u z v

y
impossible!
Jim Anderson Comp 122, Fall 2003 Single-source SPs - 1
 Lemma 24.17
Lemma
Lemma24.17:
24.17:Same Sameconditions
conditionsasasbefore.
before. Call
CallInitialize
Initialize& &repeatedly
repeatedly
call
callRelax
Relaxuntil
untild[v]d[v]==δ(s,
δ(s,v)
v)for
forall
allvvin
inV.
V. Then,
Then,GGisisaashortest-path
shortest-path
tree
treerooted
rootedatats.s.
Proof:
Key Proof Obligation: For all v in V, the unique simple path p from
s to v in G (path exists by Lemma 24.16) is a shortest path from s to v
in G.
Let p = ‹v0, v1, …, vk›, where v0 = s and vk = v.

We have d[vi] = δ(s, vi)

d[vi]  d[vi-1] + w(vi-1, vi) (reasoning as before)

Implies w(vi-1, vi)  δ(s, vi) – δ(s, vi-1).

Jim Anderson Comp 122, Fall 2003 Single-source SPs - 1
 Proof (Continued)
w(p)
k
 w(vi  1 , v i )
i 1
k
 (δ(s , v i )  δ(s , v i -1 ))
i 1

δ(s , v k )  δ(s , v 0 )
δ(s , v k )

So, equality holds and p is a shortest path.

Jim Anderson Comp 122, Fall 2003 Single-source SPs - 1

  And note that this shortest path tree will be
found after V(G) - 1 iterations of Relax.

Jim Anderson Comp 122, Fall 2003 Single-source SPs - 1

 Bellman-Ford Algorithm
Can have negative-weight edges. Will “detect” reachable negative-weight
cycles.
Initialize(G,
Initialize(G,s); s);
for
forii:= :=11toto|V[G]|
|V[G]|–1 –1do
do
for
foreach
each(u,(u,v)
v)in
inE[G]
E[G]do
do
Relax(u,
Relax(u,v,v,w)w)
od Time
od
od; Complexity
od;
for is O(VE).
foreach
each(u,(u,v)
v)ininE[G]
E[G]dodo
ififd[v]
d[v]>>d[u]
d[u]++w(u,
w(u,v)
v)then
then
return
returnfalse
false
fifi
od;
od;
return
returntrue true
Jim Anderson Comp 122, Fall 2003 Single-source SPs - 1
  So if Bellman-Ford has not converged after
V(G) - 1 iterations, then there cannot be a
shortest path tree, so there must be a negative
weight cycle.

Jim Anderson Comp 122, Fall 2003 Single-source SPs - 2

 Example
u v
5
 
–2
6 –3
8
z 0 7
–4
7 2

 
9
x y

Jim Anderson Comp 122, Fall 2003 Single-source SPs - 2

 Example
u v
5
6 
–2
6 –3
8
z 0 7
–4
7 2

7 
9
x y

Jim Anderson Comp 122, Fall 2003 Single-source SPs - 2

 Example
u v
5
6 4
–2
6 –3
8
z 0 7
–4
7 2

7 2
9
x y

Jim Anderson Comp 122, Fall 2003 Single-source SPs - 2

 Example
u v
5
2 4
–2
6 –3
8
z 0 7
–4
7 2

7 2
9
x y

Jim Anderson Comp 122, Fall 2003 Single-source SPs - 2

 Example
u v
5
2 4
–2
6 –3
8
z 0 7
–4
7 2

7 -2
9
x y

Jim Anderson Comp 122, Fall 2003 Single-source SPs - 2

 Another Look
Note: This is essentially dynamic programming.

Let d(i, j) = cost of the shortest path from s to i that is at most j hops.
0 if i = s  j = 0
 if i  s  j = 0
d(i, j) =
min({d(k, j–1) + w(k, i): i  Adj(k)}
 {d(i, j–1)}) if j > 0

i
z u v x y
1 2 3 4 5
j 0 0    
1 0 6  7 
2 0 6 4 7 2
3 0 2 4 7 2
4 0 2 4 7 –2
Jim Anderson Comp 122, Fall 2003 Single-source SPs - 2
 Lemma 24.2
Lemma
Lemma24.2:24.2:Assuming
Assumingno nonegative-weight
negative-weightcycles
cyclesreachable
reachablefrom
from
s,s,d[v]
d[v]==(s,
(s,v)
v)holds
holdsupon
upontermination
terminationfor
forall
allvertices
verticesvvreachable
reachable
from
froms.s.
Proof:
Consider a SP p, where p = ‹v0, v1, …, vk›, where v0 = s and vk = v.

Assume k  |V| – 1, otherwise p has a cycle.

Claim: d[vi] = (s, vi) holds after the ith pass over edges.
Proof follows by induction on i.

By Lemma 24.11, once d[vi] = (s, vi) holds, it continues to hold.

Jim Anderson Comp 122, Fall 2003 Single-source SPs - 2
 Correctness
Claim: Algorithm returns the correct value.
(Part of Theorem 24.4. Other parts of the theorem follow easily from earlier results.)

Case 1: There is no reachable negative-weight cycle.

Upon termination, we have for all (u, v):

d[v] = (s, v) , by lemma 24.2 (last slide) if v is reachable;
d[v] = (s, v) =  otherwise.
 (s, u) + w(u, v) , by Lemma 24.10.
= d[u] + w(u, v)

So, algorithm returns true.

Jim Anderson Comp 122, Fall 2003 Single-source SPs - 2

 Case 2
Case 2: There exists a reachable negative-weight cycle
c = ‹v0, v1, …, vk›, where v0 = vk.

We have i = 1, …, k w(vi-1, vi) < 0. (*)

Suppose algorithm returns true. Then, d[vi]  d[vi-1] + w(vi-1, vi) for
i = 1, …, k. (because Relax didn’t change any d[vi] ). Thus,

i = 1, …, k d[vi]  i = 1, …, k d[vi-1] + i = 1, …, k w(vi-1, vi)

But, i = 1, …, k d[vi] = i = 1, …, k d[vi-1].
Can show no d[vi] is infinite. Hence, 0  i = 1, …, k w(vi-1, vi).
Contradicts (*). Thus, algorithm returns false.
Jim Anderson Comp 122, Fall 2003 Single-source SPs - 2
 Shortest Paths in DAGs

Topologically
Topologicallysort
sortvertices
verticesininG;
G;
Initialize(G,
Initialize(G,s);
s);
for
foreach
eachuuin
inV[G]
V[G](in(inorder)
order)do
do
for
foreach
eachvvininAdj[u]
Adj[u]do do
Relax(u,
Relax(u,v,v,w)
w)
od
od
od
od

Jim Anderson Comp 122, Fall 2003 Single-source SPs - 3

 Example

6 1
r s t u v w
5 2 7 –1 –2
 0    

4
3
2

Jim Anderson Comp 122, Fall 2003 Single-source SPs - 3

 Example

6 1
r s t u v w
5 2 7 –1 –2
 0    

4
3
2

Jim Anderson Comp 122, Fall 2003 Single-source SPs - 3

 Example

6 1
r s t u v w
5 2 7 –1 –2
 0 2 6  

4
3
2

Jim Anderson Comp 122, Fall 2003 Single-source SPs - 3

 Example

6 1
r s t u v w
5 2 7 –1 –2
 0 2 6 6 4

4
3
2

Jim Anderson Comp 122, Fall 2003 Single-source SPs - 3

 Example

6 1
r s t u v w
5 2 7 –1 –2
 0 2 6 5 4

4
3
2

Jim Anderson Comp 122, Fall 2003 Single-source SPs - 3

 Example

6 1
r s t u v w
5 2 7 –1 –2
 0 2 6 5 3

4
3
2

Jim Anderson Comp 122, Fall 2003 Single-source SPs - 3

 Example

6 1
r s t u v w
5 2 7 –1 –2
 0 2 6 5 3

4
3
2

Jim Anderson Comp 122, Fall 2003 Single-source SPs - 3

 Dijkstra’s Algorithm
Assumes no negative-weight edges.
Maintains a set S of vertices whose SP from s has been determined.
Repeatedly selects u in V–S with minimum SP estimate (greedy choice).
Store V–S in priority queue Q.
Initialize(G,
Initialize(G,s);
s);
SS:=
:=;;
QQ:=:=V[G];
V[G];
while
whileQQ dodo
uu:=
:=Extract-Min(Q);
Extract-Min(Q);
SS:=
:=SS{u};
{u};
for
foreach
eachvvAdj[u]
Adj[u]do
do
Relax(u,
Relax(u,v,v,w)
w)
od
od
od
od
Jim Anderson Comp 122, Fall 2003 Single-source SPs - 3
 Example
u v
1
 

10
9
2 3
s 0 4 6

5 7

 
2
x y

Jim Anderson Comp 122, Fall 2003 Single-source SPs - 3

 Example
u v
1
10 

10
9
2 3
s 0 4 6

5 7

5 
2
x y

Jim Anderson Comp 122, Fall 2003 Single-source SPs - 4

 Example
u v
1
8 14

10
9
2 3
s 0 4 6

5 7

5 7
2
x y

Jim Anderson Comp 122, Fall 2003 Single-source SPs - 4

 Example
u v
1
8 13

10
9
2 3
s 0 4 6

5 7

5 7
2
x y

Jim Anderson Comp 122, Fall 2003 Single-source SPs - 4

 Example
u v
1
8 9

10
9
2 3
s 0 4 6

5 7

5 7
2
x y

Jim Anderson Comp 122, Fall 2003 Single-source SPs - 4

 Example
u v
1
8 9

10
9
2 3
s 0 4 6

5 7

5 7
2
x y

Jim Anderson Comp 122, Fall 2003 Single-source SPs - 4

 Correctness
Theorem
Theorem24.6:
24.6:Upon
Upontermination,
termination,d[u]
d[u]==δ(s,
δ(s,u)
u)for
forall
alluuin
inVV
(assuming
(assumingnon-negative
non-negativeweights).
weights).

Proof:
By Lemma 24.11, once d[u] = δ(s, u) holds, it continues to hold.
We prove: For each u in V, d[u] = (s, u) when u is inserted in S.
Suppose not. Let u be the first vertex such that d[u]  (s, u) when
inserted in S.
Note that d[s] = (s, s) = 0 when s is inserted, so u  s.
 S   just before u is inserted (in fact, s  S).

Jim Anderson Comp 122, Fall 2003 Single-source SPs - 4

 Proof (Continued)
Note that there exists a path from s to u, for otherwise
d[u] = (s, u) =  by Corollary 24.12.
 there exists a SP from s to u. SP looks like this:

p2

u
s
p1

x y
S

Jim Anderson Comp 122, Fall 2003 Single-source SPs - 4

 Proof (Continued)
Claim: d[y] = (s, y) when u is inserted into S.
We had d[x] = (s, x) when x was inserted into S.
Edge (x, y) was relaxed at that time.
By Lemma 24.14, this implies the claim.

Now, we have: d[y] = (s, y) , by Claim.

 (s, u) , nonnegative edge weights.
 d[u] , by Lemma 24.11.

Because u was added to S before y, d[u]  d[y].

Thus, d[y] = (s, y) = (s, u) = d[u].
Contradiction.
Jim Anderson Comp 122, Fall 2003 Single-source SPs - 4
 Complexity

Running time is
O(V2) using linear array for priority queue.
O((V + E) lg V) using binary heap.
O(V lg V + E) using Fibonacci heap.

(See book.)

Jim Anderson Comp 122, Fall 2003 Single-source SPs - 4