Deploying MPLS Traffic

Engineering
Haroon Ahmad Malik
haroon@corvit.com

2001, Cisco Systems, Inc. All rights reserved.

What It Is, How It Works, and

How to Use It

2001, Cisco Systems, Inc. All rights reserved.

Agenda

How MPLS-TE Works

Basic Configuration
Knobs! Knobs! Knobs!
Deploying and Designing

2001, Cisco Systems, Inc. All rights reserved.

How MPLS-TE Works

How MPLS-TE works

What good is MPLS-TE?
Information distribution
Path calculation
Path setup
Forwarding traffic down a tunnel

2001, Cisco Systems, Inc. All rights reserved.

What Good Is MPLS-TE?

There are three kinds of networks

1. Those that have plenty of bandwidth everywhere
2. Those with congestion in some places, but not in
others
3. Those with constant congestion everywhere

The first kind always evolves into the second

kind!

2001, Cisco Systems, Inc. All rights reserved.

What Good Is MPLS-TE?

MPLS-TE introduces a 4th kind:
1. Those that have plenty of bandwidth everywhere
2. Those with congestion in some places, but not in others
3. Those with constant congestion everywhere
4. Those that use all of their bandwidth to its maximum efficiency,
regardless of shortest-path routing!

MPLS-TE can help turn #2 into #4

If you have #1, you probably dont need MPLS-TEyet
If you have #3, youre stuckyou either need more
bandwidth (or less traffic)

2001, Cisco Systems, Inc. All rights reserved.

What Good Is MPLS-TE?

What Is MPLS-TE?
Multi protocol
label switching
traffic engineering

What Is It Not?
Magic problem
solving labor
substitute which is
totally effortless

This Stuff Takes Work, but Its Worth It!!!

2001, Cisco Systems, Inc. All rights reserved.

Information Distribution

You need a link-state protocol as your IGP

IS-IS or OSPF

Link-state requirement is only for

MPLS-TE!
Not a requirement for VPNs, etc!

2001, Cisco Systems, Inc. All rights reserved.

Need for a Link-State Protocol

Why do I need a link-state protocol?

To make sure info gets flooded
To build a picture of the entire network

2001, Cisco Systems, Inc. All rights reserved.

The Problem with Shortest-Path

Some links are DS3, some
are OC-3
Node
B
C
D
E
F
G

Next-Hop
B
C
C
B
B
B

Cost
10
10
20
20
30
30

Router A has 40Mb of traffic for

Route F, 40Mb of traffic for
Router G
Changing to A->C->D->E
wont help

Massive (44%) packet loss at

Router B->Router E!
Router B

Router A

Router F

35M
b

OC-3
Dro
Router
E
ps!
DS3

OC-3
ffic
a
r
T
b
M
80

DS3

OC-3
Router C
2001, Cisco Systems, Inc. All rights reserved.

DS3

Router G

OC-3

Router D
10

What MPLS-TE Address

Router A sees all links
Node
B
C
D
E
F
G

Next-Hop
B
C
C
B
Tunnel 0
Tunnel 1

Cost
10
10
20
20
30
30

Router A computes paths

on properties other than
just shortest cost
No link oversubscribed!
Router B

Router A

Router F

OC-3

OC-3

Router E

DS3

b
M
0
4

40Mb

OC-3
Router C
2001, Cisco Systems, Inc. All rights reserved.

DS3

Router G

DS3

OC-3

Router D
11

How MPLS-TE Works

How MPLS-TE works

What good is MPLS-TE?
Information distribution
Path calculation
Path setup
Forwarding traffic down a tunnel

2001, Cisco Systems, Inc. All rights reserved.

12

Information Distribution

OSPF
Uses type 10 (opaque arealocal) LSAs

2001, Cisco Systems, Inc. All rights reserved.

13

How MPLS-TE Works

How MPLS-TE works

What good is MPLS-TE?
Information distribution
Path calculation
Path setup
Forwarding traffic down a tunnel

2001, Cisco Systems, Inc. All rights reserved.

14

Path Calculation

Modified Dijkstra at tunnel head-end

Often referred to as CSPF
Constrained SPF

or PCALC (path calculation)

2001, Cisco Systems, Inc. All rights reserved.

15

Path Calculation
Node
B
C
D
E
F
G

Next-Hop
B
C
C
B
Tunnel 0
Tunnel 1

PCALC takes bandwidth, other constraints

into account

Cost
10
10
20
20
30
30

Paths calculated, resources reserved if

necessary
End result: Bandwidth used more
efficiently!

Router B
Router A

Router F

OC-3

OC-3

Router E

DS3

b
M
0
4

40Mb

OC-3
Router C
2001, Cisco Systems, Inc. All rights reserved.

DS3

Router G

DS3

OC-3

Router D
16

Path Calculation
What if theres more than one path that meets the
minimum requirements (bandwidth, etc.)?
PCALC algorithm:
Find all paths with the lowest IGP cost
Then pick the path with the highest minimum available
bandwidth along the path
Then pick the path with the lowest hop count (not IGP cost,
but hop count)
Then just pick one path at random

2001, Cisco Systems, Inc. All rights reserved.

17

How MPLS-TE Works

How MPLS-TE works

What good is MPLS-TE?
Information distribution
Path calculation
Path setup
Forwarding traffic down a tunnel

2001, Cisco Systems, Inc. All rights reserved.

18

Path Setup

Cisco MPLS-TE uses RSVP

Once the path is calculated, it is handed
to RSVP
RSVP uses PATH and RESV messages to
request an LSP along the calculated path

2001, Cisco Systems, Inc. All rights reserved.

19

Path Setup
PATH message: Can I have 40Mb along this path?
RESV message: Yes, and heres the label to use
LFIB is set up along each hop
= PATH messages
= RESV messages
Router B

Router F
Router E

Router A

Router G

Router C
2001, Cisco Systems, Inc. All rights reserved.

Router D
20

How MPLS-TE Works

How MPLS-TE works

What good is MPLS-TE?
Information distribution
Path calculation
Path setup
Forwarding traffic down a tunnel

2001, Cisco Systems, Inc. All rights reserved.

21

Forwarding Traffic Down a Tunnel

There are three ways traffic can be forwarded
down a TE tunnel
Auto-route
Static routes
Policy routing

With the first two, MPLS-TE gets you unequal

cost load balancing

2001, Cisco Systems, Inc. All rights reserved.

22

Auto-Route

Auto-route = Use the tunnel as a

directly connected link for SPF
purposes
This is not the CSPF (for path
determination), but the regular IGP
SPF (route determination)

2001, Cisco Systems, Inc. All rights reserved.

23

Auto-Route

This Is the Physical Topology

Router B

Router F

Router H

Router E

Router A

Router G

Router C
2001, Cisco Systems, Inc. All rights reserved.

Router D

Router I
24

Auto-Route
This is Router As logical topology
By default, other routers dont see
the tunnel!
Router B

Router F

Router H

Router E

Router A

Router G

Tunnel1

Router C
2001, Cisco Systems, Inc. All rights reserved.

Router D

Router I
25

Auto-Route
Node
B
C
D
E
F
G
H
I

Next-Hop
B
C
C
B
B
Tunnel 1
Tunnel 1
Tunnel 1

Router As routing table, built

via
auto-route

Cost
10
10
20
20
30
30
40
40

Everything behind the

tunnel is routed via the tunnel
Router B

Router F

Router H

Router E

Router A

Router G

Tunnel1

Router C
2001, Cisco Systems, Inc. All rights reserved.

Router D

Router I
26

Unequal Cost Load Balancing

IP routing has equal-cost load balancing,

but not unequal cost*
MPLS-TE does unequal cost load
balancing, using 16 hash buckets for nexthop, shared in rough proportion to
configured tunnel bandwidth or load-share
value
*EIGRP Has Variance, but Thats Not As Flexible
2001, Cisco Systems, Inc. All rights reserved.

27

Unequal Cost: Example

Router F
Router A

40MB

Router E
Router G

20MB
gsr1#show ip route 192.168.1.8
Routing entry for 192.168.1.8/32
Known via "isis", distance 115, metric 83, type level-2
Redistributing via isis
Last update from 192.168.1.8 on Tunnel0, 00:00:21 ago
Routing Descriptor Blocks:
* 192.168.1.8, from 192.168.1.8, via Tunnel0
Route metric is 83, traffic share count is 2
192.168.1.8, from 192.168.1.8, via Tunnel1
Route metric is 83, traffic share count is 1
2001, Cisco Systems, Inc. All rights reserved.

28

Unequal Cost: Example

Router F
Router A

40MB

Router E
Router G

20MB
gsr1#sh ip cef 192.168.1.8 internal

Load distribution: 0 1 0 1 0 1 0 1 0 1 0 0 0 0 0 0 (refcount 1)

Hash OK Interface
Address
Packets Tags imposed
1
Y
Tunnel0
point2point
0
{23}
2
Y
Tunnel1
point2point
0
{34}

Note That the Load Distribution

Is 11:5Very Close to 2:1, but Not Quite!

2001, Cisco Systems, Inc. All rights reserved.

29

Static Routing

RtrA(config)#ip route H.H.H.H

255.255.255.255 Tunnel1
Router B

Router F

Router H

Router E

Router A

Router G

Router C
2001, Cisco Systems, Inc. All rights reserved.

Router D

Router I
30

Static Routing
Node
B
C
D
E
F
G
H
I

Next-Hop
B
C
C
B
B
B
Tunnel 1
B

Router H is known via

the tunnel

Cost
10
10
20
20
30
30
40
40

Router G is not routed to over

the tunnel, even though its the
tunnel tail!
Router B

Router F

Router H

Router E

Router A

Router G

Tunnel1

Router C
2001, Cisco Systems, Inc. All rights reserved.

Router D

Router I
31

Static Routing
Router F
Router A

40MB

Router E
Router G

20MB
gsr1(config)#ip route 1.2.3.4 255.255.255.255 192.168.1.11
gsr1#sh ip cef 1.2.3.4

Load distribution: 0 1 0 1 0 1 0 1 0 1 0 0 0 0 0 0 (refcount 1)

Hash OK Interface
Address
Packets Tags imposed
1
Y
Tunnel0
point2point
0
{23}
2
Y
Tunnel1
point2point
0
{34}

Static Routes Inherit Unequal Cost Load-Sharing

When Recursing through a Tunnel
2001, Cisco Systems, Inc. All rights reserved.

32

Policy Routing
RtrA(config-if)#ip policy route-map set-tunnel
RtrA(config)#route-map set-tunnel
RtrA(config-route-map)#match ip address 101
RtrA(config-route-map)#set interface Tunnel1
Router B

Router F

Router H

Router E

Router A

Router G

Tunnel1

Router C
2001, Cisco Systems, Inc. All rights reserved.

Router D

Router I
33

Policy Routing
Node
B
C
D
E
F
G
H
I

Next-Hop
B
C
C
B
B
B
B
B

Routing table isnt affected by

policy routing

Cost
10
10
20
20
30
30
40
40

Need (12.0(16)ST or 12.2T)

or higher for set interface
tunnel to work
Router B

Router F

Router H

Router E

Router A

Router G

Tunnel1

Router C
2001, Cisco Systems, Inc. All rights reserved.

Router D

Router I
34

Forwarding Traffic down a Tunnel

You can use any combination of auto-route,
static routes, or PBR
But simple is better unless you have a
good reason
Recommendation: Either auto-route or
statics to BGP next-hops, depending on
your needs

2001, Cisco Systems, Inc. All rights reserved.

35

Basic Midpoint/Tail Configuration

(globally)
ip cef {distributed}
mpls traffic-eng tunnels
(per interface)
mpls traffic-eng tunnels

2001, Cisco Systems, Inc. All rights reserved.

36

Basic Midpoint/Tail Configuration

(if IGP = OSPF)

router ospf <x>
mpls traffic-eng router-id
Loopback0
mpls traffic-eng area <y>

2001, Cisco Systems, Inc. All rights reserved.

37

Basic Head-End Configuration

Head-end needs the 45 mid/tail

lines
But waittheres more!

2001, Cisco Systems, Inc. All rights reserved.

38

Basic Head-End Configuration

Create the tunnel interface
interface Tunnel0
ip unnumbered Loopback0
tunnel mode mpls traffic-eng
tunnel source Loopback0
tunnel destination <tunnel endpoint>
tunnel mpls traffic-eng autoroute announce
tunnel mpls traffic-eng path-option 10 dynamic

2001, Cisco Systems, Inc. All rights reserved.

39

Basic Head-End Configuration

Total configuration:
1 line globally
1 line per interface
2 lines if OSPF
+ 7 lines per tunnel at head-end

2001, Cisco Systems, Inc. All rights reserved.

40

Bandwidth

ip rsvp bandwidth <x>

Per-interface command
X = amount of reservable BW, in K
Default: X=75% of link bandwidth
2001, Cisco Systems, Inc. All rights reserved.

41

Bandwidth

tunnel mpls traffic-eng

bandwidth <Kb>

Per-tunnel command
Tunnel default: 0 Kb

2001, Cisco Systems, Inc. All rights reserved.

42

Priority

tunnel mpls traffic-eng <S> {H}

Configured on tunnel interface
S = setup priority (07)
H = holding priority (07)
Lower number is more important, or better

2001, Cisco Systems, Inc. All rights reserved.

43

Administrative Weight
mpls traffic-eng administrativeweight <X>
Per-interface command
X = 04,294,967,295
Gives a metric that be considered for use instead
of the IGP metric
This can be used as a per-tunnel
delay-sensitive metric for doing VoIP TE

2001, Cisco Systems, Inc. All rights reserved.

44

Delay-Sensitive Metric with

Administrative Weight
tunnel mpls traffic-eng pathselection metric {te|igp}
Configure admin weight = interface delay
Configure VoIP tunnels to use TE metric to
calculate the path cost (see the PCALC
algorithm earlier in these slides)

2001, Cisco Systems, Inc. All rights reserved.

45

Attributes and Affinity

mpls traffic-eng attributeflags <0x0-0xFFFFFFFF>

Per-interface command

2001, Cisco Systems, Inc. All rights reserved.

46

Attributes and Affinity

tunnel mpls traffic-eng affinity
<0x0-0xFFFFFFFF> {mask <0x00xFFFFFFFF>}
Per-tunnel command
Mask is a collection of do-care bits
affinity 0x2 mask 0xAmeans I care
about bits 1 and 3 (with the values 2 and
8); bit 1 must be set, bit 3 must be 0
2001, Cisco Systems, Inc. All rights reserved.

47

Fast Reroute
In an IP network, a link failure causes several
seconds of outage
Thing

Dependency

Link Failure Detection

Media- and
Platform-specific

Information
Propagation

IGP Timers, Network

Size, Collective
Router Load

Route Recalculation

LSDB Size, CPU Load

2001, Cisco Systems, Inc. All rights reserved.

Time
~secs (POS + APS)
~530 sec
~13 sec

48

Fast Reroute
In an MPLS network, theres more work to be
done, so a (slightly) longer outage happens
Thing

Dependency

Link Failure Detection

Media- and
Platform-specific

Information
Propagation

IGP Timers, Network

Size, Collective
Router Load

Route Recalculation

LSDB Size, CPU Load

New LSP Setup

Network Size,
CPU Load

2001, Cisco Systems, Inc. All rights reserved.

Time
~Usecs (POS + APS)
~530 sec
~13 sec
~510 sec

49

Link Protection Through FRR

TE Tunnel A -> B -> D -> E

Router A

Router B

Router D

Router E

Router C
2001, Cisco Systems, Inc. All rights reserved.

50

Link Protection
B has a pre-provisioned backup tunnel to the other
end of the protected link (Router D)
B relies on the fact that D is using global label space
Router A

Router B

Router D

Router E

Router C
2001, Cisco Systems, Inc. All rights reserved.

51

Link Protection
B -> D link fails, A -> E tunnel is encapsulated in
B -> D tunnel
Backup tunnel is used until A can re-compute tunnel
path as A -> B -> C -> D -> E (1030 seconds or so)
Router A

Router B

Router D

Router E

Router C
2001, Cisco Systems, Inc. All rights reserved.

52

Link Protection
On tunnel head-end:
tunnel mpls traffic-eng fast-reroute

Router A

Router B

Router D

Router E

On protected link:
mpls traffic-eng backup-path <backup-tunnel>
2001, Cisco Systems, Inc. All rights reserved.

53

THANKS!