QoS Best Practices

Tim Szigeti
Technical Marketing Engineer
Technology and Systems Marketing: QoS
Cisco Central Development Organization
10/5/04

IP07 QoS © 2004 Cisco Systems, Inc. All rights reserved. 1

 QoS Perception
Changing the Way Intelligent Services Are Enabled

Necessity Luxury

Security
Quality of
Service

High Availability

IP07 QoS © 2004 Cisco Systems, Inc. All rights reserved. 2

 QoS Deployment Principles

IP07 QoS © 2004 Cisco Systems, Inc. All rights reserved. 3

 How is QoS Optimally Deployed in the
Enterprise?

1) Strategically define the business objectives to be achieved

via QoS.
2) Analyze the service-level requirements of the various traffic
classes to be provisioned for.
3) Design and test the QoS policies prior to production-network
rollout.
4) Roll-out the tested QoS designs to the production-network in
phases, during scheduled downtime.
5) Monitor service levels to ensure that the QoS objectives are
being met.

IP07 QoS © 2004 Cisco Systems, Inc. All rights reserved. 4

 General QoS Design Principles
Start with the Objectives: Not the Tools

• Clearly define the organizational objectives

Protect voice? video? data? DoS/worm mitigation?

• Assign as few applications as possible to be

treated as “mission-critical”
• Seek executive endorsement of the QoS objectives
prior to design and deployment
• Determine how many classes of traffic are required
to meet the organizational objectives
More classes = more granular service-guarantees

IP07 QoS © 2004 Cisco Systems, Inc. All rights reserved. 5

 How Many Classes of Service Do I Need?
Example Strategy for Expanding the Number of Classes of Service over Time

4/5 Class Model 8 Class Model QoS Baseline Model

Voice Voice
Realtime Interactive-Video
Video Streaming Video
Call Signaling Call Signaling Call Signaling
IP Routing
Network Control
Network Management

Critical Data Mission-Critical Data

Critical Data
Transactional Data
Bulk Data Bulk Data

Best Effort Best Effort Best Effort

Scavenger Scavenger Scavenger

IP07 QoS
Time © 2004 Cisco Systems, Inc. All rights reserved. 6
 QOS REQUIREMENTS OF
VOICE, VIDEO, AND DATA

NMS-2T30
9681_05_2004_c2 © 2004 Cisco Systems, Inc. All rights reserved. 7
 Voice QoS Requirements
End-to-End Latency

Hello? Hello?
Avoid the
“Human Ethernet”

CB Zone
Satellite Quality
High Quality Fax Relay, Broadcast

0 100 200 300 400 500 600 700 800

Time (msec)
Delay Target

ITU’s G.114 Recommendation: ≤ 150msec One-Way Delay

IP07 QoS © 2004 Cisco Systems, Inc. All rights reserved. 8

 Voice QoS Requirements
Elements That Affect Latency and Jitter

PSTN

IP WAN

Campus Branch Office

Propagation
CODEC Queuing Serialization and Network Jitter Buffer

Fixed
(6.3 µs/Km) +
G.729A: 25 ms Variable Variable Network Delay 20–50 ms
(Variable)

End-to-End Delay (Must Be ≤ 150 ms)

IP07 QoS © 2004 Cisco Systems, Inc. All rights reserved. 9
 Voice QoS Requirements
Packet Loss Limitations

Voice Voice Voice Voice Voice Voice Voice Voice

4 3 2 1 4 3 2 1
Voice
3

Voice
Reconstructed Voice Sample
3

• Cisco DSP codecs can use predictor algorithms to

compensate for a single lost packet in a row
• Two lost packets in a row will cause an audible clip
in the conversation

IP07 QoS © 2004 Cisco Systems, Inc. All rights reserved. 10

 Voice QoS Requirements
Call Admission Control (CAC): Why Is It Needed?

Circuit-Switched Packet-Switched
Networks Networks

IP WAN/VPN
IP VPN Link Provisioned
PSTN
for 2 VoIP Calls

Physical
Trunks
Third Call Cisco No Physical
Rejected Router/ Call
Gateway Manager Limitation on IP Links
PBX STOP
If 3rd Call Accepted,
Voice Quality of All
Calls Degrades

CAC Limits Number of VoIP Calls on Each VPN Link

IP07 QoS © 2004 Cisco Systems, Inc. All rights reserved. 11
 Video QoS Requirements
Video Conferencing Traffic Example (384 kbps)

“I” Frame “I” Frame

1024–1518 1024–1518
Bytes Bytes
450Kbps

30pps
“P” and “B” Frames
128–256 Bytes
15pps
32Kbps

• “I” frame is a full sample of the video

• “P” and “B” frames use quantization via motion vectors
and prediction algorithms
IP07 QoS © 2004 Cisco Systems, Inc. All rights reserved. 12
 Video QoS Requirements
Video Conferencing Traffic Packet Size Breakdown

1025–1500 Bytes
37% 65–128 Bytes
1%

129–256 Bytes
513–1024 Bytes 34%
20%

257–512 Bytes
8%
IP07 QoS © 2004 Cisco Systems, Inc. All rights reserved. 13
 Data QoS Requirements
Application Differences

Oracle SAP R/3

0–64 Bytes 1024–1518

65–127 Bytes Bytes
128–252 Bytes
512–1023
Bytes 0–64
253–511 Bytes
Bytes

512–1023 253–511
Bytes Bytes
1024–1518
Bytes

128–252 65–127
Bytes Bytes

IP07 QoS © 2004 Cisco Systems, Inc. All rights reserved. 14

 Data QoS Requirements
Version Differences

SAP Sales Order

500,000
Entry Transaction
400,000
VA01
Client Version # of
Bytes 300,000

SAP GUI Release 3.0 F 14,000

200,000
SAP GUI Release 4.6C, No Cache 57,000

SAP GUI Release 4.6C, with Cache 33,000 100,000

SAP GUI for HTML, Release 4.6C 490,000

0
SAP GUI, SAP GUI, SAP GUI, SAP GUI
Release Release Release (HTML),
3.0F 4.6C, with 4.6C, no Release
Cache Cache 4.6C

• Same transaction takes over 35 times more traffic

from one version of an application to another
IP07 QoS © 2004 Cisco Systems, Inc. All rights reserved. 15
 OVERVIEW OF
DOS/WORM ATTACKS

NMS-2T30
9681_05_2004_c2 © 2004 Cisco Systems, Inc. All rights reserved. 16
 Business Security Threat Evolution
Expanding Scope of Theft and Disruption

Global
Impact
Scope of Damage

Regional
Networks Next Gen
Infrastructure
Hacking, Flash
Multiple
Networks 3rd Gen Threats,
Multi-Server Massive Worm
2nd Gen DoS, DDoS, Driven DDoS,
Individual Blended Threat Negative
Macro Viruses, (Worm+ Virus+
Networks Trojans, Email,
Payload Viruses,
Trojan), Turbo Worms and
Single Server Worms, Trojans
Individual 1st Gen DoS, Limited Widespread
Boot Viruses Targeted System
Computer
Hacking Hacking

1980’s 1990’s Today Future

Sophistication of Threats
IP07 QoS © 2004 Cisco Systems, Inc. All rights reserved. 17
 Emerging Speed of Network Attacks
Do You Have Time To React?

1980s-1990s 2000-2002 2003-Future

Usually had Weeks Attacks Progressed Attacks Progress on the
or Months to Put Defense Over Hours, Time Timeline of Seconds
in Place to Assess Danger and Impact;
Time to Implement Defense SQL Slammer Worm:
Doubled Every 8.5 Seconds
After 3 Min: 55M Scans/Sec
In Half the Time It Took to Read 1Gb Link Is Saturated After
One Minute
This Slide, Your Network
SQL Slammer Was A Warning,
and All of Your Applications Would Newer “Flash” Worms Are
Have Become Unreachable Exponentially Faster

IP07 QoS © 2004 Cisco Systems, Inc. All rights reserved. 18

 “Slammer” or the Sapphire Worm
Infected 75,000 Hosts in First 11 Minutes

• Infections doubled every 8.5 seconds

• Infected 75,000 hosts in first 11 minutes
• Caused network outages, cancelled airline
flights and ATM failures

At Peak, Scanned 55 Million

Hosts per Second

11 Minutes after Release

11 8 6 2 0
IP07 QoS © 2004 Cisco Systems, Inc. All rights reserved. 19
 Internet Worms
By the Time You Read This Slide It Will Be Out of Date

W32/ W32/
Apache/ MS-SQL Blaster MyDoom
sadmind/IIS Code Red NIMDA mod_ssl Slammer W32/Sobig W32/Bagel
Sasser

May ’01 May ’01 Sep ’01 Jul ’02 Jan ’03 Aug ’03 Jan ’04 April ’04

• More than 994 new Win32 viruses and worms were

documented in the first half of 2003, more than double
the 445 documented in the first half of 2002

http://www.symantec.com/press/2003/n031001.html
IP07 QoS © 2004 Cisco Systems, Inc. All rights reserved. 20
 Types of DoS Attacks
Spoofing vs. Slamming

• Imposter attack
Pretends to be a legitimate service but maliciously
intercepts/misdirects client requests

• Flooding attack
Exponentially generates and propagates traffic
until service resources (servers and/or network)
are overwhelmed

IP07 QoS © 2004 Cisco Systems, Inc. All rights reserved. 21

 Impact of an Internet Worm
Anatomy of a Worm: Why It Hurts

1—The Enabling
Vulnerability

2—Propagation
Mechanism

3—Payload

IP07 QoS © 2004 Cisco Systems, Inc. All rights reserved. 22

 Impact of an Internet Worm
Direct and Collateral Damage

Si
System
Under Attack
Si Si Infected
Source

Core
Si

Routers
Distribution Overloaded
Access Network Links High CPU
End Systems Overloaded Instability
Overloaded High Packet Loss Loss of Mgmt
High CPU Mission Critical
Applications Applications Impacted
Impacted

Attacks Targeted to End Systems CAN and DO

Affect the Infrastructure
IP07 QoS © 2004 Cisco Systems, Inc. All rights reserved. 23
 QoS Technologies Review

IP07 QoS © 2004 Cisco Systems, Inc. All rights reserved. 24

 QoS Technologies Review

• QoS Overview
• Classification Tools
• Scheduling Tools
• Policing and Shaping Tools
• Link-Specific Tools

IP07 QoS © 2004 Cisco Systems, Inc. All rights reserved. 25

 QoS Factors
Attributes Requiring Explicit Service Levels

Delay- Packet
Delay
(Latency) Variation Loss
(Jitter)

IP07 QoS © 2004 Cisco Systems, Inc. All rights reserved. 26

 Quality of Service Operations
How Do QoS Tools Work?

QUEUEING AND SHAPING/COMPRESSION/

CLASSIFICATION AND MARKING (SELECTIVE) DROPPING FRAGMENTATION/INTERLEAVE

IP07 QoS © 2004 Cisco Systems, Inc. All rights reserved. 27

 Classification Tools
Ethernet 802.1Q Class of Service

TAG
Pream. SFD DA SA Type PT Data FCS
4 Bytes
Ethernet Frame
Three Bits Used for CoS
(802.1p User Priority)
PRI CFI VLAN ID 802.1Q/p
Header
CoS Application
7 Reserved
• 802.1p user priority field also
6 Routing
called Class of Service (CoS)
5 Voice
• Different types of traffic are
4 Video
assigned different CoS values
3 Call Signaling
• CoS 6 and 7 are reserved for
network use 2 Critical Data
1 Bulk Data
0 Best Effort Data
IP07 QoS © 2004 Cisco Systems, Inc. All rights reserved. 28
 Classification Tools
IP Precedence and DiffServ Code Points

Version ToS
Byte Len ID Offset TTL Proto FCS IP SA IP DA Data
Length
IPv4 Packet

7 6 5 4 3 2 1 0
Standard IPv4
IP Precedence Unused
DiffServ Code Point (DSCP) IP ECN DiffServ Extensions

• IPv4: Three most significant bits of ToS byte are called IP

Precedence (IPP)—other bits unused
• DiffServ: Six most significant bits of ToS byte are called
DiffServ Code Point (DSCP)—remaining two bits used for
flow control
• DSCP is backward-compatible with IP precedence
IP07 QoS © 2004 Cisco Systems, Inc. All rights reserved. 29
 Classification Tools
DSCP Per-Hop Behaviors

• IETF RFCs have defined special keywords, called Per-Hop

Behaviors, for specific DSCP markings
• EF: Expedited Forwarding (RFC3246, formerly RFC2598)
(DSCP 46)
• CSx: Class Selector (RFC2474)
Where x corresponds to the IP Precedence value (1-7)
(DSCP 8, 16, 24, 32, 40, 48, 56)
• AFxy: Assured Forwarding (RFC2597)
Where x corresponds to the IP Precedence value
(only 1-4 are used for AF Classes)
And y corresponds to the Drop Preference value (either 1 or 2 or 3)
With the higher values denoting higher likelihood of dropping
(DSCP 10/12/14, 18/20/22, 26/28/30, 34/36/38)
• BE: Best Effort or Default Marking Value (RFC2474)
(DSCP 0)

IP07 QoS © 2004 Cisco Systems, Inc. All rights reserved. 30

 Classification Tools
Network-Based Application Recognition

TCP/UDP
Frame Segment Data Payload
IP Packet
ToS/ Source Dest Src Dst
NBAR PDLM DATA
DSCP IP IP Port Port

MAC/CoS
DE/CLP/MPLS EV 98 Supported Protocols

citrix http nntp ssh streamwork

cuseeme imap notes smtp syslog
custom irc novadigm snmp telnet
exchange kerberos pcanywhere socks secure-telnet
fasttrack ldap pop3 sqlserver tftp
ftp napster realaudio sqlnet vdolive
gnutella netshow rcmd sunrpc xwindows

IP07 QoS © 2004 Cisco Systems, Inc. All rights reserved. 31

 Policing Tools
RFC 2697 Single Rate Three Color Policer

Overflow
CIR

CBS EBS

No No
B<Tc B<Te

Packet of Yes Yes

Size B
Conform Exceed Violate

Action Action Action

IP07 QoS © 2004 Cisco Systems, Inc. All rights reserved. 32

 Policing Tools
RFC 2698 Two Rate Three Color Policer

PIR CIR

PBS CBS

No No
B<Tp B<Tc

Packet of Yes Yes

Size B
Violate Exceed Conform

Action Action Action

IP07 QoS © 2004 Cisco Systems, Inc. All rights reserved. 33

 Scheduling Tools
Queuing Algorithms

Voice 1 1 1 11
1 1 1 1 1 1 1 1 1 1 1 1 1
Video 1 1 1 1 2 12

3
1 1 1 1 1 1 13
Data

• Congestion can occur at any point in the network where there

are speed mismatches
• Routers use Cisco IOS-based software queuing
Low-Latency Queuing (LLQ) used for highest-priority traffic (voice/video)
Class-Based Weighted-Fair Queuing (CBWFQ) used for guaranteeing
bandwidth to data applications
• Cisco Catalyst® switches use hardware queuing

IP07 QoS © 2004 Cisco Systems, Inc. All rights reserved. 34

 Scheduling Tools
TCP Global Synchronization: The Need for Congestion Avoidance

All TCP Flows Synchronize in

Waves Wasting Much of the
Available Bandwidth

Bandwidth
100% Utilization

Time

Tail Drop

3 Traffic Flows Start Another Traffic Flow

at Different Times Starts at This Point
IP07 QoS © 2004 Cisco Systems, Inc. All rights reserved. 35
 Scheduling Tools
Congestion Avoidance Algorithms

TAIL
WREDDROP
Queue

3 3
1 01 2 1 2 0 2 0 3 2 1 3

0
3

0
3

• Queueing algorithms manage the front of the queue

i.e. which packets get transmitted first
• Congestion avoidance algorithms, like Weighted-Random
Early-Detect (WRED), manage the tail of the queue
i.e. which packets get dropped first when queuing buffers fill
• WRED can operate in a DiffServ compliant mode which will
drop packets according to their DSCP markings
• WRED works best with TCP-based applications, like data
IP07 QoS © 2004 Cisco Systems, Inc. All rights reserved. 36
 Scheduling Tools
DSCP-Based WRED Operation

Drop All Drop All Drop All

Drop AF13 AF12 AF11
Probability

100%

50%

Average
0 Queue
Begin Begin Begin Size
Dropping Dropping Dropping
AF13 AF12 AF11 Max Queue
Length
(Tail Drop)

AF = (RFC 2597) Assured Forwarding

IP07 QoS © 2004 Cisco Systems, Inc. All rights reserved. 37
 Congestion Avoidance Tools
IP ToS Byte Explicit Congestion Notification (ECN) Bits

Version ToS
Byte Len ID Offset TTL Proto FCS IP SA IP DA Data
Length
IPv4 Packet

7 6 5 4 3 2 1 0
DiffServ Code Point (DSCP) ECT CE

ECT Bit: CE Bit:

ECN-Capable Transport Congestion Experienced

RFC3168: IP Explicit Congestion Notification

IP07 QoS © 2004 Cisco Systems, Inc. All rights reserved. 38

 Shaping Tools
Traffic Shaping

Without Traffic Shaping

Line
Rate
With Traffic Shaping
Shaped
Rate

Traffic Shaping Limits the Transmit Rate to a Value Lower than Line Rate

• Policers typically drop traffic

• Shapers typically delay excess traffic, smoothing bursts
and preventing unnecessary drops
• Very common on Non-Broadcast Multiple-Access (NBMA)
network topologies such as Frame-Relay and ATM

IP07 QoS © 2004 Cisco Systems, Inc. All rights reserved. 39

 Link-Specific Tools
Link-Fragmentation and Interleaving

Serialization Voice Data

Can Cause
Excessive Delay
Data Data Data Voice Data

With Fragmentation and Interleaving Serialization Delay Is Minimized

• Serialization delay is the finite amount of time required to

put frames on a wire
• For links ≤ 768 kbps serialization delay is a major factor
affecting latency and jitter
• For such slow links, large data packets need to be fragmented
and interleaved with smaller, more urgent voice packets

IP07 QoS © 2004 Cisco Systems, Inc. All rights reserved. 40

 Link-Specific Tools
IP RTP Header Compression

IP Header UDP Header RTP Header Voice

Voice
20 Bytes 8 Bytes 12 Bytes Payload
Payload

cRTP Reduces L3 VoIP BW by:

~ 20% for G.711 2-5 Bytes

~ 60% for G.729

IP07 QoS © 2004 Cisco Systems, Inc. All rights reserved. 41

 QOS DESIGN PRINCIPLES
AND STRATEGIES

NMS-2T30
9681_05_2004_c2 © 2004 Cisco Systems, Inc. All rights reserved. 42
 Voice QoS Requirements
Provisioning for Voice

• Latency ≤ 150 ms
One-Way
Voice
• Jitter ≤ 30 ms Requirements

• Loss ≤ 1%
• 17–106 kbps guaranteed
priority bandwidth per call
• 150 bps (+ Layer 2 overhead) • Smooth
guaranteed bandwidth for
• Benign
Voice-Control traffic per call
• Drop sensitive
• CAC must be enabled
• Delay sensitive
• UDP priority

IP07 QoS © 2004 Cisco Systems, Inc. All rights reserved. 43

 Video QoS Requirements
Provisioning for Interactive Video

• Latency ≤ 150 ms
One-Way
Video
• Jitter ≤ 30 ms Requirements

• Loss ≤ 1%
• Minimum priority bandwidth
guarantee required is:
Video-stream + 20%
• Bursty
e.g. a 384 kbps stream would
require 460 kbps of priority • Greedy
bandwidth • Drop sensitive
• CAC must be enabled • Delay sensitive
• UDP priority

IP07 QoS © 2004 Cisco Systems, Inc. All rights reserved. 44

 Data QoS Requirements
Provisioning for Data

• Different applications have

different traffic characteristics
Data
• Different versions of the same
application can have different
traffic characteristics
• Classify data into four/five
data classes model: • Smooth/bursty
Mission-critical apps • Benign/greedy
Transactional/interactive apps • Drop insensitive
Bulk data apps
• Delay insensitive
Best effort apps
• TCP retransmits
Optional: Scavenger apps

IP07 QoS © 2004 Cisco Systems, Inc. All rights reserved. 45

 Data QoS Requirements
Provisioning for Data (Cont.)

• Use four/five main traffic classes:

Mission-critical apps—business-critical client-server applications
Transactional/interactive apps—foreground apps: client-server
apps or interactive applications
Bulk data apps—background apps: FTP, e-mail, backups,
content distribution
Best effort apps—(default class)
Optional: Scavenger apps—peer-to-peer apps, gaming traffic
• Additional optional data classes include internetwork-
control (routing) and network-management
• Most apps fall under best-effort, make sure that
adequate bandwidth is provisioned for this default class

IP07 QoS © 2004 Cisco Systems, Inc. All rights reserved. 46

 Scavenger-Class QoS DoS/Worm Mitigation Strategy
What Is the Scavenger Class?

• The Scavenger class is an Internet 2 Draft Specification

for a “less-than best effort” service
• There is an implied “good faith” commitment for the
“best effort” traffic class
It is generally assumed that at least some network resources
will be available for the default class

• Scavenger class markings can be used to distinguish

out-of-profile/abnormal traffic flows from in-
profile/normal flows
The Scavenger class marking is DSCP CS1 (8)

• Scavenger traffic is assigned a “less-than best effort”

queuing treatment whenever congestion occurs

IP07 QoS © 2004 Cisco Systems, Inc. All rights reserved. 47

 Scavenger-Class QoS DoS/Worm Mitigation Strategy
First Order Anomaly Detection

• All end systems generate traffic spikes

• Sustained traffic loads beyond ‘normal’ from each source
device are considered suspect and marked as scavenger
(DSCP CS1)
• No dropping at campus access-edge, only remarking
Police

Excess Traffic Is Remarked to Scavenger (DSCP CS1)

Normal/Abnormal Threshold

IP07 QoS © 2004 Cisco Systems, Inc. All rights reserved. 48

 Scavenger-Class QoS DoS/Worm Mitigation Strategy
Second Order Anomaly Reaction

• During ‘abnormal’ worm traffic conditions traffic, where

multiple infected hosts are causing uplink congestion,
suspect traffic—previously marked as Scavenger—is
aggressively dropped
• Stations not generating abnormal traffic volumes continue
to receive network service

Police

Throttle Scavenger
(when Congested)

IP07 QoS © 2004 Cisco Systems, Inc. All rights reserved. 49

 Scavenger-Class QoS DoS/Worm Mitigation Strategy
Preventing and Limiting the Pain

System Si
Under
Si
Attack Si
Infected
Source
Core
Si

Prevent the Attack

Distribution Cisco Guard
Firewall
Access
ACLs & NBAR
Protect the End
Systems Protect the Links Protect the Switches
Cisco Security Agent QoS CEF
Scavenger Class Rate Limiters

An Integrated Network Architecture Holistically Combines

High Availability, Quality of Service and Security
Technologies to Prevent and Limit Attacks
IP07 QoS © 2004 Cisco Systems, Inc. All rights reserved. 50
 Classification and Marking Design Principles
Where and How Should Marking Be Done?

• QoS policies (in general) should always be

performed in hardware, rather than software,
whenever a choice exists
• Classify and mark applications as close to their
sources as technically and administratively feasible
• Use DSCP markings whenever possible
• Follow standards-based DSCP PHBs to ensure
interoperation and future expansion
RFC 2474 class selector code points
RFC 2597 assured forwarding classes
RFC 3246 expedited forwarding
IP07 QoS © 2004 Cisco Systems, Inc. All rights reserved. 51
 Classification and Marking
QoS Baseline/AIT Marking Recommendations

L3 Classification L2
Application
IPP PHB DSCP CoS
Routing 6 CS6 48 6
Voice 5 EF 46 5
Video Conferencing 4 AF41 34 4
Streaming Video 4 CS4 32 4
Mission-Critical Data 3 - 25 3
Call Signaling 3 AF31 Î CS3* 26 Î 24 3

Transactional Data 2 AF21 18 2

Network Management 2 CS2 16 2

Bulk Data 1 AF11 10 1
Scavenger 1 CS1 8 1
Best Effort 0 0 0 0

IP07 QoS © 2004 Cisco Systems, Inc. All rights reserved. 52

 Policing Design Principles
Where and How Should Policing Be Done?

• Police traffic flows as close to their sources as

possible
• Perform markdown according to standards-based
rules, whenever supported
RFC 2597 specifies how assured forwarding traffic classes
should be marked down (AF11 Î AF12 Î AF13) which
should be done whenever DSCP-based WRED is supported
on egress queues
Cisco Catalyst platforms currently do not support DSCP-
based WRED, so Scavenger-class remarking is a viable
alternative
Additionally, non-AF classes do not have a standards-
based markdown scheme, so Scavenger-class remarking
is a viable option
IP07 QoS © 2004 Cisco Systems, Inc. All rights reserved. 53
 DoS/Worm Mitigation Design Principles
How Can QoS Tools Contain Attacks?

• Profile applications to determine what constitutes “normal”

vs. “abnormal” flows (within a 95% confidence interval)
• Deploy campus access-edge policers to remark abnormal
traffic to Scavenger
DSCP CS1 (8)
• Deploy a second-line of defense at the Distribution-Layer via
per-user microflow policing
Cisco Catalyst 6500 Sup720 (PFC3) only
• Provision end-to-end “less-than-Best-Effort” Scavenger-class
queuing policies
Campus + WAN + VPN
• Police-to-drop known worms/variants via NBAR on branch
routers

IP07 QoS © 2004 Cisco Systems, Inc. All rights reserved. 54

 Queuing Design Principles
Where and How Should Queuing Be Done?

• The only way to provide service GUARANTEES is to enable

queuing at any node that has the potential for congestion
Regardless of how rarely—in fact—this may occur
• At least 25 percent of a link’s bandwidth should be reserved
for the default Best Effort class
• Limit the amount of strict-priority queuing to 33 percent of a
link’s capacity
• Whenever a Scavenger queuing class is enabled, it should be
assigned a minimal amount of bandwidth
• To ensure consistent PHBs, configure consistent queuing
policies in the Campus + WAN + VPN, according to platform
capabilities
• Enable WRED on all TCP flows, whenever supported
Preferably DSCP-based WRED

IP07 QoS © 2004 Cisco Systems, Inc. All rights reserved. 55

 Campus Queuing Design
Realtime, Best Effort and Scavenger Queuing Rules

Best Effort
≥ 25% Real-Time
≤ 33%

Scavenger/Bulk
≤ 5%

Critical Data

IP07 QoS © 2004 Cisco Systems, Inc. All rights reserved. 56

 Campus and WAN/VPN Queuing Design
Compatible Four-Class and Eleven-Class Queuing Models
Following Realtime, Best Effort and Scavenger Queuing Rules

Voice 18%
Best Effort
25%

Best Effort
Scavenger ≥ 25% Real-Time
1% ≤ 33% Interactive Video
Scavenger/ 15%
Bulk 5%

Bulk 4% Critical Data

Streaming-Video Internetwork-
Control
Network Management Call-Signaling

Transactional Data Mission-Critical Data

IP07 QoS © 2004 Cisco Systems, Inc. All rights reserved. 57
 LAN/WAN/VPN QoS Design Overview

IP07 QoS © 2004 Cisco Systems, Inc. All rights reserved. 58

 Campus QoS Considerations
Where Is QoS Required Within the Campus?

FastEthernet No Trust + Policing +

Queuing
GigabitEthernet
Conditional Trust +
TenGigabitEthernet Policing + Queuing
Trust DSCP + Queuing
Per-User Microflow
Policing

Catalyst 6500 Sup720

WAN Aggregator

Server Farms IP Phones + PCs IP Phones + PCs

IP07 QoS © 2004 Cisco Systems, Inc. All rights reserved. 59

 WAN Edge QoS Design Considerations
QoS Requirements of WAN Aggregators

Campus
Distribution/Core
Queuing/Dropping/
Switches Shaping/Link-Efficiency Policies
for Campus-to-Branch Traffic

WAN Aggregator

WAN

LAN Edges WAN Edges

IP07 QoS © 2004 Cisco Systems, Inc. All rights reserved. 60

 Branch Router QoS Design
QoS Requirements for Branch Routers

Queuing/Dropping/Shaping/ Classification and Marking (+ NBAR)

Link-Efficiency Policies for Policies for Branch-to-Campus Traffic
Branch-to-Campus Traffic

Branch Router
Branch
Switch
WAN

WAN Edge LAN Edge

Optional: DSCP-to-CoS Mapping Policies

for Campus-to-Branch Traffic

IP07 QoS © 2004 Cisco Systems, Inc. All rights reserved. 61

 MPLS VPN QoS Design
Where QoS Is Required in MPLS VPN Architectures?

CE-to-PE Queuing/Shaping/Remarking/LFI Optional: Core DiffServ or MPLS TE Policies

PE Ingress Policing and Remarking

P Routers

CE Router
PE Router PE Router CE Router

MPLS VPN
PE-to-CE Queuing/Shaping/LFI

Required
Optional

IP07 QoS © 2004 Cisco Systems, Inc. All rights reserved. 62

 At-a-Glance
Summaries

NMS-2T30
9681_05_2004_c2 © 2004 Cisco Systems, Inc. All rights reserved. 63
QoS is the measure of transmission quality Policing tools can complement marking
and service availability of a network (or
internetworks). The transmission quality of
QoS Tools tools by marking metering flows
and marking-down out-of-contract traffic.
the network is determined by the following Classification can be done at Layers 2-7:
factors: Latency, Jitter and Loss.
L2 Frame L3 IP Packet L4 TCP/UDP Segment L7 Data Payload

ToS/ Source Dest Src Dst

NBAR PDLM
DSCP IP IP Port Port
Delay-
Delay-
Delay Packet
Variation
(Latency) Loss Policers meter traffic into three categories:
(Jitter) Marking can be done at Layers 2 or Layer 3:
Layer 2: 802.1Q/p CoS, MPLS EXP •Conform: traffic is within the
Layer 3: IP Precedence, DSCP and/or defined rate (green light)
QoS technologies refer to the set of tools and IP ECN •Exceed: moderate bursting is
techniques to manage network resources and allowed (yellow light)
are considered the key enabling technologies Layer 3 (IP ToS Byte) Marking Options: •Violate: no more traffic is
for the transparent convergence of voice, allowed beyond this upper-limit
7 6 5 4 3 2 1 0
video and data networks. Additionally, QoS (red light)
tools can play a strategic role in significantly IP Precedence Unused
mitigating DoS/worm attacks. DiffServ Code Point (DSCP) IP ECN Scheduling tools re-order and selectively-
drop packets whenever congestion occurs.
Cisco’s QoS toolset consists of the following: Voice
•Classification and Marking tools nn
Video
•Policing and Markdown tools RFC
RFC 2474
2474 RFC
RFC 3168
3168
oo poonn
Data
•Scheduling tools DiffServ
DiffServ Extensions
Extensions IP
IP ECN
ECN Bits
Bits pp
•Link-specific tools
Policing and
•AutoQoS tools Cisco recommends end-to-end marking at
Markdown Link-Specific tools are useful on slow-
Layer 3 with standards-based DSCP values. speed WAN/VPN links and include
Classification shaping, compression, fragmentation and
Scheduling
and Marking
(Queuing and
Link-Specific interleaving.
Traffic Shaping Mechanisms
Selective-Dropping)

AutoQoS features automatically configure

Cisco-recommend QoS on Catalyst
switches and IOS routers with just one or
two commands.
IP07 QoS © 2004 Cisco Systems, Inc. All rights reserved. 64 2004
szigeti@cisco.com
 Standards-based marking
The QoS Baseline is a strategic document
designed to unify QoS within Cisco. The
QoS Baseline provides uniform, standards-
The QoS Baseline recommendations allow for better
integration with service-provider offerings
The IP Routing class is intended for IP as well as other internetworking scenarios.
based recommendations to help ensure that
QoS products, designs and deployments are Routing protocols, such as BGP, OSPF, etc.
unified and consistent. In Cisco IOS, rate-based queuing translates
The Call-Signaling class is intended for to CBWFQ; priority queuing is LLQ.
The QoS Baseline defines up to 11 classes of
voice and/or video signaling traffic, such as DSCP-Based WRED (based on RFC 2597)
traffic that may be viewed as critical to a
Skinny, SIP, H.323, etc. drops AFx3 before AFx2, and in turn drops
given enterprise. A summary these classes
and their respective standards-based The Network Management class is intended AFx2 before AFx1. RSVP is recommended
markings and recommended QoS for network management protocols, such as (whenever supported) for Voice and/or
configurations are shown below. SNMP, Syslog, DNS, etc. Interactive-Video admission control
L3 Classification Referencing
Application Recommended Configuration
PHB DSCP Standard
Cisco products
Interactive-Video IP Routing CS6 48 RFC 2474-
2474-4.2.2 Rate-
Rate-Based Queuing + RED
that support QoS
refers to IP Video- Voice EF 46 RFC 3246 RSVP Admission Control + Priority Queuing
features will use
Conferencing; Interactive-Video AF41 34 RFC 2597 RSVP + Rate-Based Queuing + DSCP-WRED
these QoS
Streaming Video is Streaming Video CS4 32 RFC 2474-4.2.2 RSVP + Rate-Based Queuing + RED
Baseline
either unicast or Mission-Critical AF31 26 RFC 2597 Rate-Based Queuing + DSCP-WRED
recommendations
multicast uni- Call-Signaling CS3 24 RFC 2474-4.2.2 Rate-Based Queuing + RED
for marking and
directional video. Transactional Data AF21 18 RFC 2597 Rate-Based Queuing + DSCP-WRED
scheduling and
Network Mgmt CS2 16 RFC 2474-4.2.2 Rate-Based Queuing + RED
admission
Bulk Data AF11 10 RFC 2597 Rate-Based Queuing + DSCP-WRED
control.
Scavenger CS1 8 Internet 2 No BW Guarantee + RED
Best Effort 0 0 RFC 2474-
2474-4.1 BW Guarantee Rate-
Rate-Based Queuing + RED

The (Locally-Defined) Mission-Critical class is intended for a subset The Scavenger class is based on an Internet 2 draft that defines a
of Transactional Data applications that contribute most significantly “less-than-Best Effort” service. In the event of link congestion,
to the business objectives (this is a non-technical assessment). this class will be dropped the most aggressively.

The Best Effort class is also the default class. Unless an

The Transactional Data class is intended for foreground, user-
application has been assigned for preferential/deferential service,
interactive applications such as database access, transaction services,
it will remain in this default class. Most enterprises have
interactive messaging and preferred data services.
hundreds – if not thousands – of applications on their networks;
the majority of which will remain in the Best Effort service class.
The Bulk Data class is intended for background, non-interactive
traffic flows, such as large file transfers, content distribution, The QoS Baseline recommendations are intended as a standards-
database
IP07synchronization,
QoS backup operations
© 2004 Cisco and
Systems, Inc. All email.
rights reserved. based guideline for customers – not as a mandate. szigeti@cisco.com
65 2004
A successful QoS deployment includes three 3) Design and test the QoS Policies.
key phases:
QoS Best-Practices Classify, mark and police as close to the
1) Strategically defining the business traffic-sources as possible; following
2) Analyze the application service-level
objectives to be achieved via QoS. Differentiated-Services standards, such as
requirements.
2) Analyzing the service-level requirements RFC 2474, 2475, 2597, 2698 and 3246.
of the traffic classes. Voice L3 Classification
3) Designing and testing QoS policies • Predicable Flows Application PHB DSCP
• Drop + Delay Sensitive
Routing CS6 48
• UDP Priority
1) Strategically defining the business • 150 ms one-way delay
Voice EF 46

objectives to be achieved by QoS. Interactive-Video AF41 34

• 30 ms jitter
Streaming Video CS4 32
• 1% loss
Business QoS objectives need to be defined: Mission-Critical AF31 26
• 17 kbps-106 kbps VoIP +
•Is the objective to enable VoIP only or is Call-Signaling Call-Signaling CS3 24

video also required? Transactional Data AF21 18

•If so, is video-conferencing required or Network Mgmt CS2 16

streaming video? Or both? Video Bulk Data AF11 10

• Unpredicable Flows
•Are there applications that are considered Scavenger CS1 8
• Drop + Delay Sensitive
mission-critical? If so, what are they? • UDP Priority
Best Effort 0 0

•Does the organization wish to squelch • 150 ms one-way delay

Provision queuing in a consistent manner
certain types of traffic? If so, what are they? • 30 ms jitter
(according to platform capabilities).
•Does the business want to use QoS tools to 1% loss
• Voice
mitigate DoS/worm attacks? • Overprovision stream by Best
•How many classes of service are needed to 20% to account for Effort
headers + bursts
meet the business objectives? Best Real-
Scavenger Effort time
≥ 25% ≤ 33% Interactive-
Because QoS introduces a system of Video
managed unfairness, most QoS deployments Data Bulk
Critical
Data
inevitably entail political and organizational • No “one-size fits all”
repercussions when implemented. • Smooth/Bursty Routing
Streaming-
• Benign/Greedy Video Call-
To minimize the effects of these non- • TCP Retransmits/ UDP Net Mgmt
Signaling
technical obstacles to deployment, address does not
Mission-Critical
Transactional
these political and organizational issues as
early as possible, garnishing executive Thoroughly test QoS policies prior to
endorsement
IP07 QoSwhenever possible.
© 2004 Cisco Systems, Inc. All rights reserved.
production-network deployment. 66 szigeti@cisco.com 2004
DoS and worm attacks are A key point of this strategy is that
exponentially increasing in frequency,
Scavenger-Class QoS Strategy for legitimate traffic flows that
complexity and scope of damage. DoS/Worm Attack Mitigation temporarily exceed thresholds are not
QoS tools and strategic designs can penalized by Scavenger-class QoS.
mitigate the effects of worms and keep Only traffic in excess of the normal/abnormal
critical applications available during DoS Only sustained, abnormal streams
threshold is remarked to Scavenger.
attacks. generated simultaneously by multiple hosts
Scavenger (highly-indicative of DoS/worm attacks)
DSCP CS1 are subject to aggressive dropping – and
One such strategy, referred to as
Scavenger-class QoS, uses a two-step Normal/Abnormal Threshold such dropping only occurs after legitimate
tactical approach to provide first- and traffic has been fully-serviced.
second-order anomaly detection and The Campus uplinks are not the only
reaction to DoS/worm attack-generated points in the network infrastructure that
traffic. congestion could occur. Typically WAN
Campus Access-Edge policing policies are
and VPN links are the first to congest.
The first step in deploying Scavenger-class coupled with Scavenger-class queuing policies
QoS is to profile applications to determine on the uplinks to the Campus Distribution
Therefore, Scavenger-class “less-than-
what constitutes a normal vs. abnormal flow Layer.
Best-Effort” queuing should be
(within a 95% confidence interval).
provisioned on all network devices in a
Queuing policies only engage when links are
consistent manner (according to platform
Application traffic exceeding this normal congested. Therefore, only if uplinks become
capabilities).
rate will be subject to first-order anomaly congested does traffic begin to be dropped.
detection at the Campus Access-Edge, Voice
specifically: excess traffic will be marked Anomalous traffic – previously marked to Best
down to Scavenger (DSCP CS1/8). Scavenger – is dropped the most aggressively Effort
(only after all other traffic types have been Best Real-
time
Note that anomalous traffic is not dropped fully-serviced). Scavenger Effort
≥ 25% ≤ 33% Interactive-
or penalized at the edge; it is simply Video

remarked. Policing Policy Bulk

Critical
Data
Policing Policy Normal Traffic
Normal Traffic Routing
Streaming-
Video Call-
Signaling
Net Mgmt

Transactional Mission-Critical

Thoroughly test QoS policies prior to

Anomalous Traffic production-network deployment.
Anomalous
IP07 QoS Traffic © 2004 Cisco Systems, Inc. All rights reserved. Queuing Policy 67
szigeti@cisco.com 2004
QoS policies should always be enabled in Queuing policies will vary by platform:
Catalyst switch hardware – rather than
router software – whenever a choice exists.
Campus QoS Design E.g. 1P3Q1T P = Priority Queue
Q = Non-Priority Queue
Access-Edge policers, such as T = WRED Threshold
Three main types of QoS policies are this one, detect anomalous flows DSCP CoS 1P3Q1T
required within the Campus: START and remark these to Scavenger CS7 CoS 7
1) Classification and Marking (DSCP CS1). CS6 CoS 6 CoS 5 Q4
2) Policing and Markdown EF CoS 5
Priority Queue
3) Queuing Yes Yes CoS 7
VVLAN + Trust and
AF41 CoS 4
≤ 128 kbps
DSCP EF Transmit CoS 6
CS4 CoS 4
Classification, marking and policing should
be performed as close to the traffic-sources No No AF31 CoS 3 Queue 3 70%
Drop
CoS 4
as possible, specifically at the Campus CS3 CoS 3
Access-Edge. Queuing, on the other hand, Yes Yes AF21 CoS 2 CoS 3
VVLAN +
needs to be provisioned at all Campus DSCP CS3 ≤ 32 kbps Trust and CoS 2
Transmit CS2 CoS 2
Layers (Access, Distribution, Core) due to No AF11 CoS 1
oversubscription ratios. No Queue 2 25%
Remark to CS1 CoS 1 CoS 0
DSCP CS1
Yes
VVLAN
Yes 0 0 CoS 1 Queue1 5%
Classify and mark as close to the traffic- ANY
≤ 32 kbps Remark to
sources as possible following Cisco’s QoS DSCP 0 The diagram below and left shows what
Baseline marking recommendations, which No No QoS policies are needed where in the Campus.
Remark to
are based on Differentiated-Services DSCP CS1
Yes
standards, such as: RFC 2474, 2597 & 3246. DVLAN
Yes
≤ 5 Mbps
ANY Remark to
L3 Classification DSCP 0
Application No
PHB DSCP
Routing CS6 48 VVLAN = Voice VLAN Remark to
DSCP CS1
Voice EF 46 DVLAN = Data VLAN
Interactive-Video AF41 34
Streaming Video CS4 32
No Trust + Policing + Queuing
Mission-Critical AF31 26
Call-Signaling CS3 24 Conditional Trust +
Policing + Queuing
Transactional Data AF21 18
Trust DSCP + Queuing
Network Mgmt CS2 16
Bulk Data AF11 10
Per-User Microflow Policing
Scavenger CS1 8 Server Farms
Best
IP07 Effort
QoS 0© 2004 Cisco Systems,
0 Inc. All rights reserved. szigeti@cisco.com 2004 IP Phones + PCs IP Phones + PCs
68
In an enterprise network infrastructure, Link-Specific Design Recommendations:
bandwidth is scarcest – and thus most
expensive – over the WAN. Therefore, the
WAN QoS Design Leased-Line (MLP) Link
WAG
business case for efficient bandwidth Queuing Models for 5/8/11 Classes of
optimization via QoS technologies is Service are shown below:
strongest over the WAN. Branch
Voice
18%
WAN QoS policies need to be configured on Best
Effort
the WAN edges of WAN Aggregator (WAG) 25% • Use MLP LFI and cRTP on Slow-Speed links
routers and Branch routers. WAN edge QoS
Interactive- WAG Frame Relay Link
policies include queuing, shaping, selective- Video
dropping and link-specific policies. Scavenger
15%
1% Branch
The number of WAN classes of traffic is Frame Relay
Cloud
determined by the business objectives and Bulk
may be expanded over time. 4%
• Use Frame-Relay traffic shaping
QoS Baseline Call Signaling • Set CIR to 95% of guaranteed rate
5 Class Model 8 Class Model Streaming 5%
Model
Video 10% • Set Committed Burst to CIR/100
Voice Routing 3% • Set Excess Burst to 0
Voice
Realtime Network Mgmt 2% • Use FRF.12 and and cRTP on Slow-Speed links
Interactive-Video
Video Transactional
Streaming Video Data 7% Mission-Critical Data WAG ATM Link
10%
Call Signaling Call Signaling Call Signaling
IP Routing WAN QoS tools: RTP Header Compression (cRTP)
Network Control ATM Branch
Network Mgmt IP Header UDP Hdr RTP Hdr VoIP
Cloud
20 Bytes 8 Bytes 12 Bytes
Critical Data Mission-Critica
Critical Data • Use MLP LFI (via MLPoATM) and cRTP on Slow-
Transactional
Speed links
Bulk Data Bulk Data cRTP saves: • Set the ATM PVC Tx-Ring to 3 for Slow-Speed links
~ 20% for G.711
~ 60% for G.729
Best Effort cRTP Header
Best
Best Effort
Effort Best
Best Effort
Effort
2-5 Bytes WAG ATM-to-FR SIW Link
Scavenger Scavenger Scavenger
WAN QoS tools: Link Fragmentation and Interleaving ATM
Time Cloud Branch
WAN links can be categorized into three VoIP Data FR
Cloud
main speed groups: LFI tools (MLP LFI or FRF.12) fragment large data packets
and interleave these with high-priority VoIP. • Use MLP LFI (via MLPoATM and MLPoFR) for
• Slow-Speed (≤ 768 kbps) Slow-Speed Links
• Medium-Speed (> 768 kbps & ≤ T1/E1) Data Data Data VoIP Data • Optimize fragment sizes to minimize ATM
• High-Speed
IP07 QoS (≥ T1/E1) © 2004 Cisco Systems, Inc. All rights reserved. cell-padding 69 2004
szigeti@cisco.com
Branch routers are connected to central sites
3) NBAR for Known Worm Policing
via private-WAN or VPN links which often
prove to be the bottlenecks for traffic flows.
Branch QoS Design
Worms are nothing new, but they have
QoS policies at these bottlenecks align An example 10-class QoS Baseline Branch increased exponentially in frequency,
expensive WAN/VPN bandwidth utilization Router WAN Edge Queuing Model: complexity and scope of damage in recent
with business objectives. Voice years.
18% 1. The enabling code
Best
QoS designs for Branch routers are – for the Effort
25%
most part – identical to WAN Aggregator
2. The propagation
QoS designs. However, Branch routers mechanism
require three unique QoS considerations: Interactive
1) Unidirectional applications Scavenger Video
3. The payload
1% 15%
2) Ingress classification requirements
3) NBAR policies for worm policing Bulk
4%
The Branch router’s ingress LAN edge is a
Call
Each of these Branch router QoS design Signaling strategic place to use NBAR to identify &
5% drop worms, such as CodeRed, NIMDA,
considerations will be overviewed. Transactional
Data 12% Routing 3% SQL Slammer, MS-Blaster and Sasser.
Network Mgmt 2%
1) Unidirectional Applications Mission-Critical Data
15%
L2 Frame L3 IP Packet L4 Segment L7 Data Payload

Some applications (like Streaming Video) 2) Ingress Classification Worm

usually only traverse the WAN/VPN in the Branch-to-Campus traffic may not be
Campus-to-Branch direction and therefore correctly marked on the Branch Access NBAR extensions allow for custom Packet
do not require provisioning in the Branch- Data Language Modules (PDLMs) to be
Layer switch.
to-Campus direction on the Branch defined for future worms.
router’s WAN edge. These switches – which are usually lower-
Where is QoS required on Branch
end switches – may or may not have the
Bandwidth for such unidirectional capabilities to classify and mark application routers? Classification & Marking +
LLQ/CBWFQ/WRED/
application classes can be reassigned to traffic. Therefore, classification and Shaping/LFI/cRTP Policies for
NBAR Worm Policing
Policies for
other critical classes, as shown in the marking may need to be performed on the Branch-to-Campus Traffic Branch-to-Campus Traffic
following diagram. Notice that no Branch router’s LAN edge (in the ingress
Branch Router Branch
Streaming Video class is provisioned and direction). Switch
WAN/ DVLAN
the bandwidth allocated to it (on the VPN
Campus side of the WAN link) is Furthermore, Branch routers offer the
VVLAN
reallocated to the Mission-Critical and ability to use NBAR to classify and mark WAN Edge LAN Edge
Transactional Data classes. traffic flows that require stateful packet Optional: DSCP-to-CoS Mapping Policies for
IP07 QoS inspection.
© 2004 Cisco Systems, Inc. All rights reserved. szigeti@cisco.com 2004 Campus-to-Branch Traffic 70
QoS design for an enterprise subscribing to a
MPLS VPN requires a major paradigm shift
QoS Design for Example enterprise subscriber
DSCP Remarking Diagram and
from private-WAN QoS design. MPLS VPN Subscribers CE Edge Bandwidth Allocation Diagram.
This is because with private-WAN design, MPLS VPN service Enterprise
DSCP
Service Provider
Applications Classes of Service
the enterprise principally controlled QoS. providers offer classes of
The WAN Aggregator (WAG) provisioned service to enterprise Routing CS6
QoS for not only Campus-to-Branch traffic, subscribers. Voice EF REALTIME
EF
but also for Branch-to-Branch traffic 35%
(which was homed through the WAG). Interactive-Video AF41 Î CS5 CS5
Admission criteria for
WAG Branch these classes is the DSCP Streaming Video CS4 Î AF21
CS6
markings of enterprise Mission-Critical Data DSCP 25 Î AF31 AF31
CRITICAL
20%
traffic. Thus, enterprises CS3
WAN Call Signaling AF31/CS3 Î CS5
may have to remark
application traffic to gain Transactional Data AF21 Î CS3 AF21 VIDEO
15%
Branch admission into the required Network Management CS2
CS2
However, due to the any-to-any/full-mesh service provider class. BULK 5%
AF11/CS1
nature of MPLS VPNs, Branch-to-Branch Bulk Data AF11

traffic is no longer homed through the WAG. Some best practices to Scavenger CS1 Î 0 BEST EFFORT
25%
While Branch-to-MPLS VPN QoS is consider when assigning Best Effort 0
controlled by the enterprise (on their enterprise traffic to service
Customer-Edge – CE – routers), provider classes of service Enterprise
Voice 15%
MPLS VPN-to-Branch QoS is controlled by include: Applications
the service provider (on their Provider Edge – • Don’t put Voice and
PE – routers). Interactive-Video into the Best Effort
24%
Branch CE
Central CE Realtime class on slow-speed
MPLS Best
Best Effort
Effort 25%
25% Realtime
(≤ 768 kbps) CE-to-PE links 35% Interactive-
Video
VPN • Don’t put Call-Signaling Scavenger 15%
Service Provider
1%
into the Realtime class on Bulk 5% Classes of Service
slow-speed CE-to-PE links Bulk 5%
Service Provider PE Routers Video
Video
Branch CE • Don’t mix TCP Net Mgmt 15%
15%
Critical
Critical
20%
20%
applications with UDP 2% Call
Therefore, to guarantee end-to-end QoS, Signaling
applications within a single 5%
enterprises must co-manage QoS with their
service provider class
MPLS VPN service providers; their policies Streaming-Video Routing 3%
(whenever possible); UDP 13%
must be both consistent and Mission-Critical Data 12%
applications may dominate
complementary. Transactional Data 5%
IP07 QoS the class when congested
© 2004 Cisco Systems, Inc. All rights reserved. 71 2004
szigeti@cisco.com
 In order to support enterprise-subscriber QoS Design for MPLS VPN Service providers can mark at Layer 2
voice, video and data networks, service
providers must include QoS provisioning Service Providers (MPLS EXP) or at Layer 3 (DSCP).
within their MPLS VPN service offerings.
RFC 3270 presents three modes of MPLS/DiffServ marking for service providers:
This is due to the any-to-any/full-mesh 1) Uniform Mode: SP can remark customer DSCP values
nature of MPLS VPNs, where enterprise 2) Pipe Mode: SP does not remark customer DSCP values (SP uses independent MPLS
subscribers depend on their service EXP markings); final PE-to-CE policies are based on service provider’s markings
providers to provision Provider-Edge (PE) 3) Short Pipe Mode (shown below): SP does not remark customer DSCP values (SP uses
to Customer-Edge (CE) QoS policies independent MPLS EXP markings); final PE-to-CE policies are based on customer’s
consistent with their CE-to-PE policies. markings
Unshaded Areas
Shaded Area represents Service Provider DiffServ Domain represent Customer
DiffServ Domain
In addition to these PE-to-CE policies, 3) Assume a policer remarks
out-of-contract traffic’s
service providers will likely implement top-most MPLS label to 6) PE-to-CE policies
ingress policers on their PEs to identify MPLS EXP 0 MPLS VPN are based on
Customer-Markings
whether traffic flows are in- or out-of-
contract. Optionally, service providers may Provider (P)
Routers
also provision QoS policies within their core CE Router
CE Router
networks, using Differentiated Services PE Router PE Router
and/or MPLS Traffic Engineering (TE).
Optional: Core DiffServ or DSCP AF31 MPLS EXP 4 DSCP AF31
MPLS EXP 0
MPLS TE Policies
1) Packet initially MPLS EXP 4 MPLS EXP 0 7) Original customer-
PE Ingress DSCP AF31
Policing and marked to DSCP AF31 MPLS EXP 4 marked DSCP
MPLS VPN DSCP AF31 5) Topmost label Is values are preserved
Re-Marking
DSCP AF31 popped and
2) MPLS EXP values MPLS EXP value is
are set independently 4) Topmost label copied to
from DSCP values is marked down underlying label
by a policer
CE Router PE Router CE Router Direction of Packet Flow
PE Router
Required
P Routers PE-to-CE
Optional
LLQ/CBWFQ/WRED/
Shaping/LFI Service providers can guarantee service levels within their core by:
In order to guarantee end-to-end QoS, 1) Aggregate Bandwidth Overprovisioning: adding redundant links when
enterprises must co-manage QoS with utilization hits 50% (simple to implement, but expensive and inefficient)
their MPLS VPN service providers; 2) Core DiffServ policies: simplified DiffServ policies for core links
their policies must be both consistent 3) MPLS TE: TE provides granular policy-based control over traffic flows
within the core
and complementary.
IP07 QoS © 2004 Cisco Systems, Inc. All rights reserved. 72 szigeti@cisco.com 2004
IPSec VPNs achieve network segregation QoS Design for
and privacy via encryption. IPSec VPNs
are built by overlaying a point-to-point IPSec VPNs 2) Encryption/Decryption Delays
mesh over the Internet using Layer 3-
encrypted tunnels. Encryption/ decryption A marginal time element for encryption and decryption should be factored into the end-
is performed at these tunnel endpoints and to-end delay budget for realtime applications, such as VoIP. Typically these processes
the protected traffic is carried across the require 2-10 ms per hop, but may be doubled in the case of spoke-to-spoke VoIP calls that
shared network. are homed through a central VPN headend hub.

Three main QoS considerations specific to

IPSec VPNs are:
1) the additional bandwidth required by
IPSec VPN
IPSec encryption and authentication,
2) the marginal time element required at
Campus Branch Office
each point where encryption/decryption
Propagation
takes place CODEC Queuing Encrypt Serialization and Network Decrypt Jitter Buffer
3) Anti-Replay interactions
Variable Variable Fixed
10–
10–50 ms 20–
20–100 ms
(Can Be Minimal (Can Be (6.3 µs/Km) + Minimal
(Depends on (Depends on
Reduced 2–10 ms Reduced Network Delay 2–10 ms
Sample Size) Sample Size)
1) IPSec Bandwidth Overhead Using LLQ) Using LFI) (Variable)

End-
End-to-
to-End Delay (Must Be < 150 ms)
The additional bandwidth required to
encrypt and authenticate a packet needs to 3) Anti-Replay Interactions
be factored into account when
provisioning QoS policies. Anti-Relay is a standards-defined mechanism to protect IPSec VPNs from hackers. If
packets arrive outside of a 64-byte window, then they are considered hacked and are
This is especially important for VoIP, dropped prior to decryption. QoS queuing policies may re-order packets such that they
where IPSec could more than double the fall outside of the Anti-Replay window. Therefore, IPSec VPN QoS policies need to be
size of a G.729 voice packet, as shown properly tuned to minimize Anti-Replay drops.
below. Outside
Window
G.729 VoIP
IP UDP RTP
60 Bytes Voice 64 Packet Sliding Window
Hdr Hdr Hdr 64 Packet Sliding Window
Æ
1 2 4 64 65 66 67
IPSec ESP ESP GRE IP GRE IP UDP RTP ESP ESP 3
Hdr Hdr IV Hdr Hdr Hdr Voice Pad/NH Auth
Hdr
Anti-Replay
20 4 20 8 12 20 12 Drop
20 8 8 2–257
IP07ESP
IPSec QoS
Tunnel Mode G.729 VoIP - 136
© 2004 Bytes
Cisco Systems, Inc. All rights reserved. 73 2004
szigeti@cisco.com
 Q&A

NMS-2T30
9681_05_2004_c2
IP07 QoS © 2004 Cisco Systems, Inc. All rights reserved. 74
 REFERENCES

NMS-2T30
9681_05_2004_c2 © 2004 Cisco Systems, Inc. All rights reserved. 75
 Solution Reference Network Design Guides
Enterprise QoS Design Guide

http://www.cisco.com/go/srnd

IP07 QoS © 2004 Cisco Systems, Inc. All rights reserved. 76

 Solution Reference Network Design Guides
Site-to-Site V3PN Design Guide

http://www.cisco.com/go/srnd

IP07 QoS © 2004 Cisco Systems, Inc. All rights reserved. 77

 Solution Reference Network Design Guides
Teleworker V3PN Design Guide

http://www.cisco.com/go/srnd

IP07 QoS © 2004 Cisco Systems, Inc. All rights reserved. 78

 Solution Reference Network Design Guides
Service Provider QoS Design (MPLS VPNs)
http://www.cisco.com/en/US/netsol/ns341/ns396/ns172/ns103/networking_
solutions_white_paper09186a00801b1c5a.shtml

IP07 QoS © 2004 Cisco Systems, Inc. All rights reserved. 79

 Reference Materials
DiffServ Standards

• RFC 2474 “Definition of the Differentiated Services Field

(DS Field) in the IPv4 and IPv6 Headers”
http://www.ietf.org/rfc/rfc2474
• RFC 2475 “An Architecture for Differentiated Services”
http://www.ietf.org/rfc/rfc2475
• RFC 2597 “Assured Forwarding PHB Group”
http://www.ietf.org/rfc/rfc2597
• RFC 2697 “A Single Rate Three Color Marker”
http://www.ietf.org/rfc/rfc2697
• RFC 2698 “A Two Rate Three Color Marker”
http://www.ietf.org/rfc/rfc2698
• RFC 3246 “An Expedited Forwarding PHB
(Per-Hop Behavior)”
http://www.ietf.org/rfc/rfc3246

IP07 QoS © 2004 Cisco Systems, Inc. All rights reserved. 80

 Reference Materials
Campus QoS Documentation

• Cisco Catalyst 2950 QoS

http://www.cisco.com/univercd/cc/td/doc/product/lan/cat2950/12120ea2/2950scg/swqos.htm

• Cisco Catalyst 2970 QoS

http://www.cisco.com/univercd/cc/td/doc/product/lan/cat2970/12220se/2970scg/swqos.htm

• Cisco Catalyst 3550 QoS

http://www.cisco.com/univercd/cc/td/doc/product/lan/c3550/12120ea2/3550scg/swqos.htm

• Cisco Catalyst 3750 QoS

http://www.cisco.com/univercd/cc/td/doc/product/lan/cat3750/12220se/3750scg/swqos.htm

• Cisco Catalyst 4500 (Cisco IOS) QoS

http://www.cisco.com/univercd/cc/td/doc/product/lan/cat4000/12_2_18/config/qos.htm

• Cisco Catalyst 6500 (Cisco Catalyst OS)

http://www.cisco.com/univercd/cc/td/doc/product/lan/cat6000/sw_8_3/confg_gd/qos.htm

• Cisco Catalyst 6500 (Cisco IOS)

http://www.cisco.com/univercd/cc/td/doc/product/lan/cat6000/122sx/swcg/qos.htm

IP07 QoS © 2004 Cisco Systems, Inc. All rights reserved. 81

 Reference Materials
WAN/Branch Cisco IOS QoS Documentation

• Classification Tools
http://www.cisco.com/univercd/cc/td/doc/product/software/ios123/123cgcr/qos_vcg.htm#1000913

• Congestion Management (Queuing) Tools

http://www.cisco.com/univercd/cc/td/doc/product/software/ios123/123cgcr/qos_vcg.htm#1001619

• Congestion Avoidance (Selective Dropping) Tools

http://www.cisco.com/univercd/cc/td/doc/product/software/ios123/123cgcr/qos_vcg.htm#1000448

• Policing and Shaping Tools

http://www.cisco.com/univercd/cc/td/doc/product/software/ios123/123cgcr/qos_vcg.htm#1001018

• Link-Specific Tools
http://www.cisco.com/univercd/cc/td/doc/product/software/ios123/123cgcr/qos_vcg.htm#1001728

• Modular QoS CLI (MQC) Syntax

http://www.cisco.com/univercd/cc/td/doc/product/software/ios123/123cgcr/qos_vcg.htm#1001811

IP07 QoS © 2004 Cisco Systems, Inc. All rights reserved. 82

 Reference Materials
NBAR vs. Worms (SAFE White Papers)

• Code Red
http://www.cisco.com/en/US/products/hw/routers/ps359/
products_tech_note09186a00800fc176.shtml
• Nimda
http://www.cisco.com/en/US/products/sw/iosswrel/ps1835/
products_tech_note09186a0080110d17.shtml
• SQL Slammer
http://www.cisco.com/en/US/netsol/ns340/ns394/ns171/ns128/
networking_solutions_white_paper09186a00801cd7f5.shtml
• DCOM/W32/Blaster
http://www.cisco.com/en/US/netsol/ns340/ns394/ns171/ns128/
networking_solutions_white_paper09186a00801b2391.shtml
• Sasser
http://www.cisco.com/application/pdf/en/us/guest/netsol/ns441/
c664/cdccont_0900aecd800f613b.pdf
• NBAR Custom PDLM (Cisco IOS 12.3(4)T Documentation)
http://www.cisco.com/univercd/cc/td/doc/product/software/ios122/
122newft/122t/122t8/dtnbarad.htm

IP07 QoS © 2004 Cisco Systems, Inc. All rights reserved. 83

 Reference Materials
MPLS VPN Standards

• RFC 2547 “BGP/MPLS VPNs”

http://www.ietf.org/rfc/rfc2547
• RFC 2702 “Requirements for Traffic Engineering Over MPLS”
http://www.ietf.org/rfc/rfc2702
• RFC 2917 “A Core MPLS IP VPN Architecture”
http://www.ietf.org/rfc/rfc2917
• RFC 3270 “Multi-Protocol Label Switching (MPLS) Support of
Differentiated Services”
http://www.ietf.org/rfc/rfc3270
• RFC 3564 “Requirements for Support of Differentiated
Services-aware MPLS Traffic Engineering”
http://www.ietf.org/rfc/rfc3564

IP07 QoS © 2004 Cisco Systems, Inc. All rights reserved. 84

 Reference Materials
MPLS VPN QoS Documentation

• Configuring Multiprotocol Label Switching

http://www.cisco.com/univercd/cc/td/doc/product/software/ios122/
122cgcr/fswtch_c/swprt3/xcftagc.htm
• Configuring MPLS VPNs
http://www.cisco.com/univercd/cc/td/doc/product/software/ios122/
122newft/122t/122t13/ftvpn13.htm
• Configuring MPLS DiffServ Tunneling Modes
http://www.cisco.com/univercd/cc/td/doc/product/software/ios122/
122newft/122t/122t13/ftdtmode.htm
• Configuring MPLS Traffic Engineering (MPLS TE)
http://www.cisco.com/univercd/cc/td/doc/product/software/ios122/
122newft/122t/122t4/ftbwadjm.htm
• Configuring DiffServ-aware MPLS Traffic Engineering
(MPLS DS-TE)
http://www.cisco.com/univercd/cc/td/doc/product/software/ios122/
122newft/122t/122t4/ft_ds_te.htm

IP07 QoS © 2004 Cisco Systems, Inc. All rights reserved. 85

 Reference Materials
AutoQoS Documentation

• AutoQoS VoIP for the Cisco Catalyst 2950

http://www.cisco.com/univercd/cc/td/doc/product/lan/cat2950/12120ea2/2950scg/swqos.htm#wp1125412

• AutoQoS VoIP for the Cisco Catalyst 2970

http://www.cisco.com/univercd/cc/td/doc/product/lan/cat2970/12220se/2970scg/swqos.htm#wp1231112

• AutoQoS VoIP for the Cisco Catalyst 3550

http://www.cisco.com/univercd/cc/td/doc/product/lan/c3550/12120ea2/3550scg/swqos.htm#wp1185065

• AutoQoS VoIP for the Cisco Catalyst 3750

http://www.cisco.com/univercd/cc/td/doc/product/lan/cat3750/12220se/3750scg/swqos.htm#wp1231112

• AutoQoS VoIP for the Cisco Catalyst 4550

http://www.cisco.com/univercd/cc/td/doc/product/lan/cat4000/12_2_18/config/qos.htm#1281380

• AutoQoS VoIP for the Cisco Catalyst 6500 (Cisco Catalyst OS)
http://www.cisco.com/univercd/cc/td/doc/product/lan/cat6000/sw_8_3/confg_gd/autoqos.htm

• AutoQoS VoIP for Cisco IOS Routers (Cisco IOS 12.2(15)T)

http://www.cisco.com/univercd/cc/td/doc/product/software/ios122/122newft/122t/122t15/ftautoq1.htm

• AutoQoS Enterprise for Cisco IOS Routers (Cisco IOS 12.3(7)T)

http://www.cisco.com/univercd/cc/td/doc/product/software/ios123/123newft/123t/123t_7/ftautoq2.htm

IP07 QoS © 2004 Cisco Systems, Inc. All rights reserved. 86

 Reference Materials
Networkers QoS Design Techtorial

ftp://ftpeng.cisco.com/szigeti/NW2004
9-hr Techtorial (450 slides)
Detailed designs and configs

LAN
ƒCatalyst 2950
ƒCatalyst 3550
ƒCatalyst 2970/3750
ƒCatalyst 4500
ƒCatalyst 6500

WAN/Branch
ƒLeased Lines
ƒFrame Relay
ƒATM
ƒATM-to-FR SIW
ƒISDN
ƒNBAR for Worm Policing

VPN
ƒMPLS
ƒIPSec (Site-to-Site)
ƒIPSec (Teleworker)

IP07 QoS © 2004 Cisco Systems, Inc. All rights reserved. 87

 Reference Materials
Cisco Press Book: End-to-End QoS Design

http://www.ciscopress.com/title/1587051761
ISBN: 1587051761
Publish Date: Nov 9/04

LAN
ƒCatalyst 2950
ƒCatalyst 3550
ƒCatalyst 2970/3560/3750
ƒCatalyst 4500
ƒCatalyst 6500

WAN/Branch
ƒLeased Lines
ƒFrame Relay
ƒATM
ƒATM-to-FR SIW
ƒISDN
ƒNBAR for Worm Policing

VPN
ƒMPLS (for Enterprise Subscribers)
ƒMPLS (for Service Providers)
ƒIPSec (Site-to-Site)
ƒIPSec (Teleworker)
IP07 QoS © 2004 Cisco Systems, Inc. All rights reserved. 88
NMS-2T30
9681_05_2004_c2 © 2004 Cisco Systems, Inc. All rights reserved. 89