High Performance 10GE Module

Bandwidth Management and

Provisioning Algorithm
Version 1.5
Date: 30 June 2008
Author: Shay Dunevich
Product Line Management

© Nokia Siemens Networks 1 (18)

High Performance 10GE Module
Bandwidth Management and Provisioning Algorithm

The information in this document is subject to change without notice and describes only the product
defined in the introduction of this documentation. This document is intended for the use of Nokia
Siemens Networks customers only for the purposes of the agreement under which the document is
submitted, and no part of it may be used, reproduced, modified or transmitted in any form or means
without the prior written permission of Nokia Siemens Networks. The document has been prepared to
be used by professional and properly trained personnel, and the customer assumes full responsibility
when using it. Nokia Siemens Networks welcomes customer comments as part of the process of
continuous development and improvement of the documentation.
The information or statements given in this document concerning the suitability, capacity, or
performance of the mentioned hardware or software products are given “as is” and all liability arising
in connection with such hardware or software products shall be defined conclusively in a separate
agreement between Nokia Siemens Networks and the customer. However, Nokia Siemens Networks
has made all reasonable efforts to ensure that the instructions contained in the document are
adequate and free of material errors and omissions. Nokia Siemens Networks will, if deemed
necessary by Nokia Siemens Networks, explain issues which may not be covered by the document.
Nokia Siemens Networks will correct errors in the document as soon as possible. IN NO EVENT
WILL NOKIA SIEMENS NETWORKS BE LIABLE FOR ERRORS IN THIS DOCUMENT OR FOR
ANY DAMAGES, INCLUDING BUT NOT LIMITED TO SPECIAL, DIRECT, INDIRECT, INCIDENTAL
OR CONSEQUENTIAL OR ANY MONETARY LOSSES,SUCH AS BUT NOT LIMITED TO LOSS OF
PROFIT, REVENUE, BUSINESS INTERRUPTION, BUSINESS OPPORTUNITY OR DATA,THAT
MAY ARISE FROM THE USE OF THIS DOCUMENT OR THE INFORMATION IN IT
This document and the product it describes are considered protected by copyrights and other
intellectual property rights according to the applicable laws.
Wave logo is a trademark of Nokia Siemens Networks Oy. Nokia is a registered trademark of Nokia
Corporation. Siemens is a registered trademark of Siemens AG.
Other product names mentioned in this document may be trademarks of their respective owners, and
they are mentioned for identification purposes only.
Copyright © Nokia Siemens Networks 2008. All rights reserved.

2 (18) © Nokia Siemens Networks

High Performance 10GE Module
Bandwidth Management and Provisioning Algorithm

Contents

1 Introduction ............................................................................................4

2 Overview .................................................................................................5
2.1 2.5 Gig Internal Flow ................................................................................6

3 Service Provisioning Algorithm............................................................8

3.1 Load Sharing and Spare Groups..............................................................8
3.1.1 Default Values ..........................................................................................9
3.2 Service Provisioning Algorithm Example................................................10
3.3 Sub-port Configurable Thresholds for 10 G Ethernet Modules ..............16
3.4 Operational Consideration......................................................................18

List of Figures
Figure 1: 10GE High Performance Module - Internal Functions Block Diagram ..................... 6
Figure 2: Four x 2.5 Gig Internal Data Flow............................................................................. 7
Figure 3: 10 Gig Sub-port division and functional grouping..................................................... 9
Figure 4: Default sub-port to group allocation and "Fill-in" bandwidth threshold value.......... 10
Figure 5: Connection Provisioning 1 and 2............................................................................ 10
Figure 6: Connection Provisioning 2, 3 and 5........................................................................ 11
Figure 7: Connection Provisioning 6 and 7............................................................................ 12
Figure 8: Connection Provisioning 8...................................................................................... 13
Figure 9: Connection Provisioning 9...................................................................................... 14
Figure 10: Connection Provisioning 10, 11 and 12................................................................ 15
Figure 11: Connection Provisioning 13, 14, 15 and 16.......................................................... 16
Figure 12: ASPEN's Global Connection Settings Dialog ....................................................... 17
Figure 13: ASPEN's Element Manager ................................................................................. 17

© Nokia Siemens Networks 3 (18)

High Performance 10GE Module
Bandwidth Management and Provisioning Algorithm

1 Introduction
The purpose of this document is to describe in detail the 10Gigabit High
Performance Module internal bandwidth distributions and provisioning
algorithm. The document outlines the internal data flow and packet processing
engines of the 10GE module, and describes the algorithm used by the
embedded software of the A-8100 and A-4100 core devices, to provision new
services (connections) throughout this module.
This document refers to all the 10 Gigabit High Performance Modules
supported by the Nokia Siemens Networks A-8100/A-4100 series.

4 (18) © Nokia Siemens Networks

High Performance 10GE Module
Bandwidth Management and Provisioning Algorithm

2 Overview
The Nokia Siemens Networks 10 Gigabit Ethernet High Performance module
provides 10Gig/sec throughput capacity interconnecting between the A-8100
and/or A-4100 core devices, creating a true high-capacity metro core transport
infrastructure.
Figure 1 depicts the internal block diagram of the 10 GE module.
The front-end components include the 10 Gigabit pluggable or fixed optics,
and a standard 802.3ae compliant 10 Gigabit MAC layer. There are four
Network Processor (NP) engines, which are responsible for all packet
processing and forwarding decisions. Each NP is capable of processing 2.5
Gigabit/sec of data traffic in both the Ingress and Egress directions.
The MPLS Mux/Demux and Queue Management entity is used to mediate
between the four network processors and the 10Gig MAC. This entity controls
and assigns traffic streams (LSPs) of up to 2.5 Gig of total capacity to each
one of the NPs. These traffic streams are multiplexed together into a virtual
single 10 Gig/sec data stream traversing throughout the 10 Gig link.
The Subscriber Management entity serves as an off-load server entity for the
network processor, and implements all policing functionality and unique
information storage per each subscriber stream (LSP/connection).
The Ingress and Egress Queue Manager performs all queuing and priority
forwarding functions on each incoming and outgoing data packet.
The Switch Fabric Interface Module mediates between the Queue
Management entity and the switch fabric.

© Nokia Siemens Networks 5 (18)

High Performance 10GE Module
Bandwidth Management and Provisioning Algorithm

Figure 1: 10GE High Performance Module - Internal Functions Block Diagram

2.1 2.5 Gig Internal Flow

The 10 GE module supports up to 2.5 Gig of traffic designated as a single
flow. The different flows are identified as multiplexed and de-multiplexed by
their source MAC address. Each network processor is capable of processing
2.5 Gig of traffic which can either be one flow or multiple flows accumulated to
2.5 Gig total throughput.
Whenever, a certain connection is provisioned throughout the 10Gig link, the
internal provisioning algorithm decides which network processor engine out of
the four available is to be designated for this specific connection.
As mentioned above, the 10 Gig link can accommodate a full 10 Gigabit/sec of
throughput capacity. This 10 Gigabit/sec of bandwidth is usually divided
among many connections traversing this specific link. Conceptually, this can
be viewed as four groups of connections/traffic streams where each can utilize
up to 2.5 Gigabit/sec.
Figure 2 depicts a 10 GE module connected to a 10 Gig link having multiple
connections/data streams, grouped into 2.5 Gig/sec flows. Each connection in
the ingress direction is classified by the MPLS Mux/DeMux entity and directed
to one of the four 2.5 Gig network processors.
In the egress direction, traffic arriving from the switch matrix is processed by
the network processors and multiplexed together to a 10 Gig/sec traffic flow.

6 (18) © Nokia Siemens Networks

High Performance 10GE Module
Bandwidth Management and Provisioning Algorithm

Figure 2: Four x 2.5 Gig Internal Data Flow

© Nokia Siemens Networks 7 (18)

High Performance 10GE Module
Bandwidth Management and Provisioning Algorithm

3 Service Provisioning Algorithm

Whenever a service (connection) is to be provisioned through a 10GE module,
the bandwidth management and the internal Connection Admission Control
(CAC) entities are engaged to assign this connection with the appropriate
internal path routed through one of the four network processors as described
above.
Each core device A-8100 or A-4100 implements a Bandwidth Management
entity which controls the provisioning of each connection traversing a specific
core device. The Bandwidth Management runs a Provisioning Algorithm upon
every request it gets (either by ASPEN or directly from CLI) to determine the
optimal path for provisioning this connection. After the path has been
determined, an internal Cross Connect (XC) is created between the Ingress
and the Egress ports.
The following section explains the Provisioning Algorithm used to provision a
connection through the 10GE link.

3.1 Load Sharing and Spare Groups

Each 10Gig link is divided into four sub-ports. As described above, each sub-
port supports up to 2.5 Gig flow of traffic. The four sub-ports are divided into
two virtual groups:
• Load sharing
• Spare
Figure 3 depicts a 10 Gig link with four 2.5 Gig sub-ports divided into two
groups.

8 (18) © Nokia Siemens Networks

High Performance 10GE Module
Bandwidth Management and Provisioning Algorithm

Figure 3: 10 Gig Sub-port division and functional grouping

The Load Sharing group defines the available resources from which a certain
connection is to be provisioned. The Spare group defines spare resources to
be used whenever the load sharing group runs out of available resources
according to predefined thresholds.
In the example above, two sub-ports were assigned to the Load Sharing group
while the other two sub-ports were assigned to the Spare group. Initially new
connections are provisioned through those sub-ports of the Load Sharing
group. Each sub-port is assigned a specific “fill-in” bandwidth threshold, which
reduces the amount of bandwidth that can be used out of the available 2.5
Gig/sec.
Whenever all sub-ports of the Load Sharing group reach this threshold, a new
sub-port is added to the Load Sharing group. The provisioning algorithm
distributes the connections among all the available sub-ports within the Load
Sharing group.

3.1.1 Default Values

Software Release 3.5.x features non-configurable initial values for the Load
Sharing vs. Spare groups, as well as for the “fill-in” bandwidth threshold.
The default values are:
• Initial Load Sharing group = Sub-port 1
• Initial Spare group = Sub-port2, Sub-port 3 and Sub-port 4
• "Fill-in" bandwidth threshold = 1.5 Gigabit/sec of the 2.5 Gig/sec
available.
Figure 4 depicts these initial group values and the “fill-in” bandwidth thresholds
per sub-port.

© Nokia Siemens Networks 9 (18)

High Performance 10GE Module
Bandwidth Management and Provisioning Algorithm

Figure 4: Default sub-port to group allocation and "Fill-in" bandwidth threshold

value

3.2 Service Provisioning Algorithm Example

Consider a 10 GE link where multiple connections are to be provisioned
through it. As a first step, two connections, each with 200 M, should be
provisioned. The following table outlines the required services (connections) to
be provisioned:

Total Link
Service/Connection Bandwidth Sub-Port Used Sub-Port Usage
Usage

1 200M Sub-Port 1
2 200M Sub-Port 1 400M 400M

The provisioning algorithm will be using load-distribution group sub-port 1. A

total of 400 M out of the 1.5 Gig will be occupied.

Figure 5: Connection Provisioning 1 and 2

10 (18) © Nokia Siemens Networks

High Performance 10GE Module
Bandwidth Management and Provisioning Algorithm

In the next step, an additional three connections should be provisioned, two

connections having 400M and the third 300M. The table below outlines the
specifications for this provisioning stage:

Total Link
Service/Connection Bandwidth Sub-Port Used Sub-Port Usage
Usage

1 200 M Sub-Port 1
2 200 M Sub-Port 1 400 M 400 M
3 400 M Sub-Port 1
4 400 M Sub-Port 1
5 300 M Sub-Port 1 1500 M 1500 M

The total occupancy of this sub-port (sub-port 1) is now 1.5 Gig, i.e. it has
reached its “fill-in” bandwidth threshold; therefore additional sub-ports should
be moved from the Spare group to the Load Sharing group.

Figure 6: Connection Provisioning 2, 3 and 5

An additional two connections, each of 600 M, should be provisioned. Sub-
port 2 will be used as an available resource for these two connections.

Total Link
Service/Connection Bandwidth Sub-Port Used Sub-Port Usage
Usage

1 200 M Sub-port 1
2 200 M Sub-port 1 400 M 400 M
3 400 M Sub-port 1
4 400 M Sub-port 1
5 300 M Sub-port 1 1500 M 1500 M
6 600 M Sub-port 2
7 600 M Sub-port 2 1200 M 2700 M

© Nokia Siemens Networks 11 (18)

High Performance 10GE Module
Bandwidth Management and Provisioning Algorithm

The total occupancy of sub-port 2 is 1200 M, while sub-port 1 maintains

1500 M. Sub-port 2 can accept another 300 M of traffic.

Figure 7: Connection Provisioning 6 and 7

An additional 400 M connection should be provisioned. The Bandwidth
Management entity will not use sub-port 2 for this connection, since, if used,
the 1500 M "fill-in" threshold (1200+400=1600 M) will be exceeded.

Total Link
Service/Connection Bandwidth Sub-Port Used Sub-Port Usage
Usage

1 200 M Sub-port 1
2 200 M Sub-port 1 400 M 400 M
3 400 M Sub-port 1
4 400 M Sub-port 1
5 300 M Sub-port 1 1500 M 1500 M
6 600 M Sub-port 2
7 600 M Sub-port 2 1200 M 2700 M
8 400 M Sub-port 3 400 M 3100 M

Sub-port 3 will be moved to the Load Sharing group and will accept the newly
provisioned 400 M connection.

12 (18) © Nokia Siemens Networks

High Performance 10GE Module
Bandwidth Management and Provisioning Algorithm

Figure 8: Connection Provisioning 8

An additional 300 M connection is to be provisioned. According to the
algorithm, even though adding 300 M to sub-port 2 will not exceed the 1.5G
“fill-in” bandwidth threshold, the connection will be provisioned through sub-
port 3 since this sub-port is the least occupied of the three sub-ports
participating in the Load Sharing group.
The following table indicates the connection provisioning specifications:

Total Link
Service/Connection Bandwidth Sub-Port Used Sub-Port Usage
Usage

1 200 M Sub-port 1
2 200 M Sub-port 1 400 M 400 M
3 400 M Sub-port 1
4 400 M Sub-port 1
5 300 M Sub-port 1 1500 M 1500 M
6 600 M Sub-port 2
7 600 M Sub-port 2 1200 M 2700 M
8 400 M Sub-port 3 400 M 3100 M
9 300 M Sub-port 3 700 M 3400 M

Sub-port 3 occupancy is 700 M.

© Nokia Siemens Networks 13 (18)

High Performance 10GE Module
Bandwidth Management and Provisioning Algorithm

Figure 9: Connection Provisioning 9

An additional three connections should be provisioned: 500 M, 300 M and
800 M. The first two connections (300 M and 500 M) will be provisioned
through sub-port 3 since it is the least occupied at this point, and it will just
meet the 1.5 Gig “fill-in” threshold.
The third 800 M connection will be provisioned through sub-port 4 which will
be moved to the Load Sharing group as well.

Total Link
Service/Connection Bandwidth Sub-Port Used Sub-Port Usage
Usage

1 200 M Sub-port 1
2 200 M Sub-port 1 400 M 400 M
3 400 M Sub-port 1
4 400 M Sub-port 1
5 300 M Sub-port 1 1500 M 1500 M
6 600 M Sub-port 2
7 600 M Sub-port 2 1200 M 2700 M
8 400 M Sub-port 3 400 M 3100 M
9 300 M Sub-port 3 700 M 3400 M
10 500 M Sub-port 3 3900 M
11 300 M Sub-port 3 1500 M 4200 M
12 800 M Sub-port 4 800 M 5000 M

Figure 10 depicts a complete Load Sharing group, where all four sub-ports are
beginning to be used for load sharing of any additional connections to be
provisioned.

14 (18) © Nokia Siemens Networks

High Performance 10GE Module
Bandwidth Management and Provisioning Algorithm

Figure 10: Connection Provisioning 10, 11 and 12

An additional four “wide” connections of 1 Gigbit/sec each should be
provisioned. The first connection will be provisioned through sub-port 4 since
this is the least occupied of the four sub-ports. The second connection will be
provisioned through sub-port 2 and the remaining two connections will be
provisioned through sub-ports 1 and 3.
The following table outlines this provisioning stage:

Total Link
Service/Connection Bandwidth Sub-Port Used Sub-Port Usage
Usage

1 200 M Sub-port 1
2 200 M Sub-port 1 400 M 400 M
3 400 M Sub-port 1
4 400 M Sub-port 1
5 300 M Sub-port 1 1500 M 1500 M
6 600 M Sub-port 2
7 600 M Sub-port 2 1200 M 2700 M
8 400 M Sub-port 3 400 M 3100 M
9 300 M Sub-port 3 700 M 3400 M
10 500 M Sub-port 3 3900 M
11 300 M Sub-port 3 1500 M 4200 M
12 800 M Sub-port 4 800 M 5000 M
13 1000 M Sub-port 4 1800 M 6000 M
14 1000 M Sub-port 2 2200 M 7000 M
15 1000 M Sub-port 1 2500 M 8000 M
16 1000 M Sub-port 3 2500 M 9000 M

© Nokia Siemens Networks 15 (18)

High Performance 10GE Module
Bandwidth Management and Provisioning Algorithm

The current link occupancy is 9 Gig as depicted in Figure 11.

Figure 11: Connection Provisioning 13, 14, 15 and 16

3.3 Sub-port Configurable Thresholds for 10 G

Ethernet Modules
Software Version 3.6.1.e supports a new feature which enables the operator to
change the default thresholds used for traffic distribution at the connection
provisioning stage for 1-Port 10 Gigabit Ethernet, and 1-Port high performance
10 Gigabit Ethernet Interface Modules.
These thresholds can be controlled from the CLI and/or through ASPEN 3.6.1
Build 05.
When controlled through ASPEN, new global setting parameters as well as
per each core device have been implemented.
ASPEN’s global connection settings dialog (Figure 12) provides an option to
configure an Initial Sub-port Group Size and a Sub-port Initial Bandwidth Limit.
The options are separate for the 10 G interface modules and the 10 G high
performance interface modules.

16 (18) © Nokia Siemens Networks

High Performance 10GE Module
Bandwidth Management and Provisioning Algorithm

Figure 12: ASPEN's Global Connection Settings Dialog

Users can individually configure each specific A-8000 series and A-4100
device via the ASPEN Element Manager (Figure 13). For more details please
refer to the addenda document for ASPEN Documentation Release 3.6.1.

Figure 13: ASPEN's Element Manager

© Nokia Siemens Networks 17 (18)

High Performance 10GE Module
Bandwidth Management and Provisioning Algorithm

These thresholds can also be configured through CLI by running the following
commands:
Config-set mpls traffic-eng distribution-parameters initial-subport-bw-limit
Show mpls traffic-eng distribution-parameters
Config-set mpls traffic-eng distribution-parameters auto-setting
Config-set mpls traffic-eng distribution-parameters initial-subport-group-size

3.4 Operational Consideration

By using the thresholds described above, the operator can manually control
the sub-port provisioning algorithm. For a "wide" connection environment
(connections above 500 M to 1 Gig) it is advisable to set the thresholds for
1.5 Gig, i.e., the default value.
For a low-speed connection (1 M to 10 M) environment, the thresholds can be
set to a high number such as 2 Gig or 2.5 Gig.
For an EIR-reach environment, where there is a need to better unitize EIR and
the majority of the connections having a low CIR and a high EIR, the
thresholds should be set to a low number such as 10s or 100s Mbps.

18 (18) © Nokia Siemens Networks