Introduction

This project aims to simulate a Data Center Network (DCN) – which is a physical facility that is
used by organizations to house critical applications and data, based on a network of computing
and storage resources [1]. There exist many DCN topologies i.e., three tier, DCell, fat tree etc,
each hosting a number of core components such as routers, switches, firewalls, servers,
storage.
I will be implementing the Fat tree topology, proposed by M. Al-Fares et al [2], which is derived
from Clos switching network; a multistage switching architecture that reduces the number of
ports required in an interconnected fabric.
For simulation purposes, I will be using Mininet; a network emulator able to create virtual
hosts, swictes, controllers, and links [3]. Mininet supports Software Defined Networking (SDN)
and OpenFlow for custom routing.
In addition, I shall be employing the use of traffic generators for my virtual network and
measuring various parameters for evaluation and analysis.

Fat Tree Network

The Fat tree topology is based on leveraging commodity Ethernet switches for interconnecting
cluster nodes to achieve full bisection bandwidth or oversubscription ratio of 1:1. This means
that all hosts may potentially communicate with other hosts at full bandwidth of their network
interface. The distinctive feature of this topology as pointed out by M. Al Fares is its cost-
effectiveness. It is able to deliver scalable bandwidth at moderate cost as opposed to other
existing techiniques.
The Fat tree topology resembles the tree topology, where we have terminologies such as root,
parent & child. Typical data center architectures have either a two-tier or three-tier trees of
swicthes and routers. A two-tiered design has only the core and edge tiers, whilst the three-tier
also has the aggregation tier.
We shall be looking at three-tier fat tree for my project. This type of topology has three layers
as mentioned above i.e., core, aggregation, and edge. Switches (used to define devices that
perform layer-2 swicthing) eixsting at all three levels of the architecture. For the purpose of this
project, all switches have been assumed to be identical with equal number of ports.

2
The diagram below depicts a three-tier fat tree topology.

The fat tree topology provides a strong case for the use of small commodity switches as
opposed to fewer larger ones due to the price differential.

Mininet
Mininet is a network emulator which creates a network of virtual hosts, switches, controllers
and links. Mininet hosts run standard Linux network software and supports OpenFlow and
Software Defined Networking (SDN) for flexible custom routing.
Mininet is been used widely for research, development, learning, testing, debuging and other
purposes that can be simply implemented on a standard laptop/PC.
A few characteristics:

• Provides relatively simple network testbed

• System regressinon tests and complex topology testing are both supported
• Comes with standardized topologies but also supports arbitrary custom topologies
• It is a Command Line Interface (CLI) application and uses Python APIs for network
creation and experimentation
However, there are a few limitations. Mininet-based networks cannot exceed the available
bandwidth of a single server and cannot run non-Linux compatible applications.

3
Software Defined Networking
SDN is an approach to network architecture which allows the network to be centrally and
logically controlled using using software applications. There are four core characteristics of this
approach:
1. Generalized flow based forwarding as opposed to the conventional destination based
addressing (e.g. Openflow)
2. Control and data plane seperation, with a remote controller interacting with local
control agents to compute and distribute forwarding tables (centralized programming)
3. Control plane functions are transparent/external to the data plane switches (easier
network management)
4. Programmable control applications allowing for open and flexible implementation.

OpenFlow
The OpenFlow protocol is the standard communications protocol that defines the
communication between the SDN controller and network agents i.e., data plane switches,
routers etc. Through this interface, the SDN controller queries switch features, configures
switch parameters, add, delete, modify flow entries in the flow-table to allow network
administrators to partition traffic, control flows for optimal performance, and start testing new
configurations and applications [4].

Topology
I am constructing a three-layer fat tree network topology i.e., core, aggreagation, and edge
layer. In order to contsruct such a topology, we must first know certain information. Each
switch is identical for the purpose of this project with k number of ports. Thus, such a fat tree is
also called k-port fat tree network topology.
From the value of k, we’ll derive the number of core switches, aggregation switches, edge
switches and the maximum number of hosts that can be attached [5].
𝑘 𝑘
There are k pods. A set of (2) number of aggregation switches and (2) number of edge switches
are combined together and that is known as a pod.
𝑘
Number of core switches = (2)2
𝑘
Number of aggregation switches per pod = (2)
𝑘
Number of edge switches per pod = (2)
𝑘
Number of hosts connected to each pod = (2)2

4
For my project, I have chosen k=4, i.e., each switch has exactly 4 ports. We shall now obtain
other parameter for our topology.
Pods = 4
Core switches = 4
Aggregate switches per pod = 2, Total = 8
Edge switches per pod = 2, Total = 8
Hosts per pod = 4, Total = 16
Below is a snippet of the custom topology using Mininet’s graphical user interface i.e., Miniedit.

Ryu Controller
For the purpose of this project, I have used a remote/external controller i.e., Ryu controller.
This is due to the fact that Mininet’s default controller supports upto 16 individual switches.
Our simulation for k=4 has 20 switches in total.
The Ryu controller, implemented in Python is an open source project maintained by open Ryu
community on Github. It is compatible with SDN and is designed to increase the agility of the
network. The controller runs in the background (manually started and shut down) and is
connected to the custom topology on mininet. This is achieved by invoking the
RemoteController class, which acts as a proxy for the controller.
The controller is used to implement simple learning switches or ethernet bridges. Each
individual switch maintains its switch table which comprises of the MAC address of hosts,
interface to reach host etc. It will flood packets on each link/interface that are missing in the
switch table. In addition, they also flood broadcasts like ARP and DHCP. Since our topology
presents a case of multi-path routing, this results in a loop problem and may use up all the
resouces of the laptop/PC causing the system to crash.

5
Spanning Tree
This is solved using the Spanning Tree algorithm or Spanning Tree function. It suppresses the
occurrence of broadcast streams in the network having loops, and is a means to secure network
redundancy by automatically switching the path in case of a network failure.
For my project I have implemented the Spanning Tree Protocol (STP: IEEE 802.1D). It handles a
network as a logical tree and by setting the ports of each switch to transfer frame or not,
suppresses occurrence of broadcast streams in a network having a loop structure [6].

Simulation Process
Start the controller in the background and implement simple_switch_stp_13.py.

Then start the mininet fat tree topology.

By default hosts start with randomly assigned MAC addresses everytime mininet is created,
which makes debugging difficult. The --mac option sets host MAC and IP addresses to small,
easy to read, unique IDs.
Since I have assumed that my nodes are static within the data center thus do not require
dynamic mapping of IP to MAC addresses for ARP entries. The -- arp option permits that entires
are not over-ridden , ensuring stability of network communication.
Furthermore, all links bandwidth has been customized to 10Mbps, thus reflecting the
constraints in resorce availability and allocation in real life data center network topologies.

6
Displaying all nodes and links.

Each pair of host ping each other with pingall command.

7
Results & Analysis
The fat tree network topology is successfully created and all links are active and working. I will
now implement traffic generators for measuring performance parameters i.e., bisection
bandwidth, throughput, delay etc.

Iperf
I have employed the use of Iperf, which is a tool used for active measurement of maximum
achieveable bandwidth in IP networks. It supports various parameters and protocols i.e., TCP,
UDP etc. For each test it reports bandwidth, loss and other parameters.
I have conducted two type of tests: TCP and UDP based tests. The following are the parameters
measured:

• Throughput; the actual number of bits that flows through a network connection in a
given period of time.
• Rounded Trip Time (RTT) or Latency; the amount of time it takes for a packet to make
it from source to destination.
• Jitter (latency variation) or Packet Delay Variation; difference in packet delay.
• Datagram Loss
• Bisection Bandwidth; sum of the bandwidths of the minimal number of links that are
cut when splitting the system into two parts.

8
 Throughput

Zero Datagrams
Lost

Jitter/Packet
Delay Variation
UDP Specified
9 Bandwidth
 Average
Latency/RTT

10
We shall now look at the results obatined to draw conclusions.

Performance Metric Result

Throughput 9.55 Mbps
Latency 0.271 ms
Jitter 0.655 ms
Datagram Loss 0%
Bisection Width 8 links for k=4
Bisection Bandwidth 8 * 10 Mbps = 80 Mbps

Based on the results obtained, we see that throughput achieved is satisfactory, given the
limitations. However, due to significant latency and jitter, the throughput speed has been
affected considerably. One way of addressing this and ensuring full bandwidth communication
between arbitrary hosts, as described by in the paper ‘A Scalable, Commodity Data Center
Network Architecture’ – is to employ multi-path routing techniques such as ECMP. Equal-cost
multi-path routing (ECMP) is a routing strategy where packet forwarding to a single destination
can occur over multiple best paths with equal routing priority. It aims to balance load flows to
increase available bandwidth for usage.
Furthermore, I have modelled my topology to have fixed link bandwidth, this can cause the
overall bandwidth to be limited by the bandwidth available at the root of the tree heirarchy.
Single routing paths between source and destination can quickly lead to bottlenecks up and
down the fat tree, limiting the overalll performance.
Often switches concentrate traffic to another subnet through a specific port even though other
choices may exist with the same cost. This can cause congestion, especially if a small subclass of
core switches are chosen as intermediary links between pods. Once again, a need for more fine-
tuned traffic diffusion methodology is required which takes advantage of the fat tree structure.
This project enabled me to learn a great deal about the fat tree network topology, its
parameters and various dynamics, how to maximize throughput both sustainably and
efficiently, and get some valuable hands-on on tools such as mininet. In addition, I was able to
conduct thorough literature review of the emerging techniques and discussions in this domain
which has expanded my understanding of the topic area.

11
References
[1] “What is a data centre?” Cisco, 03-Sep-2021. [Online]. Available:
https://www.cisco.com/c/en_ca/solutions/data-center-virtualization/what-is-a-data-
center.html.

[2] Mohammad Al-Fares, Alexander Loukissas, and Amin Vahdat. 2008. A scalable,
commodity data center network architecture. SIGCOMM Comput. Commun. Rev. 38, 4
(October 2008), 63– 74. DOI: https://doi.org/10.1145/1402946.1402967

[3] M. P. Contributors, Mininet. [Online]. Available: http://mininet.org/.

[4] “What is openflow? definition and how it relates to Sdn.” [Online]. Available:
https://www.sdxcentral.com/networking/sdn/definitions/what-is-openflow/.

[5] C. Dehury, what is Fat Tree and how to construct it in 4-steps? 01-Jan-1970. [Online].
Available: https://blogchinmaya.blogspot.com/2017/04/what-is-fat-tree-and-how-to-
construct.html.

[6] “Spanning tree,” Spanning Tree - Ryubook 1.0 documentation. [Online]. Available:
https://osrg.github.io/ryu-book/en/html/spanning_tree.html.

12