BGP From Theory To Practice Contents
BGP From Theory To Practice Contents
applications, such as traffic management policies, scalable architectures, stability and security mechanisms,
and its role in big Enterprises and ISP networks. For each topic covered, also the implementation aspects in
Cisco and Juniper technologies are included, essential to fully understand the main mechanisms of the protocol
and its applications. The central idea behind the book is combining theory and practice, to avoid turning it only
into a (debatable) presentation of the standard. This is why, apart from explaining in detail and with many
examples how the protocol works, and its role in IP networks, the book also includes many practical
application suggestions, resulting from many decades of experience.
Flavio Luciani was born in Rome in 1981 and he Tiziano Tofoni graduated in Engineering at the University
graduated Computer Engineering at the Roma Tre of Padua, and obtained a Master’s Degree in Mathematical
University in 2005. Since 2008, he is part of the Namex Statistics at the Florida State University, Tallahassee,
team - the Internet eXchange Point in Rome - first as Florida (USA). He started his career as scholar at the
member of the technical staff, and, since 2020, as Chief Istituto di Dinamica dei Sistemi e Bioingegneria of the
Technology Officer. CNR, in Padua, and as Teaching Assistant at the Statistics
He is currently involved in several Internet Community Department of the Florida State University.
initiatives. He works with the RIPE NCC association, with Then, he joined the staff of the G. Reiss Romoli
the European inter-exchange point association EURO-IX, Post-Graduate School (which was then part of the Telecom
and he is a member of the Steering Committee of the Italian Group), where he worked in the Traffic Engineering
initiative promoted by the Internet Society (ISOC), and Technology sector for the IP networks of Service
Mutually Agreed Norms for Routing Security. Through Providers.
workshops, courses and feature articles, he promotes He held several courses at the University of L’Aquila, and
greater awareness on routing security. he is the author of the first edition of the book “BGP: From
Theory to Practice” and of the book “MPLS Services” (Ed.
Reiss Romoli). He is member of Namex Technical
Committee, honorary member of ITNOG Board, and
Antonio Prado, after an educational background in Chairman of Reiss Romoli srl.
Humanities Studies, lands a PhD in Computer Science. He
has been active in the Internet industry since 1995, and,
after a long experience as CTO for several
telecommunication operators, he works in the Italian
Public Administration, where he deals with digital
infrastructures and digital transition. As journalist, he
spreads knowledge on how the Internet works.
He has been holding classes in public institutions (Schools
and Universities) and private organizations (ISOC, Reiss
Romoli School) for over twenty years. Ambassador and
member of MANRS advisory group, and committed every
day to supporting the development of the Web in Italy, and
to spreading awareness of the Internet Governance,
through articles and conferences. € 60.00
Contents
CONTENTS
FOREWORD ............................................................................................ 1
PRESENTATION ...................................................................................... 3
1 – INTRODUCTION ................................................................................ 9
1.1 HISTORICAL NOTES ..........................................................................................................9
1.2 DEFINING AN AUTONOMOUS SYSTEM ......................................................................13
1.2.1 Single-homed AS ..........................................................................................................14
1.2.2 Multi-homed AS............................................................................................................15
1.3 BGP AND THE INTERNET ECOSYSTEM .....................................................................16
1.3.1 Relationships between ISPs: peering and transit ..........................................................18
1.3.2 Internet eXchange Point (IXP) ......................................................................................19
1.3.3 ISP Classification ..........................................................................................................20
1.4 BASIC OPERATION ..........................................................................................................21
1.4.1 BGP as Path Vector protocol .........................................................................................22
1.4.2 Selecting the best path...................................................................................................23
1.4.3 BGP Process Model ......................................................................................................24
1.4.4 Routing Policies ............................................................................................................25
SUMMARY ...........................................................................................................................28
i
BGP: from theory to practice
ii
Contents
iii
BGP: from theory to practice
iv
Contents
v
BGP: from theory to practice
vi
Contents
vii
BGP: from theory to practice
APPENDIXES........................................................................................ 578
A.1 AS4_PATH AND AS4_AGGREGATOR ATTRIBUTE ...................................................578
A.2 WIRESHARK ANALYSIS OF TCP AND BGP MESSAGES ........................................582
A.3 FINITE-STATE MACHINE .............................................................................................584
A3.1 Events at the core of the FSM .....................................................................................584
A3.2 The finite state machine...............................................................................................585
A.4 REGULAR EXPRESSIONS FOR COMMUNITIES ......................................................587
A.5 GRACEFUL RESTART OPERATION ............................................................................589
A.5.1 Description of the standard mechanism .....................................................................589
A.5.2 Implementation...........................................................................................................592
A.6 ADVANCED MED MANAGEMENT .............................................................................595
A.7 CASE STUDY on RFD’S APPLICATION ......................................................................597
A.7.1 Case Study in Cisco IOS XE environment.................................................................597
A.7.2 Case Study in JUNOS environment ...........................................................................600
A.8 BGP MONITORING PROTOCOL (BMP) ......................................................................605
A.8.1 The protocol ...............................................................................................................605
A.8.2 Router configuration...................................................................................................605
A.8.3 Monitoring station based on pmacct ..........................................................................608
A.9 xBGP.................................................................................................................................609
A.9.1 Beyond SDN ..............................................................................................................609
A.9.2 Valley-free routing......................................................................................................610
viii
FOREWORD
Back in 2011, Reiss Romoli published the first edition of the book “BGP: From Theory to
Practice”, written by Tiziano Tofoni. Many years have gone by since then, and the author of the
first edition deemed it necessary for it to undergo an extensive review, since, in the meantime,
BGP – although quite consolidated – underwent its own evolution. New techniques significantly
improved its security and convergence speed aspects – the two Achilles’s heels of the first BGP
versions.
Despite our different professional journeys, we all have a common denominator in BGP.
According to our ‘vision’, BGP is the standard protocol without which the entire Internet would
not be possible. And this has been proven over the years, since BGP has gained such consensus
that it has become the most important protocol for IP networks – the true supporting structure of
the “Internet ecosystem”.
BGP is based on simple, yet effective concepts, which allow for an extremely flexible use of this
protocol. Although it was born and designed as an inter-domain routing protocol, today BGP is
broadly employed also in other fields, such as:
● In modern public Service Provider networks, where it plays a key role in the overall
routing architecture, because – thanks to its proven scalability – it has turned out to be
a very efficient tool also to distribute external routing information within the network.
● In MPLS services control plane;
● For “painless” IPv4 to IPv6 migration, without major impacts on the backbone of the
Service Providers;
● As private network access protocol to Service Provider networks;
● As IGP in big Data Centers, where it acts as routing protocol on the underlay network,
and as transfer for several kinds of information on the overlay network.
Rather than an actual routing protocol in the traditional sense, BGP is routing policy application
protocol. Indeed, in its definition, the protocol designers did not focus on some of the typical
aspects of standard routing protocols, such as convergence speed and security. Rather, they focused
on making the exchange of large quantities of IP prefixes scalable – and they certainly succeed in
doing so, if we consider that today, in the routers used by large IP networks, BGP can manage the
exchange of routing information related to almost one million IP prefixes.
All this has driven us to follow BGP’s evolution up close, and to spread its knowledge to a vast
audience of insiders. This is how the idea of writing a second edition of the original book came
about. Following the spirit of the first edition, this edition also pursued the goal of combining
theory and practice, and tried not to be only a (debatable) presentation of the standard. This is why,
apart from explaining in detail and with many examples how the protocol works and its role in IP
networks and in the entire Internet ecosystem, the book also includes many practical application
examples, resulting from many years of experience.
In the way it is conceived, the book requires solid notions on TCP/IP architecture, and on IP
routing fundamentals in particular. Moreover, since it covers several configuration aspects, both
1
in Cisco (IOS/IOS-XE/IOS-XR) and in Juniper (JUNOS) environments, it also requires a basic
knowledge of these Operating Systems. Nevertheless, we firmly believe that being knowledgeable
about a specific Operating System is not so important, once the basic concept behind the protocol
and its services have been acquired. Jumping from one technology to another is just a question
of learning the basic commands, and understanding how the protocol was implemented by that
specific developer, with this last aspect being crucial in machine TCL scenarios.
In general, this is an upper-intermediate level book, while the notions on BGP can be read
both by readers with basic knowledge who wish to deepen its concepts, and by those with no
understanding of this standard. It is addressed to the wide audience of Internetworking experts,
both on the Service Provider network and on the private network side (see all institutions such as
Banks, Industries, Public Administrations, many of which have Corporate Networks based on the
IP/MPLS backbone).
We hope that reading it will help, apart from understanding the standard’s theoretical-practical
fundamentals, also to grasp the importance of an intensive use of BGP in IP networks.
Flavio Luciani
Antonio Prado
Tiziano Tofoni
2
PRESENTATION
I was born in 1963 and I’ve always been attracted to technology; telecommunications have always
fascinated me. I’ve always been curious about stories.
One of my favourite stories about telecommunications was about the Reiss Romoli Graduate
School.
It talked about a graduated school founded in 1976, under the initiative of the STET Group, and
then passed on to Telecom Italia, devoted to post-graduate education of young minds who would
be sent for months to L’Aquila, in a campus equipped with all amenities (labs, swimming pool,
gym, library), and there they were trained to become the new ranks of engineers and executives
of the former monopoly.
I met many of these former young people, trained at the Reiss Romoli, who, over the years, went
on to cover strategic positions in Tim and in other Italian ISP, and I’ve listened to their many
stories. Stories of a top-level school.
Stories of young people who studied hard, and who were also young people trying to enjoy that
experience in the best way possible, and so they held “harmless breakouts” at night from this
barrack-school that sometimes did not agree with their age.
Nice stories.
It was rumoured they had very good, and very passionate professors.
To me, the school and its professors were sort of legends, because I never met them. L’Aquila
is a city I’m familiar with. I lived there for a while, when I was working at the Physics Labs
underneath the Gran Sasso mountains. For some reason, I never visited the campus of the Reiss
Romoli School in L’Aquila.
Then, one day, Flavio Luciani, our CTO at Namex, talked to me about meeting one of these
mythical professors, Tiziano Tofoni, and told me about the possibility of working with him.
During an ITNOG event in Bologna, I met Tiziano and “BGP: From Theory to Practice”, the book
he wrote and published in 2011, and discovered it was a stable book in our community. Many of
the technicians and engineers employed in Italian ISPs were formed by that book.
During the conference breaks, we came up with the idea that saw us collaborate all these years
– train the employees of Namex-partner ISPs, through the great experience of the Reiss Romoli
School.
The matter was that many ISPs connecting to Namex needed to train their newly hires and hold
update courses for existing staff. Finding these courses on the market was certainly no easy task.
There weren’t many companies offering training on such specific topics, like the one ISPs are
interested in (BGP, MPLS, DNS, IPv6, etc.), on the market; and even less of them were able to
offer a quality equalling that of the Reiss Romoli School. What’s more, the cost was very high,
especially for smaller ISPs.
There were other lunches after that event. I had the chance to see the old campus at the Reiss
Romoli School, even if only from the outside, since it is no longer open.
We decided to found the Namex School Of Advanced Networking, with the motto “Training
Course for ISPs made by ISPs”.
3
It was 2019.
The first SOAN catalogue started with the classes that were part of the Reiss Romoli catalogue
(3 days, and, in some cases, exam + final certification), which we decided to enrich with
contributions/workshops by our CTO Flavio Luciani and by experts/friends from Namex-member
ISPs, such as Antonio Prado – a benchmark of the Italian ISP and PA community, whom I met
many years earlier, when he was working for one of Namex-member ISPs.
We decided to offer the classes free of charge, using part of the revenue from the services offered
by Namex to the ISPs. Tiziano’s book on BGP was the reference book of the most popular class
– that on BGP.
It was a success. Since then, Namex has provided, in all editions, over 50 classes, training hundreds
of people, and – more importantly – it fostered a moment of aggregation and debate between the
people that “deal with the Internet” in Italy.
It’s something I’m especially proud of, and I hope it will go down in Namex history (small in
absolute terms, yet so big for us living it).
The cherry on top is this new edition of the BGP book, wanted by “Admiral” Tiziano, with his
First Officers Antonio and Flavio. Namex has enthusiastically joined the sponsorship request, right
from the start, counting on the fact that it can continue to be a reference for all those professionals
dealing with interdomain interconnection, and with the Internet ecosystem in general, for many
years to come.
A big thank you to Antonio, Flavio and Tiziano, who worked on this new edition.
A thank you to the Reiss Romoli School, editor of the book, which has trained Italian
telecommunication professionals for decades, keeping the quality level very high.
And lastly, a thank you to Namex-member ISPs, which, with their feedback, prompted us to start
the Namex School Of Advanced Networking and to improve it, year after year.
Maurizio Goretti
Namex CEO
4
ACKNOWLEDGEMENTS
This book is to me a moment of professional and – above all – personal growth. And it wouldn’t
be so, without my two travel companions and friends, Tiziano Tofoni and Antonio Prado, whom I
want to thank from the bottom of my heart. To my family, for their love and patience. To my father,
who would be proud of me.
Flavio Luciani
I would love to thank hundreds of people, because I’ve learned something from each and every
one of them, during my career. The first people I want to thank are my friends Flavio and Tiziano,
with whom I shared this experience (and, I hope, many more to come), and then Mauro Angiolillo,
my inseparable sparring partner, and Professor Fabio Fioravanti, for his precious advice. Lastly,
my parents, for listening to me, my children who always bring me down to earth, and Belinda, my
wife, who has always supported me.
Antonio Prado
During the creation of this book, I benefited from the help of many people, who I’m proud to call
my Friends (with a capital F), and to whom I want to offer my deepest gratitude.
I also wish to thank the many Friends of Italian ISPs with whom I had interesting debates on the
role of BGP and of its actual applications in the networks of ISPs.
Last but not least, as in every book I ever wrote, I want to thank the two women of the house, Vicky
and Fiammix (Maria Vittoria and Fiammetta). Without their evening tiredness (which allowed me
to focus on my work), and without their patience in bearing my constant absent-mindedness, this
book probably would have never seen the light of day. I dedicate this work to them.
Tiziano Tofoni
The authors wish to thank Reiss Romoli srl, editor of the book, Namex CEO Maurizio Goretti,
for the enthusiasm with which he joined the project, and Belinda Menzietti, for her patient and
professional editing.
Last but certainly not least, a big thank you to Simone Morandini for the immense contribution
provided in the revision of this book.
5
Those who fall for practice without science are like the helmsman who enters
a ship without a rudder or compass who never has certainty where he is going.
Always practice must be built on top of good theory.
(Leonardo da Vinci)
7
Introduction
Chapter 1: Introduction
1 – INTRODUCTION
BGP (Border Gateway Protocol) was created as a standard EGP (Exterior Gateway Protocol)
protocol, that is, it was developed to exchange routing information between different Autonomous
Systems (ASes). The version currently used is number 4, defined in RFC 1771 – A Border Gateway
Protocol 4 (BGP-4), March 1995, rewritten with the same title in January 2006 as RFC 4271. Its
main characteristics are the following:
● it is a Path Vector routing protocol, which means it is conceptually similar to a Distance
Vector protocol, although with hops measured in terms of numbers of ASes, instead of
number of routers;
● it supports CIDR (Classless Inter Domain Routing);
● it determines optimal paths through a very complex selection process, based on metrics of
different kinds;
● due to the presence of different types of metrics, it allows the creation of routing policies both
for outbound and inbound traffic in the AS;
● it allows a reliable exchange of routing information, achieved through TCP connections;
● updates are event-driven.
All those features make BGP a routing policy application protocol, rather than an actual routing
protocol. In fact, protocol designers did not consider some of the typical aspects present in IGP
routing protocols when defining it, such as, for instance, speed of convergence, load balancing, etc.
Instead, they focused on making the management of large quantities of IP prefixes scalable – and
succeeded in doing so, if we consider that today, in routers installed in big IP networks, BGP is
capable of managing the exchange of routing information related to hundreds of thousands of IP
prefixes. In this regard, there’s an interesting statement by Yakov Rekhter, who, together with Kirk
Lougheed, may be considered the father of BGP:
“Kirk Lougheed and myself’s goal was to build a routing protocol able to convey 1000 routes and not
fall into pieces. If you think the total routes being today in the Internet, we pushed the envelope a bit.”
The purpose of this chapter is explaining some of the key definitions to understand BGP, and,
above all, define an operating model that will be constantly referenced in the next chapters.
9
BGP: From Theory to Practice
complexity. Eric Rosen – who worked as Engineer at Bolt Beranek and Newman Inc. at the time
– pointed out (in RFC 827 – Exterior Gateway Protocol (EGP), October 1982) the flat model’s
scalability issues, and the need to adopt a hierarchical model, dividing the Internet into a set of
ASes, i.e. networks managed by the same administration. One of the ASes – called Core AS –
comprised ARPANET and SATNET, and worked as the Internet’s Backbone. All the other ASes
– called stub AS – were connected by one or more routers (Exterior Gateways) to the Core AS.
Generally speaking, communication between stub ASes occurred through the Core AS. The
exchange of routing information between ASes was delegated to a new protocol, standardized in
RFC 827.
However, in a matter of years, due to the continuous growth of the Internet, EGP revealed many
limitations, essentially linked to the fact that it had been designed based on the Internet Hub-and-
Spoke model (all the stub ASes (Spoke) connected to a single Core AS (Hub)). Among them:
● the lack of loop avoidance mechanisms;
● the fact that only classful routing was supported (RFC 1817 – CIDR and Classful routing,
August 1995);
● the fact that the routing information communication mechanism was based, as in Distance
Vector protocols, on periodic transmission of the entire IP routing table to the nearest
neighbors (the timeframe was set to 2 minutes);
● the impossibility to define inbound and/or outbound traffic management policies in an AS.
In January 1989, at the 12th IETF Meeting in Austin, Texas, Yakov Rekhter and Kirk Lougheed –
Head Researcher at IBM’s T.J. Watson Research Center the former, and Engineer at Cisco Systems
the latter – sat down at a table (according to “The Packet” newsletter, Volume 1, Number 2,
published in Winter of 1989 by Cisco Systems, Leonard Bosack, Cisco co-founder, was also with
them) and laid the foundations for a new inter-AS routing protocol – Border Gateway Protocol
(BGP) – jokingly called The Two-Napkin Protocol, because BGP’s initial project was drawn on
some restaurant napkins (which, some say, were even soiled with ketchup!).
Lougheed himself shed some light on this episode, by stating: “After I wrote up the notes on two
napkins, I made a second copy, also on napkins. I gave Yakov that first copy and took the second
copy for myself. Apparently Yakov made photocopies of his napkins and these photocopies are
the origin of the images you’ve seen. Having no sense of history, I discarded my napkins at some
point.”
Photocopies of the drawings contained in those napkins are now displayed on the walls of the
Routing Protocol Development department, in Cisco Systems headquarters in Santa Clara, CA.
They are shown in Figure 1.1.
In a short while, the first practical implementation of BGP was created from this draft, and then
the first standard, defined in RFC 1105 – A Border Gateway Protocol (BGP), June 1989, written
by Yakov Rekhter and Kirk Lougheed.
NOTE: In Lougheed’s words: “Once back home I started drafting what eventually became
RFC 1105. Yakov and I passed that document back and forth while developing and refining
our own implementations in a classic iterative process. By the time RFC 1105 was published
there were two implementations, my Cisco router implementation and Yakov’s IBM router
implementation on the NSFnet backbone. We naturally tested for interoperability. I think the
gated implementation came out after RFC 1105.”
10
Introduction
Chapter 1: Introduction
The standardization process involved several stages, which led to the definition of a second and
third version, published in RFC 1163 of June 1990 and in RFC 1267 of October 1991, respectively.
Version 4 – the final one – was published in March 1995, in RFC 1771 – A Border Gateway
Protocol 4 (BGP-4), written by Y. Rekhter and T. Li, who worked for Cisco Systems. RFC 1771
was rewritten in January 2006 by the same authors, alongside S. Hares, in RFC 4271.
From Rekhter and Lougheed’s napkins was born the most important protocol of today’s Internet,
the one that glues the “network of networks” together. BGP-4 is de facto the standard protocol,
universally employed as inter-domain routing protocol. Over the years, it underwent continuous
updates, which enhanced its functions, stability and scalability.
Despite being created as an inter-domain routing protocol, over time, BGP expanded its field of
application, and today it is employed:
● in modern public networks of big ISPs (Internet Service Provider), where it plays a key role
in the overall routing architecture, because – thanks to its proven scalability – it has turned
out to be a very efficient tool also to distribute IP prefixes external to the AS inside an AS;
● in the control plane of VPN services based on the BGP/MPLS model;
● as an access protocol of private networks (Enterprise networks) to ISP networks.
Without fear of contradiction, we can say that BGP is the most important routing protocol for IP
networks. The “Three Napkins Protocol”
Copyright Reiss Romoli BGP: dalla teoria alla pratica Versione 2.1 Aprile 2020 1
Figure 1.1 a – The Two-Napkins Protocol
11
BGP: From Theory to Practice
Through the eyes of a modern provider, we must admit that BGP, despite its best intents, was born
with an original sin: it assumes that all Internet networks are reliable secure.
The fact that it has been created before security (in a broad sense) became an issue, has marked its
development since the 1990s. This aspect – which we will explore in depth in Chapter 10, when
we talk about security – can be easily summarized in Internet expert Randy Bush’s scathing yet
spot-on line: “You’re in Hackerville here on the Internet. Period. All of this stuff lacks formal
discipline. It’s paint and spackle”.
12
Introduction
Chapter 1: Introduction
GP GP
GP
A A
Figure 1.2 – Relationship between IGP and EGP and ASes.
Until 2007, the AS numbers available were only those taken from a 16-bit space that fell within
the 0 - 65535 interval. However, only those between 1 and 64511 could be publicly assigned (and
not every one of them, see the next note). Values between 64512 and 65534 cannot be assigned
to public ASes (i.e. directly on the Internet), and are reserved for private use. The last value,
65535, is reserved (RFC 7300 – Reservation of Last Autonomous System (AS) Numbers, July
2014). Generally speaking, they are BGP:
Copyright Reiss Romoli
assigned by ISPs to customers that use BGP
dalla teoria alla pratica
as an access
Versione 2.1 Aprile 2020
protocol to their network. Value AS=0 is reserved to certain BGP security aspects (see RFC 7607
– Codification of AS 0 Processing, August 2015), covered in Chapter 10.
NOTE: Values between 64496 and 64511 cannot be assigned to a public AS; they are reserved
to documentation (RFC 5398), and will be used extensively in this textbook. Number 112 is also
unavailable (RFC 7534 – AS112 Nameserver Operations, May 2015), as it has been destined
to the special purpose of hosting anycast instances for authoritative DNS servers for reverse
resolutions of IPv4 and IPv6 address spaces that cannot be routed on the Internet (for instance,
those described in RFC 1918 – Address Allocation for Private Internets, February 1996, but not
13
BGP: From Theory to Practice
only those). AS number 23456 (better known as AS_TRANS) cannot be freely assigned, because
it is used to facilitate communications between a router that doesn’t support the 32-bit AS
notation, and one that uses a 32-bit AS (RFC 6793 – BGP Support for Four-Octet Autonomous
System (AS) Number Space, December 2012). More details on this in Annex A.1.
The limited availability of public AS numbers led to a 32-bit expansion of the AS number,
standardized in RFC 4893 – BGP Support for Four-octet AS Number Space, May 2007.
NOTE: AS representation was done through a 16-bit number, and then through a 32-bit number
(e.g., the AS number 65551 can be represented as 65551 in the asplain format or as 1.15 in the
asdot+ format), as regulated by RFC 5396 – Textual Representation of Autonomous System (AS)
Numbers, December 2008. For further details, see Annex A.1.
Even in the 32-bit space, ASes have special use reserved numbers. Indeed, the 65536-65551 interval
is reserved to documentation, see RFC 5398 – Autonomous System (AS) Number Reservation for
Documentation Use, December 2008, the 4200000000-4294967294 interval (approx. 95 million
ASes) is for private use, see RFC 6996 – Autonomous System (AS) Reservation for Private Use,
July 2013, and the last, number 4294967295, is reserved to possible future uses, see RFC 7300 –
Reservation of Last Autonomous System (AS) Numbers, July 2014.
Based on their outgoing connection and on how transit traffic is processed, ASes can be classified
into:
● single-homed ASes: characterized by a single (stub AS) or redundant connection toward only
one other AS;
● multi-homed ASes: characterized by more than one connection toward several ASes.
NOTE: In the literature, definitions are not always concordant. Indeed, very often, redundant
connectivity toward a single AS is also defined as multi-homed. For the sake of clarity, in this
textbook, we prefer to use the term home for an AS, hence our classification .
1.2.1 Single-homed AS
A single-homed AS is characterized by a single connection, or, as it generally occurs in practice,
a redundant connection toward another AS. In case of single connection, we talk about stub AS.
A typical example of stub AS is a private network AS, or a small ISP connecting to the network of
a larger ISP, through a single connection. A stub AS with a single connection toward the ISP does
not need to know all the Internet prefixes. In fact, with a single connection pointing outside (see
Figure 1.3), reachability of the prefixes outside the AS can be guaranteed through a simple default
route on the access router connected to the ISP’s network.
Therefore, in theory, a stub AS doesn’t need to use BGP on its access router to exchange routing
information with the ISP to which it is connected. Some stub ASes use BGP anyway as access
protocol, even in similar situations, to compensate, for instance, for Level 2 network deficiencies
(e.g. access via an Ethernet network, where convergence may be slow).
14
Introduction
Chapter 1: Introduction
Access network
P
Stub AS
1.2.2 Multi-homed AS
As we were saying, typical examples of a multi-homed AS include private network ASes, or small
ISPs, that connect to the network of two or more public ISPs for reliability reasons, or ISPs that
exchange IP routing information with more than one AS (see Figure 1.4). The exchange of routing
Versione 2.1 Aprile 2020
Copyright Reiss Romoli BGP: dalla teoria alla pratica
information between ASes occurs through BGP, or rather, as we will see in the next chapter,
through BGP sessions.
We can identify two types of multi-homed AS:
● multi-homed Transit ASes: they allow exchange of traffic between different ASes, using their
own resources as transit;
● multi-homed Non-Transit ASes: they do not allow external traffic to transit on the AS.
Connetti it lti o e
NOTE: For an AS, transit traffic is defined as the set of IP packets with both source and destination
addresses that do not belong to any of the IP subnets used by the AS.
P1
A
BGP sessions
P2
A
Copyright Reiss Romoli BGP: dalla teoria alla pratica Versione 2.1 Aprile 2020
15
BGP: From Theory to Practice
P1
A
P P
P
P
BGP sessions
A P1 P2
P P P P
P2
A
The entire Internet can be modeled as a flow graph, where nodes consist of ASes, and connections
between nodes consist of BGP sessions (Figure 1.6). BGP sessions are logical connections between
routers, on which routing information is exchanged. This information, suitably propagated
between all ASes, allows reaching all system devices (hosts), and therefore the entire great ocean
of information on the Internet.
Routing information comprises pairs of the following kind: <IP prefix, prefix length>. For the sake
of simplicity, in this textbook we will refer to those pairs simply as IP prefixes, using the classic
notation “IP prefix/IP prefix length” (e.g. 203.0.113/24). Every AS injects a set of IP prefixes – that
is, IP address blocks generally assigned by a RIR to the AS administrator (or at least, that’s how
it should be) – into the system.
16
Introduction
Chapter 1: Introduction
NOTE: Observing the hierarchy when managing numerical resources (hierarchy that from
IANA proceeds to RIRs, and from those to NIR/LIR) is essential for Internet operation. In fact,
it is worth remembering that, failing to observe the hierarchy can cause service interruptions in
particular areas of the Internet, at best, and, in worst case scenarios, it can unleash illicit behavior
that constitutes punishable crimes.
Logically announced prefixes within an AS are automatically propagated (unless routers are
instructed to behave otherwise) on the different BGP sessions, following the rules described in
Paragraph 2.1. Propagation can be seen as a selective flooding mechanism, through which IP
prefixes are spread to all ASes of the Internet system.
In order for the system to work, it is not necessary to spread all IP prefixes to all the routers on the
Internet. The type of prefixes to spread and their recipients basically depend on the AS topology
and function. For instance, as mentioned earlier in Paragraph 1.2., it is not necessary to spread all
the IP prefixes of the Internet world to the routers of a stub AS. Because of the way the stub AS is
odello dell nternet
connected to the AS graph, it just requires a default route that allows it to reach one AS, which in
turn is capable of reaching all the Internet prefixes, through a given path.
BGP sessions
Figure 1.6 – Internet model.
Copyright Reiss Romoli BGP: dalla teoria alla pratica Versione 2.1 Aprile 2020
17
BGP: From Theory to Practice
Peering
S A
S
sit
S rans
it
ran
S A S
Copyright Reiss Romoli BGP: dalla teoria alla pratica Versione 2.1 Aprile 2020
18
Introduction
Chapter 1: Introduction
● Cost: fixed costs related to being associated with an IXP (including interconnection costs
toward the IXP data center and toward the Fabric) are generally lower (per exchange
bandwidth unit) compared to Internet transit costs. Most of the times, peering agreements
between participants to an IXP take place free of charge, which makes access to the Internet
cheaper, and therefore available to a larger number of end users in a certain country or region
(think about developing economies).
The typical infrastructure of an IXP – also called IXP Fabric – consists of one or more switches
to which different participants connect their routers. In addition, it might include servers through
which the IXP provider offers additional services to its participants (e.g. aggregate and target AS
traffic statistics), as well as services that allow for correct Internet operation: e.g., anycast replicas
of root name servers and analysis tools such as Routing Information Services (RIS), managed by
Regional Internet Registries and hosted precisely inside the IXP infrastructures.
NOTE: The most used switching technology in IXPs has switched from ATM (very popular in
the 1990s) to Ethernet. Some IXP migrated to more scalable solutions, such as the IP Fabric with
VXLAN transport and EVPN control plane.
19
Internet eXchan e Point IXP
BGP: From Theory to Practice
S S
S S
S Ro te er er
IXP Fabric
er er or P ser ices
S
ilateral BGP peering S S S
of ISPs within the IXP it wants to exchange routing information with, just needs a BGP peering
toward a Route Server, or rather, by redundancy, two BGP peerings toward two Route Servers, to
drastically reduce the number of BGP peerings. In this case, we talk about multilateral agreements.
As mentioned earlier, the main purpose of an IXP is providing a physical infrastructure, through
which network providers can interconnect and exchange traffic, gaining common advantages.
Over the years, the role of IXPs has changed. Through an increasingly broader and diversified
participation, they expanded their services from simple peering to facilitating a market where
participants can purchase different services they need from other participants (e.g., DDoS
mitigation services as well as access services). On one hand, this new nature is a strength for
IXPs, which can attract more and more providers (which consider participation as a new business
opportunity); on the other hand, participants can benefit from a broader service range.
20
Introduction
Chapter 1: Introduction
between Tier-2 ISPs occurs through direct private connections (PNI – Private Network
Interconnection), often achieved using the passive interconnection infrastructure (MMR,
Meet-Me-Room). In this case, we talk about private peering, as opposed to public peering,
where ISPs connect using the IXP Fabric.
● Third level (Tier-3): a network that must necessarily purchase a right to transit from other
networks (at least two) to reach the Internet. Usually, Tier-3 ISPs work within a limited
territory (usually a region), and are very aggressive in their pricing policies. Their clients
generally include retail or small businesses. Usually, Tier-3 establish peering agreements with
Content Providers; interconnection often occurs within the IXP, and prefixes are exchanged
through Route Servers (Content Providers, with their PoP scattered throughout the world,
try to limit the number of BGP sessions, by enabling bilateral sessions only above a certain
traffic threshold).
NOTE: The need to have at least two transit relations with other independent systems is one of
the requirements a RIR may request in order to assign an AS number. See document RIPE-679 –
Autonomous System (AS) Number Assignment Policies, March 2017, that reads: “A network must
be multi-homed in order to qualify for an AS Number.”. The RIR established practice consists
in deeming even the subscription of a transit agreement with two different independent systems
sufficient, without necessarily requiring the interconnections to be operational.
There are many reasons why network professionals use level hierarchy to describe the network,
the most important being a greater understanding of the political and economic reasons behind a
network, based on how and with whom it communicates.
NOTE: We should specify that the classification suggested and used herein, only aims at showing
what we observe every day in the Internet ecosystem. This means that, since the Network is
perpetually changing, an independent system can shift from one category to the other, based on
the expansion and growth policies it applies.
21
BGP: From Theory to Practice
Even the aggregation of IP prefixes – which we’ll see in Chapter 5 – can be a valid tool to ensure
stability.
Figure 1.9 below summarizes the basic operation described herein. In the figure, the BGP table is
a memory area where the router keeps the BGP advertisements received from each neighbor with
whom it has established a BGP session.
A Ad ertisement P1 P2 A
P1 P2 P P
BGP ta le
BGP ta le
P1 Ad ertisement P P P1
P2
P2
P
P
P
ithdra P P
BGP session
Even though its operation resembles a Distance Vector protocol, BGP does not actually belong
to the Distance Vector type, nor to the Link State type. It belongs to the Path Vector type. The
reason behind this, is the presence of a special attribute – the Path Vector (which, as we will see,
is actually called AS_PATH) – a sorted list (Vector) of the AS crossed by the BGP advertisements.
Path
Copyright ReissVector
Romoli indicates the path to BGP:
crossdallato reach
teoria a prefix, in terms of ASes. For
alla pratica instance,
Versione in
2.1 Aprile 2020
Figure 1.10 below, prefix 192.0.2/24, belonging to AS 64503, will be announced to AS 64501 both
by AS 64502 and by AS 64505. In the first advertisement, the Path Vector is [64502 64503], while
in the second is [64505 64504 64503]. This is why BGP is referred to as a Path Vector protocol.
22
Introduction
Chapter 1: Introduction
ert se ent
Pre i : 1 2.0.2 2
ert se ent Path Vector: 0
Pre i : 1 2.0.2 2
Path Vector: 02 0
A 0
A 02
1 2.0.2 2
A 01
ert se ent
Pre i : 1 2.0.2 2
Path Vector: 0
ert se ent
Pre i : 1 2.0.2 2
Path Vector: 0 0 0
A 0
A 0 ert se ent
BGP sessions Pre i : 1 2.0.2 2
Path Vector: 0 0
Figure 1.10 – The Path Vector attribute. Versione 2.1 Aprile 2020
Copyright Reiss Romoli BGP: dalla teoria alla pratica 1
We will see later on how the Path Vector plays an essential role in BGP operation.
From the basic operation description above, we can notice once again how BGP is conceptually
very simple, and maybe this explains also why it is so flexible. The complex part of BGP concerns
one of the essential aspects it was designed for, that is its routing policy definition applications.
And this is surely the most important and interesting aspect in this protocol.
23
BGP: From Theory to Practice
A n A out
np t Polic
tp t Polic
. BGP selection .
oc R B
process
. .
n ine
. .
A n A out
The partition of the memory used by the BGP process in Adj-RIB-in, Loc-RIB and Adj-RIB-out is
purely logical, and it is the one described in standard BGP documents (RFC 1771 and RFC 4271).
In practical implementations, each device manufacturer manages the memory areas to be assigned
to the BGP process as they best see fit. Hereinafter, for the sake of simplicity, we will define the
BGP table as the set of advertisements deemed valid for the selection process, i.e., the set of all
BGP advertisements received that have been processed by inbound routing policies.
NOTE: The best paths determined by the BGP selection process are not necessarily installed in
the IP routing table (RIB, Routing Information Base) and then transferred to the FIB (Forwarding
Information Base) to be used in traffic forwarding. Indeed, if the same prefix is announced to
24
Introduction
Chapter 1: Introduction
the router via BGP and also in other ways (dynamic routing protocol, static routing, directly
connected prefix), the router chooses which advertisement to install in the RIB, based on a level
of preference assigned by the router to each routing protocol. Please be aware that the level
of preference (known in Cisco documents as administrative distance, or in Juniper documents
as preference value), is a number assigned locally by the router to each routing protocol, and
expressing a level of preference for the protocol. For instance, let’s assume that a prefix Pfx
is announced to the router both by BGP and by OSPF. Which of the two protocols does the
router consider more reliable? In other words, what information <Pfx, Next-Hop> will be
installed in the RIB, the one announced by OSPF or the one announced by BGP? The rule used
by all manufacturers is preferring the advertisement of the protocol with the lowest preference
level. Note that preference level values are assigned by manufacturers according to different
logics, and so it is common to see completely different numbers, which can be varied through a
configuration, if needed.
The complex part in the practical implementation of BGP, consists of two routing policy blocks,
which are the core of the protocol. In Section 1.4.4 below, we will see what routing policy means,
and throughout the textbook we will go over the tools available to implement them, along with
several practical applications.
25
BGP: From Theory to Practice
Manipulation of BGP attributes allows one to change the value of BGP attributes, according to
one’s needs. Having control over BGP attribute values allows conditioning the choices of the
best path selection process, and therefore defining suitable AS inbound and/or outbound traffic
management policies.
There are many different routing policies, with obvious applications in case of multi-homed ASes
or stub ASes with redundant connections. For stub ASes with single connection, it makes little
sense to speak about routing policies, except for aspects relating to advertisements filtering, since
stub ASes have no alternative ways to manage inbound and/or outbound traffic. Some examples
of routing policies are:
● sending/receiving traffic using a first-choice AS (primary AS), and, in case of loss of
connectivity toward it, using a backup AS;
● balancing inbound and/or outbound traffic between two or more paths;
● choosing the most convenient paths, based on the prefix. For instance, in a primary/backup
configuration, it may be convenient to make traffic toward local prefixes of backup ASes pass
through a backup connection.
The practical implementation of routing policies requires complex configurations that use specific
tools made available by the BGP implementations of the different manufacturers. We will go over
the tools made available by Cisco and Juniper platform later in this book.
Figure 1.12 shows an example of a routing policy application to check the advertisements
received/propagated, and affect the results of the selection process.
Re ect 0 0 rom A 02
A 0
o not propagate 0 0
Assign all pre i es rom A 02
A 01 a grade o greater pre erence o not propagate 1 2.0.2 2
to A 0
Accept only ad ertisements o
pre i es ith mas 2 Assign to 1 2.0.2 2 to ards
A 0 etric 20
1 2.0.2 2 etric 20
1 2.0.2 2 1 . 1.100 2
20 .0.11 .0 2
0 0 de a lt ro te
in o nd o t o nd
BGP est paths
ro ting ro ting
ta le
policies policies
1 2.0.2 2 1 . 1.100 2
0 0 de a lt ro te
se the de a lt ro te o A 01
A 0
A 02
26
Introduction
Chapter 1: Introduction
By applying the inbound routing policies specified in the figure, we obtain the following results:
● the default route advertisement sent by AS 64502 is rejected, while the one sent by AS 64501
is accepted;
● the advertisement of prefix 203.0.113/26 sent by AS 64501 is rejected, because the prefix
mask is too big (greater than /24);
● the advertisements of prefix 192.0.2/24 sent by AS 64501 and 64502 are both accepted,
however, the advertisements coming from AS 64502 is assigned a greater preference value
(through the special BGP attribute Local Preference, which we’ll see in the next chapter).
The advertisements accepted are stored, along with their attributes, in the BGP table, and the
selection process is applied to them, with the following results:
NOTE: The BGP table is a set of BGP advertisements, with the purpose of providing information
on how to reach the networks of the different autonomous systems. We should highlight that each
BGP advertisement in the table is linked to the AS that originated it on the Internet.
The BGP table also includes advertisements of prefix 198.51.100/24, which becomes the best
path, as it is the only one present.
Lastly, let’s consider best path propagation only in BGP sessions toward AS 64503 and AS 64504.
Before being propagated, prefixes undergo the outbound routing policies, with the following
results:
● the best path of prefix 192.0.2/24 is propagated only to AS 64503 with metric 20 (through the
special BGP attribute MED, which we will see in the next chapter);
● the best path of prefix 198.51.100/24 is propagated to both AS 64503 and 64504;
● the best path of the default route undergoes an output filtering process and is not propagated
(although it remains in the BGP table).
This basic example shows that there can be many different routing policies, and, if they are defined
well, they can meet (almost) all network administrators’ needs.
27
BGP: From Theory to Practice
SUMMARY
In this opening chapter, we described the role BGP plays in the Internet ecosystem, and its manifold
applications in Service Provider networks; then, we paved the way for a conceptual model on
which the BGP process operation depends. Everything we presented here will be explored further
in the next chapters.
Worth remembering:
1. BGP’s role in the Internet ecosystem and its applications in the services offered by Service
Providers (e.g. BGP/MPLS services).
2. The concept of Autonomous System and its numbering method. In addition, you should
remember the AS classification into single-homed (stub AS) and multi-homed, and the
further division of the latter into transit AS and non-transit AS.
3. BGP’s basic operation, BGP sessions and the Path Vector protocol. In particular, the BGP
UPDATE message exchange, and the best path selection.
4. The operating model described in Figure 1.11, which will be the logic behind the entire
textbook.
5. And last but not least, what is perhaps BGP’s greatest value, the option of creating routing
policies on AS inbound and outbound traffic.
28
The book describes the main aspects of BGP (Border Gateway Protocol) and its most important practical
applications, such as traffic management policies, scalable architectures, stability and security mechanisms,
and its role in big Enterprises and ISP networks. For each topic covered, also the implementation aspects in
Cisco and Juniper technologies are included, essential to fully understand the main mechanisms of the protocol
and its applications. The central idea behind the book is combining theory and practice, to avoid turning it only
into a (debatable) presentation of the standard. This is why, apart from explaining in detail and with many
examples how the protocol works, and its role in IP networks, the book also includes many practical
application suggestions, resulting from many decades of experience.
Flavio Luciani was born in Rome in 1981 and he Tiziano Tofoni graduated in Engineering at the University
graduated Computer Engineering at the Roma Tre of Padua, and obtained a Master’s Degree in Mathematical
University in 2005. Since 2008, he is part of the Namex Statistics at the Florida State University, Tallahassee,
team - the Internet eXchange Point in Rome - first as Florida (USA). He started his career as scholar at the
member of the technical staff, and, since 2020, as Chief Istituto di Dinamica dei Sistemi e Bioingegneria of the
Technology Officer. CNR, in Padua, and as Teaching Assistant at the Statistics
He is currently involved in several Internet Community Department of the Florida State University.
initiatives. He works with the RIPE NCC association, with Then, he joined the staff of the G. Reiss Romoli
the European inter-exchange point association EURO-IX, Post-Graduate School (which was then part of the Telecom
and he is a member of the Steering Committee of the Italian Group), where he worked in the Traffic Engineering
initiative promoted by the Internet Society (ISOC), and Technology sector for the IP networks of Service
Mutually Agreed Norms for Routing Security. Through Providers.
workshops, courses and feature articles, he promotes He held several courses at the University of L’Aquila, and
greater awareness on routing security. he is the author of the first edition of the book “BGP: From
Theory to Practice” and of the book “MPLS Services” (Ed.
Reiss Romoli). He is member of Namex Technical
Committee, honorary member of ITNOG Board, and
Antonio Prado, after an educational background in Chairman of Reiss Romoli srl.
Humanities Studies, lands a PhD in Computer Science. He
has been active in the Internet industry since 1995, and,
after a long experience as CTO for several
telecommunication operators, he works in the Italian
Public Administration, where he deals with digital
infrastructures and digital transition. As journalist, he
spreads knowledge on how the Internet works.
He has been holding classes in public institutions (Schools
and Universities) and private organizations (ISOC, Reiss
Romoli School) for over twenty years. Ambassador and
member of MANRS advisory group, and committed every
day to supporting the development of the Web in Italy, and
to spreading awareness of the Internet Governance,
through articles and conferences. € 60.00