BGP — How the Internet Routes

BGP (Border Gateway Protocol) is the protocol that holds the entire internet together. Every ISP, every cloud provider, every large enterprise speaks BGP to its neighbors to exchange “I can reach these networks” information. When you load any website, BGP decided which path your packets took.

The core abstraction: Autonomous Systems

The internet is divided into Autonomous Systems (AS). An AS is a collection of IP networks under one administrative authority — typically an ISP, large enterprise, or cloud provider. Each AS has a globally unique number (ASN).

• AS 15169 — Google
• AS 13335 — Cloudflare
• AS 16509 — Amazon (AWS)
• AS 32934 — Meta/Facebook
• AS 7018 — AT&T

BGP’s job: tell each AS how to reach every other AS in the world.

How BGP works

Each BGP router has neighbors (called “peers”). Peers exchange routing information continuously:

Router in AS 13335 (Cloudflare) tells its peer in AS 15169 (Google):
  "I can reach 1.1.1.0/24 via AS path [13335]"

Google's router relays to its peers:
  "I can reach 1.1.1.0/24 via AS path [15169, 13335]"

And so on, until every router everywhere knows a path to 1.1.1.0/24.

Each router picks the BEST path from all the options it learns. “Best” considers AS path length, local preferences, MED, weight, etc.

The two flavors

eBGP (External BGP)

Between different AS numbers. Used at internet borders. Each ISP-to-ISP connection runs eBGP.

iBGP (Internal BGP)

Within the same AS. Used at large ISPs/enterprises to distribute external routes to all internal routers.

BGP path attributes (the decision logic)

When BGP has multiple paths to a destination, it picks the best by walking down this list:

1. Highest weight (Cisco-specific, local to one router)
2. Highest local preference (set by your AS to prefer one neighbor)
3. Locally originated routes win over learned routes
4. Shortest AS path (fewer hops between AS numbers)
5. Lowest origin code (IGP < EGP < incomplete)
6. Lowest MED (Multi-Exit Discriminator — neighbor’s hint about preferred entry)
7. External over internal
8. Lowest IGP cost to next hop
9. Oldest route
10. Lowest router ID (tiebreaker)

You don’t memorize this. You configure your network so the right rules cause your preferred outcome.

Peering vs transit

• Transit — you pay an upstream ISP to reach the rest of the internet. Default for small networks.
• Peering — two networks connect directly to exchange traffic between THEIR customers. Free, but only useful when there’s significant mutual traffic.

Big networks (Cloudflare, Google, Akamai) have thousands of peering arrangements. Small networks rely on a few transit providers.

Famous BGP outages

• Pakistan vs YouTube (2008) — Pakistan blocked YouTube domestically by announcing a more-specific route. The announcement leaked globally. YouTube was unreachable worldwide for 2 hours.
• Facebook (October 2021) — Facebook withdrew its own BGP routes during a maintenance error. Facebook, Instagram, WhatsApp went dark for 6 hours globally. Engineers couldn’t even badge into the data centers.
• Rostelecom hijack (2017) — Russian ISP briefly announced routes for major US financial networks. Almost certainly accidental but impossible to fully prove.

BGP’s fundamental problem: any AS can announce any prefix. The internet largely runs on trust, with imperfect filtering.

Defenses

• RPKI (Resource Public Key Infrastructure) — cryptographically signs which AS is allowed to announce which prefix. Deployment growing.
• BGPsec — signs the AS path itself. Theoretically beautiful, deployment painful, mostly stalled.
• Route filtering at peers — manual lists, IRR databases, prefix-list reviews.

See BGP in the wild

# Looking glass — see public BGP route data
# https://www.he.net/3v.cgi
# https://stat.ripe.net/

# Trace which ASes your packets traverse
mtr -bzw google.com
traceroute -A google.com

Running BGP yourself

You don’t need BGP for a typical server. You need it if:

• You operate an ISP or major service provider
• You’re multi-homed (two ISPs, you want failover)
• You have a Provider-Independent IP block

Software: BIRD, FRRouting (FRR), Quagga, Cisco IOS-XR, Juniper Junos.

What to learn next

OSPF — the interior routing protocol that handles routing inside one organization. Up next.