73. BGP operations: peering, scaling, and failures

73.1 Business relationships shape routing

CUSTOMER   pays you      -> you advertise their prefixes to EVERYONE
PEER       settlement-free -> you exchange only YOUR and YOUR CUSTOMERS' prefixes
PROVIDER   you pay them  -> they give you a default/full table; you advertise
                             only yourself and your customers to them

The "valley-free" rule that falls out of this:
   traffic goes  customer -> provider  (up),  then across a peer link,
   then  provider -> customer  (down).  It never goes UP after coming DOWN.

Violating this by accident is a route leak (§73.4). Connections happen at IXPs (internet exchange points) via a shared fabric and often a route server that redistributes among many participants.

73.2 Scaling iBGP

The full-mesh requirement (§72.1) is quadratic. Two solutions:

Solution	How	Trade-off
Route reflectors (RR)	Designated routers may re-advertise iBGP routes to clients. Clients peer only with the RRs.	Simple and dominant. RRs reflect only their best path, which can hide alternatives and cause suboptimal routing; use redundant RRs and `add-path` to mitigate.
Confederations	Split the AS into sub-ASes that eBGP-peer with each other internally	More complex; rare outside very large networks

Loop prevention within reflection uses ORIGINATOR_ID and CLUSTER_LIST.

73.3 Session states and why sessions won't come up

Idle -> Connect -> Active -> OpenSent -> OpenConfirm -> ESTABLISHED

Symptom	Cause
Stuck Idle	No route to the peer, or administratively shut, or damped
Flapping Active	TCP 179 blocked, wrong peer IP, ACL, or the peer isn't configured for you
Stuck OpenSent/OpenConfirm	AS number mismatch, router-ID collision, unsupported capability, or MD5/TCP-AO password mismatch
Established but no prefixes	Filters/route-maps rejecting everything, or no `network` statement; check `show bgp neighbor x advertised-routes` and `received-routes`
Session drops every ~90 s	Hold timer expiry — usually MTU (large updates dropped) or CPU

eBGP is TTL 1 by default (peers must be directly connected). For multihop peering use ebgp-multihop, and prefer GTSM (RFC 5082, ttl-security) which requires the packet to arrive with TTL ≥ 254 — a cheap anti-spoofing control.

Protect the session: TCP-MD5 (RFC 2385) or the modern TCP-AO (RFC 5925), plus infrastructure ACLs so only your peers can reach port 179.

73.4 The three ways BGP breaks the internet

1. Prefix hijack. An AS announces a prefix it doesn't own. Because longest-prefix match wins (Ch. 13.2), announcing a /24 out of someone's /16 steals the traffic globally. (Pakistan/YouTube 2008; repeated crypto-DNS thefts.)

2. Route leak. An AS re-announces routes it shouldn't — e.g. re-advertising one provider's routes to another provider, becoming an accidental transit for traffic it cannot carry. Usually a filter mistake, and usually causes congestion and blackholing rather than theft.

3. Instability / flapping. A flapping link generates constant updates that propagate globally. Route flap damping suppresses repeatedly-flapping prefixes (historically too aggressively; modern RIPE-recommended parameters are gentler).

73.5 Defences

Control	What it does
Prefix filters / IRR	Only accept prefixes a customer is registered (in RADB/RIPE) to originate. The oldest and still-essential control.
RPKI + ROV	A ROA cryptographically states "AS X may originate prefix P with max length L". Routers validate and drop Invalid routes. The single highest-value modern defence — publish ROAs for every prefix you own.
Max-prefix limits	Tear down a session that suddenly sends 500,000 routes instead of 50. Catches leaks fast.
AS-path filters	Reject paths containing your own ASN, or unreasonable lengths
Bogon filtering	Drop private/reserved/unallocated space
BGPsec	Cryptographic path validation (not just origin). Barely deployed.
MANRS	An industry programme bundling filtering, anti-spoofing (BCP 38), coordination, and validation

Peer locking and RPKI-based origin validation together stop most real incidents. If you run BGP and do nothing else, publish ROAs and reject Invalids.

73.6 Real-world example

A regional ISP's new engineer redistributed the full BGP table into OSPF and back into BGP without filters. For eleven minutes the ISP advertised ~400,000 prefixes to both its upstream providers, claiming to be the best path to much of the internet. Traffic that its 10 Gbit/s links could never carry arrived and was dropped. Two things limited the damage: one upstream had a max-prefix limit of 1,000 and shut the session in seconds; the other did not, and carried the leak for the full duration. Max-prefix limits are the cheapest insurance in networking.

Try it yourself

# is a prefix RPKI-valid?   https://rpki-validator.ripe.net  or:
curl -s "https://stat.ripe.net/data/rpki-validation/data.json?resource=AS15169&prefix=8.8.8.0/24" \
  | python3 -m json.tool | grep -i status

# who originates this prefix, and has it changed?
curl -s "https://stat.ripe.net/data/prefix-overview/data.json?resource=1.1.1.0/24" | python3 -m json.tool | head -30

# monitor your own prefixes for hijacks: BGPalerter, RIPE Atlas, or bgp.tools