BGP
Core17
What is BGP, why do we need it, and what is an autonomous system?
Say this first
BGP version 4 (RFC 4271) is the routing protocol between autonomous systems, the one that holds the internet together. An autonomous system (AS) is a set of networks run by one organisation with one routing policy, identified by an AS number. An IGP like OSPF finds the best path inside one network; BGP exchanges reachability between organisations and lets each one apply its own policy, at a scale of roughly a million IPv4 prefixes.
Go deeper
Why not run OSPF across the internet
- Scale. A link-state protocol floods every topology change to every router and runs SPF over the whole graph. That works for hundreds of routers, not for the whole internet.
- Policy. OSPF picks the lowest cost. Between companies you need rules like "never send my traffic through that network" or "prefer the cheaper transit provider". BGP decisions are driven by attributes and policy, not by a single metric.
- Privacy. BGP shares reachability (a prefix plus attributes), not your internal topology.
Path vector
BGP is a path-vector protocol. Every route carries the AS_PATH, the list of ASes it has crossed. A router that sees its own AS number in a received path drops the route. That is the loop prevention between ASes.
AS numbers
- 2-byte ASNs run from 1 to 65535. 4-byte ASNs (RFC 6793) go up to 4294967295.
- Private ranges: 64512 to 65534 and 4200000000 to 4294967294.
- Public ASNs are assigned by the regional internet registries (ARIN, RIPE NCC, APNIC, LACNIC, AFRINIC).
Where else BGP shows up
MPLS L3VPN (the VPNv4 address family), EVPN for VXLAN fabrics, and as the only routing protocol in large spine-leaf data centres (RFC 7938). Being good at BGP pays off well beyond the internet edge.
They will ask next
When does a company actually need BGP?
When it connects to two or more providers and wants control over which link carries traffic in and out, usually with its own provider-independent prefix and public ASN. A site with one ISP and ISP-assigned addresses is fine with a static default route.
Is BGP a routing protocol or an application?
Both views are fair. It runs as an application over TCP port 179, but its job is routing: it installs routes in the routing table. It depends on connected routes or the IGP to reach its peers and next hops.
Traps
- Calling BGP a distance-vector protocol. It is path vector, and best path selection uses attributes and policy, not hop count.
- Saying you need a public ASN to run BGP. Private ASNs work for iBGP, confederation sub-ASes and many single-provider customers.
How did you do?Book 4.1 Global InternetOwn pageWhat is the difference between eBGP and iBGP?
Say this first
eBGP runs between routers in different ASes, iBGP between routers in the same AS. The protocol is the same, the rules differ. On Cisco, eBGP routes get administrative distance 20 and iBGP routes 200. eBGP peers expect to be directly connected (TTL 1), while iBGP peers usually sit several hops apart on loopbacks. eBGP adds the local AS to the path and sets itself as next hop; iBGP does neither. And a route learned over iBGP is never passed to another iBGP peer.
Go deeper
eBGP iBGP Peers Different AS Same AS Cisco AD 20 200 Default TTL 1 255 AS_PATH on send Local AS prepended Unchanged NEXT_HOP on send Set to itself Unchanged by default LOCAL_PREF Not sent Sent Re-advertising learned routes To every peer Not to other iBGP peers Loop prevention Own ASN in AS_PATH iBGP split horizon (or RR attributes) Cisco IOS router bgp 64500 neighbor 192.0.2.1 remote-as 64496 neighbor 10.255.0.2 remote-as 64500 neighbor 10.255.0.2 update-source Loopback0The router decides eBGP or iBGP by comparing
remote-aswith its own AS. 192.0.2.1 is eBGP, 10.255.0.2 is iBGP.Why the AD values
An eBGP route comes straight from the AS that owns the destination, so it is trusted above any IGP. An iBGP route is second-hand knowledge about an external prefix; if the IGP also knows that prefix, the IGP is closer to the truth inside your network, so iBGP's 200 loses to OSPF's 110.
They will ask next
What is the AD of a locally originated BGP route, like an aggregate?
200 on Cisco. The three values are external, internal and local, set together with
distance bgp 20 200 200.How does MED behave across the two?
A router sends MED to its eBGP neighbor. The receiving AS can pass that MED to its iBGP peers, but it must not pass it on to another AS. MED is optional non-transitive.
Traps
- Saying iBGP peers need a direct link. They need IP reachability, usually through the IGP.
- Forgetting that iBGP keeps the external next hop. It is the classic reason iBGP routes are not installed.
How did you do?Book 4.1 Global InternetOwn pageWhat transport does BGP use, and who opens the connection?
Say this first
BGP runs over TCP port 179. Each side tries to connect to the other's port 179 from an ephemeral port. If both succeed at the same time, collision detection keeps the connection opened by the router with the higher BGP router ID and closes the other. TCP gives BGP reliable, ordered delivery, so BGP needs no retransmission of its own and sends only incremental updates after the first full exchange.
Go deeper
Seeing who connected
Output R1# show tcp brief TCB Local Address Foreign Address (state) 7F8A2C10 192.0.2.2.179 192.0.2.1.38112 ESTABR1 owns port 179 in this connection, so ISP1 (192.0.2.1) opened it. You can force one side to wait with
neighbor 192.0.2.1 transport connection-mode passive.What follows from running over TCP
- Firewalls and ACLs must allow TCP 179 between the peers, in the direction each side may connect.
- Session authentication lives in TCP options: MD5 (RFC 2385) or TCP-AO (RFC 5925).
- Path MTU matters. If the session comes up but drops a few minutes later every time, with the hold timer expiring during the initial table transfer, suspect full-size TCP segments being dropped on the path while small keepalives get through.
Message framing
Every message starts with a 19-byte header: a 16-byte marker of all ones, a 2-byte length and a 1-byte type. The maximum message size is 4096 bytes, or 65535 when both peers support extended messages (RFC 8654).
They will ask next
Why does BGP use TCP when OSPF does not?
BGP moves very large tables between organisations, often across several hops. TCP gives it reliability and flow control for free and works over any routed path. OSPF only talks to neighbors on the same link, so it runs directly on IP (protocol 89) with its own acknowledgments.
If TCP is reliable, why does BGP need keepalives?
TCP keeps quiet when nothing is sent, and it does not prove the BGP process on the other side is healthy. BGP KEEPALIVE messages and the hold timer do that.
Traps
- Saying UDP 179.
- Thinking only one side listens. Both listen and both try to connect unless one is set to passive.
What message types does BGP use, and what is in the OPEN message?
Say this first
Five types. OPEN (1) starts the session and carries the version, my AS, hold time, BGP identifier (router ID) and capabilities. UPDATE (2) advertises and withdraws routes. NOTIFICATION (3) reports an error and closes the session. KEEPALIVE (4) proves the peer is alive and confirms the OPEN. ROUTE-REFRESH (5, RFC 2918) asks the peer to send its routes again.
Go deeper
OPEN
- Version 4. My AS: 2 bytes; a 4-byte ASN puts AS_TRANS (23456) here and the real ASN in a capability.
- Hold time: the session uses the lower of the two values; it must be 0 or at least 3 seconds.
- BGP identifier: the router ID. The two peers must have different IDs.
- Capabilities: multiprotocol address families, route refresh, 4-byte AS, graceful restart, add-path. Adding a new address family later means new capabilities, which on most platforms resets the session.
UPDATE
One UPDATE carries withdrawn prefixes, one set of path attributes, and the prefixes (NLRI) that share those attributes. IPv6 and VPN routes travel inside the MP_REACH_NLRI and MP_UNREACH_NLRI attributes.
NOTIFICATION codes you will meet
Code/subcode Meaning Usual cause 2/2 OPEN error: bad peer AS remote-aswrong on one side2/3 OPEN error: bad BGP identifier Duplicate router ID 4/0 Hold timer expired Lost keepalives: link, CPU or MTU problem 6/1 Cease: maximum prefixes reached maximum-prefixtripped6/2 Cease: administrative shutdown neighbor x shutdownon the peer6/4 Cease: administrative reset clear ip bgpon the peerA wrong AS shows up in the log like this; the two bytes are the AS the peer sent, in hex (FBF0 is 64496):
Output %BGP-3-NOTIFICATION: sent to neighbor 192.0.2.1 2/2 (peer in wrong AS) 2 bytes FBF0KEEPALIVE
Only the 19-byte header. Cisco sends one every 60 seconds by default.
They will ask next
Are BGP updates periodic like RIP?
No. After the initial exchange BGP sends UPDATEs only when something changes. Keepalives are the only periodic messages.
Traps
- Saying keepalives carry routes, or that BGP refreshes its whole table periodically.
How did you do?Book 4.1 Global InternetOwn pageWalk me through the BGP neighbor states.
Say this first
Idle, Connect, Active, OpenSent, OpenConfirm, Established. Idle: not trying yet, or backing off after an error. Connect: waiting for the TCP handshake to finish. Active: the TCP attempt failed and BGP is retrying and listening. OpenSent: TCP is up and our OPEN is sent. OpenConfirm: the peer's OPEN was accepted, waiting for its KEEPALIVE. Established: UPDATEs flow.
show ip bgp summaryshows the state, or a prefix count once the session is Established.Go deeper
State What is happening If it sticks here, suspect Idle Not attempting Admin shutdown, max-prefix hold-down, or on IOS often no route to the neighbor at all Connect TCP SYN sent, waiting Normally passes in milliseconds Active TCP failed, retrying and listening Reachability, update-source, TTL, port 179 filtered, auth OpenSent OPEN sent, waiting for the peer's Brief; bouncing back means the OPEN was rejected OpenConfirm OPEN accepted, waiting for KEEPALIVE Brief Established Routes exchanged Check the prefix count, not just the state Output R1# show ip bgp summary | begin Neighbor Neighbor V AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down State/PfxRcd 10.255.0.2 4 64500 112 109 14 0 0 01:32:10 6 192.0.2.1 4 64496 0 0 1 0 0 never Idle (Admin) 198.51.100.1 4 64497 12 15 0 0 0 00:00:41 Active- 10.255.0.2 is Established with 6 prefixes accepted after inbound policy.
- 192.0.2.1 was shut down by an engineer (
neighbor 192.0.2.1 shutdown). - 198.51.100.1 was up earlier (it has message counters) and has been retrying TCP for 41 seconds.
They will ask next
What does Idle (PfxCt) mean?
The neighbor sent more prefixes than
maximum-prefixallows. The session is held down until the restart timer expires or someone runsclear ip bgp.The state keeps flipping between Idle and Active. What next?
Read the log. A session that reaches OpenSent and gets rejected (wrong AS, duplicate router ID, bad hold time) leaves a NOTIFICATION message that names the reason. A pure TCP failure leaves none.
Traps
- Reading Active as "active and working". It means the TCP session cannot be built.
- Calling a session healthy because it is Established. Zero prefixes received is a policy or peer problem.
How did you do?Book 4.1 Global InternetOwn pageWhy do we peer iBGP between loopbacks, and what does that need?
Say this first
A loopback interface never goes down while the router has any path to it, so an iBGP session between loopbacks survives the loss of a single link as long as the IGP finds another way. It needs three things: the loopbacks advertised in the IGP,
neighbor x update-source Loopback0so TCP is sourced from the loopback, and each side's neighbor statement pointing at the other side's loopback address.Go deeper
Cisco IOS interface Loopback0 ip address 10.255.0.1 255.255.255.255 ! router ospf 1 router-id 10.255.0.1 network 10.255.0.1 0.0.0.0 area 0 network 10.0.0.0 0.0.255.255 area 0 ! router bgp 64500 bgp router-id 10.255.0.1 neighbor 10.255.0.2 remote-as 64500 neighbor 10.255.0.2 update-source Loopback0R2 mirrors this with its own loopback 10.255.0.2 and
neighbor 10.255.0.1.Why it fails without update-source
TCP takes its source address from the outgoing interface, say 10.0.12.1. R2 only has a neighbor statement for 10.255.0.1, so it treats the SYN as coming from a stranger and resets it.
eBGP is different
eBGP between loopbacks also needs
ebgp-multihop 2ordisable-connected-check, and a static route to the peer's loopback, because you do not share an IGP with another AS.They will ask next
How does BGP choose its router ID?
On IOS: the configured
bgp router-id, otherwise the highest IP on a loopback, otherwise the highest IP on an active interface. Always set it explicitly, so it never changes when interfaces change.Do loopbacks remove the need for next-hop-self?
No, they solve a different problem. Loopbacks keep the session up; next-hop-self fixes the external next hop on eBGP-learned routes. Edge routers usually need both.
Traps
- Forgetting to advertise the loopback in the IGP. The session cannot come up without a route to it.
How did you do?Book 4.1 Global InternetOwn pageWhy does iBGP need a full mesh?
Say this first
An iBGP speaker never advertises a route it learned from one iBGP peer to another iBGP peer. That is the iBGP split horizon rule, and it exists because the AS_PATH does not change inside an AS, so iBGP has no way to spot a loop. The consequence is that every iBGP router must peer with every other: n(n-1)/2 sessions. Route reflectors and confederations relax the rule so big networks do not need a full mesh.
Go deeper
How fast it grows
Routers iBGP sessions 5 10 10 45 50 1225 100 4950 Every new router also means touching the configuration of every existing one. Walk through it
R1 learns 198.51.100.0/24 from ISP1 over eBGP and sends it to its iBGP peers R2 and R3. R2 does not pass it on to R4, because R2 learned it over iBGP. If R4 has no session with R1, R4 never hears of the prefix.
Which routes the rule covers
Only routes learned over iBGP. Routes from eBGP peers and routes the router originates itself go to all iBGP peers. And full mesh means BGP sessions, not physical links: the sessions ride over the IGP.
They will ask next
Without loop detection, how do route reflectors avoid loops?
With two attributes the RR adds: ORIGINATOR_ID (the router ID of the router that injected the route) and CLUSTER_LIST (the cluster IDs of each RR the route passed through). A router drops a route carrying its own ID in either.
Does the rule apply to eBGP?
No. eBGP advertises its best routes to every peer, and AS_PATH loop detection protects it.
Traps
- Saying iBGP routes are never re-advertised at all. They go to eBGP peers normally.
- Thinking a full mesh means physical links between every router.
How did you do?Book 4.1 Global InternetOwn pageWhat does next-hop-self do, and when do you need it?
Say this first
When an edge router passes an eBGP route to its iBGP peers, it keeps the original next hop: the ISP's address on the shared link. If the IGP does not carry that link subnet, internal routers mark the route inaccessible and never install it.
neighbor x next-hop-selfon the edge router rewrites the next hop to its own update-source address (its loopback) on routes it sends to that iBGP peer.Go deeper
The problem
R1 learns 198.51.100.0/24 from ISP1 at 192.0.2.1 and sends it to R3 over iBGP. R3 has no route to 192.0.2.0/30, so the route is not valid (no
*) and not best (no>):Output R3# show ip bgp | begin Network Network Next Hop Metric LocPrf Weight Path i 198.51.100.0 192.0.2.1 0 100 0 64496 iThe fix on the edge router
Cisco IOS router bgp 64500 neighbor 10.255.0.3 remote-as 64500 neighbor 10.255.0.3 update-source Loopback0 neighbor 10.255.0.3 next-hop-selfAfter
clear ip bgp 10.255.0.3 soft outon R1:Output R3# show ip bgp | begin Network Network Next Hop Metric LocPrf Weight Path *>i 198.51.100.0 10.255.0.1 0 100 0 64496 iThe other fix
Put the external link into the IGP as a passive interface. It works, but it spreads external subnets through your IGP and has to be remembered for every new peering. next-hop-self is the common choice.
They will ask next
Does a route reflector change the next hop of reflected routes?
No, not by default. The edge router that learned the route from eBGP must set next-hop-self itself.
Does next-hop-self matter in MPLS L3VPN?
PEs send VPNv4 routes to each other with their own loopback as next hop already, and that loopback must have an LDP label. Keeping the next hop a /32 loopback is what lets the label-switched path work.
Traps
- Configuring next-hop-self on the router that receives the route. It belongs on the edge router that sends it into iBGP.
How did you do?Book 4.1 Global InternetOwn pageWhat are BGP path attributes, and how are they classified?
Say this first
Attributes are the properties carried with every route; BGP uses them for loop prevention and best path selection. There are four classes. Well-known mandatory: ORIGIN, AS_PATH and NEXT_HOP, present in every UPDATE. Well-known discretionary: LOCAL_PREF and ATOMIC_AGGREGATE. Optional transitive: COMMUNITY and AGGREGATOR, passed on even by routers that do not understand them. Optional non-transitive: MED, ORIGINATOR_ID and CLUSTER_LIST, dropped if not understood. Cisco's weight is not an attribute at all: it never leaves the router.
Go deeper
Attribute Type code Class What it does ORIGIN 1 Well-known mandatory How the route entered BGP: IGP (i), EGP (e), incomplete (?) AS_PATH 2 Well-known mandatory ASes crossed; loop prevention and path length NEXT_HOP 3 Well-known mandatory Address to forward to MULTI_EXIT_DISC (MED) 4 Optional non-transitive Hint to a neighbor AS about which entry link to use LOCAL_PREF 5 Well-known discretionary Exit preference shared inside the AS ATOMIC_AGGREGATE 6 Well-known discretionary Flags that aggregation lost path detail AGGREGATOR 7 Optional transitive AS and router ID that built the aggregate COMMUNITY 8 Optional transitive Tags for policy (RFC 1997) ORIGINATOR_ID 9 Optional non-transitive Set by route reflectors CLUSTER_LIST 10 Optional non-transitive Set by route reflectors MP_REACH_NLRI, MP_UNREACH_NLRI 14, 15 Optional non-transitive Carry IPv6, VPNv4, EVPN and other families EXTENDED COMMUNITIES 16 Optional transitive Route targets, site of origin LARGE_COMMUNITY 32 Optional transitive 12-byte communities for 4-byte ASNs (RFC 8092) What the classes mean in practice
- Well-known: every BGP implementation must recognise them.
- Mandatory: every UPDATE that advertises a prefix must carry them. LOCAL_PREF is sent on iBGP only.
- Transitive: an optional attribute the router does not understand is still passed along, with the Partial bit set. That is how communities spread before every router supported them.
- Non-transitive: an unknown optional non-transitive attribute is dropped.
They will ask next
What does ATOMIC_AGGREGATE tell you?
That a router aggregated routes and did not keep the full AS path information (no
as-set). Downstream routers know the path shown is not the complete list of ASes behind the prefix.Is LOCAL_PREF ever sent to an eBGP peer?
No, except between sub-ASes of a confederation. It is how one AS agrees on its own exits.
Traps
- Listing weight as a BGP attribute. It is a Cisco-local value that is never advertised.
- Calling MED transitive. It crosses one AS boundary and stops.
How did you do?Book 4.1 Global InternetOwn pageRead me the show ip bgp table. What do the codes and columns mean?
Say this first
On the left, status codes:
*valid,>best,ilearned over iBGP, plusrRIB-failure,ssuppressed,ddampened,hhistory andmmultipath. Then the prefix, the next hop (0.0.0.0 means this router originated it), Metric (the MED), LocPrf, Weight (32768 for my own routes, 0 for learned ones), the AS path, and the origin code at the end:iIGP,eEGP,?incomplete, usually redistributed.Go deeper
R1 is in AS 64500, with eBGP to ISP1 (AS 64496) and ISP2 (AS 64497) and iBGP to R2 and R3.
Output R1# show ip bgp | begin Network Network Next Hop Metric LocPrf Weight Path *> 0.0.0.0 192.0.2.1 0 200 0 64496 i * 198.51.100.1 0 0 64497 i *>i 10.20.0.0/16 10.255.0.2 20 100 0 ? *> 203.0.113.0 0.0.0.0 0 32768 i r>i 172.16.5.0/24 10.255.0.3 0 100 0 i- 0.0.0.0: two paths. A blank Network column means "same prefix as the line above". The ISP1 path is best because an inbound route map set LOCAL_PREF 200. LocPrf is blank on the ISP2 line because eBGP routes show no value unless policy set one; they are treated as 100.
- 10.20.0.0/16: from R2 over iBGP, origin
?because R2 redistributed it from OSPF. The OSPF metric 20 became the MED. - 203.0.113.0: our own prefix from a
networkstatement. Next hop 0.0.0.0, weight 32768. - 172.16.5.0/24:
rRIB-failure. BGP picked a best path, but the routing table already holds a route with a better administrative distance, for example OSPF (110) beating iBGP (200).
IOS prints classful-boundary prefixes without a length:
203.0.113.0is the /24, while10.20.0.0/16needs the length because it is not a class A boundary.Next step
show ip bgp 0.0.0.0lists every path with all its attributes and marks which one is best.They will ask next
Why does Cisco give locally originated routes a weight of 32768?
So the router always prefers its own route over the same prefix learned from a peer. Weight is the first step of best path selection.
Which origin code does a network statement give, and which does redistribution give?
networkandaggregate-addressgivei. Redistribution gives?. Originecame from the old EGP protocol and you will not see it in practice.
Traps
- Reading next hop 0.0.0.0 as a default route. In this table it means the route was originated here.
- Assuming the first path listed is the best one. Look for the
>.
How did you do?Book 4.1 Global InternetOwn pageGive me the full BGP best path selection order on a Cisco router.
Say this first
First the next hop must be reachable. Then: 1 highest weight, 2 highest LOCAL_PREF, 3 locally originated, 4 shortest AS_PATH, 5 lowest origin (IGP before EGP before incomplete), 6 lowest MED, 7 eBGP over iBGP, 8 lowest IGP metric to the next hop, 9 multipath check, 10 oldest eBGP path, 11 lowest router ID, 12 shortest cluster list, 13 lowest neighbor address. The first step where paths differ decides. A common memory aid for the first part is "N WLLA OMNI": Next hop, Weight, Local pref, Locally originated, AS path, Origin, MED, Neighbor type, IGP metric.
Go deeper
Step Compare Prefer Notes 0 Next hop reachable Reachable Also: not looped, passed inbound policy 1 Weight Highest Cisco only, local to the router; 0 learned, 32768 local 2 LOCAL_PREF Highest Default 100; shared across the AS 3 Locally originated Local networkor redistribution, then aggregates4 AS_PATH length Shortest An AS_SET counts as 1; confederation segments do not count 5 ORIGIN i, then e, then ? 6 MED Lowest Only between paths from the same neighbor AS by default 7 Peer type eBGP Then confederation eBGP, then iBGP 8 IGP metric to NEXT_HOP Lowest Hot potato routing 9 Multipath With maximum-paths, equal paths are installed; one is still marked best10 Path age Oldest eBGP Stability; skipped with bgp bestpath compare-routerid11 Router ID Lowest ORIGINATOR_ID for reflected routes 12 CLUSTER_LIST length Shortest 13 Neighbor address Lowest The peer's IP address How I explain it in an interview
Group the steps. Weight and local preference are the knobs you control. Locally originated, AS path, origin and MED describe the route itself. eBGP versus iBGP and IGP metric describe where it enters your network. The rest are tie breakers chosen for stability and determinism.
Proving it on a device
show ip bgp <prefix>lists every path and saysbest #2. IOS-XR goes further withshow bgp <prefix> bestpath-compare, which explains why each path lost.Junos
No weight. Junos first compares protocol preference (BGP is 170), then follows broadly the same order: local preference, AS path length, origin, MED, eBGP over iBGP, IGP cost, then tie breakers.
They will ask next
Why prefer the oldest eBGP path over the lowest router ID?
To avoid churn. A new path that ties on everything else does not replace a working best path, which keeps traffic and downstream updates stable.
When does deterministic-med matter?
When paths for one prefix come from several neighbor ASes. Without it, IOS compares paths pairwise in the order received, so the winner can depend on arrival order.
bgp deterministic-medgroups paths by neighbor AS first.
Traps
- Saying lowest local preference wins, or placing MED before AS path length.
- Forgetting that weight only affects the router it is set on.
How did you do?Book 4.1 Global InternetOwn pageWeight versus local preference. When would you use each?
Say this first
Both steer outbound traffic and both prefer the higher value. Weight is Cisco-only, local to one router, default 0 (32768 for routes it originates) and never advertised. LOCAL_PREF is a standard attribute, default 100, carried to every iBGP peer, so the whole AS agrees on an exit. I use weight for a quick override on one router and local preference for AS-wide policy such as "ISP1 is primary".
Go deeper
Local preference across two edge routers
R1 peers with ISP1, R2 with ISP2, and R1 and R2 run iBGP. R1 sets LOCAL_PREF 200 on everything from ISP1:
Cisco IOS route-map ISP1-IN permit 10 set local-preference 200 ! router bgp 64500 neighbor 192.0.2.1 route-map ISP1-IN inThen
clear ip bgp 192.0.2.1 soft inapplies it. On R2:Output R2# show ip bgp | begin Network Network Next Hop Metric LocPrf Weight Path *>i 0.0.0.0 10.255.0.1 0 200 0 64496 i * 198.51.100.1 0 0 64497 iR2 prefers the iBGP path through R1 over its own eBGP path. eBGP over iBGP is step 7; local preference is step 2 and decides first.
Why weight would not do this
Setting weight on R1 changes only R1's choice. R2 would still prefer its own eBGP path to ISP2, and traffic would leave by whichever router it reached first.
Changing the default
bgp default local-preference 150changes the value given to routes without one, for the whole router.They will ask next
One path has the higher weight, the other the higher local preference. Which wins?
The higher weight. Weight is step 1.
Can local preference influence traffic coming in to my AS?
No. It never leaves your AS. Inbound needs AS path prepending, MED, provider communities or more specific prefixes.
Traps
- Thinking local preference is sent to the ISP.
- Using weight for a policy that must hold across several edge routers.
How did you do?Book 4.1 Global InternetOwn pageWhat is AS path prepending, and what are its limits?
Say this first
Prepending adds extra copies of my own ASN when I advertise to a neighbor, so that path looks longer and other networks prefer a different one. It is the usual way to make one ISP the backup for inbound traffic. Limits: it only matters where AS path length is the deciding step, so it loses to anyone's local preference; past about three prepends it rarely changes more; and a long prepended path is easier to beat with a shorter fake announcement.
Go deeper
Config
Cisco IOS ip prefix-list OUR-PREFIXES seq 5 permit 203.0.113.0/24 ! route-map ISP2-OUT permit 10 match ip address prefix-list OUR-PREFIXES set as-path prepend 64500 64500 ! router bgp 64500 neighbor 198.51.100.1 route-map ISP2-OUT outISP2 receives the path
64500 64500 64500: two prepends plus the normal one added on eBGP.Verifying
show ip bgp neighbors 198.51.100.1 advertised-routeson IOS shows the route as it sits in your table, not the prepended path. Check from the outside: a looking glass in ISP2 or elsewhere, or the peer'sshow ip bgp 203.0.113.0.Rules of thumb
- Prepend only your own ASN. Someone else's ASN in your path makes that AS reject your route as a loop.
- Start with one or two prepends and measure. Long prepend chains mostly make the table bigger.
- Inbound prepending also exists:
set as-path prepend last-as 2on a received route makes your own router like that path less. Local preference is the cleaner tool for that.
They will ask next
You prepend five times toward ISP2 and traffic still arrives on ISP2. Why?
ISP2 itself, and anyone who buys from ISP2, gives customer routes a higher local preference than peer routes. They never compare AS path length. Use ISP2's backup community to lower your route's local preference inside ISP2.
Traps
- Prepending on the primary link instead of the backup.
- Expecting
advertised-routeson IOS to show the prepends.
How did you do?Book 4.1 Global InternetOwn pageHow do you get a prefix into BGP, and why might a network statement not work?
Say this first
Three ways: a
networkstatement, redistribution, oraggregate-address. Unlike OSPF, the BGPnetworkcommand does not enable anything on interfaces. It means "advertise this prefix if an exact match, same prefix and same mask, is in the routing table". It usually fails because of a mask mismatch or no matching route. The fix is the rightmask, or a static route to Null0 that anchors the prefix.Go deeper
Cisco IOS ip route 203.0.113.0 255.255.255.0 Null0 ! router bgp 64500 network 203.0.113.0 mask 255.255.255.0Output R1# show ip bgp | begin Network Network Next Hop Metric LocPrf Weight Path *> 203.0.113.0 0.0.0.0 0 32768 iWhy the Null0 route
The /24 stays in the table as long as the router is up, so the announcement does not flap when a LAN subnet goes down. Inside, more specific routes for the real subnets win by longest match. Traffic for unused parts of the block hits Null0 and is dropped instead of looping toward the default route.
Common mistakes
network 203.0.113.0withoutmaskassumes the classful mask. Fine for a /24 in class C space, wrong for10.20.0.0which IOS would read as /8.- The RIB has 203.0.113.0/25 and /25, but you asked for the /24. No exact match, no advertisement.
Redistribution
redistribute ospf 1 route-map OSPF-TO-BGPsets origin?and copies the IGP metric into MED. Always use a route map at the internet edge; never redistribute the full BGP table into an IGP.They will ask next
Does the route behind a network statement have to be connected?
No. Any route in the routing table works: connected, static, OSPF. It has to match exactly.
Why prefer network statements at the internet edge?
They are explicit. You advertise only what you listed, with origin IGP. Redistribution can leak whatever the IGP happens to learn.
Traps
- Expecting
network 203.0.113.0 mask 255.255.255.0to summarise smaller routes. It needs the exact /24 in the RIB.
How did you do?Book 4.1 Global InternetOwn pageHow do you send or receive a default route with BGP?
Say this first
To send one, the cleanest way is
neighbor x default-originate, which sends 0.0.0.0/0 to that neighbor only; on Cisco it does not even need a default in the local routing table. The alternatives arenetwork 0.0.0.0, which needs a default in the routing table, or redistributing a default together withdefault-information originate. To receive one, I ask the ISP for default only or default plus partial routes and permit 0.0.0.0/0 in my inbound prefix list.Go deeper
Sending
Cisco IOS router bgp 64496 neighbor 192.0.2.2 remote-as 64500 neighbor 192.0.2.2 default-originateConditional version: advertise the default only while some upstream route exists in the routing table.
Cisco IOS ip prefix-list UPSTREAM seq 5 permit 198.51.100.0/24 ! route-map HAVE-UPSTREAM permit 10 match ip address prefix-list UPSTREAM ! router bgp 64496 neighbor 192.0.2.2 default-originate route-map HAVE-UPSTREAMReceiving
Cisco IOS ip prefix-list DEFAULT-ONLY seq 5 permit 0.0.0.0/0 ! router bgp 64500 neighbor 192.0.2.1 prefix-list DEFAULT-ONLY inOutput R1# show ip route | include 0.0.0.0 Gateway of last resort is 192.0.2.1 to network 0.0.0.0 B* 0.0.0.0/0 [20/0] via 192.0.2.1, 00:12:44Spreading it inside
Do not redistribute BGP into the IGP. Originate a default into OSPF on the edge router with
default-information originate, which by default only works while the router itself has a default.They will ask next
What does 0.0.0.0/0 le 32 match in a prefix list?
Every IPv4 prefix.
permit 0.0.0.0/0alone matches only the default route. Mixing these up is a classic way to accept a full table when you meant default only.When would you take full tables instead of a default?
When you are multihomed and want each destination to use the best exit, or need to see exactly how the internet reaches each prefix. It costs memory and CPU on the edge routers.
Traps
- Writing
0.0.0.0/0 le 32when you mean default only.
How did you do?Book 4.1 Global InternetOwn pageHow do prefix lists work for BGP filtering, and what do ge and le mean?
Say this first
A prefix list matches both the network bits and the prefix length, which is why it beats an ACL for routes.
permit 10.0.0.0/8matches exactly that /8.le 24extends the match to longer prefixes inside the range, up to /24;ge 25sets a minimum length. Entries are checked in sequence order, the first match wins, and an implicit deny sits at the end. I apply them withneighbor x prefix-list NAME in|outor inside a route map.Go deeper
Entry Matches permit 0.0.0.0/0The default route only permit 0.0.0.0/0 le 32Everything permit 0.0.0.0/0 le 24Everything from /0 to /24: a typical internet filter deny 10.0.0.0/8 le 3210.0.0.0/8 and every subnet of it permit 203.0.113.0/24 ge 25 le 26The /25s and /26s inside 203.0.113.0/24, not the /24 itself The lengths must satisfy: prefix length < ge <= le <= 32.
Inbound bogon filter (start of one)
Cisco IOS ip prefix-list FROM-ISP seq 5 deny 0.0.0.0/8 le 32 ip prefix-list FROM-ISP seq 10 deny 10.0.0.0/8 le 32 ip prefix-list FROM-ISP seq 15 deny 100.64.0.0/10 le 32 ip prefix-list FROM-ISP seq 20 deny 127.0.0.0/8 le 32 ip prefix-list FROM-ISP seq 25 deny 169.254.0.0/16 le 32 ip prefix-list FROM-ISP seq 30 deny 172.16.0.0/12 le 32 ip prefix-list FROM-ISP seq 35 deny 192.168.0.0/16 le 32 ip prefix-list FROM-ISP seq 40 deny 224.0.0.0/3 le 32 ip prefix-list FROM-ISP seq 100 permit 0.0.0.0/0 le 24224.0.0.0/3 covers multicast and the old class E space. In production, the list also covers the documentation ranges that these examples use as if they were real.
After changing a prefix list on IOS, run
clear ip bgp 192.0.2.1 soft inso it is applied to routes already received.They will ask next
Why not use an access list to filter routes?
A standard ACL matches only the network bits, so it cannot tell 10.0.0.0/8 from 10.0.0.0/24. Extended ACLs can be bent to match the mask too, but it is awkward. Prefix lists were made for this.
How do you filter on where a route came from instead of what it is?
With an AS path access list matched by regular expression, applied with
filter-listormatch as-path.
Traps
- Thinking
le 24alone means "exactly /24". It means any length from the prefix length up to 24. - Forgetting the implicit deny when you add a list meant only to block a few prefixes.
How did you do?Book 4.1 Global InternetOwn pageYou are dual-homed to two ISPs. How do you make sure you never become a transit AS?
Say this first
By advertising only my own prefixes. eBGP advertises every best route to every eBGP peer by default, so without a filter, routes learned from ISP1 go to ISP2, and ISP2 might start sending ISP1-bound traffic across my links. I filter outbound on both sessions with a prefix list of my own space, and back it up with an AS path filter that permits only
^$, meaning routes originated inside my AS.Go deeper
Cisco IOS ip prefix-list OUR-PREFIXES seq 5 permit 203.0.113.0/24 ip as-path access-list 1 permit ^$ ! router bgp 64500 neighbor 192.0.2.1 prefix-list OUR-PREFIXES out neighbor 192.0.2.1 filter-list 1 out neighbor 198.51.100.1 prefix-list OUR-PREFIXES out neighbor 198.51.100.1 filter-list 1 outThe outbound filter list sees the path as it is in my table, before my own AS is added on the way out, so my prefixes match
^$.Verify
Output R1# show ip bgp neighbors 198.51.100.1 advertised-routes | begin Network Network Next Hop Metric LocPrf Weight Path *> 203.0.113.0 0.0.0.0 0 32768 i Total number of prefixes 1A third layer
Tag everything learned from either ISP with NO_EXPORT inbound. Even a broken outbound filter would then not send those routes to another AS.
They will ask next
Why both a prefix list and an AS path filter?
Each catches mistakes the other misses. The prefix list stops anything that is not my space. The AS path filter stops routes learned from other ASes even if someone widens the prefix list by mistake.
Will the ISPs not filter me anyway?
They should, but not all do. Leaks by customers that providers accepted have caused major outages. Filter your side regardless.
Traps
- Writing
^$and applying it inbound. On a received route the path is never empty.
How did you do?Book 4.1 Global InternetOwn page
Working16
Which AS numbers are private, and what does remove-private-as do?
Say this first
Private ASNs are 64512 to 65534 and 4200000000 to 4294967294 (RFC 6996). They must not leak into the global table, so the provider strips them on its upstream sessions with
neighbor x remove-private-as. On Cisco IOS the plain command only strips them when the AS path contains nothing but private ASNs.remove-private-as allremoves them wherever they appear, andall replace-asswaps each one for the local AS so the path length is kept.Go deeper
Ranges worth knowing
Range Use 0, 65535, 4294967295 Reserved 23456 AS_TRANS: stands in for a 4-byte ASN when talking to a 2-byte-only speaker 64496 to 64511, 65536 to 65551 Documentation (RFC 5398) 64512 to 65534 Private, 2-byte 4200000000 to 4294967294 Private, 4-byte Typical use
A customer connected only to ISP1 runs eBGP with private AS 65010. ISP1 (AS 64496) strips 65010 toward its upstream, so the internet sees the customer prefix as originated by 64496.
Cisco IOS router bgp 64496 neighbor 192.0.2.10 remote-as 65010 neighbor 192.0.2.10 description CUSTOMER-A neighbor 198.51.100.9 remote-as 64511 neighbor 198.51.100.9 description UPSTREAM neighbor 198.51.100.9 remove-private-asThe upstream sees the path
64496instead of64496 65010.Cisco IOS behaviour
remove-private-as: removes private ASNs only if the whole path is private. A mixed path such as65010 64499is left alone.remove-private-as all: removes every private ASN anywhere in the path.remove-private-as all replace-as: replaces each private ASN with the local AS, so prepends keep working.- It applies on eBGP sessions only.
They will ask next
Why does a customer multihomed to two different ISPs usually need a public ASN?
After both ISPs strip the private ASN, the prefix appears to be originated by two different ASes. That multiple-origin situation is confusing to troubleshoot, needs ROAs for both origins, and most providers simply require a public ASN for multi-provider multihoming.
What is asdot notation?
A way to write 4-byte ASNs as two 16-bit halves: 65536 is
1.0in asdot and65536in asplain. Cisco switches the display withbgp asnotation dot. Asplain is the usual default today.
Traps
- Expecting plain
remove-private-asto clean a path that also contains public ASNs. On IOS you need theallkeyword.
How did you do?Book 4.1 Global InternetOwn pageA BGP neighbor is stuck in Active. What does that tell you, and what do you check?
Say this first
Active means BGP keeps trying to build the TCP session to port 179 and it keeps failing, so the problem is below BGP policy. I check the route to the neighbor address and back, that each side's neighbor statement matches the address the other side sources from (update-source), the eBGP TTL when peering on loopbacks, ACLs or firewalls dropping TCP 179, that the peer is configured for me at all, and MD5 password mismatches.
Go deeper
Checklist in the order I run it
- Reachability with the real addresses.
ping 10.255.0.2 source Loopback0tests exactly the pair BGP will use. A plain ping from the physical interface proves less. - Neighbor address and update-source. If R1 has no
update-source Loopback0, its SYN leaves from the physical interface address. R2 has no neighbor with that address and resets the connection. - eBGP on loopbacks. eBGP expects a directly connected peer. Without
ebgp-multihopordisable-connected-check, IOS does not even try:show ip bgp neighborsreports that the external neighbor is not directly connected and the state sits in Idle rather than Active. Worth knowing as the neighbouring symptom. - Port 179 filtered. Interface ACLs, CoPP, or a firewall in the path. Check ACL hit counters.
- Authentication. A password on one side only or a mismatch logs
%TCP-6-BADAUTHwith "No MD5 digest" or "Invalid MD5 digest" and the TCP session never completes. - The peer. Is the neighbor configured on the far side, with my address, and not shut down?
What does not cause Active
Route maps, prefix lists and filter lists cannot stop a session from forming. A wrong
remote-asgets further: TCP comes up, the OPEN is rejected with NOTIFICATION 2/2, and the log tells you so.They will ask next
Only one router has update-source configured. Does the session come up?
Often yes. The router with update-source connects from its loopback, which the other side recognises, so that connection succeeds. The reverse direction keeps failing. It works by luck of which side connects; configure both.
What is the difference between stuck in Idle and stuck in Active?
Idle means BGP is not even trying: admin shutdown, max-prefix hold-down, backing off after errors, or on IOS no route to the neighbor. Active means it tries and the TCP handshake fails.
Traps
- Digging through route maps for a neighbor in Active. Policy never blocks the session itself.
- Declaring reachability fine after a ping sourced from the wrong interface.
How did you do?Book 4.1 Global InternetOwn page- Reachability with the real addresses.
What are the default BGP timers, and how do you detect a dead neighbor faster?
Say this first
Cisco IOS defaults to a 60 second keepalive and a 180 second hold time. Junos uses 30 and 90. The hold time is negotiated to the lower of the two values in the OPEN messages. Three minutes is far too slow to notice a dead peer, so I either lower the timers, for example to 10 and 30, or better, run BFD and tie BGP to it with
neighbor x fall-over bfdfor sub-second detection.Go deeper
Timers
- Keepalive: how often a KEEPALIVE is sent. Hold: how long without any message before the peer is declared dead. A hold time of 0 disables both; otherwise it must be at least 3 seconds.
- Per neighbor:
neighbor 192.0.2.1 timers 10 30. For all neighbors:timers bgp 10 30. - New timers take effect only when the session is next established.
Fast external fallover
bgp fast-external-falloveris on by default in IOS. When the interface to a directly connected eBGP peer goes down, the session drops immediately. It does not help when the link stays up but the far end dies, for example with a provider switch or media converter in the middle.BFD
Cisco IOS interface GigabitEthernet0/0/0 bfd interval 300 min_rx 300 multiplier 3 ! router bgp 64500 neighbor 192.0.2.1 fall-over bfd300 ms times 3 gives about 900 ms detection. The peer must run BFD too, so for an ISP session it is something you agree with the provider. Sessions between loopbacks need multihop BFD, which is configured differently and not supported on every platform.
They will ask next
Why not just set the hold time to 3 seconds?
BGP keepalives are handled by the control plane CPU. A busy CPU can miss one, and losing a full table session causes a big wave of churn. BFD is a lightweight protocol, often run in hardware or on the line card, built for exactly this.
Does BFD replace the BGP hold timer?
No. BFD tells BGP that the forwarding path is down and BGP tears the session down at once. The hold timer still runs as the backstop.
Traps
- Saying the hold time is negotiated to the higher value.
- Changing timers and expecting the running session to pick them up without a reset.
How did you do?Book 4.1 Global InternetOwn pageHow do route reflectors work, and what are the reflection rules?
Say this first
A route reflector (RFC 4456) is an iBGP router allowed to re-advertise iBGP routes, which breaks split horizon in a controlled way. Clients peer only with the RR. A route from an eBGP peer goes to all clients and non-clients. A route from a client goes to all other clients and to non-clients. A route from a non-client goes only to clients. The RR adds ORIGINATOR_ID and CLUSTER_LIST to stop loops, and it does not change the next hop.
Go deeper
Reflection rules
Learned from Sent to clients Sent to non-clients Sent to eBGP eBGP peer Yes Yes Yes Client Yes Yes Yes Non-client Yes No Yes Config (only the RR changes)
Cisco IOS router bgp 64500 neighbor 10.255.0.1 remote-as 64500 neighbor 10.255.0.1 update-source Loopback0 neighbor 10.255.0.1 route-reflector-client neighbor 10.255.0.3 remote-as 64500 neighbor 10.255.0.3 update-source Loopback0 neighbor 10.255.0.3 route-reflector-clientClients are configured as ordinary iBGP peers of the RR.
Loop prevention
- ORIGINATOR_ID: router ID of the router that brought the route into the AS. A router that sees its own ID drops the route.
- CLUSTER_LIST: each RR adds its cluster ID (by default its router ID). An RR that sees its own cluster ID drops the route.
Design points
- Run two RRs. Each client peers with both, and the RRs peer with each other.
- The RR reflects only its own best path per prefix, so clients lose sight of alternatives. Add-path and best-external fix that when it matters.
- The RR picks the best path from its own position in the IGP. An RR off to one side can hand clients a poor exit; optimal route reflection (RFC 9107) addresses this.
They will ask next
How does the cluster list affect best path selection?
Late in the order: for reflected routes the ORIGINATOR_ID replaces the router ID in the router ID tie breaker, and after that the shorter CLUSTER_LIST wins.
What if the clients are already fully meshed?
The RR's client-to-client reflection is redundant.
no bgp client-to-client reflectionon the RR turns it off while still reflecting between clients and non-clients.Does an RR need to be in the forwarding path?
No. It only needs control plane reachability, which is why virtual RRs on servers are common. Just make sure its best path choice still gives clients sensible exits.
Traps
- Saying the RR sets itself as next hop. It does not; edge routers still need next-hop-self.
- Saying routes from one non-client are reflected to another non-client. Non-clients must stay fully meshed with each other.
How did you do?Book 4.1 Global InternetOwn pageHow do you influence outbound traffic versus inbound traffic with BGP?
Say this first
Outbound traffic is under my control: I decide which path to install, with local preference across the AS or weight on one router, and by filtering what I accept. Inbound traffic depends on other ASes' decisions, so I can only change what I advertise to influence them: AS path prepending, MED toward one provider, the provider's communities that set its local preference, or more specific prefixes on the preferred link. Inbound is never guaranteed, because every other AS applies its own policy first.
Go deeper
Tools by direction
Direction Tool Strength Outbound LOCAL_PREF inbound on the preferred ISP Firm, AS-wide Outbound Weight Firm, one router Outbound Accept only a default from the backup ISP Firm, simple Inbound More specific prefixes on the preferred link Strongest: longest match wins everywhere Inbound Provider community that lowers its local preference Strong inside that provider Inbound AS path prepending on the backup link Moderate; loses to anyone's local preference Inbound MED Weak; only between links to the same provider Example: prefer ISP1 in both directions
Cisco IOS route-map ISP1-IN permit 10 set local-preference 200 ! route-map ISP2-OUT permit 10 match ip address prefix-list OUR-PREFIXES set as-path prepend 64500 64500 64500Why prepending alone often disappoints
ISP2 and its customers usually prefer routes learned from customers (you) with a higher local preference than routes from peers. Your prepends are never looked at by them, so traffic from ISP2's own network still arrives over the backup link. The fix is ISP2's documented backup community.
They will ask next
Why not always announce more specifics on the preferred link?
Most networks drop IPv4 prefixes longer than /24 and IPv6 longer than /48, so you may have nothing smaller to announce. More specifics also grow the global table and need ROAs that allow them. If you do it, announce the covering aggregate on both links so the backup still works.
How do you see how inbound traffic actually splits?
Interface counters or NetFlow per uplink, plus external views: looking glasses, RIPE RIS and similar collectors show which path other networks chose.
Traps
- Using local preference to try to steer inbound traffic.
- Promising an interviewer an exact inbound split. You can influence it, not guarantee it.
How did you do?Book 4.1 Global InternetOwn pageWhat are BGP communities, and what do the well-known communities do?
Say this first
A community is a 32-bit tag, written ASN:value such as 64500:120, attached to routes so policy elsewhere can match on it. Standard communities are optional transitive (RFC 1997). The well-known ones: NO_EXPORT (do not advertise outside this AS or confederation), NO_ADVERTISE (do not advertise to any peer), NO_EXPORT_SUBCONFED, which Cisco calls local-AS (do not advertise outside this sub-AS), and Cisco's
internetkeyword, which means advertise normally. Cisco only sends communities if you addneighbor x send-community.Go deeper
Well-known values
Name Value Effect NO_EXPORT 65535:65281 Stay inside the AS (a confederation counts as one AS) NO_ADVERTISE 65535:65282 Do not advertise to any BGP peer NO_EXPORT_SUBCONFED (local-AS) 65535:65283 Stay inside the sub-AS BLACKHOLE 65535:666 Ask the neighbor to drop traffic to this prefix (RFC 7999) GRACEFUL_SHUTDOWN 65535:0 Lower preference before maintenance (RFC 8326) How providers use them
Providers publish their own sets, for example 64496:70 to set local preference 70 (backup), a value to prepend toward certain peers, or 64496:666 to blackhole a prefix under attack. The customer tags, the provider's import policy acts. Customer side:
Cisco IOS ip bgp-community new-format ! route-map TO-ISP1 permit 10 match ip address prefix-list OUR-PREFIXES set community 64496:70 ! router bgp 64500 neighbor 192.0.2.1 send-community neighbor 192.0.2.1 route-map TO-ISP1 outProvider side:
Cisco IOS ip community-list standard CUST-BACKUP permit 64496:70 ! route-map FROM-CUST-64500 permit 10 match community CUST-BACKUP set local-preference 70Other community types
- Extended (8 bytes, RFC 4360): route targets and site of origin in MPLS VPNs.
send-community extendedorsend-community both. - Large (12 bytes, RFC 8092):
ASN:value:value, made because a 4-byte ASN does not fit in a standard community.
ip bgp-community new-formatmakes IOS display communities as ASN:value instead of one 32-bit number.They will ask next
What is the difference between set community and set community additive?
set community 64496:70replaces every community on the route.set community 64496:70 additiveadds it to the ones already there. Forgettingadditivesilently wipes tags other teams depend on.When would you attach NO_EXPORT?
When you announce more specific routes to one provider for traffic engineering and want them to stay inside that provider, while the aggregate goes to the whole internet.
Traps
- Forgetting
send-communityon IOS and wondering why the provider ignores your tags. - Mixing up NO_EXPORT (stays in the AS) and NO_ADVERTISE (goes to no peer at all).
How did you do?Book 4.1 Global InternetOwn page- Extended (8 bytes, RFC 4360): route targets and site of origin in MPLS VPNs.
A prefix is in the BGP table but not in the routing table. What could cause that?
Say this first
The usual causes: the next hop is unreachable (the route has no
*); a route from another source with a better administrative distance is already there (RIB-failure,r, for example OSPF 110 beating iBGP 200); the path is valid but another path is best; it is suppressed by an aggregate (s) or dampened (d); or on an ancient config, synchronization is on.show ip bgp <prefix>andshow ip bgp rib-failuresay which.Go deeper
Working through it
show ip bgp 198.51.100.0: look at each path forinaccessibleafter the next hop, and forbest,suppressedordampened.- Next hop:
show ip route 192.0.2.1. If the IGP has no route to it, fix with next-hop-self on the edge router. - RIB-failure:
Output R1# show ip bgp rib-failure Network Next Hop RIB-failure RIB-NH Matches 172.16.5.0/24 10.255.0.3 Higher admin distance n/ashow ip route 172.16.5.0shows who won, here OSPF. - Not best: compare the paths with the best path order. The best one may point elsewhere.
A detail interviewers like
IOS still advertises RIB-failure routes to its BGP peers by default.
bgp suppress-inactivestops that, so the router only advertises what it actually forwards with.They will ask next
Is a RIB-failure always a problem?
Often not. If OSPF and BGP describe the same internal prefix, OSPF winning is normal. It matters when BGP policy expects one path but packets follow the IGP route somewhere else.
What if the prefix is not in the BGP table at all?
Then look earlier: did the peer send it (
received-routeswith soft reconfiguration, or a looking glass on the peer), did inbound policy drop it, or was it dropped as a loop because your own AS was in the path?
Traps
- Thinking
rmeans rejected. The route is valid and best in BGP, it just lost in the routing table.
How did you do?Book 4.1 Global InternetOwn pageHow does BGP aggregation work, and what do summary-only and as-set do?
Say this first
aggregate-addresscreates a summary in BGP as long as at least one more specific component is in the BGP table. By default the specifics are still advertised alongside the summary;summary-onlysuppresses them (they show ass). The aggregate starts a fresh AS path and carries ATOMIC_AGGREGATE and AGGREGATOR. Withas-setit keeps the component ASNs in an AS_SET instead, at the cost of re-advertising the aggregate whenever a component changes.Go deeper
Cisco IOS router bgp 64500 network 203.0.113.0 mask 255.255.255.128 network 203.0.113.128 mask 255.255.255.128 aggregate-address 203.0.113.0 255.255.255.0 summary-onlyBoth /25s are connected LAN subnets, so the network statements match.
Output R1# show ip bgp | begin Network Network Next Hop Metric LocPrf Weight Path *> 203.0.113.0 0.0.0.0 32768 i s> 203.0.113.0/25 0.0.0.0 0 32768 i s> 203.0.113.128/25 0.0.0.0 0 32768 iIOS also installs the aggregate in the routing table pointing to Null0 with distance 200, which prevents loops for unused parts of the block.
Finer control
suppress-mapsuppresses only some components.unsuppress-mapon one neighbor leaks chosen specifics to that peer only.as-setkeeps loop detection but is discouraged today (RFC 6472), and RPKI origin validation cannot validate an AS_SET origin.
They will ask next
How is this different from a static route to Null0 plus a network statement?
The static method always advertises the summary, even if nothing behind it is up. aggregate-address advertises only while at least one component exists in BGP. Many edge designs prefer the static anchor for stability.
Traps
- Forgetting
summary-onlyand announcing both the /24 and the /25s. - Configuring an aggregate with no component in the BGP table and wondering why nothing is advertised.
How did you do?Book 4.1 Global InternetOwn pageWhat was BGP synchronization, and why is it off today?
Say this first
The synchronization rule said a router must not use, or advertise to eBGP peers, a route learned over iBGP until the same prefix was also known through the IGP. It protected transit networks where some routers in the path did not run BGP: without it, a non-BGP router in the middle would drop traffic for external prefixes it had never heard of. Today every transit router runs BGP or the core is MPLS, so the rule only gets in the way. Cisco has had it disabled by default since IOS 12.2(8)T.
Go deeper
The problem it solved
AS 64500 has R1, R2 and R3 in a line. R1 and R3 run BGP and peer over iBGP. R2 runs only OSPF. R3 learns 198.51.100.0/24 from R1 and forwards traffic toward R1 through R2. R2 has no route for 198.51.100.0/24 and drops it. Synchronization stopped R3 from using the route until OSPF also carried it, which in practice meant redistributing BGP into the IGP.
Why it went away
- Redistributing internet routes into an IGP does not scale.
- The real fix is to run BGP on every router in the transit path (with route reflectors), or to use an MPLS core where P routers only need labels to the PE loopbacks.
An old Cisco quirk
With OSPF as the IGP, synchronization only succeeded if the OSPF router ID of the router advertising the prefix matched its BGP router ID.
They will ask next
What is a BGP-free core?
An MPLS design where the P routers run only the IGP and LDP. Edge routers label packets toward the exit PE's loopback, so the core forwards on labels and never needs the internet routes.
Traps
- Describing synchronization as keeping BGP tables in sync between peers. It is about BGP and the IGP agreeing.
How would you design and configure dual-homed internet access with two ISPs using BGP?
Say this first
I get provider-independent space and a public ASN, then run eBGP to each ISP, ideally from two edge routers joined by iBGP with next-hop-self. I announce only my own prefix, filtered outbound so I never become a transit path. Inbound I take a default plus partial routes, or full tables, with prefix filters and maximum-prefix. I pick the primary exit with local preference, make the backup less attractive for inbound traffic with AS path prepending or, better, the backup ISP's local preference community, and then test failover in both directions.
Go deeper
Choices to talk through
- Active/standby or load sharing. Standby is simpler to reason about. Sharing needs per-prefix tuning.
- One edge router or two. Two routers remove a single point of failure. They run iBGP between loopbacks,
and the inside gets a default from both (OSPF
default-information originate, or HSRP/VRRP on the LAN). - What to receive. Default only (cheap, no real path choice), default plus partial (the ISP's own and its customers' routes, a good middle ground), or full tables (best exits, needs memory for about a million IPv4 routes plus IPv6).
R1 (primary, ISP1)
Cisco IOS ip prefix-list OUR-PREFIXES seq 5 permit 203.0.113.0/24 ! ip prefix-list FROM-ISP seq 5 deny 203.0.113.0/24 le 32 ip prefix-list FROM-ISP seq 10 deny 10.0.0.0/8 le 32 ip prefix-list FROM-ISP seq 15 deny 172.16.0.0/12 le 32 ip prefix-list FROM-ISP seq 20 deny 192.168.0.0/16 le 32 ip prefix-list FROM-ISP seq 25 permit 0.0.0.0/0 le 24 ! route-map ISP1-IN permit 10 match ip address prefix-list FROM-ISP set local-preference 200 ! route-map ISP1-OUT permit 10 match ip address prefix-list OUR-PREFIXES ! ip route 203.0.113.0 255.255.255.0 Null0 ! router bgp 64500 bgp router-id 10.255.0.1 bgp log-neighbor-changes network 203.0.113.0 mask 255.255.255.0 neighbor 192.0.2.1 remote-as 64496 neighbor 192.0.2.1 description ISP1 neighbor 192.0.2.1 route-map ISP1-IN in neighbor 192.0.2.1 route-map ISP1-OUT out neighbor 192.0.2.1 maximum-prefix 2000 80 neighbor 10.255.0.2 remote-as 64500 neighbor 10.255.0.2 update-source Loopback0 neighbor 10.255.0.2 next-hop-selfR2 (backup, ISP2)
Same structure, with the same
OUR-PREFIXESandFROM-ISPlists, plus:Cisco IOS route-map ISP2-IN permit 10 match ip address prefix-list FROM-ISP ! route-map ISP2-OUT permit 10 match ip address prefix-list OUR-PREFIXES set as-path prepend 64500 64500 ! router bgp 64500 neighbor 198.51.100.1 remote-as 64497 neighbor 198.51.100.1 route-map ISP2-IN in neighbor 198.51.100.1 route-map ISP2-OUT out neighbor 198.51.100.1 maximum-prefix 2000 80permit 0.0.0.0/0 le 24accepts the default and anything up to /24. The deny lines drop our own block and RFC 1918 space; a production bogon list is longer.Testing
Shut the ISP1 session in a change window. Outbound should move to ISP2 within seconds. Inbound depends on the rest of the internet, so watch interface counters and a looking glass for a few minutes. Then restore and confirm traffic returns.
They will ask next
Why maximum-prefix on sessions where you only expect a few hundred routes?
If the ISP leaks a full table at you by mistake, the edge router could run out of memory or start carrying traffic it should not. The limit shuts the session first.
Can you announce to ISP2 only when ISP1 fails?
Yes, with conditional advertisement:
neighbor 198.51.100.1 advertise-map ADVERTISE non-exist-map WATCHannounces our prefix to ISP2 only when the route matched by WATCH (something learned from ISP1) is gone. It gives a clean standby but slower failover, since the internet has to learn the new path from scratch.What is the smallest prefix you can count on being accepted?
A /24 for IPv4 and a /48 for IPv6. Longer prefixes are filtered by most networks.
Traps
- Forgetting the outbound filter and leaking one ISP's routes to the other.
- Using weight for the primary choice when two routers are involved. Only local preference reaches both.
How did you do?Book 4.1 Global InternetOwn pageHow do AS path access lists and regular expressions work? Give me the common patterns.
Say this first
An AS path access list (
ip as-path access-list) matches the AS_PATH as a string using a regular expression, and is applied withneighbor x filter-list N in|outormatch as-pathin a route map. The patterns everyone uses:^$is my own routes (empty path),^64496_came in from neighbor 64496,_64496$was originated by 64496,^64496$was originated by 64496 and learned directly from it,_64496_passed through 64496 anywhere, and.*matches everything.Go deeper
Characters
Symbol Meaning ^$Start and end of the path _Any delimiter: space, comma, brace, start or end of string .Any single character *+?Zero or more, one or more, zero or one of the previous item [0-9]One character from a range ( )`` Patterns
Regex Matches ^$Locally originated routes .*Any path ^64496$Originated by directly connected AS 64496 ^64496_Learned from 64496 _64496$Originated by 64496, from anywhere _64496_64496 anywhere in the path ^64496_[0-9]+$One AS behind 64496: its direct customers ^64500(_64500)*$Originated by 64500, with any number of prepends ^[0-9]+$Originated by whichever neighbor sent it Test before you apply
Output R1# show ip bgp regexp ^64496_[0-9]+$Then apply:
Cisco IOS ip as-path access-list 10 permit ^64496_[0-9]+$ ip as-path access-list 10 permit ^64496$ ! router bgp 64500 neighbor 192.0.2.1 filter-list 10 inLike prefix lists, AS path access lists end with an implicit deny.
Junos
Junos AS path regex works on whole AS numbers, not characters:
.is any one AS, so64496 .*means learned from 64496.They will ask next
Why use the underscore instead of a space?
Because the path can begin or end at that point, or contain AS_SET braces and commas.
_matches all of those, so_64496_catches 64496 at the start, middle or end.What goes wrong with
^6449?It matches any path starting with an ASN that begins with 6449: 64496, 64497, 644990 and so on. Anchor the whole number with
_or$.
Traps
- Writing
64496with no anchors, which matches 164496 or 644961 as well.
How did you do?Book 4.1 Global InternetOwn pageWhat is the difference between a hard reset, soft reconfiguration inbound and route refresh?
Say this first
A hard reset (
clear ip bgp x) tears down the TCP session and loses every route from that peer until it comes back. Soft resets re-apply policy without dropping the session. Outbound is easy:clear ip bgp x soft outresends my updates. Inbound needs the peer's routes again: either from a local unmodified copy kept byneighbor x soft-reconfiguration inbound, which costs memory, or by asking the peer to resend with the ROUTE-REFRESH capability (RFC 2918). Almost everything supports route refresh, soclear ip bgp x soft inuses it.Go deeper
When you need it
IOS does not re-run changed route maps or prefix lists against routes already received or sent. After a policy change you soft clear the affected direction. Junos re-evaluates policy on commit.
Check that route refresh is negotiated
Output R1# show ip bgp neighbors 192.0.2.1 | include Route refresh Route refresh: advertised and received(new)Why people still enable soft reconfiguration inbound
On IOS,
show ip bgp neighbors 192.0.2.1 received-routesneeds it. That command shows what the peer sent before my inbound policy, whileroutesshows what survived the policy. Comparing the two answers "is the ISP not sending it, or am I dropping it?" The price is a second copy of that peer's routes in memory, which hurts with full tables.Command Session drops? Needs clear ip bgp 192.0.2.1Yes Nothing clear ip bgp 192.0.2.1 soft outNo Nothing clear ip bgp 192.0.2.1 soft inNo Route refresh, or soft reconfiguration inbound They will ask next
What does enhanced route refresh add?
RFC 7313 marks the start and end of a refresh, so the router can remove routes the peer no longer sends once the refresh completes, instead of keeping stale entries.
Traps
- Running
clear ip bgp *in production to apply a route map change. It drops every session at once.
How did you do?Book 4.1 Global InternetOwn pageWhat does maximum-prefix do, and how do you choose the value?
Say this first
neighbor x maximum-prefix N [threshold] [restart minutes | warning-only]caps how many prefixes I accept from a peer. At the threshold percentage, 75 by default, it logs a warning. Above N it sends a NOTIFICATION (Cease, maximum number of prefixes reached) and holds the session down until someone clears it or the restart timer fires. It protects the router from a leak or a fat-fingered config at the peer. I set it from what I expect plus headroom, and alert on the warning.Go deeper
Sizing
Peer Expected Reasonable limit Customer with one /24 1 10 ISP sending default plus partial 800 2000 ISP sending a full IPv4 table about 1,000,000 Comfortably above the table, reviewed as it grows The full-table case bites people: the internet grows, the limit does not, and one day every full-table session drops at once.
What it looks like when it trips
Output R1# show ip bgp summary | begin Neighbor Neighbor V AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down State/PfxRcd 192.0.2.1 4 64496 1520 233 0 0 0 00:01:12 Idle (PfxCt) 198.51.100.1 4 64497 8211 7998 3120 0 0 5d02h 1874The syslog shows
%BGP-4-MAXPFXat the warning threshold and%BGP-3-MAXPFXEXCEEDwhen the limit is crossed.Cisco IOS router bgp 64500 neighbor 192.0.2.1 maximum-prefix 2000 80 restart 15They will ask next
warning-only or restart?
On customer sessions at a provider, hard limits with a restart timer. On your own upstreams many teams use a hard limit too, because a leak toward you is exactly the case to stop.
warning-onlysuits peers where dropping the session is worse than the risk.It tripped. What do you do?
Do not just raise the number. Find out why the peer sent more: a leak, a new customer, a normal growth step. Then fix the cause or the limit, and clear the session.
Traps
- Setting a full-table limit once and never revisiting it.
How did you do?Book 4.1 Global InternetOwn pageWhen do you need ebgp-multihop, and what are the alternatives?
Say this first
eBGP sends with TTL 1 and expects the peer on a directly connected subnet. If I peer between loopbacks, for example to share load across parallel links, or through another device, I need
neighbor x ebgp-multihop Nto raise the TTL, plus a route to the peer's address, usually static. Between loopbacks of directly connected routers,disable-connected-checkis tidier because it keeps TTL 1.ttl-security hops N(GTSM) is the security opposite: send with TTL 255 and only accept packets that arrive with a TTL close to 255.Go deeper
Loopback peering across two parallel links
R1 (AS 64500) and R5 (AS 64510) are joined by two links, 10.0.15.0/30 and 10.0.15.4/30:
Cisco IOS interface Loopback0 ip address 10.255.0.1 255.255.255.255 ! ip route 10.255.0.5 255.255.255.255 10.0.15.2 ip route 10.255.0.5 255.255.255.255 10.0.15.6 ! router bgp 64500 neighbor 10.255.0.5 remote-as 64510 neighbor 10.255.0.5 update-source Loopback0 neighbor 10.255.0.5 disable-connected-checkR5 mirrors it. The two static routes give equal-cost paths to the loopback, so one BGP session survives a link failure and traffic shares both links.
Through a firewall or another router
Cisco IOS router bgp 64500 neighbor 10.255.0.5 ebgp-multihop 2Use the smallest hop count that works.
GTSM
neighbor 192.0.2.1 ttl-security hops 1makes both sides send with TTL 255 and drop BGP packets that arrive with a lower TTL than a directly connected peer could produce. A spoofed packet from far away cannot arrive with such a high TTL. It cannot be combined withebgp-multihopon the same neighbor.They will ask next
Why not set ebgp-multihop 255 and forget about it?
The session could then form over unexpected long paths after a failure, and you give up the protection that a low TTL gives. Set what the topology needs.
What goes wrong with multihop eBGP through a firewall?
The firewall must allow TCP 179 in both directions and must not strip TCP options, or MD5 authentication breaks. Some firewalls also randomise sequence numbers, which breaks MD5 too.
Traps
- Forgetting the route to the peer's loopback. Multihop does not create reachability.
How did you do?Book 4.1 Global InternetOwn pageHow do you secure a BGP session and the router running it?
Say this first
In layers. Authenticate the TCP session (MD5, or TCP-AO where supported). Use GTSM (
ttl-security hops 1) so only a directly connected device can reach port 179. Allow 179 only from configured peers with an infrastructure ACL and control plane policing. Filter prefixes both ways: only my space out, bogons and too-specific routes in. Cap each session with maximum-prefix, and validate origins with RPKI. Most real incidents are leaks and hijacks, so route filtering matters more than the password.Go deeper
Cisco IOS router bgp 64500 neighbor 192.0.2.1 remote-as 64496 neighbor 192.0.2.1 password Example-Key-2026 neighbor 192.0.2.1 ttl-security hops 1 neighbor 192.0.2.1 prefix-list FROM-ISP in neighbor 192.0.2.1 prefix-list OUR-PREFIXES out neighbor 192.0.2.1 maximum-prefix 2000 80Threat Control Spoofed TCP resets or session hijack MD5 or TCP-AO, GTSM CPU exhaustion through port 179 Infrastructure ACL, CoPP Peer sends junk or a leak Inbound prefix filters, maximum-prefix, RPKI You leak by mistake Outbound prefix and AS path filters Someone announces your space ROAs for your prefixes, monitoring MANRS
The industry baseline for network operators, MANRS, asks for four actions: filtering, anti-spoofing (BCP 38 source address validation), coordination (up-to-date contacts) and global validation (publishing IRR and RPKI data).
They will ask next
What is wrong with MD5?
MD5 is a weak hash and the option has no clean key rollover. TCP-AO (RFC 5925) fixes both, but support is uneven, so MD5 is still what most ISPs offer.
Traps
- Treating the MD5 password as the main protection. It stops spoofed resets, not bad routes.
How did you do?Book 4.1 Global InternetBook 8.5 Example SystemsOwn pageWhat is MP-BGP, and what are address families?
Say this first
Multiprotocol BGP (RFC 4760) lets one BGP session carry routes for many kinds of network by tagging each with an AFI/SAFI pair: IPv4 unicast, IPv6 unicast, VPNv4 for MPLS L3VPN, L2VPN EVPN for VXLAN fabrics, multicast, flowspec and more. The routes travel in the MP_REACH_NLRI and MP_UNREACH_NLRI attributes, each with its own next hop, and the peers agree on the families they support through capabilities in the OPEN.
Go deeper
Common AFI/SAFI pairs
Family AFI SAFI IPv4 unicast 1 1 IPv6 unicast 2 1 IPv4 labeled unicast (BGP-LU) 1 4 VPNv4 1 128 L2VPN EVPN 25 70 IOS config with separate IPv4 and IPv6 sessions
Cisco IOS router bgp 64500 no bgp default ipv4-unicast neighbor 192.0.2.1 remote-as 64496 neighbor 2001:DB8:0:1::1 remote-as 64496 ! address-family ipv4 unicast network 203.0.113.0 mask 255.255.255.0 neighbor 192.0.2.1 activate exit-address-family ! address-family ipv6 unicast network 2001:DB8:500::/48 neighbor 2001:DB8:0:1::1 activate exit-address-familyno bgp default ipv4-unicaststops IOS from activating every new neighbor for IPv4 automatically, so each family is turned on deliberately. Both network statements need an exact matching route, for example static routes to Null0 (ipv6 route 2001:DB8:500::/48 Null0for the IPv6 one).Output R1# show bgp ipv4 unicast summary | begin Neighbor Neighbor V AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down State/PfxRcd 192.0.2.1 4 64496 2310 2290 57 0 0 1d04h 3 R1# show bgp ipv6 unicast summary | begin Neighbor Neighbor V AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down State/PfxRcd 2001:DB8:0:1::1 4 64496 2307 2291 12 0 0 1d04h 2They will ask next
One session for both families, or separate IPv4 and IPv6 sessions?
Separate sessions are the norm with ISPs: each family fails and is debugged on its own, and IPv6 routes get an IPv6 next hop naturally. Carrying IPv6 over an IPv4 session works but needs care with the next hop.
What happens when you activate a new address family on a running session?
The capabilities are exchanged only in the OPEN, so most platforms reset the session to renegotiate. Plan it like any other session reset.
Traps
- Forgetting
activateunder the address family and wondering why a session is up with no routes in that family.
How did you do?Book 4.1 Global InternetBook 4.2 IP Version 6Own page
Expert7
What is a BGP confederation, and how does it compare with route reflection?
Say this first
A confederation (RFC 5065) splits one AS into sub-ASes, usually with private numbers. Inside each sub-AS you run normal iBGP. Between sub-ASes you run confederation eBGP: loop prevention works like eBGP, with sub-AS numbers recorded in the AS_CONFED_SEQUENCE, but NEXT_HOP, LOCAL_PREF and MED are kept as in iBGP. Outside peers see one AS. Route reflectors are far more common; confederations appear in very large networks and after mergers.
Go deeper
Config
R1 sits in sub-AS 65001 of confederation 64500 and peers with R4 in sub-AS 65002:
Cisco IOS router bgp 65001 bgp confederation identifier 64500 bgp confederation peers 65002 neighbor 10.255.0.2 remote-as 65001 neighbor 10.255.0.2 update-source Loopback0 neighbor 10.255.0.4 remote-as 65002 neighbor 10.255.0.4 update-source Loopback0 neighbor 10.255.0.4 ebgp-multihop 2External neighbors configure
remote-as 64500and never see 65001 or 65002.Details interviewers probe
- The sub-AS path shows in parentheses, for example
(65002) 64496 i. Confederation segments do not count toward AS path length. - Peer type order: eBGP beats confederation eBGP, which beats iBGP.
- Confederation eBGP sessions use TTL 1 like real eBGP, so loopback peering needs
ebgp-multihop.
Route reflectors versus confederations
Route reflectors Confederations Routers to reconfigure Only the RRs Every router (new router bgpnumber)Migration effort Low High Policy between parts Limited eBGP-style policy between sub-ASes Seen today Almost everywhere Large ISPs, merged networks They will ask next
Can you use both together?
Yes. Large confederations often run route reflectors inside each sub-AS.
Why might a merger use a confederation?
Two companies with different ASNs can join under one external AS while keeping their internal BGP structure, then clean up later.
Traps
- Thinking outside peers see the sub-AS numbers. The confederation segments are removed at the edge.
How did you do?Book 4.1 Global InternetOwn page- The sub-AS path shows in parentheses, for example
How does MED work, and what are the gotchas?
Say this first
MED (MULTI_EXIT_DISC) is a value you send to a neighbor AS to suggest which of your links it should use to reach you: lower is better. It is only a hint, it sits below LOCAL_PREF and AS path length, and by default Cisco compares MED only between paths from the same neighbor AS. Gotchas: a missing MED counts as 0, the best value, unless
bgp bestpath med missing-as-worstis set; results can depend on arrival order withoutbgp deterministic-med; comparing across different ASes needsbgp always-compare-med.Go deeper
Typical use: two links to the same provider
Cisco IOS route-map ISP1-PRIMARY-OUT permit 10 match ip address prefix-list OUR-PREFIXES set metric 10 ! route-map ISP1-BACKUP-OUT permit 10 match ip address prefix-list OUR-PREFIXES set metric 50The provider sees both paths from AS 64500 and, if it honours MED, sends traffic down the MED 10 link. Its own local preference still comes first, and some providers reset customer MEDs.
Non-transitive
The receiving AS passes your MED to its iBGP peers but must not pass it to another AS.
Order dependence
Without deterministic-med, IOS compares paths pairwise starting from the newest. MED is skipped whenever the two paths come from different ASes, so with three or more paths the result can change with arrival order. With
bgp deterministic-med, paths are grouped by neighbor AS, the best of each group is chosen, then groups are compared. Configure it the same on every router in the AS.Useful variants
set metric-type internalsets the MED to the IGP metric to the next hop, telling the neighbor which entry point is closest to the destination inside your network.They will ask next
Is MED compared before or after eBGP versus iBGP?
Before. MED is step 6, peer type is step 7.
Why do providers often ignore customer MEDs?
MED lets the customer push traffic onto expensive parts of the provider's network. Many providers prefer their own local preference rules or offer communities instead.
Traps
- Expecting MED to choose between two different ISPs. They never compare it across ASes by default.
- Expecting your MED to beat the neighbor's local preference.
How did you do?Book 4.1 Global InternetOwn pageTwo paths tie on everything down to MED. How does the router decide from there?
Say this first
Next it prefers eBGP over iBGP, then the path whose next hop is closest by IGP metric (hot potato). If multipath is configured it installs the tied paths. Then it keeps the oldest eBGP path for stability, then picks the lowest router ID (the ORIGINATOR_ID for reflected routes), then the shortest cluster list, and finally the lowest neighbor address.
Go deeper
Example
R2 in AS 64500 has its own eBGP session to ISP1 at 192.0.2.5, and also hears the same prefix over iBGP from R1:
Output R2# show ip bgp 198.51.100.0 BGP routing table entry for 198.51.100.0/24, version 31 Paths: (2 available, best #2, table default) Advertised to update-groups: 2 Refresh Epoch 1 64496 10.255.0.1 (metric 3) from 10.255.0.1 (10.255.0.1) Origin IGP, metric 0, localpref 100, valid, internal rx pathid: 0, tx pathid: 0 Refresh Epoch 1 64496 192.0.2.5 from 192.0.2.5 (192.0.2.253) Origin IGP, metric 0, localpref 100, valid, external, best rx pathid: 0, tx pathid: 0x0Same local preference, same AS path length, same origin, same MED from the same AS. Step 7 decides: external beats internal. The
(metric 3)is the IGP cost to R1's loopback, used at step 8 if both were internal. The address in parentheses afterfromis the peer's router ID, used at step 11.Why each step exists
- eBGP over iBGP: an external path leaves the AS right here instead of crossing it.
- IGP metric: hand the traffic to the other AS at the nearest exit (hot potato).
- Oldest path: do not flap the best path every time an equal new one appears.
- Router ID, cluster list, neighbor address: deterministic tie breakers. The neighbor address matters when two sessions go to the same router, for example two links to one ISP router.
They will ask next
What is hot potato versus cold potato routing?
Hot potato hands traffic to the next AS as early as possible, which is what the IGP metric step does. Cold potato carries it inside your own network to the exit closest to the destination, usually based on MEDs or communities from the neighbor.
When does the oldest path rule cause trouble?
After a reboot or flap, which path is oldest is random, so two routers can choose different exits and your traffic pattern changes from one day to the next.
bgp bestpath compare-routeridmakes it deterministic.
Traps
- Comparing the local router's own router ID. The step compares the router IDs of the peers that sent the paths.
How did you do?Book 4.1 Global InternetOwn pageHow does BGP load sharing work with maximum-paths, and what has to match?
Say this first
By default BGP installs one best path.
maximum-paths nfor eBGP ormaximum-paths ibgp nfor iBGP lets it install up to n paths. On Cisco the paths must tie down to the IGP metric step: same weight, local preference, AS path length, origin and MED, and by default the same AS path content; iBGP paths also need equal IGP cost to their next hops. One path is still marked best and only that one is advertised to peers.bgp bestpath as-path multipath-relaxaccepts different AS paths of equal length.Go deeper
R1 has two links to ISP1 (192.0.2.1 and 192.0.2.5) and
maximum-paths 2under BGP:Output R1# show ip bgp | begin Network Network Next Hop Metric LocPrf Weight Path *m 198.51.100.0 192.0.2.5 0 0 64496 i *> 192.0.2.1 0 0 64496 i R1# show ip route bgp | begin Gateway Gateway of last resort is not set B 198.51.100.0/24 [20/0] via 192.0.2.5, 00:02:10 [20/0] via 192.0.2.1, 00:02:10mmarks the extra multipath entry;>is still the single best path used for advertising.Across two different ISPs
The AS paths differ (
64496 ...versus64497 ...), so they are not multipath candidates by default.bgp bestpath as-path multipath-relaxallows it when the lengths are equal. It is a hidden or less documented command on some IOS releases.Unequal sharing
The link bandwidth extended community (
bgp dmzlink-bwon IOS) can weight the split by link speed.They will ask next
Does multipath change what the router advertises?
No. Peers still receive one best path. Advertising several paths needs add-path.
Why might iBGP multipath not work behind a route reflector?
The RR reflects only its best path, so clients never see the second path. Add-path on the RR, or in MPLS VPNs a unique RD per PE, gives clients both paths.
Traps
- Expecting load sharing across two ISPs with plain
maximum-paths. - Configuring
maximum-pathswithout theibgpkeyword for iBGP paths.
How did you do?Book 4.1 Global InternetOwn pageWhat is RPKI, and how does route origin validation work?
Say this first
RPKI is a certificate system rooted at the five RIRs. The holder of address space signs a Route Origin Authorization (ROA), for example "AS 64500 may originate 203.0.113.0/24, up to length /24". Validator software fetches and checks the ROAs and feeds the results to routers over the RTR protocol. Each BGP route then gets a state: Valid (a ROA covers it and origin and length match), Invalid (covered, but wrong origin or too specific) or NotFound (no ROA covers it). The common policy is drop Invalid and accept the rest. It checks the origin AS only, not the path.
Go deeper
Moving parts
- ROA: created in the RIR portal by the address holder.
- Validator (relying party): Routinator, rpki-client, FORT and others download the repositories and verify signatures.
- RTR (RFC 8210): routers pull the validated list from the validator.
- Router policy: what to do with each state.
IOS-XE example
Cisco IOS router bgp 64500 bgp rpki server tcp 10.10.0.50 port 3323 refresh 600 neighbor 192.0.2.1 route-map ISP1-IN in ! route-map ISP1-IN deny 5 match rpki invalid route-map ISP1-IN permit 10 set local-preference 200Port 3323 is the RTR port Routinator uses by default. Some releases also factor validation state into best path by themselves; an explicit deny makes the intent obvious. Check the session with
show ip bgp rpki servers.The maxLength trap
If your ROA says max length /24 and you announce the /24, a hijacker's /25 is Invalid. If the ROA allowed /25 while you only announce the /24, an attacker can announce a /25 with your AS as the forged origin and it shows Valid. RFC 9319 recommends ROAs that match exactly what you announce.
They will ask next
Which attacks does origin validation not stop?
A forged origin (the attacker puts your AS at the end of its fake path) and route leaks, where the origin is correct but the path is not. BGPsec (RFC 8205) is barely deployed; ASPA, which lets an AS publish its providers, is being standardised for leak detection.
What if all validators go down?
Routers keep the last data for a while, then routes fall back to NotFound and are accepted. It fails open, which is the right default, but run at least two validators.
Traps
- Saying RPKI encrypts or signs BGP updates. It only lets routers check the origin AS against signed data.
- Assuming every Invalid route is an attack. Many are stale ROAs after a customer moved providers.
How did you do?Book 4.1 Global InternetBook 8.5 Example SystemsOwn pageWhat is the difference between a BGP hijack and a route leak? Give an example.
Say this first
A hijack is announcing address space you do not hold: the exact prefix, which splits traffic based on AS path, or a more specific one, which wins everywhere because of longest match. A route leak is announcing real routes beyond their intended scope, typically a customer passing routes from one provider to another (RFC 7908). The classic hijack: in February 2008 Pakistan Telecom announced a more specific part of YouTube's address block to block YouTube inside Pakistan, its upstream propagated it, and YouTube was unreachable for much of the internet for about two hours.
Go deeper
How they differ
Hijack Leak Origin AS Wrong, or forged Correct Usual cause Mistake or malice Policy mistake Main defences RPKI origin validation, prefix filters built from IRR and ROA data Customer prefix filters, maximum-prefix, BGP roles (RFC 9234), ASPA A well-known leak
In June 2019 a BGP optimizer at a small ISP generated more specific routes meant for internal use. A customer of that ISP leaked them to its other provider, Verizon, which accepted and propagated them. Traffic for Cloudflare and others was pulled through the small networks for a couple of hours.
The valley-free rule behind leaks
Routes from customers may be sent to everyone. Routes from peers and providers may be sent only to customers. A leak breaks that rule, for example provider routes sent to another provider.
If someone hijacks your prefix
- Confirm with external views: BGP monitoring alerts, RIPE RIS, looking glasses.
- Announce more specifics if the hijack is shorter than /24, so you win by longest match.
- Contact the offending network and its upstreams, and your own providers.
- Make sure your ROAs exist, so networks that drop Invalid routes ignore the hijack.
They will ask next
Why is a more specific hijack so effective?
Routers forward on longest match before BGP preferences matter. A /24 inside your /22 attracts that traffic from every network that accepts it, whatever their path preferences.
What does RFC 9234 add?
BGP roles (provider, customer, peer, route server) negotiated in the OPEN, and the Only to Customer attribute, so routers can detect and stop leaks automatically.
Traps
- Claiming RPKI origin validation solves route leaks. In a leak the origin is correct, so the routes are Valid.
How did you do?Book 4.1 Global InternetBook 8.1 Trust and ThreatsOwn pageBGP is slow to converge. What makes it slow, and how do you speed it up?
Say this first
Four things. Detecting the failure (a 180 second hold timer). Recomputing, including path hunting through alternatives that are also failing. Propagating (the advertisement interval, 30 seconds for eBGP on Cisco, plus the time to send many prefixes). And reprogramming the forwarding table prefix by prefix. Fixes map onto each: BFD and fast external fallover for detection, next hop tracking so IGP changes trigger BGP at once, add-path or best-external so backup paths are already known, and BGP PIC so the FIB switches all affected prefixes to a precomputed backup next hop in one step.
Go deeper
Detection
bgp fast-external-fallover(on by default in IOS) for directly connected eBGP.- BFD with
neighbor x fall-over bfdfor everything else.
Reacting to IGP changes
Next hop tracking watches the routes to BGP next hops and recalculates when they change, instead of waiting for a periodic scan. On IOS it is on by default, with a short trigger delay (5 seconds).
Knowing the backup in advance
A route reflector advertises one best path, so after a failure its clients wait for the RR to compute and send a new one. Add-path sends extra paths; best-external makes a router advertise its own external path even when an internal one is best.
Programming the FIB
Without PIC, a router with 900,000 prefixes behind a failed next hop rewrites 900,000 entries. With BGP PIC (prefix independent convergence), prefixes share next hop objects with a precomputed backup, so one change moves them all.
Planned work
Graceful shutdown: before maintenance, tag routes with GRACEFUL_SHUTDOWN (65535:0) or lower their local preference and wait for traffic to move, then shut the session. Graceful restart and NSR keep forwarding while a control plane restarts.
They will ask next
What is route flap dampening, and why is it used less now?
It adds a penalty each time a prefix flaps (1000 per flap on Cisco) and suppresses it above 2000 until the penalty decays below 750, with a 15 minute half-life and 60 minute maximum suppression. Aggressive settings punished normal convergence events, so operators now use it lightly or not at all.
How do you take an edge router out of service without dropping traffic?
Make its paths less preferred first (graceful shutdown community, lower local preference, or prepend outbound), wait until traffic has moved to the other edge, then shut the sessions.
Traps
- Focusing only on timers. Big tables converge slowly because of the FIB, which is why PIC exists.
How did you do?Book 4.1 Global InternetOwn page