Octet

Branch fails over to the backup tunnel, but HQ keeps replying over the dead link

TroubleshootingWorking

The ticket

Branch router BR1 reaches HQ1 over a carrier Metro Ethernet circuit (10.1.1.0/30) with a GRE tunnel over the internet as backup (Tunnel0, 172.31.0.0/30). Both sides use static routes.

BR1:

Cisco IOS
ip sla 1
 icmp-echo 10.1.1.1 source-interface GigabitEthernet0/0/0
 frequency 5
ip sla schedule 1 life forever start-time now
track 1 ip sla 1 reachability
ip route 10.0.0.0 255.255.0.0 10.1.1.1 track 1
ip route 10.0.0.0 255.255.0.0 172.31.0.1 250

HQ1:

Cisco IOS
ip route 10.50.0.0 255.255.255.0 10.1.1.2
ip route 10.50.0.0 255.255.255.0 172.31.0.2 250

At 09:10 the carrier had a fault inside its network. The Ethernet handoffs at both ends stayed up. BR1 logged %TRACK-6-STATE: 1 ip sla 1 reachability Up -> Down and moved to the tunnel, as designed. Even so, branch users could not reach anything at HQ (10.0.0.0/16) until the carrier fixed the fault at 11:30.

Gi0/0/1 .1Gi0/0/0 .2Gi0/0/2Gi0/0/1Gi0/0/2HQ1HQ LAN 10.0.0.0/16CARRIERMetro E 10.1.1.0/30INETGRE Tunnel0 172.31.0.0/30BR1PC1branch 10.50.0.0/24

Book 3.3 Internet (IP)

Step 1 of 4

BR1 failed over correctly and the tunnel is up. Where do you look next?