Octet

Build an AWS site-to-site VPN with BGP to an IOS-XE router

AWSWorking

Tasks

  1. Create the customer gateway, the virtual private gateway, and attach the VGW to the VPC.
  2. Create a BGP-based VPN connection with IKEv2 only and fixed inside CIDRs per tunnel.
  3. Enable VGW route propagation on the app route table.
  4. Configure IKEv2, the IPsec profile and two tunnel interfaces on R1.
  5. Run eBGP to both AWS inside addresses and advertise the on-prem summary.

Solution

  1. 1Customer gateway, VGW and VPN connection
    AWS
    CGW=$(aws ec2 create-customer-gateway --type ipsec.1 \
  --public-ip 198.51.100.10 --bgp-asn 65010 \
  --query CustomerGateway.CustomerGatewayId --output text)

VGW=$(aws ec2 create-vpn-gateway --type ipsec.1 --amazon-side-asn 64512 \
  --query VpnGateway.VpnGatewayId --output text)
aws ec2 attach-vpn-gateway --vpn-gateway-id $VGW --vpc-id vpc-0c5d6e7f8091a2b34

VPN=$(aws ec2 create-vpn-connection --type ipsec.1 \
  --customer-gateway-id $CGW --vpn-gateway-id $VGW \
  --options '{"StaticRoutesOnly":false,"TunnelOptions":[
    {"TunnelInsideCidr":"169.254.10.0/30","PreSharedKey":"<psk-tunnel-1>","IKEVersions":[{"Value":"ikev2"}]},
    {"TunnelInsideCidr":"169.254.20.0/30","PreSharedKey":"<psk-tunnel-2>","IKEVersions":[{"Value":"ikev2"}]}]}' \
  --query VpnConnection.VpnConnectionId --output text)

aws ec2 enable-vgw-route-propagation --route-table-id rtb-0d6e7f8091a2b3c45 --gateway-id $VGW

    StaticRoutesOnly: false means BGP. Read the AWS outside addresses with aws ec2 describe-vpn-connections --vpn-connection-ids $VPN --query "VpnConnections[0].Options.TunnelOptions[].[OutsideIpAddress,TunnelInsideCidr]" --output text. AWS can also generate a device config with get-vpn-connection-device-sample-configuration.

  2. 2IKEv2 and IPsec on R1R1
    R1 on Cisco IOS
    crypto ikev2 proposal AWS-PROP
 encryption aes-cbc-256
 integrity sha256
 group 14
!
crypto ikev2 policy AWS-POL
 proposal AWS-PROP
!
crypto ikev2 keyring AWS-KEYS
 peer AWS-TUN1
  address 203.0.113.11
  pre-shared-key <psk-tunnel-1>
 peer AWS-TUN2
  address 203.0.113.12
  pre-shared-key <psk-tunnel-2>
!
crypto ikev2 profile AWS-IKEV2
 match identity remote address 203.0.113.11 255.255.255.255
 match identity remote address 203.0.113.12 255.255.255.255
 identity local address 198.51.100.10
 authentication remote pre-share
 authentication local pre-share
 keyring local AWS-KEYS
 lifetime 28800
 dpd 10 3 on-demand
!
crypto ipsec transform-set AWS-TS esp-aes 256 esp-sha256-hmac
 mode tunnel
!
crypto ipsec profile AWS-IPSEC
 set transform-set AWS-TS
 set pfs group14
 set ikev2-profile AWS-IKEV2

    AES-256, SHA-256 and DH group 14 are all in AWS's accepted set. Lifetime 28800 seconds matches the AWS phase 1 default.

  3. 3Tunnel interfacesR1
    R1 on Cisco IOS
    interface Tunnel1
 description AWS tunnel 1
 ip address 169.254.10.2 255.255.255.252
 ip tcp adjust-mss 1379
 tunnel source GigabitEthernet0/0/0
 tunnel mode ipsec ipv4
 tunnel destination 203.0.113.11
 tunnel protection ipsec profile AWS-IPSEC
!
interface Tunnel2
 description AWS tunnel 2
 ip address 169.254.20.2 255.255.255.252
 ip tcp adjust-mss 1379
 tunnel source GigabitEthernet0/0/0
 tunnel mode ipsec ipv4
 tunnel destination 203.0.113.12
 tunnel protection ipsec profile AWS-IPSEC

    R1 takes the second address of each /30; AWS has the first. The MSS clamp of 1379 is the value AWS uses in its sample configs.

  4. 4eBGP over both tunnelsR1
    R1 on Cisco IOS
    ip route 192.168.0.0 255.255.0.0 Null0
!
router bgp 65010
 bgp log-neighbor-changes
 neighbor 169.254.10.1 remote-as 64512
 neighbor 169.254.10.1 timers 10 30 30
 neighbor 169.254.20.1 remote-as 64512
 neighbor 169.254.20.1 timers 10 30 30
 !
 address-family ipv4
  network 192.168.0.0 mask 255.255.0.0
  neighbor 169.254.10.1 activate
  neighbor 169.254.20.1 activate
 exit-address-family

    The Null0 route lets the network statement match an exact RIB entry. Keepalive 10 and hold 30 seconds detect a dead tunnel faster than the defaults.

Verify

R1 output
R1# show crypto ikev2 sa
 IPv4 Crypto IKEv2  SA

Tunnel-id Local                 Remote                fvrf/ivrf            Status
1         198.51.100.10/500     203.0.113.11/500      none/none            READY
      Encr: AES-CBC, keysize: 256, PRF: SHA256, Hash: SHA256, DH Grp:14, Auth sign: PSK, Auth verify: PSK
      Life/Active Time: 28800/742 sec

Tunnel-id Local                 Remote                fvrf/ivrf            Status
2         198.51.100.10/500     203.0.113.12/500      none/none            READY
      Encr: AES-CBC, keysize: 256, PRF: SHA256, Hash: SHA256, DH Grp:14, Auth sign: PSK, Auth verify: PSK
      Life/Active Time: 28800/736 sec

Two IKEv2 SAs in READY, one per AWS endpoint, on UDP 500 because there is no NAT in the path.

R1 output
R1# show ip bgp summary
BGP router identifier 198.51.100.10, local AS number 65010
BGP table version is 3, main routing table version 3

Neighbor        V           AS MsgRcvd MsgSent   TblVer  InQ OutQ Up/Down  State/PfxRcd
169.254.10.1    4        64512      76      74        3    0    0 00:12:09        1
169.254.20.1    4        64512      75      73        3    0    0 00:12:03        1

Both neighbors established with one prefix each, the VPC CIDR 10.30.0.0/16.

Output
aws ec2 describe-vpn-connections --vpn-connection-ids $VPN \
  --query "VpnConnections[0].VgwTelemetry[].[OutsideIpAddress,Status,StatusMessage]" --output text

203.0.113.11    UP      1 BGP ROUTES
203.0.113.12    UP      1 BGP ROUTES

AWS sees both tunnels UP and accepted the on-prem summary on each.

R1 output
R1# ping 10.30.10.20 source GigabitEthernet0/0/1
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 10.30.10.20, timeout is 2 seconds:
Packet sent with a source address of 192.168.0.1
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 11/12/14 ms

End to end through the tunnel. Needs the propagated route in the VPC and a security group on APP1 that allows ICMP from 192.168.0.0/16.

Mistakes people make

  • Configuring only one tunnel. AWS maintains one endpoint at a time and expects both up.
  • Forgetting VGW route propagation, so the VPN is up but replies have no route back.
  • Swapping the inside addresses. AWS is always the first host of the /30, the customer the second.
  • Advertising a network that is not in the RIB, so BGP never sends it. The Null0 route fixes that.
  • Leaving MSS unclamped and seeing large transfers stall through the tunnel.

On other platforms

Azure models the on-prem side as a local network gateway and connects it to a route-based VPN gateway in GatewaySubnet. With BGP, the local network gateway carries R1's ASN and BGP peer address, and R1 peers with the gateway's BGP address inside GatewaySubnet, which is not directly connected, so R1 needs a static route to it over the tunnel and eBGP multihop.

Azure
az network local-gateway create -g rg-net -n lng-hq \
  --gateway-ip-address 198.51.100.10 \
  --local-address-prefixes 192.168.0.1/32 \
  --asn 65010 --bgp-peering-address 192.168.0.1
az network vpn-connection create -g rg-net -n cn-hq \
  --vnet-gateway1 vgw-hub --local-gateway2 lng-hq \
  --shared-key <psk> --enable-bgp

Say it in the interview

"AWS gives me two tunnels per VPN connection, to two endpoints in different AZs. I build both as VTIs with IKEv2 and run eBGP to the first address of each /30, advertising the on-prem summary backed by a Null0 route. On the AWS side I enable route propagation on the route tables, otherwise the tunnels come up and nothing flows. I clamp MSS on the tunnels and check both SAs, both BGP sessions and an end-to-end ping."

Book 8.5 Example Systems