Connect two Azure spokes through a hub firewall with peering and UDRs

AzureWorking

Peer two spoke VNets to an existing hub that has a VPN gateway and an Azure Firewall. Spokes must use the hub's VPN gateway for on-prem, and all spoke-to-spoke, spoke-to-on-prem and internet traffic must pass the firewall at 10.0.1.4.

VNet	Address space	Subnets
vnet-hub	10.0.0.0/16	AzureFirewallSubnet 10.0.1.0/26 (firewall 10.0.1.4), GatewaySubnet 10.0.255.0/27 (vgw-hub)
vnet-spoke1	10.1.0.0/16	snet-app 10.1.1.0/24
vnet-spoke2	10.2.0.0/16	snet-app 10.2.1.0/24

Resource group rg-net, region westeurope. The hub, its VPN gateway and the firewall already exist. On-prem is 192.168.0.0/16.

Tasks

Create the two spoke VNets.
Peer each spoke with the hub in both directions, with gateway transit from the hub.
Give each spoke subnet a route table that sends the other spoke, on-prem and the internet to the firewall.
Make on-prem traffic to the spokes enter through the firewall too, so the path is symmetric.
Verify peering state, effective routes and the next hop.

Solution

1Spoke VNets

Azure

az network vnet create -g rg-net -n vnet-spoke1 -l westeurope \
  --address-prefixes 10.1.0.0/16 --subnet-name snet-app --subnet-prefixes 10.1.1.0/24
az network vnet create -g rg-net -n vnet-spoke2 -l westeurope \
  --address-prefixes 10.2.0.0/16 --subnet-name snet-app --subnet-prefixes 10.2.1.0/24

2Peering in both directions

Azure

az network vnet peering create -g rg-net -n hub-to-spoke1 \
  --vnet-name vnet-hub --remote-vnet vnet-spoke1 \
  --allow-vnet-access --allow-forwarded-traffic --allow-gateway-transit
az network vnet peering create -g rg-net -n spoke1-to-hub \
  --vnet-name vnet-spoke1 --remote-vnet vnet-hub \
  --allow-vnet-access --allow-forwarded-traffic --use-remote-gateways

az network vnet peering create -g rg-net -n hub-to-spoke2 \
  --vnet-name vnet-hub --remote-vnet vnet-spoke2 \
  --allow-vnet-access --allow-forwarded-traffic --allow-gateway-transit
az network vnet peering create -g rg-net -n spoke2-to-hub \
  --vnet-name vnet-spoke2 --remote-vnet vnet-hub \
  --allow-vnet-access --allow-forwarded-traffic --use-remote-gateways

Create the hub side with gateway transit first. The spoke side's use remote gateways fails if the hub does not offer transit or has no gateway.

3Spoke route tables pointing at the firewall

Azure

az network route-table create -g rg-net -n rt-spoke1 -l westeurope \
  --disable-bgp-route-propagation true
az network route-table route create -g rg-net --route-table-name rt-spoke1 -n to-spoke2 \
  --address-prefix 10.2.0.0/16 --next-hop-type VirtualAppliance --next-hop-ip-address 10.0.1.4
az network route-table route create -g rg-net --route-table-name rt-spoke1 -n to-onprem \
  --address-prefix 192.168.0.0/16 --next-hop-type VirtualAppliance --next-hop-ip-address 10.0.1.4
az network route-table route create -g rg-net --route-table-name rt-spoke1 -n default \
  --address-prefix 0.0.0.0/0 --next-hop-type VirtualAppliance --next-hop-ip-address 10.0.1.4
az network vnet subnet update -g rg-net --vnet-name vnet-spoke1 -n snet-app \
  --route-table rt-spoke1

Repeat for rt-spoke2 with to-spoke1 (10.1.0.0/16). Disabling BGP route propagation stops the on-prem routes learned by the VPN gateway from reaching the spoke directly, so the to-onprem UDR to the firewall is the only path.

4Route on-prem to spoke traffic through the firewall

Azure

az network route-table create -g rg-net -n rt-gateway -l westeurope
az network route-table route create -g rg-net --route-table-name rt-gateway -n to-spoke1 \
  --address-prefix 10.1.0.0/16 --next-hop-type VirtualAppliance --next-hop-ip-address 10.0.1.4
az network route-table route create -g rg-net --route-table-name rt-gateway -n to-spoke2 \
  --address-prefix 10.2.0.0/16 --next-hop-type VirtualAppliance --next-hop-ip-address 10.0.1.4
az network vnet subnet update -g rg-net --vnet-name vnet-hub -n GatewaySubnet \
  --route-table rt-gateway

Without this, on-prem traffic to a spoke goes straight from the gateway over the peering while the reply goes through the firewall, and the firewall drops the asymmetric flow. Keep BGP propagation enabled on the GatewaySubnet route table.

Verify

Output

az network vnet peering list -g rg-net --vnet-name vnet-hub \
  --query "[].{Name:name,PeeringState:peeringState,AllowForwardedTraffic:allowForwardedTraffic,AllowGatewayTransit:allowGatewayTransit}" \
  -o table

Name           PeeringState    AllowForwardedTraffic    AllowGatewayTransit
-------------  --------------  -----------------------  ---------------------
hub-to-spoke1  Connected       True                     True
hub-to-spoke2  Connected       True                     True

Both hub-side peerings are Connected, which only happens once the spoke-side peering also exists.

Output

az network nic show-effective-route-table -g rg-net -n vm-app1-nic -o table
Source    State    Address Prefix    Next Hop Type     Next Hop IP
--------  -------  ----------------  ----------------  -------------
Default   Active   10.1.0.0/16       VnetLocal
Default   Active   10.0.0.0/16       VNetPeering
Default   Invalid  0.0.0.0/0         Internet
User      Active   10.2.0.0/16       VirtualAppliance  10.0.1.4
User      Active   192.168.0.0/16    VirtualAppliance  10.0.1.4
User      Active   0.0.0.0/0         VirtualAppliance  10.0.1.4

The user routes are active and the default internet route is Invalid because the UDR overrides it. No VirtualNetworkGateway routes appear because propagation is off. (Other system routes with next hop None are trimmed.)

Output

az network watcher show-next-hop -g rg-net --vm vm-app1 \
  --source-ip 10.1.1.4 --dest-ip 10.2.1.4 -o table

NextHopIpAddress    NextHopType       RouteTableId
------------------  ----------------  ---------------------------------------------------------------------------------------------------------------------------
10.0.1.4            VirtualAppliance  /subscriptions/3f2a7c1e-5b4d-4e8a-9c6f-1d2e3f4a5b6c/resourceGroups/rg-net/providers/Microsoft.Network/routeTables/rt-spoke1

Traffic from APP1 to APP2 goes to the firewall, chosen by rt-spoke1. The firewall also needs a network rule allowing the flow.

Mistakes people make

Creating the peering on one side only. It stays Initiated and passes nothing.
Forgetting --allow-forwarded-traffic, so traffic forwarded by the firewall is dropped at the peering.
Adding the UDR in spoke1 but not the return UDR in spoke2, which sends replies around the firewall.
Leaving GatewaySubnet without a route table, which makes on-prem flows asymmetric through the firewall.
Overlapping spoke address spaces. Peering refuses to connect VNets that overlap.

On other platforms

In AWS the same hub and spoke is a transit gateway. Spokes attach to it and send traffic to it from their route tables; an inspection VPC plays the firewall's role.

AWS

aws ec2 create-transit-gateway --description hub \
  --options AmazonSideAsn=64512,DefaultRouteTableAssociation=enable,DefaultRouteTablePropagation=enable
aws ec2 create-transit-gateway-vpc-attachment --transit-gateway-id tgw-0a1b2c3d4e5f60718 \
  --vpc-id vpc-0b2c3d4e5f6a70819 --subnet-ids subnet-0c3d4e5f6a7b80912 subnet-0d4e5f6a7b8c90123
aws ec2 create-route --route-table-id rtb-0e5f6a7b8c9d01234 \
  --destination-cidr-block 10.0.0.0/8 --transit-gateway-id tgw-0a1b2c3d4e5f60718

Say it in the interview

"Peering is not transitive, so the hub's firewall has to be put in the path with UDRs. Each spoke peers with the hub in both directions, uses the hub's VPN gateway through gateway transit, and has a route table that sends the other spoke, on-prem and the internet to the firewall IP. I also put a route table on GatewaySubnet so on-prem traffic enters through the firewall, which keeps every flow symmetric."