How would you connect two data centers? Talk me through the options and the risks of stretching layer 2.
Show the answer
Say this first
I prefer layer 3 between sites, with applications running active/active behind DNS or anycast. If layer 2 really has to stretch, I use VXLAN EVPN Multi-Site with border gateways, or EVPN across the WAN, rather than bridging VLANs over dark fiber with back-to-back vPC. A stretched layer 2 joins the failure domains (one loop or broadcast storm hits both sites), causes traffic tromboning, and risks split brain in clusters when the link fails.
Go deeper
Transport
Dark fiber or DWDM (lowest latency, you own the optics), carrier MPLS or Metro Ethernet, and internet IPsec as a backup.
Layer 3 only
Each site has its own subnets. BGP between sites. Service failover through GSLB, DNS or anycast. Replication in the database or application. The cleanest failure isolation.
Layer 2 extension options
| Option | Notes |
|---|---|
| Back-to-back vPC or MLAG over fiber | Simple, but one STP and broadcast domain; storm control is essential |
| OTV | Cisco legacy, replaced by VXLAN EVPN Multi-Site |
| VXLAN EVPN Multi-Site | Border gateways end each fabric's VXLAN and re-encapsulate between sites; separate underlays and control planes; BUM can be rate limited |
| EVPN over MPLS, VPLS | Service provider style |
Risks of stretched layer 2
- Shared failure domain. A loop in one site floods the other.
- Tromboning. A host in site B uses a gateway in site A. Fix with an anycast gateway in both sites.
- Ingress asymmetry. Traffic enters through site A's firewall and leaves through site B's, and stateful firewalls drop it.
- Split brain. Clusters need a quorum witness in a third location.
- Latency. Synchronous replication and live migration have distance limits.
They will ask next
How do you stop ingress tromboning for a subnet that lives in both sites?
Advertise host routes (/32 from EVPN type 2 or type 5) toward the WAN from the site where each host actually is, or steer clients with DNS. The longest match then brings traffic to the right site.
Traps
- Proposing "just trunk the VLAN over the fiber" without storm control, loop protection or a plan for link failure.