What MTU do you get in AWS, and where do large packets get into trouble?
Show the answer
Say this first
Current instance types use jumbo frames, 9001 bytes, inside a VPC. Traffic through an internet gateway is limited to 1500. A transit gateway carries up to 8500 bytes between VPCs and Direct Connect, and drops anything larger. A site-to-site VPN supports an MTU of 1446 and MSS of 1406. Path MTU discovery needs the ICMP fragmentation-needed message, so security groups must allow it or large transfers hang.
Go deeper
AWS numbers
| Path | MTU |
|---|---|
| Inside a VPC (current instance types) | 9001 |
| Through an internet gateway | 1500 |
| Transit gateway between VPCs, Direct Connect, peering | 8500 |
| Site-to-site VPN | 1446 (MSS 1406) |
| Direct Connect private VIF | 1500 or 9001 if jumbo is enabled |
| Direct Connect transit VIF | 1500 or 8500 if jumbo is enabled |
Where it hurts
- An instance with MTU 9001 sends large segments to a host across the transit gateway or the internet. If ICMP type 3 code 4 is blocked, the sender never learns to shrink, and transfers stall after the handshake.
- On-prem over VPN: the tunnel overhead leaves less than 1500 for the inner packet. Clamp MSS on the tunnel interface of your router.
What I do
- Allow ICMP "destination unreachable, fragmentation needed" inbound in security groups for paths that cross gateways.
- For instances that mostly talk off-VPC, set the interface MTU to 1500:
Shell ip link show eth0 sudo ip link set dev eth0 mtu 1500 - Clamp MSS on VPN routers. AWS's sample Cisco config uses
ip tcp adjust-mss 1379.
GCP
VPC networks default to an MTU of 1460 and can be raised up to 8896. The guest OS must match the VPC setting.
They will ask next
How do you prove an MTU problem from an instance?
ping -M do -s 1472 <dst>works (1500 total) andping -M do -s 8972 <dst>fails, ortracepathreports the path MTU. Small requests succeeding while large responses hang is the classic sign.Why does SSH work but scp stalls?
SSH login packets are small. The file transfer fills full-size segments, which hit the smaller MTU and are dropped with no ICMP getting back.
Traps
- Blocking all ICMP in security groups and NACLs "for security".
- Assuming 1500 everywhere and missing that instances default to 9001 inside the VPC.