How do you work out the usable MTU inside a VTI or GRE over IPsec tunnel?
Show the answer
Say this first
Subtract every header the tunnel adds from the underlay MTU. GRE adds 24 bytes. ESP adds 8 bytes of
header, the IV, 2 bytes of trailer, the ICV and padding to the cipher block; tunnel mode adds a 20-byte
outer IP header and NAT-T adds 8 bytes of UDP. With AES-256-CBC and HMAC-SHA-256 on a 1500-byte path,
a VTI carries up to 1438 bytes, GRE over IPsec in transport mode 1434, in tunnel mode 1414, and in
tunnel mode behind NAT only 1398. The usual ip mtu 1400 with MSS 1360 fits every case except that
last one.
Go deeper
The ESP pieces (AES-256-CBC, HMAC-SHA-256)
| Piece | Bytes |
|---|---|
| ESP header (SPI and sequence) | 8 |
| IV | 16 |
| Pad length and next header | 2 |
| Padding | 0 to 15, so payload plus trailer is a multiple of 16 |
| ICV | 16 |
Worked example: VTI, no NAT
1500 minus 20 (outer IP), 8 (ESP header), 16 (IV) and 16 (ICV) leaves 1440 for the encrypted part.
1440 is a multiple of 16, and 2 bytes go to the trailer, so the largest inner packet is 1438. IOS
shows this as plaintext mtu 1438 in show crypto ipsec sa.
Largest inner packet on a 1500-byte underlay
| Tunnel | No NAT | Behind NAT (NAT-T) |
|---|---|---|
| Plain GRE | 1476 | not used (PAT cannot carry GRE) |
| VTI (IPsec tunnel mode) | 1438 | 1422 |
| GRE over IPsec, transport mode | 1434 | 1418 |
| GRE over IPsec, tunnel mode | 1414 | 1398 |
Adjustments
- AES-GCM uses an 8-byte IV and pads only to 4 bytes, so it leaves slightly more room.
- A PPPoE underlay is 1492, not 1500: redo the math rather than subtracting 8. Because CBC pads to 16-byte blocks, the IPsec numbers drop by 16 or not at all (a VTI goes from 1438 to 1422); plain GRE drops by exactly 8, to 1468.
- For GRE over IPsec in tunnel mode behind NAT, use
ip mtu 1380andip tcp adjust-mss 1340.
They will ask next
Why does padding make the overhead variable?
CBC encrypts whole 16-byte blocks, so the payload plus the 2-byte trailer is padded up to the next multiple of 16. The overhead for a given packet depends on its length, anywhere in a 15-byte range.
Why is transport mode the better choice for GRE over IPsec here?
It drops the second 20-byte IP header. The GRE packet already says public address to public address, so the extra header buys nothing.
Traps
- Quoting "IPsec adds 50 to 60 bytes" as a fixed number. It depends on cipher, integrity, mode, NAT-T and padding.