A capture on a Linux server shows TCP segments of 20 to 60 KB and checksum errors, yet the MTU is 1500. Is something broken?
Show the answer
Say this first
No. tcpdump sees packets where the kernel hands them to the driver, not on the wire. On transmit, TSO and GSO let the kernel pass one large segment that the NIC (or the last software step) cuts into MTU-sized frames, and checksum offload means the NIC fills in the checksum after tcpdump saw it, hence "incorrect". On receive, GRO merges consecutive segments before the capture point. The wire carries normal 1500-byte frames. To see the real frames, capture on a SPAN port or switch the offloads off for the test.
Go deeper
What it looks like
10:02:31.114502 IP 10.0.10.21.443 > 10.0.10.30.51022: Flags [P.], seq 1:28961, ack 518, win 501, length 2896028960 bytes is 20 segments of 1448. With -v, outgoing packets show cksum 0x2e4f (incorrect -> 0x8a1b).
Check the offloads
$ ethtool -k ens192 | grep -E 'checksumming|segmentation-offload|receive-offload'
rx-checksumming: on
tx-checksumming: on
tcp-segmentation-offload: on
generic-segmentation-offload: on
generic-receive-offload: on
large-receive-offload: offSee wire-sized packets for a test
ethtool -K ens192 tso off gso off gro off
# capture, then put them back
ethtool -K ens192 tso on gso on gro onThis costs CPU and changes behaviour slightly, so do it briefly and not on a loaded production box.
They will ask next
Can receive offloads break a Linux router or bridge?
LRO can: it merges packets in a way that cannot be undone exactly, so merged frames could not be forwarded as they arrived. The kernel disables LRO on devices that forward or are bridge ports. GRO is safe because the kernel re-segments with GSO on the way out.
Where do you capture if you need exactly what was on the wire?
Off the host: a SPAN or ERSPAN session or a tap on the switch side. The capture point on the host is above the NIC, so it always reflects offloads.
Traps
- Opening a ticket for "giant packets" or "bad checksums" seen in a host capture.
- Disabling offloads permanently to make captures look tidy.