Octet

A capture on a Linux server shows TCP segments of 20 to 60 KB and checksum errors, yet the MTU is 1500. Is something broken?

Show the answer

Say this first

No. tcpdump sees packets where the kernel hands them to the driver, not on the wire. On transmit, TSO and GSO let the kernel pass one large segment that the NIC (or the last software step) cuts into MTU-sized frames, and checksum offload means the NIC fills in the checksum after tcpdump saw it, hence "incorrect". On receive, GRO merges consecutive segments before the capture point. The wire carries normal 1500-byte frames. To see the real frames, capture on a SPAN port or switch the offloads off for the test.

Go deeper

What it looks like

Output
10:02:31.114502 IP 10.0.10.21.443 > 10.0.10.30.51022: Flags [P.], seq 1:28961, ack 518, win 501, length 28960

28960 bytes is 20 segments of 1448. With -v, outgoing packets show cksum 0x2e4f (incorrect -> 0x8a1b).

Check the offloads

Output
$ ethtool -k ens192 | grep -E 'checksumming|segmentation-offload|receive-offload'
rx-checksumming: on
tx-checksumming: on
tcp-segmentation-offload: on
generic-segmentation-offload: on
generic-receive-offload: on
large-receive-offload: off

See wire-sized packets for a test

Shell
ethtool -K ens192 tso off gso off gro off
# capture, then put them back
ethtool -K ens192 tso on gso on gro on

This costs CPU and changes behaviour slightly, so do it briefly and not on a loaded production box.

Traps

  • Opening a ticket for "giant packets" or "bad checksums" seen in a host capture.
  • Disabling offloads permanently to make captures look tidy.
How did you do?