What are Nagle's algorithm and delayed ACK, and why do they sometimes add 40 or 200 ms of latency together?
Show the answer
Say this first
Nagle's algorithm (RFC 896) holds back a small segment while earlier data is still unacknowledged, so many tiny writes become fewer full segments. Delayed ACK has the receiver wait a short time before acknowledging, hoping to combine the ACK with a reply or with the next segment. When a sender does two small writes and then waits for a reply, Nagle holds the second write until the first is acknowledged, and the receiver delays that ACK because it has no reply yet. Both wait until the delayed ACK timer fires: about 40 ms on Linux, 200 ms on Windows.
Go deeper
The stall step by step
- Client writes a 100-byte header. No data is outstanding, so it goes immediately.
- Client writes a 50-byte body. Data is outstanding and the segment is small, so Nagle holds it.
- Server gets the header. It cannot answer until it has the body, so there is no reply to piggyback on, and it delays its ACK.
- The delayed ACK timer fires, the ACK goes, Nagle releases the body, the server answers. Every request pays the timer.
Fixes
- Write the whole message in one call (or use writev), so there is no second small write.
- Set TCP_NODELAY on the sender for request and response or interactive protocols. SSH and many RPC libraries do this; Go sets it by default.
- On Linux the receiver can use TCP_QUICKACK, which is temporary and needs resetting.
Delayed ACK rules
A receiver acknowledges at least every second full-sized segment, and must not delay more than 500 ms (RFC 1122). Linux adapts its delay between about 40 ms and 200 ms.
They will ask next
Does this affect bulk transfers?
No. Bulk transfers send full-sized segments, which Nagle never holds, and the receiver ACKs every second segment. It hurts small request and response exchanges.
What is TCP_CORK?
A Linux option that is the opposite of NODELAY: hold all partial segments until the cork is removed or a full segment is ready. Web servers use it to send headers and file data in full segments.
Traps
- Disabling delayed ACK everywhere. It roughly halves ACK traffic and is not the problem in bulk flows.