How does a SYN flood work, and how do SYN cookies defend against it without breaking TCP?
Show the answer
Say this first
A SYN flood sends huge numbers of SYNs, usually from spoofed addresses, so the server allocates a half-open entry for each and retransmits SYN-ACKs that are never answered. The SYN queue fills and real clients are refused. With SYN cookies the server stores nothing: it encodes the connection's details and a keyed hash into its own initial sequence number. When the client's ACK returns, the server checks the hash in ack minus one and rebuilds the connection from it.
Go deeper
Two queues on a Linux listener
- SYN queue: half-open connections in SYN_RCVD, waiting for the final ACK.
- Accept queue: completed connections waiting for the application to call accept(). Its size is the
listen() backlog, capped by
net.core.somaxconn(4096 by default since kernel 5.4).
A SYN flood fills the first. A slow application fills the second, which shows as ListenOverflows in
nstat and as Recv-Q at or near Send-Q on the LISTEN line of ss -lnt.
How cookies work
The server sets its ISN from a time counter, a small index for the client's MSS, and a cryptographic hash of the four-tuple with a secret. It keeps no state. A real client's ACK carries that ISN plus one; the server checks the hash and the time counter, then creates the connection. Spoofed sources never send the ACK, so they cost the server only an outgoing SYN-ACK.
The trade-off
Only a few MSS values fit in the cookie, and window scale and SACK would be lost. Linux stores those
options in the TCP timestamp when timestamps are on, which avoids most of the loss. Linux only switches to
cookies when the SYN queue overflows (net.ipv4.tcp_syncookies=1, the default), and the kernel log
then shows a "Possible SYN flooding on port ... Sending cookies" message.
Other layers of defence
SYN proxy on a firewall or load balancer, which completes the handshake itself before opening the server side; rate limits; and upstream scrubbing for volumetric floods that fill the link itself.
They will ask next
How do you tell a SYN flood from an application that is just slow to accept?
A flood shows many SYN-RECV sockets and rising
TcpExtSyncookiesSent. A slow application shows few SYN-RECV, a full accept queue on the LISTEN socket inss -lnt, and risingTcpExtListenOverflows.Why do victims of spoofed SYN floods sometimes complain about traffic they never sent?
The attacked server sends SYN-ACKs to the spoofed source addresses. Those hosts receive unexpected SYN-ACKs (backscatter) and answer with RST.
Traps
- Saying SYN cookies are always on. Linux uses them only when the queue overflows.