Name the TCP states and tell me what a pile-up in each one usually means when you run ss or netstat.
Show the answer
Say this first
The states are LISTEN, SYN_SENT, SYN_RCVD, ESTABLISHED, FIN_WAIT_1, FIN_WAIT_2, CLOSE_WAIT, CLOSING, LAST_ACK, TIME_WAIT and CLOSED. Many SYN_SENT means SYNs are not answered; many SYN_RCVD points at a SYN flood or lost final ACKs; many CLOSE_WAIT is almost always an application that does not close its sockets; many TIME_WAIT is normal on the side that closes first.
Go deeper
What each pile-up tells you
| State | Side | Many of them usually means |
|---|---|---|
| SYN_SENT | Client | Peer down, route missing, or a firewall dropping SYNs silently |
| SYN_RCVD | Server | SYN flood, or the client's final ACK not arriving (asymmetric path through a stateful firewall) |
| ESTABLISHED | Both | Normal. Steady growth is a connection leak or a slow backend |
| FIN_WAIT_1 | Active closer | Our FIN is not acknowledged: peer gone, or unsent data stuck behind a zero window |
| FIN_WAIT_2 | Active closer | Peer acknowledged our FIN but its application never closed |
| CLOSE_WAIT | Passive closer | Our application received end of stream and never called close() |
| LAST_ACK | Passive closer | Our FIN is not acknowledged, peer probably gone |
| CLOSING | Both | Simultaneous close, rare and short-lived |
| TIME_WAIT | Active closer | Normal. A problem only if ephemeral ports run out |
Naming differs by tool
netstat prints SYN_SENT, ESTABLISHED, CLOSE_WAIT. ss prints SYN-SENT, SYN-RECV, ESTAB,
CLOSE-WAIT, TIME-WAIT, and UNCONN for unconnected UDP sockets.
Quick count on Linux
ss -tan | awk 'NR>1 {print $1}' | sort | uniq -c | sort -rnThey will ask next
Which states have no timer at all?
ESTABLISHED (unless keepalive is on) and CLOSE_WAIT. A socket in CLOSE_WAIT stays there until the process closes it or exits, which is why a leak grows until the process hits its file descriptor limit.
You see thousands of SYN-RECV on a server behind a firewall, but traffic is low. What else could it be?
Asymmetric routing. The SYN-ACK leaves by a different path, the client's ACK comes back through a stateful firewall that never saw the SYN-ACK, and the firewall drops it. The server keeps retrying SYN-ACKs from SYN_RCVD.
Traps
- Blaming the network for CLOSE_WAIT. The kernel has done its part; the local application has not.