An interface averages 20 percent utilization but its output drops keep rising. Explain it and how you would prove it.
Show the answer
Say this first
Averages hide bursts. A 30-second or 5-minute counter averages millions of packets, while a microburst lasts milliseconds. When several fast ports send to one egress port at once, or a 10G port sends toward a 1G port, the egress buffer fills within a few milliseconds and drops, while the average stays low. I prove it with fine-grained data: platform buffer or microburst monitoring, per-queue drop counters, or a timestamped capture showing line-rate trains. Fixes are QoS and buffer tuning, removing the speed mismatch, or spreading the load.
Go deeper
The arithmetic
A 1G port drains 125 KB per millisecond. A 10G port can deliver 1.25 MB toward it in that same millisecond. If the queue's buffer is a few hundred KB, a burst of a couple of milliseconds overflows it. Over 30 seconds, the port still averages a small number.
Typical sources
- Speed mismatch: a 10G or 25G server sending to a 1G client.
- Incast: many servers answering one request at the same moment, as with storage or distributed databases.
- Backups and replication jobs that all start on the hour.
Proving it
- Platform tools built for this, such as Arista LANZ or micro-burst monitoring on Nexus 9000, or streaming telemetry at short intervals.
- Per-queue drop counters, which show which class is dropping.
- A capture with precise timestamps at the receiver: trains of back-to-back packets, then loss and retransmissions.
Fixes
- QoS, so bursts land in a queue with room and drops fall on bulk traffic.
- More buffer per queue where supported (on Catalyst 9000,
qos queue-softmax-multiplier 1200). - Remove the speed mismatch by upgrading the egress port.
- Spread incast across more links or pace it in the application.
They will ask next
Why does a deeper buffer not always help?
Deep buffers absorb bursts but add queuing delay, which hurts latency-sensitive traffic. Storage and bulk transfer like deep buffers; voice and trading prefer short queues.
Traps
- Concluding "the link is only 20 percent used, so it cannot be congestion".