A fiber link stays down even though both optics and the fiber test fine. What else can keep it down?
Show the answer
Say this first
Settings that must agree at layer 1. On 1 Gbps fiber, 1000BASE-X autonegotiation on one side and
speed nonegotiate on the other can leave the link down on one end. On 25G and 100G, a forward error
correction mismatch (RS-FEC on one side, BASE-R FEC or none on the other) keeps the link down. Also check a
port speed that does not match the optic, a breakout that was not configured, and an err-disabled port from
UDLD or an unsupported module.
Go deeper
Checklist
| Check | Why |
|---|---|
| Autonegotiation (1000BASE-X) | It agrees duplex and pause, not speed. Both ends must use it or both must not. Cisco disables it with speed nonegotiate. |
| FEC mode | 25GBASE-SR and 100GBASE-SR4 require RS-FEC. DACs at 25G may use RS-FEC, BASE-R (FC) FEC or none. Both ends must match. |
| Port speed | A 1G optic in a port fixed at 10G (or the reverse) stays down. |
| Breakout | A 4 x 25G cable on a port still configured as 100G stays down. |
| Err-disable | show interfaces status err-disabled for gbic-invalid, udld and others. |
| Polarity | Tx must land on Rx. Swap the strands at one end as a test. |
Checking FEC on other platforms
SW1(config)# interface Ethernet1
SW1(config-if-Et1)# error-correction encoding reed-solomonethtool --show-fec ens1f0
ethtool --set-fec ens1f0 encoding rsServer NICs and switches often pick different FEC defaults, which is why switch-to-server 25G links are the classic case.
They will ask next
What does FEC cost you?
A little latency (on the order of 100 ns for RS-FEC) and some line overhead, in exchange for correcting bit errors that would otherwise corrupt frames. At 25G and above over multimode, the optics are designed assuming FEC is on.
Traps
- Swapping optics repeatedly before comparing the configuration on both ends.