Octet

The nightly Netmiko config backup fails on 9 of 120 devices

TroubleshootingCore

The ticket

A Python script on SRV1 (10.0.50.10) backs up the running config of 120 Cisco devices every night with Netmiko. Last night it finished with 9 failures. Nobody changed the script. The summary at the end of the log:

Output
2026-11-09 01:00:44 FAIL ASW17 10.20.0.17 NetmikoAuthenticationException
2026-11-09 01:00:45 FAIL ASW18 10.20.0.18 NetmikoAuthenticationException
2026-11-09 01:00:45 FAIL ASW19 10.20.0.19 NetmikoAuthenticationException
2026-11-09 01:00:46 FAIL ASW20 10.20.0.20 NetmikoAuthenticationException
2026-11-09 01:01:13 FAIL ASW22 10.30.0.22 NetmikoTimeoutException
2026-11-09 01:01:13 FAIL ASW23 10.30.0.23 NetmikoTimeoutException
2026-11-09 01:01:14 FAIL ASW24 10.30.0.24 NetmikoTimeoutException
2026-11-09 01:02:05 FAIL CORE1 10.0.0.1 ReadTimeout
2026-11-09 01:02:06 FAIL CORE2 10.0.0.2 ReadTimeout
SRV1Backup host 10.0.50.10CORE1Large configFW1Building C edgeASW17New floor 10.20.0.17ASW22Building C 10.30.0.22

Step 1 of 4

Nine failures across very different devices. How do you approach it?