How do you plan a migration or cutover, for example replacing a core switch pair, so it does not become an outage?
Show the answer
Say this first
As phases with exit criteria: discovery and baseline, design and lab test, a written method of procedure, pre-checks, a change window with a go or no-go decision, defined rollback triggers with a time box, post-checks compared against the baseline, then a period of hypercare. Rollback must be as rehearsed as the change and must fit inside the window. Communication goes out before, during and after.
Go deeper
Phases
- Discovery: inventory, configs, every port mapped with CDP, LLDP and MAC tables, and hidden dependencies: static routes to old addresses, SNMP and syslog sources, firewall rules that name the old devices, RADIUS clients.
- Design and test: new configs built from templates, tested in a lab, peer reviewed.
- Method of procedure (MOP): numbered steps with exact commands, expected output, who does what, timings and checkpoints.
- Pre-checks: capture interface states, routing neighbors and route counts, ARP and MAC counts, STP root and FHRP state, a ping matrix to key hosts, and application tests by their owners.
- Window: go or no-go meeting, other changes frozen, console or out-of-band access confirmed,
safety nets such as
reload inorcommit confirmedwhere they fit. - Rollback: triggers written in advance, for example "any critical application test fails and is not fixed within 15 minutes" or "not complete by 03:00". The rollback time is reserved inside the window.
- Post-checks: the same captures, compared line by line, and application owners confirm.
- Hypercare: extra monitoring and someone on site the first business morning.
- Close: update diagrams, IPAM, monitoring and the documentation, and hold a short review.
Techniques that lower risk
- Build new beside old and move links in batches instead of one big bang.
- Drain traffic before touching a device, for example OSPF
max-metric router-lsaor higher costs, so traffic has already moved when you cut. - Lower DNS TTLs days before any change that moves service addresses.
- Pilot with a small, friendly group first.
They will ask next
Big bang or phased?
Phased when the old and new can coexist, because each step is small and reversible. Big bang only when coexistence is impossible or costs more than the risk, and then with a full rehearsal and a tested rollback.
Who decides to roll back?
A named change lead, using the triggers agreed before the window. Deciding in the moment under pressure is how a 30 minute problem becomes a 4 hour outage.
Traps
- A plan with no rollback trigger, only rollback steps. Nobody knows when to pull the cord.
- Skipping the pre-check capture, then having nothing to compare the result with.