Octet

How do you plan a migration or cutover, for example replacing a core switch pair, so it does not become an outage?

Show the answer

Say this first

As phases with exit criteria: discovery and baseline, design and lab test, a written method of procedure, pre-checks, a change window with a go or no-go decision, defined rollback triggers with a time box, post-checks compared against the baseline, then a period of hypercare. Rollback must be as rehearsed as the change and must fit inside the window. Communication goes out before, during and after.

Go deeper

Phases

  1. Discovery: inventory, configs, every port mapped with CDP, LLDP and MAC tables, and hidden dependencies: static routes to old addresses, SNMP and syslog sources, firewall rules that name the old devices, RADIUS clients.
  2. Design and test: new configs built from templates, tested in a lab, peer reviewed.
  3. Method of procedure (MOP): numbered steps with exact commands, expected output, who does what, timings and checkpoints.
  4. Pre-checks: capture interface states, routing neighbors and route counts, ARP and MAC counts, STP root and FHRP state, a ping matrix to key hosts, and application tests by their owners.
  5. Window: go or no-go meeting, other changes frozen, console or out-of-band access confirmed, safety nets such as reload in or commit confirmed where they fit.
  6. Rollback: triggers written in advance, for example "any critical application test fails and is not fixed within 15 minutes" or "not complete by 03:00". The rollback time is reserved inside the window.
  7. Post-checks: the same captures, compared line by line, and application owners confirm.
  8. Hypercare: extra monitoring and someone on site the first business morning.
  9. Close: update diagrams, IPAM, monitoring and the documentation, and hold a short review.

Techniques that lower risk

  • Build new beside old and move links in batches instead of one big bang.
  • Drain traffic before touching a device, for example OSPF max-metric router-lsa or higher costs, so traffic has already moved when you cut.
  • Lower DNS TTLs days before any change that moves service addresses.
  • Pilot with a small, friendly group first.

Traps

  • A plan with no rollback trigger, only rollback steps. Nobody knows when to pull the cord.
  • Skipping the pre-check capture, then having nothing to compare the result with.
How did you do?