← All work
Disaster recovery test automation
Outcome: On the strength of the automation, leadership brought DR testing back in-house, ending the recurring 150-person staffing contract while keeping the standing team the same size.
The problem
Every disaster recovery test was a huge manual effort. Running one meant contracting about 150 supplemental staff through an outside firm, so each cycle was expensive, slow to coordinate, and dependent on people who did not own the environment.
What I did
- Built automation that ran DR failover across the entire virtual server fleet, with no per-server intervention.
- Turned the runbooks the contractors had followed by hand into repeatable, auditable code.
- Ran it cycle after cycle until the standing team, using the automation, outperformed the contracted approach.