The database cutover checklist I actually run before flipping traffic

Database cutovers fail for social reasons as often as technical ones: unclear ownership, no rollback criteria, and “we will watch dashboards” without a written abort line. This article is a checklist you can paste into a runbook for moving traffic between database endpoints (primary change, major version, or vendor move). Treat it as practice discipline for a staging dry-run if you are not mid-migration.

Scope and non-goals

In scope: application traffic cutover to a new primary endpoint with comparable schema.

Out of scope here: multi-region active-active design, full data remodeling, or storage engine debates.

Compatibility work before the night

[ ] Schema diff: new primary has required tables, indexes, types
[ ] User/grants: app roles exist with least privilege
[ ] Connection flags: TLS, timezone, SQL mode, max_connections budget
[ ] Migration history table matches (no surprise pending DDL)
[ ] Feature flags ready to disable heavy jobs
[ ] Backups + tested restore point on the new side

Verification queries (examples)

-- row counts for critical tables (expect small drift if async replica)
SELECT 'orders' AS t, COUNT(*) AS c FROM orders
UNION ALL
SELECT 'customers', COUNT(*) FROM customers;

-- migration version
SELECT MAX(version) FROM schema_migrations;

-- connections budget
SHOW VARIABLES LIKE 'max_connections';

Record outputs in the ticket before you start.

Dual-write / dual-read: when to bother

Strategy	Use when	Skip when
Dual-write	Long coexistence, high risk	Short maintenance window + full resync done
Shadow reads	Compare results on sampling	You already have logical replication lag SLO met
Hard cut	Rehearsed, reversible DNS/config	Never rehearsed

If you skip dual-write, you are accepting a sharper cut. That is fine when the checklist and rehearsal are real.

Rehearsal (non-optional)

Run the full procedure on staging with production-sized data if possible.

Things that commonly fail in rehearsal:

Connection string typos / wrong secret version
ORM still pointing at old host via cached env
Long migrations that only show up on prod scale
Autoscale spawning instances with old config mid-cut
Cron jobs writing to the old primary after cut

Write down each failure and the fix. If rehearsal was “we clicked around,” you did not rehearse.

Cutover sequence (owners in parentheses)

Freeze risky jobs (scheduler owner)
Final sync / lag check (DBA) — lag under agreed threshold
Stop writes or enter read-only if required (app owner)
Promote / switch endpoint (DBA / platform)
App config flip (deploy pool size + host) (app owner)
Smoke: login, pay path, admin read (app owner)
Data diff sample (DBA)
Unfreeze jobs (scheduler owner)
Watch window 30–120 minutes with named watcher

Smoke queries / checks

App health and readiness (see health-check article)
One write + read-your-write on a canary tenant
Error rate and lag graphs

Rollback triggers (write these before you start)

Abort and roll back if any hold:

[ ] Lag exceeds threshold after T minutes
[ ] Error rate on critical route > X% for Y minutes
[ ] Data diff on checksum tables beyond agreed tolerance
[ ] Cannot complete smoke write/read in Z minutes
[ ] On-call cannot reach a required owner

Rollback is not shameful. Undefined rollback is.

Rollback sketch

Re-point app to old primary (config/flag)
Confirm writes land on old primary
Preserve new primary for forensics; do not immediately destroy
Schedule postmortem

Watch window

Minimum signals:

DB CPU, connections, lag
App 5xx, p95 on write routes
Queue depth for async workers
Disk growth if triggers/backfills run

Name a human who can call abort without a committee.

Summary checklist card

Phase	Done?
Compatibility + grants + budgets
Verification query baselines saved
Rehearsal completed with notes
Owners assigned per step
Rollback triggers written
Smoke + diff + watch window

Cutover is project management with SQL. The checklist is the product; the vendor console is just a tool.