Feature flags as kill switches: ownership, defaults, and cleanup

Feature flags are often sold as experimentation. In production backend systems their highest value is a kill switch: turn off a dangerous path without rebuilding binaries under stress. That only works if evaluation order is correct, defaults are safe, and someone owns cleanup.

When a flag is justified

Use a flag when:

You need instant disable independent of deploy cadence
Rollout should be percentage or allowlist based
Two implementations must coexist briefly

Skip a flag when:

The change is a pure bugfix with no new path
You will not remove it within a sprint (it will become permanent architecture by accident)

Evaluation order (hard rule)

1) killSwitch / hard off
2) explicit allowlist (canary tenants)
3) percentage rollout
4) defaultEnabled

Never let a percentage rollout override a kill switch. The lab encodes that order.

node content/labs/feature-flags-kill-switches/demo.mjs

Sample behavior:

t_canary allowed even at low percentage
other tenants follow percentage hashing
after killSwitch=true, everyone is false—including canary

That last line is the point of a kill switch.

Safe defaults

Flag type	Default in prod	Why
New risky path	off	fail closed
Removing old path	old path on until verified	avoid forced new code
Kill switch	off (meaning “not killing”)	do not ship already-killed features silently

Document fail open vs fail closed when the flag service itself is down. Many outages come from “flag lookup timeout ⇒ enable new code.”

Operational controls

Who can toggle prod? break-glass role, not every engineer in the org chat
Audit: who flipped what, when, previous value
Change windows: percentage ramps with observation time
Dashboards: error rate by flag_on dimension when possible

If flags live only in a JSON file in the repo, you do not have a kill switch—you have another deploy.

Lifecycle and debt

Create with owner + removal date
Roll out allowlist → % → default on
Remove old code path
Delete flag config

Track open flags like production debt. A quarterly “flag inventory” that never deletes anything is theater.

Case study (lab narrative)

checkout_v2 rolls to 20% + canary tenant. Latency regresses. Operator sets killSwitch=true. All tenants immediately leave v2 without waiting for CI. Later the percentage is irrelevant until the switch is cleared and a fix ships.

Summary

Practice	Reason
Kill switch first in evaluation	Abort beats experiment
Safe defaults	Outages love fail-open flags
Auth + audit on toggles	Flags are production control plane
Delete flags	Prevent permanent branching

Flags are operational controls. Treat them with the same seriousness as deploy credentials.