Connection pool math that keeps databases alive under load

Database connection pools fail in a boring way: every application instance opens “just 25 connections,” traffic spikes, and the database hits max_connections while application threads wait on checkout. The outage looks like random timeouts. The math is usually sitting in plain sight.

This article is a working budget for pools: a small simulator in the repo, a formula you can defend in a review, and the metrics that tell you the budget is wrong before users do.

The failure mode in one picture

Suppose you run:

Knob	Example value
App instances (pods / VMs)	12
Pool size per instance	25
Database max_connections	150
Reserved for admin / migrations / BI	10

Maximum concurrent app connections if every pool fills:

\[ 12 \times 25 = 300 \]

Usable connections for apps:

\[ 150 - 10 = 140 \]

300 > 140. The configuration is unsafe even if average traffic is fine. The first serious fan-out—retries, a slow query, a deploy that doubles replicas—turns idle headroom into connection refused / “too many connections” / pool wait timeouts.

You do not need a dramatic story for this to be real. The misconfiguration is the story.

Run the lab

node content/labs/connection-pool-math/simulate.mjs
node content/labs/connection-pool-math/simulate.mjs --instances 20 --pool 30 --max 200 --reserved 15

Default output flags the 12 × 25 vs 150 case as UNSAFE and prints a starting pool size:

pool size ≈ floor(usable / instances)

That is an upper bound for capacity, not a guarantee of low latency. It only answers: “Can the fleet physically open more sockets than the database will accept?”

A budget you can defend

1. Cap total demand

\[ \text{instances} \times \text{pool\_size} \le \text{max\_connections} - \text{reserved} \]

reserved is not optional. Humans, migrations, logical replication, and emergency sessions need room when the app is already sick.

2. Size for concurrency, not marketing RPS

Little’s law for connection checkouts:

\[ \text{concurrency} \approx \text{arrivals\_per\_sec} \times \text{avg\_hold\_seconds} \]

If each instance handles 80 RPS and a connection is held 40 ms on average:

\[ 12 \times 80 \times 0.040 = 38.4 \]

You only need on the order of 40 concurrent checkouts fleet-wide for that average—not 300. Pools still need headroom for bursts and slow queries, but 25 × 12 is not “because Postgres likes 25.”

3. Prefer smaller pools + queueing visibility

A smaller pool with metrics beats a large pool that silently starves the database:

• pool active / idle / waiting
• checkout wait time (p50 / p95)
• DB Threads_connected / numbackends
• application 5xx and latency

If checkout wait climbs while DB CPU is idle, you may be under-pooled or stuck on locks. If Threads_connected rides the max_connections ceiling, you are over-pooled or leaking connections.

Symptoms when the math is wrong

Signal	Over-pooled (DB exhausted)	Under-pooled (app wait)
DB logs	too many connections	quiet
App errors	connect failures, retries	pool timeout / wait queue
Latency	mixed; often cascade	checkout wait dominates
After scaling pods	gets worse	may improve until DB caps

The nasty case is autoscaling: each new instance multiplies pool_size. Horizontal scale without a global connection budget is how a healthy database becomes the bottleneck.

Worked example from the simulator defaults

instances          : 12
pool size          : 25
demand             : 300
max_connections    : 150
reserved           : 10
usable             : 140
headroom           : -160  → UNSAFE
safe pool start    : floor(140/12) = 11

A practical rollout sequence:

1. Set pool size to 11 (or lower) on a canary.
2. Watch checkout wait and DB connections under peak.
3. If wait is high and DB still has headroom, raise carefully (12 → 15), never straight to 25 “because the library default said so.”
4. Cap HPA/replica count so max_replicas × pool_size still fits the budget.

For PgBouncer / ProxySQL style pools, the same inequality applies to server-side connections; client-side pools must not assume they each deserve a private set of server backends.

What to put in config reviews

Paste a table, not a vibe:

Item	Value
max_connections	150
reserved	10
usable	140
max app instances	12
pool size	11
worst-case demand	132
headroom	8

If someone wants more instances, they must change the table: lower pool size, raise max_connections with memory analysis, or introduce a pooler that multiplexes.

Monitoring that proves the budget

Minimum dashboards:

1. DB connections used vs max (alert before the ceiling, not at it).
2. Pool wait time p95 (alert when user latency is mostly waiting for a connection).
3. Replica count × configured pool size (a derived “theoretical demand” metric).

A useful synthetic check after deploy:

node content/labs/connection-pool-math/simulate.mjs \
  --instances "$CURRENT_REPLICAS" \
  --pool "$POOL_SIZE" \
  --max "$DB_MAX_CONNECTIONS" \
  --reserved 10

Fail CI or the deploy note if the script prints UNSAFE.

Summary

• Fleet demand is instances × pool_size, not “whatever one box needs.”
• Stay under max_connections - reserved even at max replicas.
• Average concurrency is closer to RPS × hold_time; giant pools often only help during pile-ups—and pile-ups are when you kill the database.
• Measure checkout wait and DB connection count; adjust with numbers, not library defaults.

The simulator will not replace a load test. It will stop the most common foot-gun: shipping a pool size that cannot possibly fit the cluster you already run.