Scaling Evolution Agent Skill

Scaling Evolution

Grow a design one bottleneck at a time. A system that serves 1k users and one

that serves 10M users are different architectures, but you do not jump between

them — you walk a path where each step removes the *current* ceiling and exposes

the next. Getting this wrong means either over-building day one (paying multi-

region complexity for 1k users) or freezing when load doubles because the design

was a memorized end-state, not a sequence of justified moves (GUIDE #7).

When to reach for this

A load increase is on the table ("what if traffic 10×?", "scale to millions"),

the user asks where the bottleneck is or what breaks first, or a single-box

design has outgrown one machine. Reach here to sequence the next two or three

moves — never the whole roadmap at once.

When NOT to

Do not pre-build steps the numbers do not yet demand (YAGNI). Sharding,

multi-region, and a message queue are *late* moves; proposing them for a system

that fits on two boxes is the over-indexing this skill defends against. If the

current load fits comfortably on a vertically-scaled box with a replica, stop —

that is the cheapest design that meets the constraint, and it wins. Naming the

next five tiers when only one is needed is a red flag, not foresight.

Clarify first

The path is driven entirely by numbers and constraints, so pin these down before

moving (most come from requirements-scoping and back-of-the-envelope):

• Current and target scale — today's QPS/data and the multiple you must hit

(2×? 100×?). The multiple decides how many steps you take now.

• Read:write ratio — read-heavy systems scale with replicas + cache; write-

heavy systems hit the master/storage ceiling and need sharding far sooner.

• Where it hurts now — is the *symptom* compute (CPU saturated), storage

(DB/disk saturated), or network (bandwidth/connections)? Diagnose before adding.

• Consistency and staleness budget — replicas and multi-region trade freshness

for scale; if reads must be current, that constrains the path (→ consistency-coordination).

• State — is anything pinned to a server (sessions, local files)? Stateful

tiers block horizontal scaling.

The method: walk the bottleneck ladder

Each rung removes one ceiling. Apply the next rung the numbers justify, not

the whole ladder. Full triggers and worked thresholds are in

references/scaling-ladder.md.

• Single server. Web, app, DB, cache on one box. Correct for low traffic and

early validation. *Breaks when* one machine can't hold the load or the data.

• Split the tiers. Move the database (and later cache) onto its own host so

web and data scale independently. *Breaks when* the single web box or single DB

saturates, or either becomes a single point of failure.

• Vertical scale + replicate for reads. First scale *up* (bigger box — simple,

no app changes) until the hard limit or cost knee. Add read replicas to spread

reads off the primary (→ data-storage owns replication). *Breaks when* writes

saturate the primary, or replica lag breaks freshness.

• Add a cache. Put hot reads in front of the DB once a number shows reads

dominate (→ caching). Highest-leverage move for read-heavy systems. *Breaks

when* writes are the bottleneck, or the working set no longer fits.

• Push static/edge to a CDN. Offload images/JS/CSS/video to edge servers

close to users (→ content-delivery). *Breaks when* the bottleneck is dynamic

requests, not static assets.

• Make the web tier stateless + load-balance. Move session/state to a shared

store so any request hits any server; put a load balancer in front and

autoscale the fleet (→ load-balancing). This is the unlock for cheap

horizontal scale. *Breaks when* the data tier (now the bottleneck) can't keep up.

• Decouple with async. Move slow/bursty work (uploads, encoding, fan-out)

behind a queue so producers and consumers scale independently and spikes are

absorbed (→ messaging-streaming). *Breaks when* even the sync path or storage

is the limit.

• Multi-DC / multi-region. Geo-route users to the nearest healthy data center

for latency and disaster survival; replicate across regions. *Breaks when*

cross-region data sync, conflict resolution, or a single dataset too big for one

region forces the last rung.

• Shard the data tier. Partition data across nodes when one primary can no

longer hold the writes/data (→ data-storage owns sharding/partitioning;

consistency-coordination owns consistent hashing). The most complex move —

last, not first.

Vertical scaling (scale *up*: more CPU/RAM) is the cheap early move with no app

changes but a hard ceiling and no redundancy. Horizontal scaling (scale *out*:

more boxes) is the durable answer — better availability, near-unlimited headroom —

but demands statelessness and adds coordination cost. Climb vertically until the

knee, then go horizontal.

Diagnose the bottleneck before adding anything

"Add more servers" without a diagnosis is guessing (GUIDE #3, #7). Classify the

pressure first, then act on *that* resource:

• Compute-bound (CPU/threads pegged, latency rises with request rate): add app

servers / autoscale; check for an O(n) hot path before buying hardware.

• Storage-bound (DB CPU/IO pegged, slow queries, replica lag growing): add

read replicas, cache, then shard. Adding web servers here makes it *worse* —

more connections onto an already-saturated DB.

• Network-bound (bandwidth saturated, connection limits, cross-region RTT):

CDN for egress, compression, connection pooling, keep chatty traffic in one DC.

The classic failure is sharding the database when the bottleneck is compute — a

new failure mode added to fix the wrong layer. Read the symptom, name the

resource, then pick the rung.

Treat each rung as a *hypothesis*, not a destination: "this design holds until

writes exceed X / a region is lost / the working set outgrows RAM." Saying the

breaking point out loud is the move that distinguishes reasoning from defending a

diagram. When a constraint changes (load doubles, latency target tightens, a DC

is lost), revisit the assumption and climb — calmly, not by patching the old

shape onto a problem it no longer fits.

Dos and don'ts

Distilled from the ladder, the diagnosis step, and where the technique misleads.

Do

• Diagnose the resource before adding capacity — name compute vs storage vs

network, then act on *that* tier. "Add servers" without a symptom is guessing.

• Apply only the next rung the numbers justify, and state its breaking point

out loud ("holds until writes exceed X / a region is lost").

• Climb vertically until the knee, then go horizontal — bigger box first

(no app changes), scale-out once the ceiling or cost knee is hit.

• Make the web tier stateless before scaling it out — move sessions to a

shared store so autoscaling can't drop them.

• Pair every rung with the failure mode it adds (cache stampede, replica

promotion, region failover, hot shard) → resilience-failure.

Don't

• Don't treat the ladder as a checklist — it is a menu; most systems live

happily at rung 4–6 forever and never shard.

• Don't skip rungs to the "impressive" answer — jumping to sharding or

multi-region signals a memorized diagram, not a crossed ceiling.

• Don't scale the layer that isn't the bottleneck — more web servers onto a

saturated DB just adds connections and amplifies the load downstream.

• Don't route read-your-writes to a lagging replica — replica lag reads like

data loss; pin those reads to the primary (→ consistency-coordination).

• Don't build the end-state on day one — multi-region for 1k users is cost

and complexity with no payoff. Match the rung to today's number.

Numbers that matter

Numbers decide *which* rung is next; don't restate the tables — read

back-of-the-envelope. The ceilings that trigger a climb: a single RDBMS node

handles roughly 1k QPS, a key-value node ~10k, a cache node

~100k–1M; one ~64-core box is ~64k req/s of pure compute before IO. When

an estimate crosses one of these, that crossing *is* the next bottleneck. Peak is

typically ~2× average, so size to peak. A useful framing: the rung you need is

roughly set by the *order of magnitude* of target QPS and dataset size — 1k QPS

fits one box, ~10k wants replicas and a cache, ~100k+ forces a stateless

horizontal tier, and a dataset past one node's RAM forces sharding. Each extra

"nine" of availability costs a disproportionate jump in redundancy (replicas →

multi-AZ → multi-region) — tie the target to the requirement, not ambition

(→ back-of-the-envelope, resilience-failure).

Diagram

To visualize the current architecture and the one-rung-ahead version side by

side (so the breaking point and the next move are explicit), use the in-plugin

architecture-diagram skill. Sketch only the rung you're on plus the next one —

not the whole ladder.

Related building blocks

• back-of-the-envelope — *depends on* it for the ceilings; a number crossing one names the next bottleneck.
• data-storage — *owned-concept lives there*: replication and sharding/partitioning, the heaviest rungs.
• caching — *pairs with* this as the cache rung; owns what to cache and how it fails under load.
• load-balancing — *feeds into* the stateless horizontal tier; sits at its front.
• content-delivery — *pairs with* this as the CDN/edge rung for static and geo-distributed content.
• resilience-failure — *pairs with* every rung; each one adds a failure mode to degrade gracefully.
• consistency-coordination — *owned-concept lives there*: the freshness/coordination cost of replicas, multi-region, and shards.
• system-design — *orchestrated by* it; this skill runs at its "scale the design" step.

References

• references/scaling-ladder.md — the full rung-by-rung ladder with concrete

trigger thresholds, the compute/storage/network diagnosis checklist, the

vertical-vs-horizontal decision, and the multi-region sync gotchas. Read when

sequencing the next moves or diagnosing what breaks first.