Write Tests Agent Skill

Write Tests

A good test fails only when real behavior breaks, and passes through every

refactor or config change that preserves it. Most bad tests fail the opposite

way: red on harmless changes, green while the real path is broken. Every rule

below serves that one goal.

Workflow: tracer bullets, not a batch

• Write ONE test at a time. Assert first, watch it go red on the un-fixed

code, make the code earn green, learn, then write the next. Never a batch up

front: a batch written against *imagined* behavior pins what you guessed —

those tests pass when the mechanism breaks and fail when it's fine. Each

green cycle tells you what the next test should actually assert.

• Iterate on the fastest focused runner (one file, one test name), and run

the full suite only as a final gate before handing off. Check the exit code,

not just the output — a runner that prints nothing and a green run look the

same. On failures, read EVERY red test before fixing one; they often share a

root cause.

• Before calling it done, prove the test can fail (below) and walk the

review checklist.

What to assert

• Observable behavior through the outermost practical entry point — return

values, exit codes, persisted rows, HTTP responses, rendered output — never

which internal functions ran or how a value is computed. A test on the public

surface survives a rewrite of everything underneath; a test that reaches into

internals breaks on every refactor and pins implementation, not behavior.

Reserve isolated unit tests for genuinely tricky pure logic (parsers,

schedulers, state machines).

• Nothing the compiler already guarantees. A test that re-asserts a type

signature — field shapes, rejected argument types — can only fail if the

compiler failed first. Spend the budget on business rules, arithmetic,

branching, ordering, edge cases, side effects.

• Actual values, not collection sizes. For dedup/normalize/idempotency

paths, length == 1 passes even when normalization is broken; also assert

the stored value equals the expected canonical form.

• The smallest scale that can show the behavior. Two entities and one

mechanism before crowds and integration; small tests fail fast with readable

state and don't entangle five behaviors in one assert.

• The design contract, not current behavior. When a test goes red, the

reflex is to re-measure and pin the new number — resist it: a bar calibrated

to whatever the code currently does silently encodes bugs as baseline. Write

the assert from the stated contract and make the code earn it; if no contract

exists, that's a question for the owner, not a number to measure-and-pin.

Recalibrating is legitimate only when the contract itself changed.

• Deterministic claims tight, stochastic claims as distributions. An

invariant that holds every run gets an exact threshold. Anything noisy or

tuning-dependent ("A usually beats B", "load balances evenly") must be

asserted over a set of runs/seeds as a band — a single-sample pin on a

stochastic outcome is not a weak test, it is a blind one: it certifies

whatever the lucky sample did and can mask a systematic bias for months. If

you must assert a noisy differential, widen the margin and name it

chaos-marginal in a comment.

Seams and mocks

• Don't couple tests to config — mock the seam. A test keyed to a live

config value (which flag is on, which tier is default) breaks when someone

legitimately edits config. The fix is never skipIf/conditionals — a skipped

assertion hides the coupling and stops covering the path. Feed the function

fixed inputs, or stub the lookup so fixture ids resolve to fixed values.

Litmus: "would this break if a config value changed with no logic change?"

• Don't over-mock — every mock is a frozen assumption. Mock at the system's

edges (network, clock, filesystem, third-party SDK, config lookup), never

internal collaborators. Stubbing an internal hard-codes its current contract;

refactor it and the test lies. Never make "was called with" the primary

assertion when an observable outcome exists. Mocking three internal modules

to test one function means: test one layer up, where they're real.

• Harnesses must wire the system the way production does. If a bug only

showed up against a real environment, the harness skipped an input production

always sets — fix the harness, don't just fix the bug.

Control variables and probes

• One variable per comparison. Pin everything else — same seed, same

counts, same configuration on both arms; disable mechanisms that aren't the

subject. When a comparison test breaks, first ask whether an unrelated

mechanism leaked into the experiment before touching the code under test.

• Validate, don't assume. Never reason your way to a conclusion about

*cause* — which path fired, where the failures come from — and act on it.

Write a throwaway probe that reads public state and prints; the plausible

story is wrong often enough to burn a session, and one probe redirects the

whole effort. Probes are scratch: put them where they cannot be committed,

delete them the moment the question is answered, or promote them into a real

test if the behavior deserves a permanent guard.

Tests run on someone's live machine

A suite that grabs the desktop is a suite people stop running. Whatever the test

needs, it takes the least intrusive form of it:

• Never take focus. No activating the app, no makeKeyAndOrderFront, no

raising, no moving the pointer, no simulated clicks into the session, no audio

playback. A UI a person must not be interrupted by is still fully testable:

build the window, drive its model, and assert on the rendered view. Where a

screenshot is the evidence, capture the window's own image offscreen rather

than photographing the screen. Keep activation on the code path a person

triggers, and exercise it by calling the handler, not by launching the app

repeatedly.

• Write nowhere the person keeps their own state. Point every test at a

scratch home, a scratch preferences domain or suite, and scratch install

locations. A test must never modify the real config, library, login items or

installed applications. Production may read an override the tests set; it must

behave normally when that override is absent.

• Batch anything unavoidably visible. If a check genuinely needs a real

launch, do all of its observations in one run instead of one relaunch per

assertion, and say in the evidence that it was visible.

• Leave nothing running. Every process, window, server and temporary file the

test started is stopped and removed, including on failure.

Prove the test can fail

A test you never saw fail is decoration. For any regression test — especially

one written *after* the fix — falsify it once: revert or break the production

code the way the bug would, confirm red *for the expected reason*, restore,

confirm green. Verify the revert actually took: a stash or checkout with a

wrong pathspec reverts nothing, silently, and the "red" run quietly tests the

fixed code. The tell: the "red" numbers equal the green numbers.

When a test goes red after a change

Triage in order of likelihood before editing anything:

• The test encodes deleted semantics — it pinned an accident of the old

implementation. Re-spec it to the contract's real claim.

• The experiment lost control of a variable — a new mechanism leaks in.

Pin the variable.

• The margin was chaos-tight. Widen it, with a comment saying so.
• The code is actually wrong. Probe the state, find the mechanism.

Never tune constants to make one test pass without rerunning the neighbors:

coupled systems reshuffle. If two consecutive tweaks each break different

tests, stop poking — the control surface is wrong; find the mechanism.

Review checklist

Walk this on any test diff, apply fixes in the same pass, re-run the suite:

• Does an assertion restate a type guarantee? → delete it.
• Would it break on a config edit with no logic change, or is it

skipIf-conditional on config? → mock the seam.

• Are assertions mostly mocked-internal "called with"? → test a layer up

against observable output.

• Does it pin a single sample of a stochastic outcome? → assert a

distribution.

• Does it assert a count without the value on a dedup/normalize path? → add

the value assertion.

• Is the function under test still called in production? → if the only

callers are tests, delete the function and the test together.

• Can it fail for the right reason? → falsify once to confirm.
• Does it take focus, move the pointer, play sound, or write the person's real

config, library or applications? → drive the model and a scratch location

instead, and capture windows offscreen.