Purple Team

Purple Team

The combined red+blue loop — the outer orchestration the red-team skill says "lives outside it."

The artifact is a target system (frozen *within* a phase, patched *between* phases); the feedback signal

is how many new failure classes a fresh attack pass finds against the patched target. Each cycle

runs three strictly separated phases — find (red-team), fix (blue-team), re-verify (a fresh

red-team pass) — and you repeat until a fresh find stays dry (zero new classes), meaning the target

is hardened. Red and blue run as independent agents so the attacker that wrote a catalogue never

grades its own patch. On stop it opens a pull request with the cycle history and the patch set.

When to use

Use to harden a guardrail/classifier/filter/prompt/API the user owns or is authorized to test, when the

goal is an actually-hardened target plus a reviewable patch — not just a catalogue (that is red-team

alone) and not just closing a pre-existing catalogue (that is blue-team alone). It needs a runnable

oracle for the objective signal, and the two sibling skills installed.

Default: spawn red and blue as separate subagents per phase. Escape hatch: on hosts without subagent

dispatch, run each phase inline (serial) per the role files — still correct, but the same context plays

both sides, so be deliberate about not letting the fix bias the re-verify. Not for unauthorized targets.

Setup

Resolve bindings interactively. If loop.run.yaml exists, load it, confirm the values in one line, and

skip to the loop. Otherwise: on Claude Code (the AskUserQuestion tool is available) infer a likely

value per binding and recommend it; on other hosts ask each as a quoted prompt. Then write

loop.run.yaml (format: examples/run.example.yaml) and confirm before creating any other files.

The phases reuse the sibling skills, so the bindings are their union — one shared loop.run.yaml drives

both. The same file is <target_files> to blue (writable) and the program behind <target_cmd> to red

(read-only); the same path is red's <failures_log> and blue's <catalogue>.

| binding | meaning | default | how to infer |

|---|---|---|---|

| <target_cmd> | run the target on one stdin input → a verdict (what red attacks) | — | the guardrail/classifier/API entrypoint |

| <target_files> | the source file(s) blue may edit to fix the target | — | the file(s) behind <target_cmd> |

| <oracle_cmd> | ground-truth verdict for the same input (frozen) | — | a reference checker / policy impl |

| <gate_cmd> | functional tests that must stay green through a fix (exits 0) | — | the target's test command; else rely on <holdout> |

| <holdout> | benign + clearly-correct inputs blue must not break | <sandbox_root>/holdout.jsonl | known-good inputs the oracle agrees on |

| <catalogue> | shared failures file: red writes it, blue closes it, JSONL {id,text,class,...} | <sandbox_root>/failures.jsonl | — |

| <iter_strategy> | branches (commit per fix → PR) or snapshots | branches | dirty/non-git tree → snapshots |

| <pr_branch> | branch the fixes land on and the PR opens from | purple-team/<tag> | today's date as <tag> |

| <sandbox_root> | where catalogues, snapshots, ledgers live | ./sandbox | — |

| <cycle_budget> | max find→fix→re-verify cycles | 4 | — |

| <find_budget>, <fix_budget> | inner per-phase budgets passed to red / blue | 8 | — |

<skill_dir> is this skill's installed folder. The phases run the sibling skills' tools

(red-team/tools/harness.py, blue-team/tools/verify.py); the role files name them.

The loop

Copy this checklist and tick items off each cycle. Cycles are numbered from 0; per-cycle artifacts go in

<sandbox_root> with the cycle index in the name so nothing is overwritten: the find catalogue is

failures.cycle<N>.jsonl (cycle 0 may use <catalogue> directly) and the re-verify result is

failures.cycle<N>_reverify.jsonl. The **catalogue blue fixes in cycle N is the failures file from that

cycle's find/re-verify pass** — never append back into one shared file, so each cycle's accounting is clean.

• [ ] Cycle 0 — Find. Run the red-team phase against the frozen target (spawn-or-degrade,

roles/red-find.md), writing failures.cycle0.jsonl. Read back its distinct failure classes.

• [ ] If the catalogue is empty on cycle 0, stop — the target is already dry; report clean, no PR.
• [ ] Fix. Run the blue-team phase on this cycle's failures file (spawn-or-degrade,

roles/blue-fix.md): it patches <target_files> one class per iteration under the gate +

regression guard, committing kept fixes on <pr_branch>. Read back the classes it closed and

any residuals.

• [ ] Re-verify. Run a fresh red-team phase against the patched target into

failures.cycle<N>_reverify.jsonl. This confirms the closed classes no longer reproduce and

surfaces any new classes the fix introduced (e.g. an over-block).

• [ ] Append one cycle row to the ledger. If the re-verify pass found new classes, they become the

next cycle's catalogue (failures.cycle<N+1>.jsonl) — loop to Find/Fix for cycle N+1.

Repeat find→fix→re-verify until a re-verify pass stays dry (no new classes) or

<cycle_budget> is hit.

• [ ] Handoff. Open the pull request with the full cycle history (see

Handoff).

Phase independence (why three separated phases). The target is read-only ground truth for the

duration of a red pass, so patches happen only *between* passes — mutating it mid-pass would break

reproducibility and the class accounting. Spawn red and blue as separate subagents so neither grades

its own work; the orchestrator only passes artifacts (the catalogue, the closed/residual summary)

between them. Launch a phase's subagent and wait for its structured return before the next phase.

Ledger

<sandbox_root>/cycle_ledger.tsv, tab-separated, never commas in free text. Header

cycle found fixed residual regressions_introduced net_open:

cycle	found	fixed	residual	regressions_introduced	net_open
0	5	5	0	1	1
1	1	1	0	0	0

All counts are distinct classes, not inner iterations: found = classes red surfaced this cycle;

fixed = classes blue *closed* (a class blue attempted several times still counts once); residual =

classes blue could not close; regressions_introduced = new classes the re-verify pass found that the

fix caused; net_open = classes still open entering the next cycle (residual + regressions_introduced,

the next cycle's catalogue size). Convergence is a re-verify pass with net_open = 0. Report the cycle

at which the target went dry (or the best net_open reached).

Constraints

• Authorized targets only. This drives real attacks against the target; only run it on a system the

user owns or is explicitly authorized to test (inherited from red-team).

• Keep the phases independent and the ground truth frozen. Never let one phase edit the oracle,

<gate_cmd> tests, <holdout>, or either tool; never patch the target inside a red pass. These keep

the find/fix/re-verify accounting honest.

• Re-verify is a *fresh* attack, not a replay. It must be free to find new classes (including ones

the fix introduced), not just re-check the old list — concretely, at least half its candidates should

be new payloads and it should try at least two attack angles not in the prior catalogue (this matters

most in degrade mode, where the same context plays both sides). That is what makes the loop converge to dry

rather than to "the original five are gone."

• One change per inner iteration (handled by the sub-loops); the orchestrator changes nothing

itself except moving artifacts between phases.

• Stay inside the repo and <sandbox_root>; do not pause between phases to ask whether to continue —

run until dry or <cycle_budget>.

Handoff: the pull request

The deliverable is one pull request for the whole hardening run — the communication interface to the

target's owner, who keeps final approval. With the tree at blue's best state and the kept fixes already

committed on <pr_branch>:

• Open it with gh pr create; body = the cycle ledger (found → fixed → re-verified per cycle), one

reproducible example per closed class, and any residuals still open. Confirm once before opening —

it is outward-facing; never auto-push silently.

• Degrade, don't fail: with no remote / no gh, leave commits on <pr_branch> and write

git format-patch + PR_BODY.md into <sandbox_root>; in snapshots mode emit a unified diff +

PR_BODY.md. Tell the user the one command to open the PR themselves.

Roles

• roles/red-find.md — runs the red-team phase against the target and returns the catalogue + classes.
• roles/blue-fix.md — runs the blue-team phase on the catalogue and returns closed classes + residuals.

Both are spawn-or-degrade: spawn a real isolated subagent on Claude Code (the Agent/Task tool),

else adopt the role inline. Each delegates to the sibling skill (red-team / blue-team) bound to the

shared loop.run.yaml; if a sibling is not installed, ask the user to install it (`npx skills add

gaasher/agent-loop-skills`) rather than re-implementing it here.