Check Agent Skill
Run CIAgent regression checks after changing an AI agent's code, prompts, or knowledge base in a repo that has agentci_spec.yaml, and interpret the results. Use after editing agent logic, before committing agent changes, or when the user asks whether the agent still works.
Install
one command, takes just this skill from the repositorynpx skills add https://github.com/davepoon/buildwithclaude --skill checkWhat it tells the agent to use
found in the instruction textThe instruction itself
4 sections, as written by the authorRun CIAgent checks on this repo's agent
The repo has agentci_spec.yaml (if it does not, use the onboard skill
instead). Your job: run the right check for the change that was just made,
read the result correctly, and never paper over a failure.
Which command
| Situation | Command |
|---|---|
| Spec or wiring changed, or no API keys | ciagent test --mock |
| Agent code / prompt / retrieval changed | ciagent test --yes --format json |
| Result differs from last run, or flakiness suspected | ciagent test --runs 3 --yes |
| Knowledge base changed | ciagent generate-checks --dry-run, review, then apply |
| The LLM judge's verdicts look wrong | ciagent judge-audit |
Live runs (test without --mock, judge-audit, generate-checks) call model
APIs on the user's keys. Mock mode is free. If the user has not already
approved live runs in this session, prefer --mock or ask.
Reading results
Exit codes: 0 pass (including flaky-but-passing), 1 correctness failure
(with --runs N: failed in every run), 2 infra or config error — fix the
setup, not the agent.
With --format json: per-query entries carry layer results (correctness /
path / cost) and the answer text; with --runs N a top-level stability block
lists flipped queries with flip_source.
Flip sources route the work:
agent-variance— the agent's answer changed between runs → fix the agent
(prompt, retrieval, temperature).
judge-flake— same answer, the LLM judge changed its verdict → fix the
eval (tighten the rubric or replace with a deterministic check).
infra-error— a judge API call failed → retry; fix nothing.mixed— ambiguous; look at the answers yourself.
Rules
- A correctness failure means the agent lost a fact it used to state. Fix the
agent, or — only if the check itself is factually wrong — fix the check.
Never weaken or delete a correct check or baseline to make a run green;
report the failure to the user instead.
- After intentionally changing agent behavior, re-record the affected golden:
delete its baseline file and rerun
ciagent bootstrap --runner <runner> --queries <file> --yes for that query,
or update the spec's expectations — with the user's confirmation.
- Report results in one or two sentences: score, what failed and in which
layer, flip sources if any, and the command you ran.
Other skills for the same job
different authors, same section of the catalogue
This skill should be used when the user asks about "plugin settings", "store plugin configuration", "user-configurable plugin", ".local.md files", "plugin state files", "read YAML frontmatter", "per-project plugin settings", or wants to make plugin behavior configurable. Documents the .claude/plugin-name.local.md pattern for storing plugin-specific configuration with YAML frontmatter and markdown content.
-Automatically convert documentation websites, GitHub repositories, and PDFs into Claude AI skills in minutes.
This skill should be used when the user asks about "plugin settings", "store plugin configuration", "user-configurable plugin", ".local.md files", "plugin state files", "read YAML frontmatter", "per-project plugin settings", or wants to make plugin behavior configurable. Documents the .claude/plugin-name.local.md pattern for storing plugin-specific configuration with YAML frontmatter and markdown content.
Standardize how an AI-collaboration project turns work into reusable knowledge. Use when initializing or retrofitting Project Cairn in a project, recording progress after meaningful work, maintaining AGENTS/CLAUDE/cairn docs, auditing project knowledge for drift or missing records, pulling and citing external knowledge, or graduating validated project experience into a reusable knowledge base.
Persistent memory and context management for AI agents using OpenContext. Keep context across sessions/repos/dates, store conclusions, and provide document search workflows.
> PARA method knowledge management for Obsidian vaults. Use this skill whenever the user wants to organize notes using PARA (Projects, Areas, Resources, Archive), classify a note into a PARA category, route a note to the right vault folder, normalize frontmatter fields, run a PARA hygiene review, suggest archiving, audit vault structure, or process new knowledge inputs into an existing PARA-based vault. Also trigger when the user mentions inbox processing, vault cleanup, note classification, PARA review, or asks "where does this note belong?". Works with existing Obsidian skills (obsidian-markdown, obsidian-cli) — never replaces them.
Keeps implementation and specs in sync. Use when working on a feature that has a spec in .agents/specs/, when the user says /spec, or when starting implementation of a documented feature. Also use when the user asks to verify implementation against a spec or update a spec after changes.
Build and query AI-powered knowledge bases from claude-mem observations. Use when users want to create focused "brains" from their observation history, ask questions about past work patterns, or compile expertise on specific topics.
How to use it
Take davepoon/check from the repository into ~/.claude/skills for personal
use, or into .claude/skills inside a project.
The agent identifies a skill by the name field in its header. Two skills with the
same name cannot sit side by side — one of them will be ignored.