Claude Skills
The open format is called Agent Skills and works in Claude Code, Codex, Cursor and other agents — most people know it as Claude Skills.
Every Agent Skill we could find on GitHub, deduplicated by content. 80 149 files from 1 774 authors, of which 62 489 are unique — the rest is the same skill repackaged into someone else's repository. For each one: what it weighs in tokens, whether it ships runnable scripts, and which MCP servers it needs.
Reviews tool configurations for correctness.
Reviews tool configurations for correctness.
An example skill for testing diff-skill evaluation.
| docs/wip plan OR a standalone task (bug fix, GitHub issue, one-off change). Provides structured execution with profile detection, dependency handling, review checkpoints.
An example skill for testing diff-skill evaluation.
| After implementing a new feature or fixing a bug, make sure to document the changes. Use when writing documentation, after finishing the implementation phase for a feature or a bug-fix.
| Compare and merge two design docs for the same feature into a single source of truth. Use when you have competing or complementary design/implementation docs (e.g. from separate design runs) that need reconciling into one unified document.
| Code review of current git changes with an expert senior-engineer lens. Detects SOLID violations, security risks, and proposes actionable improvements. Use when performing code reviews.
| Review design, implementation, and task documents produced by design. Evaluates document quality, internal consistency, and technical soundness. Use after design completes and before starting implement.
| Use after implementing tasks or mid-feature to verify code matches design docs and ensure they are in sync. Detects spec deviations, missing implementations, doc inconsistencies, and outdated docs in design and implementation documentation.
> Use when the user has a known, already-observed anomaly in their data — a metric spike or drop, an outlier, an unexpected number — and wants its root cause diagnosed, not guessed. Forms a slate of candidate causes, tests each against the data, and eliminates the ones the data refutes, narrowing the live candidates until exactly one survives refutation and passes a positive confirming test. The result is an investigation log with the confirmed root cause and the evidence that ruled out the alternatives. Not for open-ended discovery over a dataset with no specific anomaly in hand (that is data-analysis), and not for checking an external claim against sources (that is claim-verify) — this is reactive diagnosis of one anomaly you already know about.
> Use when the user wants an autonomous ML research loop that explores the space broadly rather than several wild, diverse swings (full rewrites, different architectures/regimes) early, then enters an adaptive phase that picks swing / merge / exploit per iteration — with a hard stagnation guard that bans further small-step exploits once they run too long, forcing a pivot back to a swing or merge. Tracks an approaches.md registry and a move_type per iteration; analyses every run before the next move. One change per iteration; loops forever until interrupted. Not for the standard analysis-first ml-autoresearch (which lets analysis alone choose each change), one-off training runs, or sweeps.
> Use when the user wants to evolve an ML model/program through population-based search rather than a single sequential refine loop — a generational evolution where parallel proposers each apply one small SEARCH/REPLACE diff to a parent, scored by a cascade-evaluated training run, and children are kept in a MAP-Elites archive across islands (with migration + checkpointing) so diverse high performers survive. A finite, bounded-parallelism re-creation of AlphaEvolve/OpenEvolve, bent for ML autoresearch. Runs to a fixed compute budget or until interrupted. Not for the sequential single-thread autoresearch loops (one change → measure → keep/revert), and not for verifying a known bug or external claim — this is parallel, diversity-preserving search over a program.
> Use when the user wants to generate and literature-vet a pool of novel, testable research hypotheses LiteratureScout grounds each in real retrieved literature (already known? closest prior work? what gap does it fill?), and a Judge scores them against a fixed rubric and keeps the strong, non-duplicate ones; rounds repeat — mutating toward the open gaps — until fresh rounds stop adding keepers. Not for sharpening or decomposing a research question (no grounding/scoring there), and not for grading an existing written proposal against the literature.
> Use when the user wants an iterative, self-checking exploratory analysis of a dataset — surfacing findings that are each verified by re-running the computation, not asserted. Proposes one specific hypothesis at a time, writes and runs analysis code to test it, and records the finding only if the numbers support it at a meaningful effect size; loops until no new verified finding appears or the budget is hit. The result is a findings report where every claim is backed by a reproducible number. Not for diagnosing a single known anomaly or pipeline failure, and not for verifying an external claim against sources (that is a claim-verification task) — this is open-ended discovery over a bound dataset.
> Use when the user wants two approaches raced head-to-head on a single shared metric — e.g. a classical/algorithmic lane vs an ML/learned lane, or any two strategies for the same task. Each lane runs its own analysis-first research loop confined to its lane, the lanes share a scoreboard and may borrow ideas across the boundary without abandoning their identity, and a shared eval keeps the head-to-head honest; loops until interrupted, reporting the current leader. Not for improving a single approach in isolation (use a single-track research loop), and not for picking between two finished artifacts in one shot (that is a one-time comparison).
> Use when the user has a results draft or a set of data-backed claims and wants each one adversarially verified against the underlying dataset before publishing — a pre-publication red-team of the findings. Extracts the discrete checkable claims from the draft, reproduces each claim's number against the data, stress-tests it against the threats most likely to kill it (outliers, confounds, Simpson's reversals, tiny subgroups, alternative specifications), and marks it verified, fragile, or refuted; fragile and refuted claims are revised — hedged, scoped, or retracted — until every claim is verified or appropriately qualified. The result is a draft where every surviving claim has been reproduced and survived a stress test. Not for open-ended discovery of new findings over a dataset (that is a data-analysis task), and not for diagnosing a single known anomaly or pipeline failure — this is a gate over an existing draft.
> Use when the user has concrete failing cases in code or a guardrail/classifier/filter/prompt/API they own — a red-team failure catalogue OR a CI/CD test-failure report (failing pytest/JUnit tests) — and wants the target patched until those failures are closed without breaking what already works. It points straight at the failed cases (normalize any source with tools/ingest.py), fixes one root-cause class per iteration, and re-checks with tools/verify.py — oracle mode against a red-team oracle, or tests mode against the test suite — keeping a patch only if it closes a class while nothing that passed before regresses, else reverting; loops until every class is closed (dry) or the budget runs out, then opens a pull request with the patch set. The defensive fixer half of a find→fix setup. Not for discovering new failures (that is red-team), and not for editing the oracle, tests, or holdout that define ground truth.
> Use when a loop needs scholarly literature — paper discovery, novelty checks, full-text snippet search, citation-graph traversal, single-paper reads, or experimental-result extraction. A shared, stdlib-only CLI (`tools/lit_search.py`) over Semantic Scholar + arXiv (plus optional OpenAlex, of them call it instead of vendoring their own copy. Degrades to the caller's WebSearch/WebFetch.
> Use when the user wants a structured, saturating literature survey on a question — not a one-shot summary, but an evidence/contradiction matrix (sources × claims) built by iterative search until coverage stops growing. Each round expands the search (new sub-topic queries plus citation-graph walks), admits new sources, extracts their claims with verbatim snippets, and records where every source stands on each claim (supports / contradicts / qualifies); the feedback signal is how many new matrix-changing sources a round adds, and it stops at saturation, patience, or budget. The output is the matrix plus a synthesis of consensus, disputes, and gaps, every cell backed by a real citation. Not for grading a written proposal against the literature (that is a proposal-evaluation task) and not for generating new hypotheses — this maps what the literature already says.
> training code, runs it, and keeps changes that lower a single scalar metric (e.g. val_bpb). One agent proposes one change at a time, runs training in the user's env, keeps it only if the metric improves (advancing a git branch) else reverts, and loops forever until the human interrupts. A faithful adaptation of Karpathy's autoresearch. Not for the analysis-first variant that profiles before editing (that is ml-autoresearch), and not for a budgeted, plateau-stopping refactor.
> Use when the user wants an autonomous ML research loop that does more than blindly try changes. After every training run the agent analyses what actually happened inside the model — gradients, activations, embeddings, errors, data — and grounds the next change in that evidence. A `<literature>` searches papers, grades the evidence, and implements only what prior work supports. One change per run; loops forever until interrupted. Not for one-off training runs or hyperparameter sweeps.
> Use when the user has a coding or engineering prompt and wants it refined into a detailed, executable plan before any code is written — the planning stage of a prompt → plan → execute → debug pipeline. It decomposes the prompt from first principles (objective, end state, environment, building blocks, tools, packages), breaks the work into PR-sized tasks each tied to a component with its files, tests, and dependencies, orders them topologically, splits each into atomic subtasks, then a separate principal-engineer agent critiques the plan for alignment, coverage, sizing, and executability; it revises until the critique passes, emitting a plan.md layout and a structured tasks.json that a junior engineer or a smaller model can execute correctly. Not for executing, scaffolding, or debugging the plan (those are downstream loops), and not for research proposals or experiment plans.
> Use when the user wants to iteratively improve an artifact under a hard correctness bound while minimizing a measured cost — refactoring a code module to cut complexity while its test suite stays green, OR speeding up a SQL query while it returns the same rows. Each iteration applies one focused change, checks a correctness gate that must pass, measures a metric that must drop, and keeps the change only if both hold, else reverts; loops to a plateau or budget. Not for adding features, fixing bugs, or any change that is allowed to alter behaviour or results.
> Use when the user is planning a two-arm comparison (an A/B test, a simple RCT, a behavioral study, or a two-model/two-config evaluation) and needs to size it and preregister it before collecting data — finding the per-group sample size that hits target statistical power for the smallest effect worth detecting, auditing the design against a validity checklist, and locking it in a preregistration. Only for a single two-arm comparison with one primary outcome. Not for factorial, repeated-measures, clustered/multilevel, time-series, adaptive, or survival designs; not for analyzing data already collected; not for choosing the outcome or manipulation from domain knowledge.
> Use when the user has a prompt that feeds a system they can already score, and wants that prompt automatically improved to raise the score against their own evaluation command. Makes one targeted quality edit per iteration — clarity, context, specificity, structure, examples, decomposition, guardrails — re-runs the user's eval to measure the metric, and keeps the edit only if the metric improves, else reverts; loops to a target, plateau, or budget. The metric is whatever the user's eval command prints (task accuracy, an LLM-judge score, a pass rate, a tool-call success rate); the loop is metric-agnostic and never edits the eval. Not for writing a prompt from scratch, not for tuning model weights or hyperparameters, and not for one-off manual prompt edits without a score.
> Use when the user wants to automatically harden a guardrail, classifier, content filter, prompt, or API they own by running attack and defense together as a closed loop, not just one or the other. It classes against the frozen target, blue patches the target to close them under a regression gate, then a fresh red pass re-verifies — confirming each class is closed and surfacing any new ones the fix introduced. The find→fix→re-verify cycle repeats until a fresh attack pass stays dry (the target is hardened) or a cycle budget is hit, then it opens a pull request with the patch set. Not for attacking a system the user is not authorized to test, and not for a one-shot scan — use red-team alone to only find, or blue-team alone to only fix.
> Use when the user has a research proposal (problem + proposed methodology + planned experiments) and wants it iteratively strengthened until it clears a passing grade. ScholarEval grades the proposal against the literature (Soundness + Contribution), a Judge scores that feedback 0-100 on a fixed rubric, and a Reviser rewrites the proposal to fix the worst points without diluting the research question; loops until the grade passes or the budget is hit. Not for generating a proposal from scratch, and not for running a literature survey on its own.
> Use when the user wants to adversarially stress-test a guardrail, classifier, prompt, or API they own or are authorized to test, to surface the distinct ways it fails. Generates adversarial inputs, runs them through the target and a ground-truth oracle, logs every target-vs-oracle disagreement as a failure de-duplicated by technique class, and loops until rounds stop surfacing new classes. Produces a catalogue of distinct, reproducible failures — the attacker half of a find→fix setup. Not for patching the target, and not for attacking systems the user does not own or have permission to test.
> Use when the user has a scientific draft (with its dataset, figures, and optional analysis code) and wants it iteratively revised until it clears a quality bar. Five specialist judges (figures, scientific content, style, formatting, code) critique the draft; a fresh, independent peer_reviewer grades it on those same axes (1-5 each → a percentage) with honesty guardrails; a scientific_writer revises prose, figures, and code — regenerating figures by running the user's plot command — until the peer-review score clears the threshold or the budget is hit. Not for writing a paper from a blank page, and not for a standalone literature survey.
> Use when the user has a coding prompt — a feature, bug fix, or refactor — and wants it implemented end to end by a self-checking software loop, not a single pass. It refines the prompt into an subagent implements each PR-sized task editing only source, a separate QA subagent authors the tests that prove the task's acceptance criteria (or grades quality only when tests already cover them), runs the task's tests plus the full accumulated regression suite, and grades the code against a strict conciseness/readability/style-match rubric; the Engineer and QA loop on each task until the tests pass and the quality gate holds, then it commits and opens a pull request for the task — stacked PRs (one per task, each based on the prior) by default, or one combined PR — and moves to the next task. Author and critic have disjoint write scopes (Engineer owns source, QA owns tests) so neither can game the gate. Not for producing a plan without building it (that is plan-loop), and not for tuning a metric on an existing artifact under a fixed correctness bound (that is optimize-loop).
> Use when the user has a vague topic or area of interest and wants it sharpened into a few strong, novel, feasible research questions. Drafts candidate questions, scores each against a fixed rubric (Specific, Answerable, Novel, Feasible, Significant) with a light literature/web novelty check, and revises the weakest axis until enough questions clear the bar. Not for grading a full written proposal (use the research-proposal loop), and not for turning a question into testable predictions (use the hypothesis-generation loop).
> Use when the user has scientific data (or a prompt alluding to scientific data) and wants a publication-quality figure made from it. A generator drafts and renders a figure that lands a frozen communication goal; an adversarial critic critiques it hard and grades it 1-5 per axis against a fixed rubric (message, aesthetic, clarity, integrity, and a conditional domain-completeness axis), aggregates to 0-100, and decides pass; the generator revises against the critic's findings until the grade clears a threshold or the budget is hit. Both roles may consult the literature (Semantic Scholar + arXiv) to verify domain content (e.g. a pathway figure's gene set, or a benchmark's reported numbers), and conform to / grade against a named journal's figure spec fetched via web search. Not for writing a paper or analyzing a dataset, and not for editing an existing finished image — this renders a figure from data/brief and iterates on it.
> Use when the user has a messy tabular data dump (CSV/TSV/parquet/Excel/JSON) and wants it iteratively cleaned to an inferred data contract — a checklist of deterministic pass/fail checks, not a quality score. A single agent profiles the table, synthesizes a per-column contract compiled into binary checks (types, nulls, duplicates, inconsistent categories, format/range violations, outliers), then applies one targeted transform at a time, keeping it only if it reduces its target check's violations with no regression and no guardrail breach. Stops deterministically when every check passes, every remaining check is an unfixable residual, or a budget is hit; emits a replayable pipeline and an auditable ledger. Not for open-ended analysis of an already-clean dataset, diagnosing one known anomaly, or verifying a claim against sources — those are analytical loops; this rewrites the data to a contract.
> Use when the user wants an autonomous ML research loop that pressure-tests competing ideas before spending compute — several research subagents each propose one architecture change, a self-calibrating Judge critiques them against a rubric, the proposers refine, and the Judge picks the single change to run. The Judge learns to pick better over time by scoring its own predictions against realized metric deltas, recording predicted-vs-realized in a calibration ledger and refining its working rubric. The result is an experiment ledger where each iteration's change won a de-biased tournament. Not for running a single pre-decided experiment, and not for analysis-only exploration — for one hypothesis proposed and run per iteration without competition, use the sibling ml-autoresearch loop.
Convert markdown documents and chat summaries into formatted EPUB ebook files that can be read on any device or uploaded to Kindle.
Analyze LLM experiment results. Handles single or comparative experiments, exploratory or Q&A modes. Use when user says "analyze experiment", "compare experiments", "analyze against baseline", or provides one or two experiment IDs for analysis.
Datadog skills for AI agents. Essential monitoring, logging, tracing and observability.
End-to-end Agent Observability pipeline for an instrumented ml_app — classify production traces, root-cause failures, bootstrap evaluators, then (optionally) sample + publish a dataset, generate + run an experiment, and analyze results. Six narrated phases with a standardized banner and a "continue" checkpoint between each. Pure orchestration over the agent-observability sub-skills (`agent-observability-session-classify`, `agent-observability-trace-rca`, `agent-observability-eval-bootstrap`, `agent-observability-experiment-py-bootstrap`, `agent-observability-experiment-analyzer`). Use when user says "run the eval pipeline", "go from traces to evals", "bootstrap evals end to end", "classify then RCA then bootstrap", "build an eval set from scratch", "onboard me to datasets and experiments", "walk me through experiments", "I have an ml_app, now what", "Agent Observability onboarding", "guided experiment setup", "from traces to experiments", or wants a deterministic, narrated tour from production data through evaluators, datasets, and experiments. Stop early with `--stop-after <phase>` to short-circuit at evaluators or dataset, or resume mid-flow with `--start-at <phase>`.
>- Use when a developer wants to iterate on ONE specific Agent Observability / LLM Obs trace whose output they didn't like — re-running that trace against their LOCAL code, seeing a concise diff of the old vs new output, and looping (change code → replay → diff) until satisfied. Invoked as a trace"; "this trace's output is wrong, fix it and re-run"; "re-run trace <id> with <change>"; pasting a trace id from the Agent Observability UI with a description of what to fix. It fetches the trace via the datadog-llmo MCP or the pup CLI, edits code, re-runs the app to emit a NEW trace, and diffs the two — no local server, no browser. For agents traced with ddtrace / LLM Obs (Python first-class), with JSON-serializable entry agent-observability-replay-experiment), building an experiment from a dataset/CSV, writing evaluators, root-causing failed traces, or RUM/HTTP session replay.
Generates a self-contained Python experiment client that uses the ddtrace.llmobs SDK. Emits either a runnable .py script or a Jupyter .ipynb notebook matching the canonical DataDog reference notebook style. Use when the user says "generate Python experiment", "write an SDK experiment", "create a ddtrace experiment", "Python notebook experiment", "use the Agent Observability SDK", or has `ddtrace` installed and wants idiomatic SDK code.
> Classify whether user intent was satisfied in a Datadog Agent Observability trace or session. (2) trace_id — classify a single Agent Observability trace without RUM; (3) ml_app — sample and classify multiple sessions or traces from a given LLM app. Output is compact by default (verdict + one-sentence reason). Use when evaluating satisfaction, classifying sessions/traces, labeling data, or generating signal for agent-observability-eval-pipeline or agent-observability-trace-rca.
>- Run an iterative code-improvement hill-climb against real Datadog LLM-Obs data, locally, with Claude Code as the agent. Establishes a baseline eval, makes one focused change, re-scores with the same harness, keeps the change if it improves the score in the goal's direction (labeling within-noise gains tentative), and repeats. Use when the user says "run an auto experiment", "hill-climb this code", "iteratively improve X and measure the delta", "optimize this prompt/file against my traces", "auto-optimize against LLM-Obs", or wants the local equivalent of the auto_experiments worker. Works from a local dataset file, an ml_app, a dataset_id, or a list of trace_ids.
Bootstrap evaluators from production traces — by default propose online LLM-judge evaluators and, after you confirm, create them in Datadog as disabled drafts (never auto-enabled); on request emit Python SDK code or a framework-agnostic JSON spec instead. Use when user says "bootstrap evaluators", "generate evaluators", "create evals from traces", "eval bootstrap", "write evaluators", "build eval suite", "publish evaluators", or wants to generate BaseEvaluator/LLMJudge code or online judge configs from production LLM trace data. Works with ml_app and optional RCA report or failure hypothesis.
Install the Datadog Agent on Kubernetes using the Datadog Operator — required before enabling Single Step Instrumentation (SSI), which automatically instruments applications for APM without code changes. Only use if no Datadog Agent is deployed on the cluster yet.
Generate a live Single Step Instrumentation (SSI) onboarding confirmation report — verifies APM instrumentation is working end-to-end with deep links into the Datadog UI. Only use after agent-install and enable-ssi have both completed successfully.
Root cause analysis on production LLM traces. Diagnoses why an LLM application is failing — works from eval judge verdicts, runtime errors, or structural anomalies depending on what signals are present. Walks the span tree from symptom to root cause. Use when user says "what's wrong with my app", "why is my eval failing", "analyze errors", "root cause analysis", "diagnose failures", or wants to understand production failure patterns.
APM - install, onboard, instrument, enable, set up, configure, traces, services, dependencies, performance analysis. Use for any request involving Datadog APM setup, instrumentation (SSI, ddtrace, agent install), or analysis.
Install the Datadog Agent on Linux hosts via SSH with Single Step Instrumentation (SSI) enabled — SSI automatically instruments applications for APM without code changes. Only use if no agent is installed yet.
Diagnose and fix Single Step Instrumentation (SSI) issues on Kubernetes — SSI automatically instruments applications for APM without code changes. Only use if the agent and SSI are already configured but traces are missing or instrumentation is not working.
Enable Single Step Instrumentation (SSI) on Kubernetes — automatically instruments applications for APM without code changes. Only use if the Datadog Agent is already running on the cluster — if not, use agent-install first.
Verify Single Step Instrumentation (SSI) is working end-to-end on Kubernetes — SSI automatically instruments applications for APM without code changes. Only use after enable-ssi has run.
Generate a live Single Step Instrumentation (SSI) onboarding confirmation report for Linux hosts — verifies APM instrumentation is working end-to-end with deep links into the Datadog UI. Only use after agent-install and enable-ssi have both completed.
Configure Unified Service Tags and verify Single Step Instrumentation (SSI) injection on Linux hosts — SSI automatically instruments applications for APM without code changes. Only use if the Datadog Agent is already installed.
Verify Single Step Instrumentation (SSI) is working end-to-end on Linux hosts — SSI automatically instruments applications for APM without code changes. Only use after enable-ssi has run.
Guides developers building Datadog Apps with TypeScript, React, the @datadog/apps scaffolder, and @datadog/vite-plugin. Use when a user wants to scaffold, run, debug, upgrade, build, upload, publish, upload without publishing (draft upload), add an upload-no-publish script, set up CI/CD, use OAuth or API/application key auth, trigger/poll Workflow Automation, choose DDSQL or Action Catalog for backend data access, or query app datastores with DDSQL, including backend function troubleshooting.
Audit Trail investigations - who changed what, key compromise, cost spike root cause, compliance evidence (SOC 2/PCI), and AI activity auditing.
Diagnose and fix Single Step Instrumentation (SSI) issues on Linux hosts — SSI automatically instruments applications for APM without code changes. Only use if the agent and SSI are configured but traces are missing or instrumentation is not working.
Create and manage APM service remapping rules — rewrite service names at ingestion time to collapse noisy inferred entities, clean up auto-generated names, handle org renames, or normalize naming conventions. Use for any request involving service renaming, service mapping, inferred service cleanup, peer.service normalization, or collapsing fragmented service names.
Audit what the Bits AI assistant (MCP server) has done in your Datadog org — tool calls by user, resources accessed, and anomaly flags for AI governance.
Claude Skills — questions
Answers built from the skills we actually parsed.