Model Card Agent Skill

Model-Card Skill

Purpose

This skill produces the documentation an engineer-built medical-imaging model must carry: a

Model Card (intended use, out-of-scope use, training data, per-subgroup performance, caveats), a

Datasheet for its dataset (provenance, composition, collection, labelling, consent), and a

METRIC-informed data-quality pass. It fills the templates from facts the user supplies — it

never invents a number, a provenance detail, a consent status, or a licence — and ships a deterministic

gate that no required section is missing or left as an unfilled [NEEDS INPUT] placeholder.

It is the reporting seam of the model-engineering lane: after /model-validation audits the design

and /model-evaluation produces the numbers, this skill records them in a portable, auditable card that

/write-paper and /check-reporting consume. It mirrors /version-dataset structurally (generate +

deterministic verify).

When to use

• A trained model needs a Model Card / Datasheet for a repo, Hugging Face card, or manuscript supplement.

When NOT to use

• Auditing the validation design / metrics → /model-validation, /model-evaluation.
• Versioning the dataset bytes → /version-dataset; tabular variable docs → /generate-codebook.
• Item-by-item reporting-guideline compliance of the manuscript → /check-reporting.
• Building / training the model → /model-scaffold.

Workflow

Phase 1 — Collect the facts

Gather, from the user / the model's developers: task + architecture + provenance + licence; intended use

and out-of-scope use; training and evaluation cohorts; the reference standard and inter-reader agreement;

overall and per-subgroup performance; data collection, consent, and de-identification. Anything not

supplied stays [NEEDS INPUT] — never guess.

Phase 2 — Fill the Model Card

Copy ${CLAUDE_SKILL_DIR}/references/model_card_template.md to MODEL_CARD.md and fill each section

from the facts. Keep the headings. Numbers come only from /model-evaluation / executed results.

Phase 3 — Fill the Datasheet

Copy ${CLAUDE_SKILL_DIR}/references/datasheet_template.md to DATASHEET.md and fill the seven

question groups (Motivation, Composition, Collection, Preprocessing/Labeling, Uses, Distribution,

Maintenance).

Phase 4 — METRIC data-quality pass

Walk ${CLAUDE_SKILL_DIR}/references/metric_dimensions.md (completeness, correctness, consistency,

representativeness, timeliness, provenance, label provenance, fairness/coverage, leakage safety) and

record each finding in the Datasheet. Anything that affects the headline metric's validity is also a

/model-validation finding — cross-check there.

Phase 5 — Verify completeness (deterministic gate)

python3 ${CLAUDE_SKILL_DIR}/scripts/check_model_card_complete.py \
  --card MODEL_CARD.md --datasheet DATASHEET.md --strict

MISSING_SECTION / EMPTY_REQUIRED_SECTION must be zero before the card ships.

Phase 6 — Hand off

Carry the card into /write-paper (the Methods / supplement reference it), /check-reporting

(CLAIM 2024 / TRIPOD+AI item audit of the manuscript), and /self-review.

Anti-Hallucination

• Never invent evaluation numbers, subgroup results, or dataset provenance. Every figure comes from

/model-evaluation or the user's executed results; every provenance / consent / licence statement is

user-confirmed. Unknown → [NEEDS INPUT], which the gate flags.

• Never mark a section complete without user-supplied content, and never auto-fill a placeholder to

pass the gate.

• Never assert a licence or consent status the user did not confirm.
• The gate checks presence, not truth — a complete card can still contain a wrong number; validity

is /model-validation and the human's responsibility.

Deterministic gate

scripts/check_model_card_complete.py — verifies every required Model Card / Datasheet section is

present and non-empty (stdlib, network-free). Reproducible challenge:

bash ${CLAUDE_SKILL_DIR}/scripts/check_model_card_complete_challenge/verify.sh.

Note on classification

Model Cards (Mitchell et al. 2019) and Datasheets (Gebru et al. 2021) are documentation standards,

not clinical reporting guidelines, so they live here as references/ templates (uncounted), not in

/check-reporting's counted checklist set — the same way appraisal_tools/METRICS.md is kept separate.

/check-reporting still owns the manuscript-level CLAIM 2024 / TRIPOD+AI item audit.

Boundaries

model-validation (audit design) + model-evaluation (metrics)
  └─ model-card (this skill: Model Card + Datasheet + METRIC pass, completeness-gated)
       └─ write-paper + check-reporting (manuscript) ; version-dataset (dataset bytes)