Nat Evaluation

nvidia/nat-evaluation

Use when designing, configuring, running, or troubleshooting NeMo Agent Toolkit evaluations, datasets, evaluator selection, ATIF surfaces, quality gates, custom evaluators, and `nat eval`.

30k tokens

context cost

the whole folder, loaded on every use

files

instructions only

copies elsewhere

how many repositories repackaged it

2551

stars on the repo

on the repository, not the skill itself

Install

one command, takes just this skill from the repository

npx skills add https://github.com/NVIDIA/NeMo-Agent-Toolkit --skill nat-evaluation

What comes with it

120 329 bytes besides the instruction

references/agent-eval-framework.md

references/code-patterns.md

references/evaluation-contract.md

references/evaluation-surfaces.md

references/evaluators/evaluator-langsmith-judge.md

references/evaluators/evaluator-langsmith.md

references/evaluators/evaluator-ragas.md

references/evaluators/evaluator-red-teaming.md

references/evaluators/evaluator-trajectory.md

references/evaluators/evaluator-tunable-rag.md

references/methodology.md

references/operating-mode.md

The instruction itself

3 sections, as written by the author

NeMo Agent Toolkit Evaluation

Use this skill for measuring agent quality and behavior.

Workflow

Decide the evaluation surface and output format.
Decompose quality goals into separate evaluators.
Choose built-in evaluators before writing custom evaluators.
Keep datasets small and explicit for local validation.
Run nat eval and inspect generated artifacts.

References

references/operating-mode.md
references/methodology.md
references/agent-eval-framework.md
references/evaluation-surfaces.md
references/evaluation-contract.md
references/evaluators/
references/code-patterns.md

How to use it

Copy the folder

Take nvidia/nat-evaluation from the repository into ~/.claude/skills for personal use, or into .claude/skills inside a project.

Check the name does not clash

The agent identifies a skill by the name field in its header. Two skills with the same name cannot sit side by side — one of them will be ignored.