Eagle3 Validate Agent Skill
> Validate that an EAGLE3 pipeline run completed successfully end-to-end. Checks all 4 steps produced expected artifacts, verifies acceptance rate meets threshold (>= 2.1), and produces a summary report. Use when user wants to verify a pipeline run or check benchmark results.
Install
one command, takes just this skill from the repositorynpx skills add https://github.com/NVIDIA/Model-Optimizer --skill eagle3-validateThe instruction itself
8 sections, as written by the authorEAGLE3 Pipeline Validation
Verify that an EAGLE3 pipeline run completed successfully and meets quality criteria.
Step 0 — Identify the experiment
Find the most recent experiment directory (or ask the user for the path):
ls -td experiments/cicd/cicd_* | head -5
Each experiment directory has one subdirectory per task (numbered 0–3), each containing a
log file whose name varies by launch mode (Slurm: sbatch_*.out, local Docker: *.log).
Step 1 — Check task outcomes
Match the log files generally and read the tail of each:
find experiments/<exp_id>/ -type f \( -name '*.out' -o -name '*.log' \) | sort | while read -r f; do
echo "=== $f ==="; tail -50 "$f"; echo
done
All 4 tasks must complete without error. Look for:
exit code: 0or no error — successDUE TO TIME LIMIT— timeoutFAILED/signal/ exception traceback — failure
If any task failed, suggest running /eagle3-triage instead.
Step 2 — Verify artifacts exist
Check each step produced the expected output (artifacts live on the cluster at /scratchspace/).
Confirm via log messages:
| Step | Expected log evidence | Artifact |
|------|-----------------------|----------|
| task_0 | "Saved N samples" or progress bar completing | /scratchspace/data/*.jsonl |
| task_1 | "Successfully processed N conversations" | /scratchspace/offline_hidden_states/*.pt |
| task_2 | Training loss decreasing, "export complete" | /scratchspace/eagle3/model.safetensors, /scratchspace/export/ |
| task_3 | Average Acceptance Length ... ratio: X.XX | JSON result files |
Step 3 — Check acceptance rate
In the task_3 log, find:
Average Acceptance Length {'accept': X, 'count': Y, 'ratio': Z.ZZ}
The ratio field is the acceptance rate (AR).
| Criterion | Threshold | Status |
|-----------|-----------|--------|
| AR (MT-Bench) | >= 2.1 | PASS / FAIL |
If the log shows AR ... < lower bound, the run already triggered a threshold failure (exit code 1).
Step 4 — Check training quality
In the task_2 log look for:
- Final training loss — should be decreasing, not NaN
- AR validation during training (if
training.ar_validate_stepswas set) - Number of training steps — confirms full training duration
Step 5 — Produce validation report
## EAGLE3 Pipeline Validation Report
**Experiment:** <exp_dir>
**Model:** <model_name>
**Date:** <date>
**Pipeline config:** <yaml_path>
### Step Status
| Step | Task | Status | Notes |
|------|------|--------|-------|
| 0 | Data synthesis | PASS/FAIL/TIMEOUT | N samples generated |
| 1 | Hidden state dump | PASS/FAIL | N .pt files |
| 2 | Training + export | PASS/FAIL | Final loss: X.XX |
| 3 | Benchmark | PASS/FAIL | AR: X.XX |
### Acceptance Rate
- MT-Bench AR: X.XX (threshold: >= 2.1) — PASS/FAIL
### Training Summary
- Final loss: X.XX
- Training steps: N
- AR during training: X.XX (if validated)
### Overall: PASS / FAIL
<one-line summary>
Step 6 — Suggest next steps
If PASS:
- Record the verified result (and checkpoint path) in the team's internal triage tracker
- This model is now a candidate to add as a launcher example in a dedicated PR
If FAIL:
- Identify which step or metric failed
- Suggest running
/eagle3-triagefor diagnosis - For a low AR, diagnose the specific cause from the run (training loss curve, data
volume/quality, draft-head capacity, hyperparameters) and suggest fixes targeted to that
scenario — low AR can have many causes, so avoid a generic checklist.
Other skills for the same job
different authors, same section of the catalogue
Production-ready patterns for building LLM applications. Covers RAG pipelines, agent architectures, prompt IDEs, and LLMOps monitoring. Use when designing AI applications, implementing RAG, building agents, or setting up LLM observability.
Build production ML systems with PyTorch 2.x, TensorFlow, and modern ML frameworks. Implements model serving, feature engineering, A/B testing, and monitoring. Use PROACTIVELY for ML model deployment, inference optimization, or production ML infrastructure.
World-class ML engineering skill for productionizing ML models, MLOps, and building scalable ML systems. Expertise in PyTorch, TensorFlow, model deployment, feature stores, model monitoring, and ML infrastructure. Includes LLM integration, fine-tuning, RAG systems, and agentic AI. Use when deploying ML models, building ML platforms, implementing MLOps, or integrating LLMs into production systems.
Expert in Langfuse - the open-source LLM observability platform. Covers tracing, prompt management, evaluation, datasets, and integration with LangChain, LlamaIndex, and OpenAI. Essential for debugging, monitoring, and improving LLM applications in production. Use when: langfuse, llm observability, llm tracing, prompt management, llm evaluation.
Use this skill for reinforcement learning tasks including training RL agents (PPO, SAC, DQN, TD3, DDPG, A2C, etc.), creating custom Gym environments, implementing callbacks for monitoring and control, using vectorized environments for parallel training, and integrating with deep RL workflows. This skill should be used when users request RL algorithm implementation, agent training, environment design, or RL experimentation.
Managed vector database for production AI applications. Fully managed, auto-scaling, with hybrid search (dense + sparse), metadata filtering, and namespaces. Low latency (<100ms p95). Use for production RAG, recommendation systems, or semantic search at scale. Best for serverless, managed infrastructure.
Deploy, evaluate, fine-tune, and manage Foundry agents end-to-end with azd: hosted agent scaffold/run/deploy, prompt agent create, batch eval, continuous eval, prompt optimizer, Agent Optimizer scaffold, agent.yaml, dataset curation from traces, model fine-tuning (SFT/DPO/RFT). USE FOR: azd ai agent, azd provision/deploy, deploy agent, hosted agent, create agent, add tool to agent, invoke agent, evaluate agent, continuous eval, continuous monitoring, agent CI/CD, optimize prompt, improve prompt, optimize agent instructions, agent optimizer, deploy model, Foundry project, RBAC, role assignment, permissions, quota, capacity, region, troubleshoot agent, deployment failure, AI Services, create Foundry resource, provision, knowledge index, customize deployment, onboard, availability, fine-tune, SFT, DPO, RFT, training-data, grader, distillation, fine-tuned model, large file upload. DO NOT USE FOR: Azure Functions, App Service, general Azure deploy (use azure-deploy), general Azure prep (use azure-prepare).
Cost optimization patterns for LLM API usage — model routing by task complexity, budget tracking, retry logic, and prompt caching.
How to use it
Take nvidia/eagle3-validate from the repository into ~/.claude/skills for personal
use, or into .claude/skills inside a project.
The agent identifies a skill by the name field in its header. Two skills with the
same name cannot sit side by side — one of them will be ignored.