Analyze Evals Agent Skill
> Analyze exported evaluation results from Copilot Studio's Evaluate tab. The user provides a CSV file exported from the Copilot Studio UI; this skill parses it, identifies failures, and proposes YAML fixes. No API access or published agent required — just the exported CSV.
Install
one command, takes just this skill from the repositorynpx skills add https://github.com/microsoft/skills-for-copilot-studio --skill analyze-evalsWhat it tells the agent to use
found in the instruction textThe instruction itself
4 sections, as written by the authorAnalyze Copilot Studio Evaluation Results
Analyze evaluation results exported from the Copilot Studio UI as CSV.
Phase 1: Get Results
- Ask the user for the CSV file path if not already provided. The file is typically exported from Copilot Studio's Evaluate tab and named
Evaluate <agent name> <date>.csvin their Downloads folder.
- Read the CSV file. The in-product evaluation CSV has these columns:
| Column | Meaning |
|--------|---------|
| question | The test utterance |
| expectedResponse | Expected response (may be empty) |
| actualResponse | What the agent responded |
| testMethodType_1 | Eval method (e.g., GeneralQuality) |
| result_1 | Pass or Fail |
| passingScore_1 | Score threshold (may be empty) |
| explanation_1 | Why it passed/failed (e.g., "Seems relevant; Seems incomplete; Knowledge sources not cited") |
The _1 suffix indicates the first eval method. There may be additional methods (_2, _3, etc.) with the same column pattern.
Phase 2: Analyze Results
- Focus on failed evaluations (
result_1=Fail, or anyresult_N=Fail).
- For each failure, use the
explanationcolumn to understand the issue: - "Question not answered" — The agent couldn't handle the question. Check if there's a matching topic or knowledge source.
- "Knowledge sources not cited" — The agent responded but didn't cite sources. Check knowledge source configuration and
SearchAndSummarizeContentnodes. - "Seems incomplete" — The response was partial. Check topic flow for early exits, missing branches, or incomplete
SendActivitymessages. - Error messages in
actualResponse(e.g.,GenAIToolPlannerRateLimitReached) — These are runtime errors, not authoring issues. Flag them to the user as transient failures to retry.
Phase 3: Propose Fixes
- For each failure, identify the relevant YAML file(s):
- Auto-discover the agent:
Glob: **/agent.mcs.yml - Find the relevant topic by matching the test utterance against trigger phrases and model descriptions
- Read the topic file to understand the current flow
- Propose specific YAML changes to fix each failure. Present them to the user as a summary:
- Which test(s) failed and why
- Which file(s) need changes
- What the proposed change is (show the diff)
- Wait for user decision. The user can:
- Accept all — apply all proposed changes
- Accept partially — apply only some changes (ask which ones)
- Reject — discard proposed changes and discuss alternative approaches
- Apply accepted changes using the Edit tool. After applying, remind the user to push and publish again before re-running evaluations.
Other skills for the same job
different authors, same section of the catalogue
Meta-skill for publication-ready figures. Use when creating journal submission figures requiring multi-panel layouts, significance annotations, error bars, colorblind-safe palettes, and specific journal formatting (Nature, Science, Cell). Orchestrates matplotlib/seaborn/plotly with publication styles. For quick exploration use seaborn or plotly directly.
Coding Agent Account Manager - Sub-100ms account switching for AI coding CLIs with fixed-cost subscriptions. Vault profiles, isolated profiles for parallel sessions, smart rotation with health scoring, cooldown tracking, automatic failover, TUI dashboard. Go CLI.
Use this umbrella skill when the request spans multiple Cloud Claw user-facing domains, especially launching a new AltClaw or OpenClaw VM and then managing lifecycle, logs, renewal, or dashboard access through the local altllm cloud-claw-* commands in this repository.
Creating interactive data visualisations using d3.js. This skill should be used when creating custom charts, graphs, network diagrams, geographic visualisations, or any complex SVG-based data visua...
Use this skill when you need to report on a troubleshooting session
Query and browse evaluation results stored in MLflow. Use when the user wants to look up runs by invocation ID, compare metrics across models, fetch artifacts (configs, logs, results), or set up the MLflow MCP server. ALWAYS triggers on mentions of MLflow, experiment results, run comparison, invocation IDs in the context of results, or MLflow MCP setup.
Guided journey from an app people sign up for and then quietly abandon to a sealed retention engine with a habit loop, an activated first run, and one metric the whole team trusts. Orchestrates eight skills phase by phase - hooked-ux, improve-retention, continuous-discovery, lean-ux, inspired-product, lean-analytics, microinteractions, drive-motivation - asking the user questions at every decision point and recording results in the project docs/ folder (PRODUCT.md, METRICS.md, GROW-APP-PLAN.md) so the journey resumes across sessions. Use when the user wants to lift activation and retention, design a habit loop, fix a leaky onboarding funnel, or says ''users sign up then disappear''. Do not use to fix broken UX or performance that no engagement mechanic can paper over - run improve-app first; if there is no app yet, use create-app. For one framework in isolation, invoke that skill directly.
Guided journey from a stalled, plateaued business to one with an honest diagnosis, a working operating rhythm, and offers repriced to real value. Orchestrates eight skills phase by phase - good-strategy-bad-strategy, traction-eos, high-output-management, team-topologies, drive-motivation, lean-analytics, negotiation, monetizing-innovation - asking the user questions at every decision point and recording results in the project docs/ folder (STRATEGY.md, OPERATIONS.md, METRICS.md, IMPROVE-BUSINESS-PLAN.md) so the journey resumes across sessions. Use when the user wants to fix a business that has plateaued, diagnose why growth stalled, tighten strategy and execution, re-motivate a team, or says ''revenue is flat and I do not know why''. Starting from scratch with no customers: use create-business. Once the fundamentals work and the goal is expansion: use grow-business. When the product itself drags the business down: use improve-app. For one framework in isolation, invoke that skill directly.
How to use it
Take microsoft/analyze-evals from the repository into ~/.claude/skills for personal
use, or into .claude/skills inside a project.
The agent identifies a skill by the name field in its header. Two skills with the
same name cannot sit side by side — one of them will be ignored.