Eval Consistency Agent Skill
测试 use-persona 的角色扮演一致性。给定 persona + 10 个对话场景，生成回复并按 5 个维度评分，输出一致性报告。
Install
one command, takes just this skill from the repositorynpx skills add https://github.com/YIKUAIBANZI/forge-skill --skill eval-consistencyThe instruction itself
7 sections, as written by the author/eval-consistency — 角色扮演一致性评测
你的任务是对 use-persona 的角色扮演质量做一次系统性评测，全程在当前对话中完成，不需要调用任何外部 API。
Step 0：加载测试资源
- 读取测试用例文件：
evals/test_cases/persona_consistency_cases.yaml - 根据
persona_name字段，读取对应 persona：
personas/others/{persona_name}/persona.json
- 从 persona.json 中提取 chat-card 关键内容：
- L0 硬性特征
- L2 表达风格（语言特征 + 沟通模式，重点是 signature_phrases 和消息长度偏好）
- L4 互动模式（关键场景下的表现）
正在加载 {persona_name} 的 persona 和测试用例...
共 {N} 个场景待测试。
Step 1：逐场景测试
对每个测试用例，执行两步：
1a. 生成角色扮演回复
以 persona 的身份回复用户消息。只输出回复本身，不加任何解释。
内部模板（不展示给用户）：
你是 {persona_name}。
[chat-card 关键内容]
用户发来消息："{user_message}"
以你的身份回复，只输出回复本身。
1b. 评分（内部执行，立即给出）
生成回复后，立刻按以下 5 个维度给自己打分（每项 0-20 分）：
| 维度 | 评分标准 |
|------|----------|
| 消息长度 | 回复长度是否符合 L2 的消息长度偏好？短消息风格但回了长段落扣分 |
| 口头禅命中 | 是否自然用到了 L2 的 signature_phrases？完全没有扣分 |
| 标点风格 | 标点和语气是否符合 persona 的风格描述？ |
| 互动模式 | 在这个具体场景下，互动方式是否符合 L4 的 scene_responses？ |
| 边界遵守 | 有没有违反 L0 的硬性特征？违反则此项得 0 分 |
给出每项分数 + 一句话说明。
Step 2：输出完整报告
所有场景跑完后，输出评测报告：
===================================
角色扮演一致性评测报告 — {persona_name}
===================================
## 逐场景结果
[c01] {场景简述}
回复："{生成的回复}"
得分：{total}/100
✅/⚠️ 消息长度：{score}/20 — {说明}
✅/⚠️ 口头禅命中：{score}/20 — {说明}
✅/⚠️ 标点风格：{score}/20 — {说明}
✅/⚠️ 互动模式：{score}/20 — {说明}
✅/⚠️ 边界遵守：{score}/20 — {说明}
[c02] ...
---
## 汇总
平均分：{avg}/100 {✅ 通过 / ❌ 未达标（目标 70+）}
各维度平均：
消息长度 {avg}/20
口头禅命中 {avg}/20
标点风格 {avg}/20
互动模式 {avg}/20
边界遵守 {avg}/20
## 主要问题
{如果平均分 < 70，列出最常见的失分点}
## 建议
{如果某维度平均分 < 12，给出 1-2 条具体改进建议，指向 persona 的哪一层需要补充}
Step 3：保存结果（可选）
询问用户是否保存：
要把这次结果存入 evals/results/ 吗？
以后优化后可以对比。（y/n）
如果确认，写入 evals/results/consistency_{YYYYMMDD}.md。
注意
- 全程不需要 API Key：评分是你自己执行的，不是另起一个 LLM
- 评分要诚实：对自己生成的回复该扣分就扣分，不要因为是自己生成的就打高分
- 用例是基于小美的，如果用户指定了其他 persona，根据那个 persona 的 L2/L4 调整评分标准
Other skills for the same job
different authors, same section of the catalogue
Toolkit for interacting with and testing local web applications using Playwright. Supports verifying frontend functionality, debugging UI behavior, capturing browser screenshots, and viewing browser logs.
Use when implementation is complete, all tests pass, and you need to decide how to integrate the work - guides completion of development work by presenting structured options for merge, PR, or cleanup
Use when implementing any feature or bugfix, before writing implementation code
Use when encountering any bug, test failure, or unexpected behavior, before proposing fixes
Use when about to claim work is complete, fixed, or passing, before committing or creating PRs - requires running verification commands and confirming output before making any success claims; evidence before assertions always
Expert guidance for systematic backtesting of trading strategies. Use when developing, testing, stress-testing, or validating quantitative trading strategies. Covers "beating ideas to death" methodology, parameter robustness testing, slippage modeling, bias prevention, and interpreting backtest results. Applicable when user asks about backtesting, strategy validation, robustness testing, avoiding overfitting, or systematic trading development.
Cloud laboratory platform for automated protein testing and validation. Use when designing proteins and needing experimental validation including binding assays, expression testing, thermostability measurements, enzyme activity assays, or protein sequence optimization. Also use for submitting experiments via API, tracking experiment status, downloading results, optimizing protein sequences for better expression using computational tools (NetSolP, SoluProt, SolubleMPNN, ESM), or managing protein design workflows with wet-lab validation.
This skill should be used for time series machine learning tasks including classification, regression, clustering, forecasting, anomaly detection, segmentation, and similarity search. Use when working with temporal data, sequential patterns, or time-indexed observations requiring specialized algorithms beyond standard ML approaches. Particularly suited for univariate and multivariate time series analysis with scikit-learn compatible APIs.
How to use it
Take yikuaibanzi/eval-consistency from the repository into ~/.claude/skills for personal
use, or into .claude/skills inside a project.
The agent identifies a skill by the name field in its header. Two skills with the
same name cannot sit side by side — one of them will be ignored.