Agent Eval runs on your own machine — the client starts it, so there is no endpoint to ping. 187 installs a week from pypi. Last commit 25 Aug 2026.
Statistical regression testing for LLM agents: p-value, effect size, and CI on behavior change.
What the code does
We read the source, 11 h ago · rules 3dff92dd89df
What this server is able to do. For an MCP server this is often the job itself — a terminal server runs commands because that is what it is for. Listed so you know what you are plugging in, not as an accusation.
return spawnSync(command, args, { stdio: 'inherit', shell: useShell });
result = subprocess.run( # noqa: S603 -- fixed argv list, shell=True never used
Is this your server and something here is wrong? Tell us — corrections are free and do not require a plan.
This code can reach further than it looks
We found places where it runs commands, builds paths or queries from values it is given. None of that is a flaw by itself — it becomes one when the code changes, and code changes quietly between releases. We re-read it on every one.
Connect this server
This server runs on your own machine — install it with the package manager and the client starts it for you. Package name taken from the official registry entry.
claude mcp add agent-eval -- uvx agent-regress-cli{
"mcpServers": {
"agent-eval": {
"args": [
"agent-regress-cli"
],
"command": "uvx"
}
}
}[mcp_servers.agent-eval]
command = "uvx"
args = ["agent-regress-cli"]{
"mcpServers": {
"agent-eval": {
"args": [
"agent-regress-cli"
],
"command": "uvx"
}
}
}{
"mcpServers": {
"agent-eval": {
"args": [
"agent-regress-cli"
],
"command": "uvx"
}
}
}Alternatives to Agent Eval
same job, measured the same wayVerified statistical inference for AI agents: hypothesis tests, correlation, effect sizes, power.
Regression testing for AI agents. Golden baselines, CI/CD, LangGraph, CrewAI, OpenAI, Claude.
Visual regression & snapshot testing for AI agents — list runs, read semantic diffs, review.
Local multi-channel message emulator for E2E testing, with MCP tools for LLM agents.
Local test execution and coverage summaries for onboarding agents.
Find test gaps, generate grounded tests, and dynamically prove behavior with mutation testing.
Find test gaps, generate grounded tests, and dynamically prove behavior with mutation testing.
Snapshot tests for AI tool-call traces: diff, validate, and catch silent regressions.
Agent Eval — questions
Answers built from our own checks of this server.