Agent Eval runs on your own machine — the client starts it, so there is no endpoint to ping. 187 installs a week from pypi. Last commit 25 Sep 2026.
Statistical regression testing for LLM agents: p-value, effect size, and CI on behavior change.
What the code does
We read the source, 18 h ago · rules ccca72095570
What this server is able to do. For an MCP server this is often the job itself — a terminal server runs commands because that is what it is for. Listed so you know what you are plugging in, not as an accusation.
return spawnSync(command, args, { stdio: 'inherit', shell: useShell });
result = subprocess.run( # noqa: S603 -- fixed argv list, shell=True never used
Is this your server and something here is wrong? Tell us — corrections are free and do not require a plan.
This code can reach further than it looks
We found places where it runs commands, builds paths or queries from values it is given. None of that is a flaw by itself — it becomes one when the code changes, and code changes quietly between releases. We re-read it on every one.
Connect this server
This server runs on your own machine — install it with the package manager and the client starts it for you. Package name taken from the official registry entry.
claude mcp add agent-eval -- uvx agent-regress-cli{
"mcpServers": {
"agent-eval": {
"args": [
"agent-regress-cli"
],
"command": "uvx"
}
}
}[mcp_servers.agent-eval]
command = "uvx"
args = ["agent-regress-cli"]{
"mcpServers": {
"agent-eval": {
"args": [
"agent-regress-cli"
],
"command": "uvx"
}
}
}{
"mcpServers": {
"agent-eval": {
"args": [
"agent-regress-cli"
],
"command": "uvx"
}
}
}Alternatives to Agent Eval
same job, measured the same wayChange-aware CI validation and affected-test guidance for coding agents.
Fix production agent behavior with tested changes and a production verdict on every fix.
Regression testing for AI agents. Golden baselines, CI/CD, LangGraph, CrewAI, OpenAI, Claude.
Visual regression & snapshot testing for AI agents — list runs, read semantic diffs, review.
Local multi-channel message emulator for E2E testing, with MCP tools for LLM agents.
Local test execution and coverage summaries for onboarding agents.
Verified statistical inference for AI agents: hypothesis tests, sequential testing, power.
Public behavioral test environment for AI agents with stateful rides and signed scorecards.
Agent Eval — questions
Answers built from our own checks of this server.