MCP Eval Runner runs on your own machine — the client starts it, so there is no endpoint to ping. 35 installs a week from npm. Last commit 23 Sep 2026.
A standardized testing harness for MCP servers and agent workflows
What the code does
We read the source, 5 d ago · rules 995398eba69f
What this server is able to do. For an MCP server this is often the job itself — a terminal server runs commands because that is what it is for. Listed so you know what you are plugging in, not as an accusation.
const destPath = path.join(dest, fileName);
this.proc = spawn(command, args, {
Is this your server and something here is wrong? Tell us — corrections are free and do not require a plan.
This code can reach further than it looks
We found places where it runs commands, builds paths or queries from values it is given. None of that is a flaw by itself — it becomes one when the code changes, and code changes quietly between releases. We re-read it on every one.
Connect this server
This server runs on your own machine — install it with the package manager and the client starts it for you. Package name taken from the official registry entry.
claude mcp add mcp-eval-runner -e 'YOUR_API_KEY=<YOUR_API_KEY>' -- npx -y mcp-eval-runner{
"mcpServers": {
"mcp-eval-runner": {
"command": "npx",
"args": [
"-y",
"mcp-eval-runner"
],
"env": {
"YOUR_API_KEY": "<YOUR_API_KEY>"
}
}
}
}[mcp_servers.mcp-eval-runner]
command = "npx"
args = ["-y", "mcp-eval-runner"]
env = { YOUR_API_KEY = "<YOUR_API_KEY>" }{
"mcpServers": {
"mcp-eval-runner": {
"command": "npx",
"args": [
"-y",
"mcp-eval-runner"
],
"env": {
"YOUR_API_KEY": "<YOUR_API_KEY>"
}
}
}
}{
"servers": {
"mcp-eval-runner": {
"type": "stdio",
"command": "npx",
"args": [
"-y",
"mcp-eval-runner"
],
"env": {
"YOUR_API_KEY": "<YOUR_API_KEY>"
}
}
}
}Put your own value in place of
<YOUR_API_KEY> (Your API key for the service).
The author’s registry entry marks it as required.
Alternatives to MCP Eval Runner
same job, measured the same wayMCP server and CLI for Android development and change-aware Android and Flutter testing workflows.
MCP server for Testkube - Manage test workflows, executions, and artifacts via AI assistants
A sample MCP server for testing
Local QA workflows, test evidence, screenshot maps and confidence reports for AI coding agents.
My first MCP server for testing
A/B testing MCP server for pagent: create, run and review A/B tests on your live website.
Ingeniux CTnP MCP server for personalization, A/B testing, and audience management.
A MCP server for TestDino
MCP Eval Runner — questions
Answers built from our own checks of this server.