mcpbeat

AI Eval MCP Server

io.github.lazymac2x/ai-eval
answering

AI Eval is answering right now. Last checked 10 min ago. It exposes 3 tools. Last commit 17 May 2026.

Cloudflare Workers MCP server: ai-eval

Uptime history 41 hours of history
41 hours agonow
100.0%
Uptime 24h
91 of 91 checks
3
Tools
read from the server
153 ms
Response time
average over 24h
0
Stars
last commit 17 May 2026

Connect this server

Endpoint below is the one we actually reach during checks — not the one copied from a README. Last verified 10 min ago.

run in your terminal
claude mcp add ai-eval --transport http https://api.lazy-mac.com/ai-eval/mcp
~/Library/Application Support/Claude/claude_desktop_config.json
{
  "mcpServers": {
    "ai-eval": {
      "url": "https://api.lazy-mac.com/ai-eval/mcp"
    }
  }
}
~/.codex/config.toml
[mcp_servers.ai-eval]
url = "https://api.lazy-mac.com/ai-eval/mcp"
.cursor/mcp.json
{
  "mcpServers": {
    "ai-eval": {
      "url": "https://api.lazy-mac.com/ai-eval/mcp"
    }
  }
}
.vscode/mcp.json
{
  "mcpServers": {
    "ai-eval": {
      "url": "https://api.lazy-mac.com/ai-eval/mcp"
    }
  }
}

Available tools 3

Read directly from the server with tools/list, grouped by what they act on. If a tool disappears, we record the date.

compare
compare_responses
Compare and rank multiple AI responses to the same prompt
score
score_response
Score an AI response against a prompt using heuristic metrics (length, relevance, structure, completeness)
text
text_metrics
Get text quality metrics: word count, sentence count, estimated tokens, readability grade

Endpoints

URLTransportStateLatencyChecked
https://api.lazy-mac.com/ai-eval/mcp streamable-http answering 160 ms 10 min ago

AI Eval — questions

Answers built from our own checks of this server.

What can AI Eval do?
It exposes 3 tools, read directly from the server on our last check. Among them: compare_responses, score_response, text_metrics. The full list with descriptions is on this page — we take it from the server itself via tools/list, not from a README. How MCP servers expose tools in the first place →
Is AI Eval working right now?
We send a real MCP handshake every 15 minutes. Over the last 24 hours 91 of 91 checks got a reply (100.0%), average response time 153 ms. The bar chart above shows every period we have measured.
How do I connect AI Eval?
Copy the ready config from this page — we generate it for Claude Code, Claude Desktop, Codex, Cursor and VS Code, each with the file path that client actually reads. It is a remote server, so there is nothing to install — the client connects to the address.
Does AI Eval need an API key?
No. AI Eval completed a full MCP handshake with us as an anonymous client and listed its tools without asking for anything. All 3 of them are readable on this page. This is what we observed, not what the docs claim.
How fast is AI Eval?
It answers our handshake in 153 ms on average, which is faster than 73% of all working MCP servers we measure. The comparison comes from our own checks across the whole registry, every 15 minutes.
Is AI Eval open source?
Yes — 0 stars on GitHub. The source link is on this page, so you can read exactly what it does with your data before you connect it.