The Aggregate — LLM benchmark aggregate MCP Server

by theaggregate Your server? Claim it

answering

The Aggregate — LLM benchmark aggregate is answering right now. Last checked 3 min ago. It exposes 8 tools.

Fused LLM rankings: one IRT/Elo scale across ~5,000 public benchmark leaderboards, updated daily.

Uptime history 52 days of history · worst day 92%

52 days agonow

92.3%

Uptime 24h

84 of 91 checks

Tools

read from the server

335 ms

Response time

average over 24h

open, no key

Access

streamable-http

What changed 6

Every tool that appeared, vanished or quietly changed what it asks for. Recorded since 15 August 2026. No other catalogue keeps this.

12 Sep a tool description was rewritten get_leaderboard

7 Sep a tool description was rewritten get_leaderboard

15 Aug 2 tool descriptions were rewritten get_leaderboard, search_models

15 Aug 2 tools changed the parameters they ask for get_leaderboard, search_models

The Aggregate — LLM benchmark aggregate does not always answer

Over the last week it answered 98.1% of our checks. We check every 15 minutes, so you hear about the next outage within the hour — not from your users.

Connect this server

Endpoint below is the one we actually reach during checks — not the one copied from a README. Last verified 3 min ago.

Claude Code Claude Desktop Codex Cursor VS Code

run in your terminal

claude mcp add the-aggregate --transport http https://theaggregate.ai/mcp

~/Library/Application Support/Claude/claude_desktop_config.json

{
  "mcpServers": {
    "the-aggregate": {
      "url": "https://theaggregate.ai/mcp"
    }
  }
}

~/.codex/config.toml

[mcp_servers.the-aggregate]
url = "https://theaggregate.ai/mcp"

.cursor/mcp.json

{
  "mcpServers": {
    "the-aggregate": {
      "url": "https://theaggregate.ai/mcp"
    }
  }
}

.vscode/mcp.json

{
  "mcpServers": {
    "the-aggregate": {
      "url": "https://theaggregate.ai/mcp"
    }
  }
}

Available tools 8

Read directly from the server with tools/list, grouped by what they act on. If a tool disappears, we record the date.

about

about_the_aggregate

What this data is: how the IRT fusion works, current coverage counts, update cadence, and how to cite it.

benchmark

get_benchmark

One benchmark in depth: what it measures, the original source leaderboard URL, IRT stats (difficulty, noise, model coverage), skill weights, and the current top models on it.

benchmarks

search_benchmarks

Find benchmarks in the aggregate by (partial) name. Returns model coverage, difficulty on the Elo scale, and the benchmark page URL.

compare

compare_models

Head-to-head between 2-4 models: aggregate ranks, Elo gap with a significance note based on the standard errors, and notable benchmarks they share.

leaderboard

get_leaderboard

Top of the cross-benchmark aggregate ranking: every model placed on one Elo scale by an IRT model fit over public benchmark leaderboards (call about_the_aggregate for the current coverage counts). One row per model by default, fused across reasoning-effort settings. Supports paging via limit/offset.

model

get_model

One model in depth: aggregate rank, Elo with standard error, provider, what it is, cost per task where known, and its most notable benchmark results (with percentiles).

models

search_models

Find ranked models by (partial) name or provider. Returns rank, Elo and the model page URL. One row per model by default, fused across reasoning-effort settings.

prediction

get_prediction_duel

Guesswork — the public prediction duel: every day frontier LLMs and The Aggregate's own IRT model predict newly scraped benchmark scores before seeing them, and the errors are scored. Returns the current monthly standings, wins and losses included.

Endpoints

URL	Transport	State	Latency	Checked
https://theaggregate.ai/mcp	streamable-http	answering	285 ms	3 min ago

Alternatives to The Aggregate — LLM benchmark aggregate

same job, measured the same way

Bottrade

by bot-trade

Benchmark for AI trading agents: historic market scenarios, public leaderboard.

19 tools answering

ERP Report Engine

by gulmezeren2-byte

Read-only MCP for the SQL database behind an ERP: provably read-only guard, public benchmark.

46 installs/wk local only

b/cited

by bcited

AEO tools: scan any URL for AI-citation readiness, read the public leaderboard and citation panel.

6 tools answering

News Monitor MCP

by malkreide

Aggregated news monitoring across Swiss public media RSS feeds

142 installs/wk local only

F1Laps

by f1laps

Read-only F1 game laps, telemetry, setups, leaderboard benchmarks, and progress.

answering

Badge MCP Server

by thehomer87

Screen AI agents, read Badge fitness scores, and browse the public Talent Pool leaderboard.

30 installs/wk local only

Summoned Games

by summoned

Build, publish and update browser games with saves, leaderboards and realtime multiplayer built in.

answering

MCP Lever

by smeet666

Search public Lever job boards: resolve a company, read its openings, read one in full.

53 installs/wk local only

The Aggregate — LLM benchmark aggregate — questions

Answers built from our own checks of this server.

What can The Aggregate — LLM benchmark aggregate do?

It exposes 8 tools, read directly from the server on our last check. Among them: about_the_aggregate, compare_models, get_benchmark, get_leaderboard, get_model, get_prediction_duel and 2 more. The full list with descriptions is on this page — we take it from the server itself via tools/list, not from a README. How MCP servers expose tools in the first place →

Is The Aggregate — LLM benchmark aggregate working right now?

We send a real MCP handshake every 15 minutes. Over the last 24 hours 84 of 91 checks got a reply (92.3%), average response time 335 ms. The bar chart above shows every period we have measured.

How do I connect The Aggregate — LLM benchmark aggregate?

Copy the ready config from this page — we generate it for Claude Code, Claude Desktop, Codex, Cursor and VS Code, each with the file path that client actually reads. It is a remote server, so there is nothing to install — the client connects to the address.

Does The Aggregate — LLM benchmark aggregate need an API key?

No. The Aggregate — LLM benchmark aggregate completed a full MCP handshake with us as an anonymous client and listed its tools without asking for anything. All 8 of them are readable on this page. This is what we observed, not what the docs claim.

How fast is The Aggregate — LLM benchmark aggregate?

It answers our handshake in 335 ms on average, which is faster than 48% of all working MCP servers we measure. The comparison comes from our own checks across the whole registry, every 15 minutes.