ML Referee is answering right now. Last checked 5 min ago. It exposes 26 tools.
Evaluate ML features beyond a baseline and test bounded feature repairs. Invite-only beta.
Uptime history
8 days of history
8 days agonow
100.0%
Uptime 24h
92 of 92 checks
26
Tools
read from the server
617 ms
Response time
average over 24h
open, no key
Access
streamable-http
Nothing serious here today
Today is the operative word: we check ML Referee every 15 minutes and re-read its code on every release. Watch it and you find out the day that stops being true.
Connect this server
Endpoint below is the one we actually reach during checks — not the one copied from a README. Last verified 5 min ago.
run in your terminal
claude mcp add mlreferee --transport http https://api.mlreferee.com/mcpSettings → Connectors → Add custom connector, then paste this address
https://api.mlreferee.com/mcp~/.codex/config.toml
[mcp_servers.mlreferee]
url = "https://api.mlreferee.com/mcp".cursor/mcp.json
{
"mcpServers": {
"mlreferee": {
"url": "https://api.mlreferee.com/mcp"
}
}
}.vscode/mcp.json
{
"servers": {
"mlreferee": {
"type": "http",
"url": "https://api.mlreferee.com/mcp"
}
}
}Available tools 26
Read directly from the server with tools/list, grouped by what they act on.
If a tool disappears, we record the date.
audit
delete_audit
Delete retained inputs and results for an audit owned by this tenant.
get_audit_receipt
Read a tenant-private report fingerprint retained after report expiry or deletion. Older reports may have no record. This does not retrieve report contents.
get_audit_result
Read a completed tenant-private result and signed output manifest.
get_audit_status
Read the state of an audit owned by the authenticated tenant.
upload
delete_upload
Delete a tenant-private upload that is no longer needed.
upload_campaign_shard
Encrypt one of up to 64 private feature shards for a large campaign; retain for at least 4,200 seconds.
upload_private_dataset
Encrypt a de-identified CSV/JSON payload for a short tenant-private retention window.
upload_repair_matrix
Encrypt one complete de-identified repair matrix of up to 100 MB and 1,000 diagnostic features.
quote
quote_audit
Create a contract-bound quote for one audit credit; baseline mode and evidence cannot be substituted.
quote_campaign
Preflight all private feature shards, time a pilot, and quote credits and ETA for up to 1,000 candidates in one corrected search family.
quote_repair
Preflight and quote label-based feature repair; large jobs return a measured ETA and credit requirement.
preflight
preflight_dataset
Validate a tenant-private upload and its baseline evidence/row identity without spending a credit.
preflight_repair
Validate a tenant-private feature-repair matrix without spending a credit.
referee
get_referee_readiness
Report connection, credential, audit-credit, and submission readiness with the next action.
get_referee_usage
Read aggregate, privacy-preserving Referee activity and outcome evidence for the selected audience.
beta
get_beta_request_status
Read the authenticated tenant's beta-access request status.
credits
get_credits
Read beta or paid audit-credit availability and usage.
describe
describe_referee
Discover capabilities, privacy, and the approved public catalog.
input
get_input_schema
Get neutral column roles and validation defaults for a vertical.
record
record_audit_followup
Record whether a completed audit changed a decision or was confirmed prospectively.
redeem
redeem_beta_invite
Redeem one operator-issued invite for free beta audit credits before January 1, 2027.
repair
get_repair_schema
Get the feature-only contract for diagnosing degradation and testing bounded repairs.
request
request_beta_access
Request operator-reviewed beta access without purchasing or granting credits.
submit
submit_audit
Accept a quote and atomically reserve its disclosed audit credits.
verify
verify_audit_report
Compare a locally computed SHA-256 with the retained tenant-private receipt. Use the exact downloaded result JSON bytes (not the API envelope), or UTF-8 result.report text with representation report_text. Never copy the expected hash as the candidate: compute it from the supplied artifact. A digest match does not validate a summary, factual correctness, or absence of harm. No report is uploaded.
verticals
list_verticals
List the three supported evaluation profiles and their discovery tags.
Endpoints
| URL | Transport | State | Latency | Checked |
|---|---|---|---|---|
| https://api.mlreferee.com/mcp | streamable-http | answering | 638 ms | 5 min ago |
Alternatives to ML Referee
same job, measured the same wayLastest
by las-team
Run visual regression tests, review diffs, and manage baselines on a Lastest instance.
Optux
by optux
Create and run A/B tests, patches, split tests and feature flags on your website
F
First Dollar Test
by first-dollar-test
Base wallet-agent beta: deposit 1 USDC; PASS returns 1 USDC; terminal failure is refunded.
S
SMX by Space Markets (testnet)
by space-markets-hq
SMX (Space Markets) testnet event markets and the Space Markets Index (SMI). Base Sepolia only.
EdgeDepth Research
by edgedepth
Test crypto-perpetual claims using recorded counts, baselines, and replay.
Code Factory
by zrk222
Read-only audit evidence for security, tests, native coverage, runtime failures and repair status.
Agenthub Monitor
by nevelsk65
Renders a pipeline DAG and test-harness dashboard inline as an MCP App
ResiliReplay
by aliengineering-byte
Run bounded, deterministic MCP reliability tests and generate executable regressions.
ML Referee — questions
Answers built from our own checks of this server.
What can ML Referee do?
It exposes 26 tools, read directly from the server on our last check. Among them: delete_audit, delete_upload, describe_referee, get_audit_receipt, get_audit_result, get_audit_status and 20 more. The full list with descriptions is on this page — we take it from the server itself via tools/list, not from a README.
How MCP servers expose tools in the first place →
What is ML Referee mostly used for?
Its tools cluster around upload, audit and quote. That is what this server is built to work with — the grouping comes from the actual tool names, not from a category we assigned.
Is ML Referee working right now?
We send a real MCP handshake every 15 minutes. Over the last 24 hours 92 of 92 checks got a reply (100.0%), average response time 617 ms. The bar chart above shows every period we have measured.
How do I connect ML Referee?
Copy the ready config from this page. We generate it for Claude Code, Claude Desktop, Codex, Cursor and VS Code, each in the form that client actually reads. It is a remote server, so there is nothing to install — the client connects to the address.
Does ML Referee need an API key?
No. ML Referee completed a full MCP handshake with us as an anonymous client and listed its tools without asking for anything. All 26 of them are readable on this page. This is what we observed, not what the docs claim.
How fast is ML Referee?
It answers our handshake in 617 ms on average, which is faster than 17% of all working MCP servers we measure. That is on the slow side — worth knowing if the tool sits inside an interactive loop. The comparison comes from our own checks across the whole registry, every 15 minutes.