RAGScore runs on your own machine — the client starts it, so there is no endpoint to ping. 126 installs a week from pypi. Last commit 25 May 2026.
Generate QA datasets & evaluate RAG systems with failure diagnosis. Any LLM.
Nothing serious here today
Today is the operative word: we check RAGScore every 15 minutes and re-read its code on every release. Watch it and you find out the day that stops being true.
Connect this server
This server runs on your own machine — install it with the package manager and the client starts it for you. Package name taken from the official registry entry.
claude mcp add ragscore -- uvx ragscore{
"mcpServers": {
"ragscore": {
"command": "uvx",
"args": [
"ragscore"
]
}
}
}[mcp_servers.ragscore]
command = "uvx"
args = ["ragscore"]{
"mcpServers": {
"ragscore": {
"command": "uvx",
"args": [
"ragscore"
]
}
}
}{
"servers": {
"ragscore": {
"type": "stdio",
"command": "uvx",
"args": [
"ragscore"
]
}
}
}Optional settings from the registry entry, left out of the block above:
OPENAI_API_KEY (OpenAI API key (if using OpenAI provider)), ANTHROPIC_API_KEY (Anthropic API key (if using Anthropic provider)).
Alternatives to RAGScore
same job, measured the same wayTrace, evaluate, and optimize your LLM, RAG, and agent apps with Noveum observability.
Version-controlled golden datasets and RAG evaluation, no API key needed.
Evaluate RAG retrieval, agent tool calls and prompt regressions with deterministic metrics.
SysML v2 systems engineering: generate, verify, simulate and export system models from text.
Local FAISS vector database for RAG with document ingestion, semantic search, and MCP prompts.
Crawl any documentation site and return heading-bounded, LLM-ready chunks with source metadata.
AI assistant with RAG, web search, filesystem and memory. MCP sub-agent for Claude Code.
Reasoning-based RAG system for chatting with long PDFs. Supports local and online files.
RAGScore — questions
Answers built from our own checks of this server.