Commoncrawl is answering right now. Last checked 4 min ago. 442 installs a week from npm. It exposes 47 tools. Last commit 31 Oct 2026.
Common Crawl's public web archive — find every time a URL was crawled since 2008 and read the…
Every tool that appeared, vanished or quietly changed what it asks for. Recorded since 3 October 2026. No other catalogue keeps this.
Today is the operative word: we check Commoncrawl every 15 minutes and re-read its code on every release. Watch it and you find out the day that stops being true.
Endpoint below is the one we actually reach during checks — not the one copied from a README. Last verified 4 min ago.
claude mcp add commoncrawl --transport http https://gateway.pipeworx.io/commoncrawl/mcp
https://gateway.pipeworx.io/commoncrawl/mcp
[mcp_servers.commoncrawl]
url = "https://gateway.pipeworx.io/commoncrawl/mcp"
{
"mcpServers": {
"commoncrawl": {
"url": "https://gateway.pipeworx.io/commoncrawl/mcp"
}
}
}
{
"servers": {
"commoncrawl": {
"type": "http",
"url": "https://gateway.pipeworx.io/commoncrawl/mcp"
}
}
}
Read directly from the server with tools/list, grouped by what they act on.
If a tool disappears, we record the date.
polymarket_arbitrage
polymarket_edge_tracker
polymarket_edges
polymarket_fill_risk
polymarket_kalshi_spread
ask_pipeworx
ask_pipeworx_beta
ask_pipeworx_grounded
commoncrawl_crawls
commoncrawl_fetch_record
commoncrawl_index_search
archive_compare
archive_evidence
company_facts
company_register_search
pipeworx_feedback
pipeworx_trending
recent_alerts
recent_changes
resolution_audit
resolution_diff
scan_competitor_ai_presence
scan_dependency
updown_divergence_rate
updown_venue_compare
bet_research
classify_row
compare_entities
deep_research
discover_tools
entity_profile
forget
generate_llms_txt
kalshi_weather_edge
prediction_calibration
recall
release_calendar_markets
remember
research_row
resolve_entity
subscribe
list_subscriptions
suggest_questions
unsubscribe
validate_claim
ai_visibility_check
search_within
| URL | Transport | State | Latency | Checked |
|---|---|---|---|---|
| https://gateway.pipeworx.io/commoncrawl/mcp | streamable-http | answering | 188 ms | 4 min ago |
What search and AI crawlers extract from any public URL. Five of nine tools need no token.
Read web pages as markdown, search the web, list a site's URLs and run a real Chrome.
CrawlGraph MCP — backlink intelligence on the public Common Crawl webgraph (4.4B edges, 120M…
Read, map and crawl the web, and drive a real browser. Pay per call, no key, no signup.
Read, crawl and act on websites, signed in as the user, and turn any site into an API.
Scrape, search and extract the public web. 3,000 free credits at https://vera.waggle.ae/signup
Web search, URL content extraction to Markdown, site mapping, and recursive web crawler.
Public website health scan, AI-crawler and structured-data readiness, and a local presence score.
Answers built from our own checks of this server.