Deepcrawl runs on your own machine — the client starts it, so there is no endpoint to ping. Last commit 28 Oct 2026.
Clone websites, scrape pages to markdown and crawl whole sites. A free Firecrawl alternative.
Today is the operative word: we check Deepcrawl every 15 minutes and re-read its code on every release. Watch it and you find out the day that stops being true.
Web pages and whole sites to clean Markdown for RAG and AI agents. Crawls links and sitemaps.
Scrape, crawl, and map websites to Markdown or JSON via local CLI.
Scrape, crawl, and map websites to Markdown or JSON via local CLI.
Crawl and scrape websites to clean markdown, and track what changed between crawls.
AI-native web scraper: scrape, crawl and map any site to clean markdown over stdio. MIT-licensed.
Web pages to Markdown, metadata and small crawls for agents. Free daily calls, then USDC x402/MPP.
Web tools for AI agents: scrape pages to Markdown, audit SEO, detect tech stacks, check sitemaps
Web scraping for agents: scrape, crawl, map, search and extract pages as clean markdown or JSON.
Answers built from our own checks of this server.