Crawl4ai

brettdavies/crawl4ai

Use when scraping JavaScript-heavy pages or SPAs, crawling multiple URLs concurrently, extracting structured data with reusable CSS/JSON schemas, or building automated web data pipelines. Wraps the Crawl4AI library (`crwl` CLI and Python SDK) with schema-generation patterns for LLM-free extraction. Triggers on crawl4ai, crwl, scrape JS-heavy site, scrape SPA, headless browser scrape, schema-based extraction, batch crawl, sitemap crawl, web data pipeline. SKIP when a static HTML page can be read with `defuddle` / `fetch-web` — those are faster cold-start and don't need a browser.

100k tokens

context cost

the whole folder, loaded on every use

files

ships runnable scripts

copies elsewhere

how many repositories repackaged it

stars on the repo

on the repository, not the skill itself

Install

one command, takes just this skill from the repository

npx skills add https://github.com/brettdavies/crawl4ai-skill --skill crawl4ai

How to use it

Copy the folder

Take brettdavies/crawl4ai from the repository into ~/.claude/skills for personal use, or into .claude/skills inside a project.

Check the name does not clash

The agent identifies a skill by the name field in its header. Two skills with the same name cannot sit side by side — one of them will be ignored.

Install what it needs

The instructions reference pip. Without those the skill loads but fails at the first command.