Firecrawl Crawl

firecrawl/firecrawl-crawl

| Bulk extract content from an entire website or site section. Use this skill when the user wants to crawl a site, extract all pages from a docs section, bulk-scrape multiple pages following links, or says "crawl", "get all the pages", "extract everything under /docs", "bulk extract", or needs content from many pages on the same site. Handles depth limits, path filtering, and concurrent extraction.

This is a copy. The original lives at aaaaqwq/firecrawl-crawl.

676 tokens

context cost

the whole folder, loaded on every use

files

instructions only

copies elsewhere

how many repositories repackaged it

stars on the repo

on the repository, not the skill itself

Install

one command, takes just this skill from the repository

npx skills add https://github.com/firecrawl/cli --skill firecrawl-crawl

What it tells the agent to use

found in the instruction text

Bash runs shell commands — read the instruction before connecting

The instruction itself

6 sections, as written by the author

firecrawl crawl

Bulk extract content from a website. Crawls pages following links up to a depth/limit.

When to use

You need content from many pages on a site (e.g., all /docs/)
You want to extract an entire site section
Step 4 in the workflow escalation pattern: search → scrape → map → crawl → interact

Quick start

# Crawl a docs section
firecrawl crawl "<url>" --include-paths /docs --limit 50 --wait -o .firecrawl/crawl.json

# Full crawl with depth limit
firecrawl crawl "<url>" --max-depth 3 --wait --progress -o .firecrawl/crawl.json

# Check status of a running crawl
firecrawl crawl <job-id>

Options

| Option | Description |

| ------------------------- | ------------------------------------------- |

| --wait | Wait for crawl to complete before returning |

| --progress | Show progress while waiting |

| --limit <n> | Max pages to crawl |

| --max-depth <n> | Max link depth to follow |

| --include-paths <paths> | Only crawl URLs matching these paths |

| --exclude-paths <paths> | Skip URLs matching these paths |

| --delay <ms> | Delay between requests |

| --max-concurrency <n> | Max parallel crawl workers |

| --pretty | Pretty print JSON output |

| -o, --output <path> | Output file path |

Tips

Always use --wait when you need the results immediately. Without it, crawl returns a job ID for async polling.
Use --include-paths to scope the crawl — don't crawl an entire site when you only need one section.
Crawl consumes credits per page. Check firecrawl credit-usage before large crawls.

Repackaged in 2 other repositories

same content, different owner

aAAaqwq/AGI-Super-Team open on GitHub →

aiskillstore/marketplace open on GitHub →

How to use it

Copy the folder

Take firecrawl/firecrawl-crawl from the repository into ~/.claude/skills for personal use, or into .claude/skills inside a project.

Check the name does not clash

The agent identifies a skill by the name field in its header. Two skills with the same name cannot sit side by side — one of them will be ignored.

Install what it needs

The instructions reference npx. Without those the skill loads but fails at the first command.