mcpbeat Sign in

Tavily Map Agent Skill

by tavily-ai

| Discover and list all URLs on a website without extracting content, via the Tavily CLI. Use this skill when the user wants to find a specific page on a large site, list all URLs, see the site structure, find where something is on a domain, or says "map the site", "find the URL for", "what pages are on", "list all pages", or "site structure". Faster than crawling — returns URLs only. Essential when you know the site but not the exact page. Combine with extract for targeted content retrieval.

796 tokens
context cost
the whole folder, loaded on every use
1
files
instructions only
0
copies elsewhere
how many repositories repackaged it
443
stars on the repo
on the repository, not the skill itself

Install

one command, takes just this skill from the repository 
npx skills add https://github.com/tavily-ai/skills --skill tavily-map

What it tells the agent to use

found in the instruction text
Bash runs shell commands — read the instruction before connecting

The instruction itself

8 sections, as written by the author

tavily map

Discover URLs on a website without extracting content. Faster than crawling.

Before running any command

If tvly is not found on PATH, install it first:

curl -fsSL https://cli.tavily.com/install.sh | bash && tvly login

Do not skip this step or fall back to other tools.

See tavily-cli for alternative install methods and auth options.

When to use

  • You need to find a specific subpage on a large site
  • You want a list of all URLs before deciding what to extract or crawl
  • Step 3 in the workflow: search → extract → map → crawl → research

Quick start

# Discover all URLs
tvly map "https://docs.example.com" --json

# With natural language filtering
tvly map "https://docs.example.com" --instructions "Find API docs and guides" --json

# Filter by path
tvly map "https://example.com" --select-paths "/blog/.*" --limit 500 --json

# Deep map
tvly map "https://example.com" --max-depth 3 --limit 200 --json

Options

| Option | Description |

|--------|-------------|

| --max-depth | Levels deep (1-5, default: 1) |

| --max-breadth | Links per page (default: 20) |

| --limit | Max URLs to discover (default: 50) |

| --instructions | Natural language guidance for URL filtering |

| --select-paths | Comma-separated regex patterns to include |

| --exclude-paths | Comma-separated regex patterns to exclude |

| --select-domains | Comma-separated regex for domains to include |

| --exclude-domains | Comma-separated regex for domains to exclude |

| --allow-external / --no-external | Include external links |

| --timeout | Max wait (10-150 seconds) |

| -o, --output | Save output to file |

| --json | Structured JSON output |

Map + Extract pattern

Use map to find the right page, then extract it. This is often more efficient than crawling an entire site:

# Step 1: Find the authentication docs
tvly map "https://docs.example.com" --instructions "authentication" --json

# Step 2: Extract the specific page you found
tvly extract "https://docs.example.com/api/authentication" --json

Tips

  • Map is URL discovery only — no content extraction. Use extract or crawl for content.
  • Map + extract beats crawl when you only need a few specific pages from a large site.
  • Use --instructions for semantic filtering when path patterns aren't enough.

See also

  • tavily-extract — extract content from URLs you discover
  • tavily-crawl — bulk extract when you need many pages

Other skills for the same job

different authors, same section of the catalogue
Firecrawl Build Onboarding
by firecrawl
vendor ×1

Get Firecrawl credentials and SDK setup into a project. Use when an application needs `FIRECRAWL_API_KEY`, when an agent should add Firecrawl to `.env`, when the user wants to authenticate Firecrawl for app code, or when choosing the first SDK and docs for a new Firecrawl integration. This skill includes its own browser auth flow, so it does not depend on the website onboarding skill.

2k tokens
Facebook Ads Library Search
by browser-act

Searches Meta Ad Library (Facebook/Instagram/WhatsApp ads) by keyword or Facebook page ID and extracts ad details including creatives, copy, CTA, publisher platforms, spend, impressions, reach estimates, and page transparency info. Use when user mentions Meta Ad Library, Facebook ads scraper, Instagram ads data, FB ad library, search Facebook ads, get ads from a Facebook page, scrape Meta ads, Facebook advertising data, ad creative extraction, competitor ads analysis, brand ads monitoring, Meta advertising transparency, political ads Facebook, housing ads Facebook, view all ads from a page, facebook ads search, fb ads library api, facebook ad archive, instagram ad data, get ad creatives.

4k tokens scripts
Linkedin Job Scraper
by gooseworks-ai

> Scrapes LinkedIn job postings using the JobSpy library (python-jobspy). Use this skill whenever the user wants to find jobs on LinkedIn, search for open roles, pull job listings, build a job pipeline, source job targets for GTM research, or monitor hiring signals. Even if the user just says "find me some jobs" or "what roles is [company] hiring for", use this skill. It runs a local Python script that outputs a CSV of job postings with title, company, location, salary, job type, description, and direct URLs.

2k tokens scripts
Indexing
by kostja94

When the user wants to fix indexing issues from Search Console, use noindex, or implement Google Indexing API. Also use when the user mentions "fix indexing," "not indexed," "Crawled - currently not indexed," "discovered - currently not indexed," "index coverage," "noindex," "noindex tag," "pages not indexed," "why not indexed," "request indexing," or "Google Indexing API." For sitemap, use xml-sitemap.

2k tokens
X Twitter Scraper
by aiskillstore

Use when the user needs X (Twitter) data or confirmation-gated X actions through Xquik: tweet search, user lookup, follower extraction, media download, monitoring, webhooks, MCP, SDKs, posting, likes, DMs, and profile updates. Requires a Xquik API key. Never ask for X login material.

62k tokens
Apify Public Registries
by apify
vendor

Look up official company data from European public registries across 11 countries/regions (CZ, SK, PL, DE, UK, NL, RO, HR, SE + EU-level + ESG). Covers company registration, ownership, financial filings, VAT status, ESG data. Use when the user asks to "look up a company", "check registry", "find company info", "look up IČO/KRS/LEI/CRN", "company due diligence", "check VAT status", "find ownership structure", or needs official data from European registries. Reads tracked companies from data/companies.json. Some lookups use Python scripts (stdlib), some fall back to Apify actors for scraping-based registries.

39k tokens scripts
X Twitter Scraper
by lingxling

Use Xquik for X data workflows: tweet search, user lookup, follower export, media downloads, monitors, webhooks, REST API, MCP, SDK setup, and approval-gated account actions.

2k tokens
X Twitter Scraper
by Xquik-dev

Use Xquik for X/Twitter REST, MCP, SDKs, search, filtered exports, monitoring & approved publishing. Not affiliated with X Corp. Trigger for X API alternatives, pricing comparisons, tweet search, user lookup, timelines, follower exports, media, webhooks, bulk extraction, giveaways, or MCP setup. Read-only by default. Require explicit approval for writes, private reads, monitors, webhooks & metered bulk jobs.

62k tokens

How to use it

Copy the folder

Take tavily-ai/tavily-map from the repository into ~/.claude/skills for personal use, or into .claude/skills inside a project.

Check the name does not clash

The agent identifies a skill by the name field in its header. Two skills with the same name cannot sit side by side — one of them will be ignored.