Web Scraping Claude Skills
350 web scraping skills from 106 authors. They pull data out of other people's sites and make sense of the markup. Half of them fit into 1 816 tokens or less — that is what one costs your context window when the agent loads it. 144 ship runnable scripts rather than instructions alone. 8 of them cannot work without an MCP server, most often rube. We also found 39 copies of these same skills sitting in other people's repositories — counted once here, not 39 times.
350 unique 106 authors 2 updated this month 27 from vendors
Use Xquik for X data and confirmation-gated X actions: tweet search, user lookup, follower export, media download, monitors, webhooks, MCP, and SDK workflows.
构建一个全自动化的AI驱动数据收集代理，适用于任何公共来源——招聘网站、价格信息、新闻、GitHub、体育赛事等任何内容。按计划进行抓取，使用免费LLM（Gemini Flash）丰富数据，将结果存储在Notion/Sheets/Supabase中，并从用户反馈中学习。完全免费在GitHub Actions上运行。适用于用户希望自动监控、收集或跟踪任何公共数据的场景。
Python library for scraping Twitter/X data using GraphQL API with account rotation and session management. Use when extracting tweets, user profiles, followers, trends, or building social media monitoring tools.
Fetch web content, take screenshots, extract structured data, search the web, and crawl documentation sites. Use when the user needs current web information, asks to scrape a URL, wants a screenshot, needs to extract specific data from a page, or wants to learn about a framework or library.
多功能网页抓取和数据提取工具，支持同步抓取、搜索、网站地图获取和异步爬取
When the user wants to implement, audit, or validate structured data (schema markup) on their website. Use when the user mentions 'structured data,' 'schema.org,' 'JSON-LD,' 'rich results,' 'rich snippets,' 'schema markup,' 'FAQ schema,' 'Product schema,' 'HowTo schema,' or 'structured data errors in Search Console.' Also use when someone asks why their content isn't showing rich results or wants to improve AI search visibility. NOT for general SEO audits (use seo-audit) or technical SEO crawl issues (use site-architecture).
Develop, debug, and deploy Apify Actors - serverless cloud programs for web scraping, automation, and data processing. Use when creating new Actors, modifying existing ones, or troubleshooting Actor code.
Scrape e-commerce data for pricing intelligence, customer reviews, and seller discovery across Amazon, Walmart, eBay, IKEA, and 50+ marketplaces. Use when user asks to monitor prices, track competitors, analyze reviews, research products, or find sellers.
Generates B2B/B2C leads by scraping Google Maps, websites, Instagram, TikTok, Facebook, LinkedIn, YouTube, and Google Search. Use when user asks to find leads, prospects, businesses, build lead lists, enrich contacts, or scrape profiles for sales outreach.
Universal AI-powered web scraper for any platform. Scrape data from Instagram, Facebook, TikTok, YouTube, Google Maps, Google Search, Google Trends, Booking.com, and TripAdvisor. Use for lead generation, brand monitoring, competitor analysis, influencer discovery, trend research, content analytics, audience analysis, or any data extraction task.
Build and scale partner ecosystems that drive revenue and platform adoption. Use when building partner programs from scratch, tiering partnerships, managing co-marketing, making build-vs-partner decisions, or structuring crawl-walk-run partner deployment.
Twitter 推文爬取器 - 指定用户名爬取推文，保存为 Markdown 格式，支持自定义数量和字段
Use this skill for web search, extraction, mapping, crawling, and research via Tavily’s REST API when web searches are needed and no built-in tool is available, or when Tavily’s LLM-friendly format is beneficial.
Scan ICE-Crawler extraction logs, pick promising algorithms/tools, and emit skill creation proposals (name, scope, source files, next steps).
Automate ICE-Crawler ingestion → registry update → proposal stub creation. Use when you want a full harvest loop that ends with a ready-to-review skill idea.
Run ICE-Crawler’s Frost→Glacier→Crystal pipeline to ingest repositories safely, emit bounded artifact bundles, and hand off sealed fossils for downstream agents.
| Build a complete web scraping Actor with Crawlee and deploy to Apify. data extraction, dataset output, and platform deployment. "crawlee scraper", "apify main workflow".
| Deploy Apify Actors and integrate scraping into external applications. Use when deploying Actors to the platform, integrating Actor results into web apps, or connecting Apify with external services. "apify production deploy", "integrate apify results", "apify API endpoint".
| Run your first Apify Actor and retrieve results via apify-client. Use when starting a new Apify integration, testing connectivity, or learning the Actor call/dataset retrieval pattern. "apify quick start", "first apify scrape".
| Use when Actors are slow, consuming too much memory, or being blocked by target sites. "crawlee concurrency", "apify memory tuning", "scraper performance".
| Production-grade architecture patterns for Apify-powered applications. Use when designing scraping infrastructure, building multi-Actor pipelines, or integrating Apify into a larger system architecture. "apify project structure", "scraping architecture", "apify system design".
| Implement Apify webhooks for Actor run notifications and event-driven pipelines. Use when setting up run completion alerts, building event-driven scraping pipelines, or configuring ad-hoc webhooks for individual runs. "apify run succeeded webhook", "apify ad-hoc webhook".
| Scrape structured data with Bright Data Scraping Browser using Playwright/Puppeteer. Use when scraping JavaScript-rendered pages, SPAs, or sites requiring browser interaction. Trigger with phrases like "brightdata scraping browser", "brightdata playwright", "brightdata puppeteer", "scrape SPA with brightdata", "browser scraping".
| Research competitor YouTube channels, niches, and trending topics for your content strategy. Use this skill whenever the user says "research channels", "analyze competitors", "find trending topics", "niche analysis", "competitive research", "what are other creators doing", "scrape YouTube channels", or wants to understand the competitive landscape for a specific tool or topic area. Use when working with yt research. Trigger with 'yt', 'research'.
Firecrawl v2.5 API for web scraping/crawling to LLM-ready markdown. Use for site extraction, dynamic content, or encountering JavaScript rendering, bot detection, content loading errors.
Automated cold email pipeline. Finds target companies, enriches contacts, scrapes websites, and generates personalized cold emails using AI. One API call does it all: search → enrich → scrape → write.
Find the papers that answer a research query with Firecrawl Research, using semantic search, semantic and structural expansion, and in-body verification. Always use this skill for any literature-finding / paper-retrieval task — single-paper lookups or full multi-paper sets.
Integrate Firecrawl into application code whenever a product, agent, or workflow needs web data inside the app: web search, live search results, page scraping, structured extraction, or browser interaction. Use when building any feature that needs data from the web in code, even if the user does not mention Firecrawl explicitly and only describes wanting web data, website content, search, scraping, or interaction in an application. Trigger for Firecrawl requests, "fire girl" shorthand, and generic app-level web-data needs that should map to `/scrape`, `/search`, or `/interact`. Do not use this skill for one-off terminal-only web tasks during the current session; use `firecrawl/cli` for those.
Looks up a Turkish academic's official YOKSIS-backed profile, current affiliation, unvan (academic title), publications, research projects, and supervised theses on the YOK Akademik portal (akademik.yok.gov.tr/AkademikArama/), a server-rendered JSP app with no public JSON API, by scraping its verified endpoints (AkademisyenArama POST search, viewAuthor.jsp profile, AkademisyenProjeBilgileri, AkademisyenYonTezBilgileri) keyed by an opaque authorId, with Turkish-character (Iı Şş Ğğ Çç Öö Üü) normalization for name matching. Use when the request is to verify a Turkish academic's current institution, find a researcher on YOK Akademik, confirm affiliation for authorship or a recommendation letter, or list someone's supervised theses or projects. For admission statistics (kontenjan, taban puan) use alterlab-yokatlas; for the national thesis full-text archive use alterlab-yok-tez; for publication metadata enrichment use alterlab-openalex. Part of the AlterLab Academic Skills suite.
This skill should be used when the user asks to "research web data", "scrape websites", "extract web data", "perform market research", "analyze competitors", "monitor prices", "collect product information", "search and analyze web content", or mentions Bright Data MCP, web scraping, web data extraction, or automated research. Provides comprehensive web research workflows using Bright Data MCP tools including search, scraping, extraction, and browser automation capabilities.
| web-data agents, scrapes into Delta tables, and produces an AI/BI dashboard and/or a deployed Databricks App — a table → dashboard → app workflow, for production data products or quick demos. Use whenever a request pairs live or scraped web data WITH a Databricks destination — e.g. "scrape Amazon/Walmart prices into a Delta table and build a dashboard", "load Zillow/Instagram/Maps/search results into Databricks and build a dashboard or app", "showcase Nimble + Databricks to a prospect". Prefer it over nimble-web-expert or competitor-intel when the data lands in Databricks. Do NOT use for one-off web fetches or CSV exports with no Databricks destination — use nimble-web-expert instead. Do NOT use for competitor or company research briefings — use competitor-intel or company-deep-dive instead. Do NOT use for generic Databricks work with no Nimble/web-data angle — use the official databricks-* skills instead.
| Extracts structured practitioner data from healthcare practice websites. Returns names, credentials, specialties, contact info, and education for every provider on a practice's site. Use when user asks to extract, pull, or list doctors, providers, or staff from", "who are the providers at", "build a provider database", "list all doctors at", "scrape the team page", "get practitioner data from". Accepts practice URLs (pasted, CSV, Google Sheet) or discovers practices via Google Maps when given specialty + location. Single sites or 100+ URLs. Do NOT use for filling data gaps — use healthcare-providers-enrich instead. Do NOT use for credential validation — use healthcare-providers-verify instead. Do NOT use for discovering practices — use market-finder or local-places instead. Do NOT use for general extraction — use nimble-web-expert instead.
| Reference for nimble crawl command. Load when bulk-crawling many pages asynchronously.
| Reference for nimble extract command. Load when fetching URLs or scraping pages. parser schemas, geo targeting, async, parallelization.
| Reference for nimble map command. Load when discovering URLs on a site before bulk extraction. response structure {links[].url/title/description}, map→filter→extract pattern, map vs crawl comparison.
| Reference for nimble tasks and batches commands. Load when polling async task status, tracking batch progress, or fetching results. crawl (per-page tasks), search async, map async.
Use when needing to scrape documentation websites into markdown for AI context. Triggers on "scrape docs", "download documentation", "get docs for [library]", or creating local copies of online documentation. CRITICAL - always analyze sitemap first before scraping.
GTM prospecting, enrichment, outreach, scoring, CSVs, and plays. Discovery: deepline-pre-research. Providers: adyntel,ai_ark,allegrow,apify,attention,attio,aviato,bettercontact,bloomberry,bluesky,bounceban,browserbase,builtwith,cloudflare,contactout,crustdata,crustdata-v2,crustdata-v3,customer_db,dataforseo,datagma,deepline_ip_to_company,deepline_native,deeplineagent,discolike,dropleads,emailbison,emailguard,enformion,exa,findymail,firecrawl,firmable,forager,fullenrich,generic_http,gong,google_ads_audiences,google_workspace,hackernews,heyreach,hubspot,hunter,icypeas,instantly,intercom,ipqs,kernel,leadmagic,lemlist,limadata,linkedin_ads_audiences,linkedin_scraper,lusha,meta_audiences,nooks,openmart,opensosdata,openwebninja,outreach,parallel,peopledatalabs,podscan,predictleads,prospeo,rocketreach,salesforce,salesforge,scrapecreators,sentrion,serper,slack,smartlead,snowflake,sumble,theirstack,trestle,twitterapi,upcell,wiza,wizleads,zerobounce.
> Conduct deep OSINT research on individuals. Build full digital footprint, psychoprofile (MBTI/Big Five), career history, social graph with confidence scores. Recursive self-evaluation until completeness threshold is met. Includes internal intelligence (Telegram history, email, vault contacts) before going external. "разведка", "due diligence", "background check", "digital footprint", "найди всё про", "собери информацию", "кто это", "профиль человека". market research, content generation, or general web scraping tasks.
业务诊断助手适合市场营销、运营、产品、销售在用户提出“这件事该怎么做”这类问题，需要快速拆解目标、判断重点并形成可执行结果时使用，帮助基于输入材料生成摘要、诊断结论、行动建议和可复用交付物。
业务诊断助手适合市场营销、运营、产品、销售在用户提出“这件事该怎么做”这类问题，需要快速拆解目标、判断重点并形成可执行结果时使用，帮助基于输入材料生成摘要、诊断结论、行动建议和可复用交付物。
业务诊断助手适合市场营销、运营、产品、销售在用户提出“这件事该怎么做”这类问题，需要快速拆解目标、判断重点并形成可执行结果时使用，帮助基于输入材料生成摘要、诊断结论、行动建议和可复用交付物。
业务诊断助手适合市场营销、运营、产品、销售在用户提出“这件事该怎么做”这类问题，需要快速拆解目标、判断重点并形成可执行结果时使用，帮助基于输入材料生成摘要、诊断结论、行动建议和可复用交付物。
Scrape web pages and save as HTML or Markdown (with text and images). Minimal dependencies - only requests and beautifulsoup4. Use when the user provides a URL and wants to download/archive the content locally.
Build structured ICP documentation by scraping case studies, testimonials, and solutions pages from your website. Produces firmographics, champion + economic buyer personas, pain points, decision criteria, anti-ICP, customer proof points, and voice-of-customer synthesis. Writes to marketing/icp/ICP.md as the canonical ICP every downstream skill reads. Triggers - "icp research", "ideal customer profile", "persona research", "customer segments", "who are our customers", "build ICP
Use when scraping JavaScript-heavy pages or SPAs, crawling multiple URLs concurrently, extracting structured data with reusable CSS/JSON schemas, or building automated web data pipelines. Wraps the Crawl4AI library (`crwl` CLI and Python SDK) with schema-generation patterns for LLM-free extraction. Triggers on crawl4ai, crwl, scrape JS-heavy site, scrape SPA, headless browser scrape, schema-based extraction, batch crawl, sitemap crawl, web data pipeline. SKIP when a static HTML page can be read with `defuddle` / `fetch-web` — those are faster cold-start and don't need a browser.
World Cup match prediction and pre-match intel for football (soccer) fans: predict who will win and a likely scoreline, backed by structured, verifiable evidence -- player profiles, squad dossiers, current form, head-to-head records, and past-tournament history -- using AnyCap web search and crawl. Weighs FIFA ranking, recent form, head-to-head, and key players, plus any user-defined dimensions (home advantage, injuries, rest, weather, vibes), then makes a clearly-labeled for-fun call with a confidence level and a mandatory disclaimer. Built on the 'search less, not faster' strategy: narrow the search space first, retrieve only what matters, then verify against authoritative sources. Use when a fan wants a World Cup match prediction, predicted score, or who-will-win call; wants a player's profile, bio, form, or stats; wants a full World Cup squad or national-team lineup table; wants to size up two teams before a match; or wants head-to-head and past World Cup context. Trigger on: World Cup prediction, predict the score, who will win, match prediction, predicted scoreline, World Cup, World Cup squad, national team lineup, pre-match analysis, matchup, player form, footballer stats, player profile, head-to-head, or scout a team.
Build production web scraping on Bedrock AgentCore Browser — connect Playwright over signed CDP WebSocket, drive extraction with an LLM agent over fixed tool primitives (navigate, scroll, extract-by-selector, screenshot), reuse login state via Browser Profiles with a DCV live-view login flow, detect login walls without false positives, and route through an external proxy. Use when scraping dynamic or login-gated sites (X/Twitter, Reddit, Instagram, YouTube, forums) with AgentCore Browser, when scraped results come back empty on lazy-loaded pages, when Google SSO fails silently in the cloud browser, or when a target site blocks AWS egress IPs.