Web Scraping Claude Skills
350 web scraping skills from 106 authors. They pull data out of other people's sites and make sense of the markup. Half of them fit into 1 816 tokens or less — that is what one costs your context window when the agent loads it. 144 ship runnable scripts rather than instructions alone. 8 of them cannot work without an MCP server, most often rube. We also found 39 copies of these same skills sitting in other people's repositories — counted once here, not 39 times.
350 unique 106 authors 2 updated this month 27 from vendors
Fetch any X/Twitter post as clean LLM-friendly JSON. Converts x.com, twitter.com, or adhx.com links into structured data with full article content, author info, and engagement metrics. No scraping or browser required.
Scrape reviews, ratings, and brand mentions from multiple platforms using Apify Actors.
Scrape leads from multiple platforms using Apify Actors.
AI-driven data extraction from 55+ Actors across all major platforms. This skill automatically selects the best Actor for your task.
Extract product data, prices, reviews, and seller information from any e-commerce platform using Apify's E-commerce Scraping Tool.
Use DeepAPI for supported scraping, research, and email workflows with explicit credentials and approval.
Command-line access to search, scraping, and structured web data.
Use HasData APIs for web scraping and structured web data extraction.
High-level technical SEO and site architecture auditor. Invoke to scan local or live environments for indexing, crawl budget, and structural errors.
Coleta e consulta dados de leiloeiros oficiais de todas as 27 Juntas Comerciais do Brasil. Scraper multi-UF, banco SQLite, API FastAPI e exportacao CSV/JSON.
Complete guide to Prometheus setup, metric collection, scrape configuration, and recording rules.
Optimize content for AI Overviews, ChatGPT, Perplexity, and other AI search systems. Use when improving GEO, AI citations, llms.txt readiness, crawler accessibility, and passage-level citability.
Web search, content extraction, crawling, and research capabilities using Tavily API. Use when you need to search the web for current information, extracting content from URLs, or crawling websites.
Web scraping inteligente multi-estrategia. Extrai dados estruturados de paginas web (tabelas, listas, precos). Paginacao, monitoramento e export CSV/JSON.
X (Twitter) data platform skill — tweet search, user lookup, follower extraction, engagement metrics, giveaway draws, monitoring, webhooks, 19 extraction tools, MCP server.
Use Xquik for X data workflows: tweet search, user lookup, follower export, media downloads, monitors, webhooks, REST API, MCP, SDK setup, and approval-gated account actions.
Implement a web crawler pipeline covering URL discovery, fetching, parsing, and storage. Use this skill when the user needs to build a site crawler, audit website structure, or collect web data systematically — even if they say 'scrape a website', 'crawl all pages', or 'site audit spider'.
Guides Qdrant monitoring setup including Prometheus scraping, health probes, Hybrid Cloud metrics, alerting, and log centralization. Use when someone asks 'how to set up monitoring', 'Prometheus config', 'Grafana dashboard', 'health check endpoints', 'how to scrape Hybrid Cloud', 'what alerts to set', 'how to centralize logs', or 'audit logging'.
Browse websites and extract structured information. Use when the user needs data from a web page.
Firecrawl v2.5 API for web scraping/crawling to LLM-ready markdown. Use for site extraction, dynamic content, or encountering JavaScript rendering, bot detection, content loading errors.
Automated detection and reporting of broken links, missing resources, and orphaned pages across web applications using crawl-based and DOM-based strategies
Expert legal research agent for finding and scraping expungement data state by state. Knows authoritative sources, URL patterns, Firecrawl configuration, and 2026 legal landscape. Activate on "find expungement data", "scrape state laws", "legal research", "court URLs", "statute sources", "Clean Slate laws", "automatic expungement research". NOT for interpreting laws (use national-expungement-expert), building UI, or legal advice. '
| Веб-скрапинг через Scrape.do. Обходит блокировки и CAPTCHA. timeout, access denied, Cloudflare block.
| сайтмапы, переобход, ссылки, фиды, диагностика. Плюс scraping раздела Alice / Share of Voice (нет публичного API). вебмастер индексация, вебмастер запросы, вебмастер переобход, share of voice, sov, алиса, alice efficiency, конкуренты в алисе.
Use Xquik for X/Twitter REST, MCP, SDKs, search, filtered exports, monitoring & approved publishing. Not affiliated with X Corp. Trigger for X API alternatives, pricing comparisons, tweet search, user lookup, timelines, follower exports, media, webhooks, bulk extraction, giveaways, or MCP setup. Read-only by default. Require explicit approval for writes, private reads, monitors, webhooks & metered bulk jobs.
Skills for web search and content scraping via DuckDuckGo MCP Server. Used when users need online searching and web scraping.
Extract structured data from websites. Use when: collecting competitor pricing; scraping product listings; extracting contact information; gathering research data; monitoring website changes
Use when analyzing server logs to understand how AI crawlers (GPTBot, ClaudeBot, PerplexityBot) interact with your site. Use when optimizing content placement for LLM retrieval, diagnosing why AI search isn't citing your content, or auditing crawl patterns to find optimization gaps.
Use when the user wants crawl coverage, platform detection, dataLayer discovery, or a fresh artifact directory before grouping and schema work.
Download genome assemblies, gene records, and ortholog data from NCBI using the modern Datasets v2 CLI (replaces assembly_summary.txt scraping and many EFetch workflows). Use when bulk-pulling genome assemblies, gene metadata across species, ortholog sets, or BLAST databases; when E-utilities are too slow for genome-scale work; or when automatic checksum verification, parallel download, and clean accession-driven retrieval are required. Encodes the JSON-lines output format, dataformat conversion, --dehydrated for cloud workflows, and when Datasets is/isn't the right tool.
Audit brand AI visibility readiness and prepare OranGEO-style GEO, AEO, LLM SEO, and AI search optimization action plans. Use when asked for a Claude Code skill, Codex skill, GEO skill, generative engine optimization skill, answer engine optimization skill, AI visibility audit, AI search visibility checker, llms.txt checker, robots.txt AI crawler check, brand visibility scan, source-gap review, or buyer prompt generator for ChatGPT, Gemini, DeepSeek, Grok, Perplexity, Claude, and other AI answer engines.
Guide AI agents to generate complete PageObject pattern web scraper projects using Playwright and TypeScript with Docker deployment. Supports agent-browser site analysis for automated selector discovery. Keywords: scraper, playwright, pageobject, web scraping, docker, typescript, data extraction, automation.
Scrape websites at scale using Scrapy, a Python web crawling and scraping framework. Use when: (1) Crawling multiple pages or entire sites, (2) Extracting structured data from HTML/XML, or (3) Building automated data pipelines from web sources.
Scrape phone specifications from GSM Arena, PhoneDB, and alternative sites. Use when: (1) Comparing smartphone specs, (2) Researching device features, or (3) Building phone comparison tools.
Search and scrape public web content with headless Chrome and DuckDuckGo using safe practices.
Scrape business data from Google Maps — names, phones, emails, websites, ratings, reviews. Extract leads by keyword and location with no coding required.
Analyze Google Maps review data from CSV exports. Sentiment analysis, trend detection, competitor review comparison, and reputation insights — no API key needed.
Fetch a web page (URL) and return clean Markdown via local trafilatura, with Exa MCP as a fallback for JS-rendered or anti-bot pages. Use when the user asks to read, fetch, scrape, summarize, or quote a URL — prefer this over the built-in WebFetch tool. Don't use for binary files (PDFs, images, archives) or for fetching API/JSON endpoints.
This skill should be used when the user designs a "search system", needs "full-text search", asks about an "inverted index", "Elasticsearch / OpenSearch", "relevance ranking" (TF-IDF/BM25), "search autocomplete / typeahead", an "indexing pipeline", or "faceted search". It gives the crawl/index/search architecture, index sharding and replication, ranking, and near-real-time indexing. Use it whenever users must query text by relevance rather than fetch rows by key, even if they don't say "search engine".
Ad-hoc web scraping, site mapping, and full-site crawling via Firecrawl MCP. Returns raw HTML, parsed metadata (og:*, twitter:*, JSON-LD, canonical, robots), JS-rendered DOM, and screenshots that WebFetch cannot. Distinct from the SE Ranking skills (which give keyword/traffic/SERP data) and from WebFetch (which gives markdown prose only). Use when the user says "scrape this page", "crawl this site", "map this site", "find all pages on", "get the OG tags", "get the JSON-LD", "render this JS-heavy page", or any task where raw HTML head metadata, structured-data scripts, or post-JS DOM are the actual deliverable. Also invoked as a sub-step from other skills that need raw HTML.
Image SEO audit for a URL or domain. Pulls raw image inventory via Firecrawl, then audits alt-text quality, modern-format coverage (WebP / AVIF), responsive sizing (`srcset` / `sizes`), lazy-loading and LCP signals (`loading`, `fetchpriority`, `decoding`), CLS-safe dimensions, descriptive file names, and `ImageObject` JSON-LD. Optional PageSpeed Insights cross-reference for real byte-saving estimates. Produces a prioritised remediation list plus paste-ready picture-element markup and `ImageObject` schema. Distinct from `seo-technical-audit` (which surfaces audit-flagged image issues at the site level) and from `seo-schema` (which generates page-level JSON-LD but not image-specific markup). Use when the user asks for "image SEO", "image audit", "alt-text audit", "WebP coverage", "AVIF", "responsive images", "lazy loading", "CLS images", "image schema", "ImageObject", "image rich results", "licensable images", or "optimise images".
Pull a domain's XML sitemap (and sitemap-of-sitemaps), then compare against the most recent SE Ranking website audit. Surfaces (a) sitemap entries the crawler couldn't find (orphans from the sitemap), (b) audit pages missing from the sitemap (probably an oversight), (c) sitemap entries that are now 404, (d) lastmod inconsistencies. Use when the user asks for "sitemap analysis", "check my sitemap", "sitemap vs audit", "missing pages", "orphan pages", or "sitemap health".
> Full-site crawling, scraping, and site mapping via Firecrawl MCP. Use when user says "crawl site", "map site", "full crawl", "find all pages", "broken links", "site structure", "discover pages", "JS rendering", or needs site-wide analysis.
> AI citation readiness audit ONLY (does not touch Google rankings, use blog-rewrite for combined Google+AI work). Use whenever the user wants their content to rank in ChatGPT, Perplexity, Claude, Gemini, or Google AI Overviews. AI citation optimization audit scoring blog posts for ChatGPT, Perplexity, and Google AI Overview citability. Evaluates passage-level citability, Q&A formatting, entity clarity, structured data, and AI crawler accessibility. Generates citation capsules and a 0-100 AI Citation Readiness score. Use when user says "geo", "ai citation", "ai optimization", "citation audit", "aeo", "perplexity optimization", "chatgpt citation".
> Optimize content for AI Overviews (formerly SGE), ChatGPT web search, Perplexity, and other AI-powered search experiences. Generative Engine Optimization (GEO) analysis including brand mention signals, AI crawler accessibility, llms.txt compliance, passage-level citability scoring, and platform-specific optimization. Use when user says "AI Overviews", "SGE", "GEO", "AI search", "LLM optimization", "Perplexity", "AI citations", "ChatGPT search", or "AI visibility".
| Add new sources to your knowledge base or re-scrape existing ones to pick up changes. Supports Notion, Slack, Confluence, and local files. Can be run anytime after /setup-knowledge-base.
> Conduct deep OSINT research on individuals. Build full digital footprint, psychoprofile (MBTI/Big Five), career history, social graph with confidence scores. Recursive self-evaluation until completeness threshold is met. Includes internal intelligence (Telegram history, email, vault contacts) before going external. "разведка", "due diligence", "background check", "digital footprint", "найди всё про", "собери информацию", "кто это", "профиль человека". market research, content generation, or general web scraping tasks.