Web Scraping Skills

自动化爬取网站数据和 API 接口。当用户需要抓取网页内容、调用 API、解析数据或创建爬虫脚本时使用此技能。

网络搜索与网页内容获取。当用户需要搜索互联网信息、获取网页内容、查找实时数据、进行 websearch 时使用此技能。支持多种搜索工具：WebFetch、Firecrawl skill、Tavily skill。

This skill activates for web scraping and Actor development. It proactively discovers APIs via traffic interception, recommends optimal strategy (traffic interception/sitemap/API/DOM scraping/hybrid), and implements iteratively. For production, it guides TypeScript Actor creation via Apify CLI.

Analyze HitKeep AI visibility data. Use for AI crawler fetches, AI-referred visits, assistant names or families, citation yield, resource types, top paths, fetch failures, Search Console context, GPTBot, ClaudeBot, ChatGPT, Claude, Perplexity, AI search, or whether assistants are discovering and sending traffic to a site.

> Receive and verify Scrapfly webhooks. Use when setting up Scrapfly webhook handlers for async scrape, extraction, screenshot, or crawler jobs, debugging X-Scrapfly-Webhook-Signature verification, or routing on X-Scrapfly-Webhook-Resource-Type.

> Your For You page for content creators. Scrapes tracked competitors across social media platforms, scores what's performing, and turns it into actionable, differentiated content ideas backed by real engagement data. Use this whenever the user wants competitor/creator research, a content feed or "for you" page, trending-topic ideas in their niche, to see what's working on social, to track what creators are posting, or to generate video/post briefs from what's performing — even if they don't say "find ideas." First run walks through setup.

How to perform data lookups in Excel using INDEX and MATCH. Use this skill when needing to extract data from a source table based on two conditions (e.g., Row header and Column header).

Fetch web content efficiently by checking llms.txt first, then Cloudflare markdown endpoints, then falling back to HTML. Reduces token usage by 80% on sites that support clean markdown delivery. No external dependencies — installs a single Python script. Trigger words: fetch URL, web content, read website, scrape page, download page, get webpage, read this link.

Automate web crawling and data extraction with Firecrawl -- scrape pages, crawl sites, extract structured data, batch scrape URLs, and map website structures through the Composio Firecrawl integration.

Deep web scraping, screenshots, PDF parsing, and website crawling using Firecrawl API

Automate lead generation, web scraping, and social media data extraction workflows through PhantomBuster's cloud platform via Composio

Automate Scrape Do tasks via Rube MCP (Composio). Always search tools first for current schemas.

X (Twitter) data platform skill — tweet search, user lookup, follower extraction, engagement metrics, giveaway draws, monitoring, webhooks, 19 extraction tools, MCP server.

Develop, debug, and deploy Apify Actors - serverless cloud programs for web scraping, automation, and data processing. Use when creating new Actors, modifying existing ones, or troubleshooting Acto...

Scrape e-commerce data for pricing intelligence, customer reviews, and seller discovery across Amazon, Walmart, eBay, IKEA, and 50+ marketplaces. Use when user asks to monitor prices, track competi...

Generates B2B/B2C leads by scraping Google Maps, websites, Instagram, TikTok, Facebook, LinkedIn, YouTube, and Google Search. Use when user asks to find leads, prospects, businesses, build lead lis...

Universal AI-powered web scraper for any platform. Scrape data from Instagram, Facebook, TikTok, YouTube, Google Maps, Google Search, Google Trends, Booking.com, and TripAdvisor. Use for lead gener...

Web search, content extraction, crawling, and research capabilities using Tavily API

Run end-to-end competitor research and monitoring through the Hyper MCP — pick the set, scrape every public surface (site, blog, pricing, organic social, search rank, mentions, demand) via Firecrawl + HyperSEO + the Apify scrapers, diff against last run, and synthesize a brief. Use when the user wants to research competitors, build a battle card or comparison page, run a weekly digest, or asks "what's [competitor] doing?" or "how do we compare?".

Create and maintain a single brand-context.md — positioning, audience, personas, pain points, customer language, voice, proof points — that every other marketing skill reads before asking the user to re-explain their business. Auto-drafts from the brand's site via Firecrawl, or builds the doc through a short interview. Use when the user wants to set up brand context, onboard a new brand, define positioning / brand voice / ICP, or complains about re-answering the same brand questions.

Research competitor Facebook and Instagram ads from the Meta Ads Library via the Hyper MCP — search by keyword, pull full ad creative and metadata, enrich with page contact info for lead generation, and surface structured ad-intelligence summaries in chat. Use when the user wants to scrape the Meta Ads Library, spy on competitor ads, monitor new ads in a category, build a lead list from advertisers, or surface creative trends across an industry.

Research Reddit discussions with high signal using scrape_reddit_leads and scrape_reddit — pain points, intent discovery, and trend tracking. Use when the user wants to mine subreddits for leads, find threads worth replying to, or track what a community says about a topic.

Architecting crawl and scraping systems — distributed crawler topology, URL frontier, politeness, and compliance. Architecture-only (no execution code). Don't use for single-page scraping (Vector) or ETL pipelines (Stream).

Crawl GitHub trending repositories, analyze with LLM for Chinese insights, categorize by themes, compute diffs against history, and generate Markdown reports. Default brief mode stops at trend analysis; optional detailed mode appends per-project analysis. Supports incremental gap-filling and selective re-analysis with caching.

微信公众号文章抓取与导出。自动处理 mp.weixin.qq.com 的登录态获取与续期， 支持按公众号搜索、抓取文章列表与正文、按日期窗口导出 Markdown / JSON / CSV。 Trigger when the user wants to crawl a WeChat public account, export recent articles, or 提到 "wcx"、"微信公众号"、"公众号文章"、"mp.weixin"、"抓公众号"、 "crawl wechat official account"、"wxmp"、"最近十天的文章"。

Migrate Exa, Tavily, Perplexity, or Firecrawl web-data integrations completely to the appropriate Parallel products while preserving application behavior. Use when replacing these providers' SDKs or REST calls, dependencies, environment variables, request parameters, response parsing, model tools, search-plus-scrape paths, full-content or answer-synthesis paths, tests, and documentation; separating unsupported research-index, crawl, browser, file-parse, monitor, or other non-search capabilities; auditing for leftover provider usage; or finishing and verifying an in-progress provider migration.

Primitive web crawling and scraping for one or more URLs. Use when a user shares links, asks to ingest or archive web content, or needs raw source artifacts normalized into reusable local records before feed-building or synthesis.

Turn a crawler-produced Tapestry artifact into a source-appropriate structured feed. Use when a user wants a rigorous feed entry, normalized text output, or source-specific organization rather than a loose free-form summary.

AI-native web intelligence workflow for crawling, organizing, and synthesizing web content from multiple platforms (Zhihu, Reddit, HN, X/Twitter, Xiaohongshu, Weibo). Use when users share URLs, want to archive web content, build knowledge bases, or analyze online discussions.

Automate web scraping and data extraction with Apify -- run Actors, manage datasets, create reusable tasks, and retrieve crawl results through the Composio Apify integration.

Automate web crawling and data extraction with Firecrawl -- scrape pages, crawl sites, extract structured data, batch scrape URLs, and map website structures through the Composio Firecrawl integration.

Automate lead generation, web scraping, and social media data extraction workflows through PhantomBuster's cloud platform via Composio

Automate Scrape Do tasks via Rube MCP (Composio). Always search tools first for current schemas.

Search Google, scrape web pages, Amazon product pages, YouTube subtitles, or Reddit (post/subreddit) using the Decodo Scraper OpenClaw Skill.

Complete toolkit for web crawling and data extraction using Crawl4AI. This skill should be used when users need to scrape websites, extract structured data, handle JavaScript-heavy pages, crawl multiple URLs, or build automated web data pipelines. Includes optimized extraction patterns with schema generation for efficient, LLM-free extraction.

Use Xquik for X data and confirmation-gated X actions: tweet search, user lookup, follower export, media download, monitors, webhooks, MCP, and SDK workflows.

构建一个全自动化的AI驱动数据收集代理，适用于任何公共来源——招聘网站、价格信息、新闻、GitHub、体育赛事等任何内容。按计划进行抓取，使用免费LLM（Gemini Flash）丰富数据，将结果存储在Notion/Sheets/Supabase中，并从用户反馈中学习。完全免费在GitHub Actions上运行。适用于用户希望自动监控、收集或跟踪任何公共数据的场景。

Python library for scraping Twitter/X data using GraphQL API with account rotation and session management. Use when extracting tweets, user profiles, followers, trends, or building social media monitoring tools.

Fetch web content, take screenshots, extract structured data, search the web, and crawl documentation sites. Use when the user needs current web information, asks to scrape a URL, wants a screenshot, needs to extract specific data from a page, or wants to learn about a framework or library.

多功能网页抓取和数据提取工具，支持同步抓取、搜索、网站地图获取和异步爬取

When the user wants to implement, audit, or validate structured data (schema markup) on their website. Use when the user mentions 'structured data,' 'schema.org,' 'JSON-LD,' 'rich results,' 'rich snippets,' 'schema markup,' 'FAQ schema,' 'Product schema,' 'HowTo schema,' or 'structured data errors in Search Console.' Also use when someone asks why their content isn't showing rich results or wants to improve AI search visibility. NOT for general SEO audits (use seo-audit) or technical SEO crawl issues (use site-architecture).

Develop, debug, and deploy Apify Actors - serverless cloud programs for web scraping, automation, and data processing. Use when creating new Actors, modifying existing ones, or troubleshooting Actor code.

Scrape e-commerce data for pricing intelligence, customer reviews, and seller discovery across Amazon, Walmart, eBay, IKEA, and 50+ marketplaces. Use when user asks to monitor prices, track competitors, analyze reviews, research products, or find sellers.