Collects every tweet in an X (Twitter) conversation thread given a conversation id (root tweet id) — the focal tweet plus all replies, sub-replies, and quote chains — and returns normalized per-tweet data with text, author, engagement counts, media, hashtags, mentions, in_reply_to mapping, and cursor for pagination. Use when user mentions Twitter conversation, X conversation thread, conversation_id, thread scraper, scrape Twitter replies, all replies to a tweet, replies under a tweet, sub-replies, nested replies, thread harvester, get replies of a tweet, scrape comments on Twitter, scrape comments on X, full thread extraction, conversation export, conversation tree, reply chain, thread dump, X tweet thread, twitter thread scrape, comment scraping twitter, comment scraping x, focal tweet plus context, root tweet plus replies. Also applies to sentiment analysis on a single viral tweet, controversy mapping, harvesting community Q&A threads, capturing AMA threads, recovering long-running discussions, and any paginated bulk reply collection driven by a conversation id.
npx skills add https://github.com/browser-act/skills --skill x-tweet-by-conversation
> Conversation id (root tweet id) → normalized list of every tweet in the thread (focal tweet + replies + sub-replies), with author, engagement, media, cursor.
All process output to user (progress updates, process notifications) follows the user's language.
Given an X conversation id (which equals the root tweet id), collect the focal tweet and every reply / sub-reply visible to the logged-in session, returning structured per-tweet data with pagination cursors.
If browser-act has been confirmed available in the current session → skip this step.
Invoke browser-act via Skill tool to load usage. If installation or configuration issues arise, follow its guidance to resolve then retry.
If login status for X has been confirmed in the current session → skip this step.
Otherwise: open https://x.com and observe the left sidebar:
User refuses or cannot log in → terminate execution.
> This Skill's operational boundary = what the user can manually do in their browser. It only reads tweet data already shown to the user, never bypassing authentication. The browser's own JS signs the GraphQL request; the Skill triggers it via URL navigation and reads the response from network traffic. Python scripts under scripts/ only build URLs and parse responses — they do not call X directly. Run them through the bash tool.
Step 1 — build the tweet-detail URL for the conversation root:
URL=$(python scripts/build-conversation-url.py '{conversation_id}' [--handle {handle}])
Parameters:
conversation_id (positional): the X conversation id, which equals the root tweet id (the same number you would see in https://x.com/i/status/<conversation_id>).--handle: the root tweet's author handle if known; if unknown, leave the default — x.com/i/status/<id> resolves to the same focal tweet because X canonicalises the URL.Step 2 — navigate and capture the first page:
network requests --clearnavigate "$URL"wait stable --timeout 25000 (timeout is normal on X; proceed)network requests --type xhr,fetch --filter TweetDetail → take the latest entry's request_idnetwork request <request_id> → save full output to a file (e.g. tmp/x-conversation-page-1.txt)python scripts/parse-tweets.py --json-file tmp/x-conversation-page-1.txt --source tweet_detail → emits JSON {tweets, count, cursor_top, cursor_bottom}. The focal tweet is the first element; subsequent elements are replies (in display order). Each reply carries is_reply: true, in_reply_to_id, in_reply_to_user, and conversation_id == <root tweet id>, which makes the reply tree reconstructable downstream.Endpoint characteristic: URL contains /i/api/graphql/<hash>/TweetDetail. The query hash rotates; always filter by name.
Step 3 — paginate via scroll to load deeper replies and sub-replies:
network requests --clearscroll down --amount 5000wait stable --timeout 10000network requests --type xhr,fetch --filter TweetDetail → newest entry's request_idnetwork request <request_id> → save to tmp/x-conversation-page-N.txtpython scripts/parse-tweets.py --json-file tmp/x-conversation-page-N.txt --source tweet_detailRepeat Step 3 until any termination condition is met:
count == 0 on the current page.cursor_bottom is unchanged across two consecutive pages.state reveals it; the Agent may click <index> to expand more, then resume scrolling.Error handling:
TweetTombstone entry; parse-tweets.py filters tombstones, so the result will be count: 0 — report to the user and stop.TweetDetail request appears after a scroll, wait 3 s and retry once. Persistent absence after two scrolls means the thread has been fully loaded.state as an authorization prompt), pause and ask the user.Output example:
{
"tweets": [
{
"type": "tweet",
"id": "2069990565530214798",
"url": "https://x.com/Rothmus/status/2069990565530214798",
"twitter_url": "https://twitter.com/Rothmus/status/2069990565530214798",
"text": "\"We are going to have a multi-racial nation in Singapore. ...\"",
"created_at": "Tue Jun 24 23:01:50 +0000 2026",
"lang": "en",
"source": "Twitter Web App",
"retweet_count": 91,
"reply_count": 84,
"like_count": 567,
"quote_count": 12,
"bookmark_count": 50,
"view_count": 64140,
"is_reply": false,
"is_retweet": false,
"is_quote": false,
"quote_id": null,
"quote_url": null,
"in_reply_to_id": null,
"in_reply_to_user": null,
"in_reply_to_user_id": null,
"conversation_id": "2069990565530214798",
"hashtags": [],
"mentions": [],
"urls": [],
"media": [],
"card": null,
"place": null,
"author": {
"id": "987654321",
"user_name": "Rothmus",
"name": "Rothmus",
"url": "https://x.com/Rothmus",
"is_verified": false,
"is_blue_verified": true,
"verified_type": null,
"profile_picture": "https://pbs.twimg.com/profile_images/.../photo.jpg",
"description": "Singapore observer ...",
"location": "Singapore",
"followers": 12345,
"following": 678,
"created_at": "Wed Jun 16 10:00:00 +0000 2021"
}
},
{
"type": "tweet",
"id": "2070027327409655973",
"url": "https://x.com/DinoLeadingNews/status/2070027327409655973",
"text": "@Rothmus @elonmusk Singapore proves that identity ...",
"is_reply": true,
"in_reply_to_id": "2069990565530214798",
"in_reply_to_user": "Rothmus",
"in_reply_to_user_id": "987654321",
"conversation_id": "2069990565530214798",
"like_count": 0,
"reply_count": 0,
"retweet_count": 0,
"quote_count": 0,
"bookmark_count": 0,
"view_count": 19,
"author": {"user_name": "DinoLeadingNews", "name": "Dino Leading News", "...": "..."},
"...": "..."
}
],
"count": 40,
"cursor_top": null,
"cursor_bottom": "DAACCgABHLoz..."
}
Network Capture Pagination: triggered by scroll down. X's page JS inserts the previous response's cursor_bottom into the next TweetDetail request's variables.cursor. Some sub-reply branches require clicking an in-page "Show replies" expander (state to locate, click <index>) before the next scroll surfaces them. Termination: count == 0, cursor_bottom does not advance across two consecutive pages, or user target reached.
count >= 1 on the first page (the focal tweet must be present unless the conversation root is deleted) AND every tweet has non-null id, text, created_at, author.user_name, like_count, retweet_count, reply_count, conversation_id AND each reply tweet has is_reply == true and in_reply_to_id pointing to a tweet inside the thread.
count: 0.x-tweet-search-by-query with filter:quote conversation_id:<id> if needed.view_count is null for very new replies where X has not emitted views.count.network-idle; wait stable will frequently time out — proceed to read network anyway.cursor_bottom and accumulated reply IDs per conversation after every page.id: the focal tweet may appear in multiple paginated responses (some endpoints repeat the focal tweet at the top of every page); merge by id.Path: {working-directory}/browser-act-skill-forge-memories/x-tweet-scraper-x-tweet-by-conversation.memory.md (working directory is determined by the Agent running the Skill, typically the project root or current working directory)
Before execution: If the file exists, read it first — it records unexpected situations encountered during past executions (e.g., a strategy has become ineffective); adjust strategy order accordingly.
After execution: If an unexpected situation is encountered (strategy became ineffective, page redesigned, anti-scraping upgraded, better path discovered), append a line:
{YYYY-MM-DD}: {what happened} → {conclusion}
Normal execution does not write to the file. Do not record which conversation was scraped or how many replies were returned — those are task outputs, not experience.
Automate web scraping and data extraction with Apify -- run Actors, manage datasets, create reusable tasks, and retrieve crawl results through the Composio Apify integration.
Scrapes content based on a preset URL list, filters high-quality technical information, and generates daily Markdown reports.
Web search, content extraction, crawling, and research capabilities using Tavily API
| Bulk extract content from an entire website or site section. Use this skill when the user wants to crawl a site, extract all pages from a docs section, bulk-scrape multiple pages following links, or says "crawl", "get all the pages", "extract everything under /docs", "bulk extract", or needs content from many pages on the same site. Handles depth limits, path filtering, and concurrent extraction.
Deep web scraping, screenshots, PDF parsing, and website crawling using Firecrawl API. Use when you need deep content extraction from web pages, page interaction is required (clicking, scrolling, etc.), or you want screenshots or PDF parsing.
Build a fully automated AI-powered data collection agent for any public source — job boards, prices, news, GitHub, sports, anything. Scrapes on a schedule, enriches data with a free LLM (Gemini Flash), stores results in Notion/Sheets/Supabase, and learns from user feedback. Runs 100% free on GitHub Actions. Use when the user wants to monitor, collect, or track any public data automatically.
Scrapes content based on a preset URL list, filters high-quality technical information, and generates daily Markdown reports.
Scrape, crawl, search, and extract structured data from websites using Firecrawl API - converts web pages to LLM-ready markdown
Take browser-act/x-tweet-by-conversation from the repository into ~/.claude/skills for personal
use, or into .claude/skills inside a project.
The agent identifies a skill by the name field in its header. Two skills with the
same name cannot sit side by side — one of them will be ignored.