Bypasses anti-bot protections using Oxylabs Web Unblocker, an AI-powered proxy that handles fingerprinting, JavaScript rendering, and retries automatically. Use when the user needs to scrape protected websites, bypass CAPTCHAs, access blocked content, or when regular proxies fail due to anti-bot measures.
npx skills add https://github.com/oxylabs/agent-skills --skill web-unblocker
AI-powered proxy solution that automatically manages fingerprinting, headers, retries, and JavaScript rendering.
https://unblock.oxylabs.io:60000
HTTP Basic Auth via proxy credentials:
curl -k -x "https://unblock.oxylabs.io:60000" \
-U "$OXYLABS_USERNAME:$OXYLABS_PASSWORD" \
"https://example.com"
Basic request:
curl -k -x "https://unblock.oxylabs.io:60000" \
-U "$OXYLABS_USERNAME:$OXYLABS_PASSWORD" \
"https://ip.oxylabs.io/headers"
With JavaScript rendering:
curl -k -x "https://unblock.oxylabs.io:60000" \
-U "$OXYLABS_USERNAME:$OXYLABS_PASSWORD" \
-H "x-oxylabs-render: html" \
"https://example.com/spa-page"
| Header | Description |
|--------|-------------|
| x-oxylabs-render | html for rendered HTML, png for raw PNG bytes; empty value disables automatic forced rendering |
| X-Oxylabs-Session-Id | Reuse same IP across requests (any random string) |
| X-Oxylabs-Geo-Location | Target country, city/state, ZIP/postcode, or coordinates |
| x-oxylabs-force-headers: 1 | Enable custom header passthrough |
| x-oxylabs-force-cookies: 1 | Enable custom cookie passthrough |
| X-Oxylabs-Successful-Status-Codes | Define custom success codes to prevent retries |
| x-oxylabs-browser-instructions | JSON-escaped browser actions; requires x-oxylabs-render: html |
Reuse the same IP across multiple requests:
curl -k -x "https://unblock.oxylabs.io:60000" \
-U "$OXYLABS_USERNAME:$OXYLABS_PASSWORD" \
-H "X-Oxylabs-Session-Id: my-session-123" \
"https://example.com/page1"
curl -k -x "https://unblock.oxylabs.io:60000" \
-U "$OXYLABS_USERNAME:$OXYLABS_PASSWORD" \
-H "X-Oxylabs-Geo-Location: Germany" \
"https://example.com"
Use values such as Germany, 90210, California,United States, New York,New York,United States, or lat: 40.7128, lng: -74.0060, rad: 50.
Use normal HTTP methods and request bodies through the proxy; Web Unblocker supports both GET and POST.
| Scenario | Use |
|----------|-----|
| Sites with anti-bot protection | Web Unblocker |
| CAPTCHAs, fingerprint detection | Web Unblocker |
| JavaScript-heavy SPAs | Web Unblocker with x-oxylabs-render: html |
| Simple requests, no protection | Regular Proxies |
| High volume, price sensitive | Regular Proxies |
-k flag (or disable SSL verification) - the proxy uses its own certificatesx-oxylabs-render: html if experiencing empty content or low success rates; set client timeouts near 180 seconds for rendered requestsX-Oxylabs-Final-Url in response headers when redirects matterx-oxylabs-render: htmlFor code examples in Python, Node.js, PHP, Go, Java, and C#, see examples.md.
Automate web scraping and data extraction with Apify -- run Actors, manage datasets, create reusable tasks, and retrieve crawl results through the Composio Apify integration.
Scrapes content based on a preset URL list, filters high-quality technical information, and generates daily Markdown reports.
Web search, content extraction, crawling, and research capabilities using Tavily API
| Bulk extract content from an entire website or site section. Use this skill when the user wants to crawl a site, extract all pages from a docs section, bulk-scrape multiple pages following links, or says "crawl", "get all the pages", "extract everything under /docs", "bulk extract", or needs content from many pages on the same site. Handles depth limits, path filtering, and concurrent extraction.
Deep web scraping, screenshots, PDF parsing, and website crawling using Firecrawl API. Use when you need deep content extraction from web pages, page interaction is required (clicking, scrolling, etc.), or you want screenshots or PDF parsing.
Build a fully automated AI-powered data collection agent for any public source — job boards, prices, news, GitHub, sports, anything. Scrapes on a schedule, enriches data with a free LLM (Gemini Flash), stores results in Notion/Sheets/Supabase, and learns from user feedback. Runs 100% free on GitHub Actions. Use when the user wants to monitor, collect, or track any public data automatically.
Scrapes content based on a preset URL list, filters high-quality technical information, and generates daily Markdown reports.
Scrape, crawl, search, and extract structured data from websites using Firecrawl API - converts web pages to LLM-ready markdown
Take oxylabs/web-unblocker from the repository into ~/.claude/skills for personal
use, or into .claude/skills inside a project.
The agent identifies a skill by the name field in its header. Two skills with the
same name cannot sit side by side — one of them will be ignored.