Using Webctl

oaustegard/using-webctl

Browser automation via webctl CLI in Claude.ai containers with authenticated proxy support. Use when users mention webctl, browser automation, Playwright browsing, web scraping, or headless Chrome in container environments.

4k tokens

context cost

the whole folder, loaded on every use

files

ships runnable scripts

copies elsewhere

how many repositories repackaged it

137

stars on the repo

on the repository, not the skill itself

Install

one command, takes just this skill from the repository

npx skills add https://github.com/oaustegard/claude-skills --skill using-webctl

The instruction itself

23 sections, as written by the author

Using webctl in Claude.ai Containers

Browser automation using webctl CLI with automatic proxy authentication handling for Claude.ai's egress-controlled environment.

When This Skill Applies

User requests browser automation, web scraping, or page interaction
webctl commands fail with ERR_TUNNEL_CONNECTION_FAILED
Playwright/Chromium needs to work through authenticated HTTP proxy

Core Problem

Claude.ai containers route traffic through an authenticated egress proxy (HTTP_PROXY env var with JWT credentials). Chromium doesn't properly handle proxy authentication for HTTPS CONNECT tunnels, causing all HTTPS navigation to fail even though curl works.

Solution

A local forwarding proxy (port 18080) intercepts Chromium connections and injects Proxy-Authorization headers before forwarding to the real egress proxy.

Setup Procedure

1. Install webctl

pip install webctl --break-system-packages
webctl setup  # Downloads Chromium if needed

2. Deploy Auth Proxy Module

Copy scripts/auth_proxy.py to webctl's daemon directory:

cp /mnt/skills/user/using-webctl/scripts/auth_proxy.py \
   /usr/local/lib/python3.12/dist-packages/webctl/daemon/

3. Patch Session Manager

Apply this patch to /usr/local/lib/python3.12/dist-packages/webctl/daemon/session_manager.py.

Recent webctl versions split context creation into two branches (mobile emulation vs desktop). Both must receive the proxy config. Find the block (search for cfg = WebctlConfig.load() followed by the if mode == "unattended" and cfg.mobile_emulation: branch — around line 190):

cfg = WebctlConfig.load()
if mode == "unattended" and cfg.mobile_emulation:
    # Mobile emulation: cleaner pages, fewer ads, simpler layouts
    device = self._playwright.devices["Pixel 7"]
    context = await browser.new_context(
        storage_state=storage_state,
        **device,
    )
else:
    context = await browser.new_context(
        storage_state=storage_state, viewport={"width": 1280, "height": 720}
    )

Replace with:

cfg = WebctlConfig.load()
from .auth_proxy import get_local_proxy_url
proxy_url = get_local_proxy_url()
proxy_config = {"server": proxy_url} if proxy_url else None
if mode == "unattended" and cfg.mobile_emulation:
    # Mobile emulation: cleaner pages, fewer ads, simpler layouts
    device = self._playwright.devices["Pixel 7"]
    context = await browser.new_context(
        storage_state=storage_state,
        proxy=proxy_config,
        **device,
    )
else:
    context = await browser.new_context(
        storage_state=storage_state, viewport={"width": 1280, "height": 720},
        proxy=proxy_config
    )

If the file only has a single new_context call (older webctl), inject proxy=proxy_config into that one and place the three proxy_url lines just above it.

4. Verify

webctl start --mode unattended
webctl --quiet navigate "https://github.com"
webctl snapshot --interactive-only --limit 10
webctl stop

Quick Reference

Session Management

webctl start --mode unattended    # Headless browser
webctl stop                       # Full shutdown (use --keep-daemon to leave daemon running)
webctl status                     # Current state + console error counts

webctl navigate "https://..."
webctl back / webctl forward / webctl reload

Observation

webctl snapshot --interactive-only --limit 30   # Buttons, links, inputs
webctl snapshot --within "role=main"            # Scope to container
webctl query "role=button name~=Submit"         # Debug queries
webctl screenshot --path shot.png

Interaction

webctl click 'role=button name~="Submit"'
webctl type 'role=textbox name~="Email"' "[email protected]"
webctl type 'role=textbox name~="Search"' "query" --submit
webctl select 'role=combobox name~="Country"' --label "Germany"

Query Syntax

role=button — By ARIA role
name~="partial" — Contains (preferred, more robust)
name="exact" — Exact match
nth=0 — Select first when multiple matches

Wait Conditions

webctl wait network-idle
webctl wait 'exists:role=button name~="Continue"'
webctl wait 'url-contains:"/dashboard"'

Troubleshooting

ERR_TUNNEL_CONNECTION_FAILED

Auth proxy not loaded. Verify:

auth_proxy.py exists in webctl daemon directory
Session manager is patched
Restart daemon: webctl stop && webctl start --mode unattended

Multiple matches error

Add specificity or use nth=0:

webctl click 'role=link name="Sign in" nth=0'

Verify proxy is running

netstat -tlnp | grep 18080

Limitations

Patch is session-local (container resets clear it)
Only allowed domains in network config are accessible
No Firefox support (download blocked by egress policy)

Domain Restrictions

Check <network_configuration> in system prompt for allowed domains. Common allowed: *.github.com, *.bsky.app, allowed API endpoints.

Output Filtering

Reduce context consumption:

webctl snapshot --interactive-only --limit 30    # Cap elements
webctl snapshot | grep -i "submit"               # Unix filtering
webctl --quiet navigate "..."                    # Suppress events

How to use it

Copy the folder

Take oaustegard/using-webctl from the repository into ~/.claude/skills for personal use, or into .claude/skills inside a project.

Check the name does not clash

The agent identifies a skill by the name field in its header. Two skills with the same name cannot sit side by side — one of them will be ignored.

Install what it needs

The instructions reference pip. Without those the skill loads but fails at the first command.