coco-research/browser-automation
Browser automation for AI agents. Two providers — agent-browser (local CLI with Playwright) and agentic-browser (cloud via inference.sh). Both use the same @e ref-based workflow for navigating pages, filling forms, clicking buttons, taking screenshots, extracting data, and automating browser tasks.
npx skills add https://github.com/coco-research/coco --skill browser-automation
Browser automation for AI agents with two provider options. Both share the same core workflow: navigate, snapshot, interact using @e refs, re-snapshot after changes.
| Provider | Runtime | Best For |
|----------|---------|----------|
| agent-browser | Local (Playwright CLI) | Local testing, iOS Simulator, file:// URLs |
| agentic-browser | Cloud (inference.sh) | Video recording, cloud execution, parallel sessions |
Every browser automation follows this pattern:
@e refs for interactive elementsImportant: Refs are invalidated after navigation. Always re-snapshot after clicking links/buttons, form submissions, or dynamic content loading.
agent-browser open https://example.com/form
agent-browser snapshot -i
# Output: @e1 [input type="email"], @e2 [input type="password"], @e3 [button] "Submit"
agent-browser fill @e1 "[email protected]"
agent-browser fill @e2 "password123"
agent-browser click @e3
agent-browser wait --load networkidle
agent-browser snapshot -i # Check result
# Navigation
agent-browser open <url> # Navigate
agent-browser close # Close browser
# Snapshot
agent-browser snapshot -i # Interactive elements with refs
agent-browser snapshot -i -C # Include cursor-interactive elements
agent-browser snapshot -s "#selector" # Scope to CSS selector
# Interaction (use @refs from snapshot)
agent-browser click @e1 # Click element
agent-browser fill @e2 "text" # Clear and type text
agent-browser type @e2 "text" # Type without clearing
agent-browser select @e1 "option" # Select dropdown option
agent-browser check @e1 # Check checkbox
agent-browser press Enter # Press key
agent-browser scroll down 500 # Scroll page
# Get information
agent-browser get text @e1 # Get element text
agent-browser get url # Get current URL
agent-browser get title # Get page title
# Wait
agent-browser wait @e1 # Wait for element
agent-browser wait --load networkidle # Wait for network idle
agent-browser wait --url "**/page" # Wait for URL pattern
agent-browser wait 2000 # Wait milliseconds
# Capture
agent-browser screenshot # Screenshot to temp dir
agent-browser screenshot --full # Full page screenshot
agent-browser pdf output.pdf # Save as PDF
# Login once and save state
agent-browser open https://app.example.com/login
agent-browser snapshot -i
agent-browser fill @e1 "$USERNAME"
agent-browser fill @e2 "$PASSWORD"
agent-browser click @e3
agent-browser wait --url "**/dashboard"
agent-browser state save auth.json
# Reuse in future sessions
agent-browser state load auth.json
agent-browser open https://app.example.com/dashboard
agent-browser --session site1 open https://site-a.com
agent-browser --session site2 open https://site-b.com
agent-browser session list
agent-browser --headed open https://example.com
agent-browser highlight @e1
agent-browser record start demo.webm
agent-browser --allow-file-access open file:///path/to/document.pdf
agent-browser --allow-file-access open file:///path/to/page.html
agent-browser screenshot output.png
# List available iOS simulators
agent-browser device list
# Launch Safari on a specific device
agent-browser -p ios --device "iPhone 16 Pro" open https://example.com
# Same workflow — snapshot, interact, re-snapshot
agent-browser -p ios snapshot -i
agent-browser -p ios tap @e1
agent-browser -p ios fill @e2 "text"
agent-browser -p ios swipe up
agent-browser -p ios screenshot mobile.png
agent-browser -p ios close
Requirements: macOS with Xcode, Appium (npm install -g appium && appium driver install xcuitest)
agent-browser find text "Sign In" click
agent-browser find label "Email" fill "[email protected]"
agent-browser find role button click --name "Submit"
agent-browser find placeholder "Search" type "query"
agent-browser find testid "submit-btn" click
# Install CLI
curl -fsSL https://cli.inference.sh | sh && infsh login
# Open a page
infsh app run agentic-browser --function open --input '{"url": "https://example.com"}' --session new
| Function | Description |
|----------|-------------|
| open | Navigate to URL, configure browser (viewport, proxy, video) |
| snapshot | Re-fetch page state with @e refs after DOM changes |
| interact | Perform actions using @e refs |
| screenshot | Take page screenshot (viewport or full page) |
| execute | Run JavaScript code on the page |
| close | Close session, returns video if recording enabled |
| Action | Description | Required Fields |
|--------|-------------|-----------------|
| click | Click element | ref |
| dblclick | Double-click | ref |
| fill | Clear and type text | ref, text |
| type | Type without clearing | text |
| press | Press key (Enter, Tab) | text |
| select | Select dropdown option | ref, text |
| hover | Hover over element | ref |
| check / uncheck | Toggle checkbox | ref |
| drag | Drag and drop | ref, target_ref |
| upload | Upload file(s) | ref, file_paths |
| scroll | Scroll page | direction, scroll_amount |
| back | Go back in history | - |
| wait | Wait milliseconds | wait_ms |
| goto | Navigate to URL | url |
# Start session
RESULT=$(infsh app run agentic-browser --function open --session new --input '{
"url": "https://example.com/login"
}')
SESSION_ID=$(echo $RESULT | jq -r '.session_id')
# Fill and submit
infsh app run agentic-browser --function interact --session $SESSION_ID --input '{
"action": "fill", "ref": "@e1", "text": "[email protected]"
}'
infsh app run agentic-browser --function interact --session $SESSION_ID --input '{
"action": "fill", "ref": "@e2", "text": "password123"
}'
infsh app run agentic-browser --function interact --session $SESSION_ID --input '{
"action": "click", "ref": "@e3"
}'
# Re-snapshot after navigation
infsh app run agentic-browser --function snapshot --session $SESSION_ID --input '{}'
# Close when done
infsh app run agentic-browser --function close --session $SESSION_ID --input '{}'
# Start with recording enabled
SESSION=$(infsh app run agentic-browser --function open --session new --input '{
"url": "https://example.com",
"record_video": true,
"show_cursor": true
}' | jq -r '.session_id')
# ... perform actions ...
# Close to get the video file
infsh app run agentic-browser --function close --session $SESSION --input '{}'
# Returns: {"success": true, "video": <File>}
infsh app run agentic-browser --function open --session new --input '{
"url": "https://example.com",
"proxy_url": "http://proxy.example.com:8080",
"proxy_username": "user",
"proxy_password": "pass"
}'
infsh app run agentic-browser --function interact --session $SESSION --input '{
"action": "upload",
"ref": "@e5",
"file_paths": ["/path/to/file.pdf"]
}'
infsh app run agentic-browser --function execute --session $SESSION --input '{
"code": "document.querySelectorAll(\"h2\").length"
}'
# Returns: {"result": "5", "screenshot": <File>}
| Reference | Description |
|-----------|-------------|
| references/commands.md | Full command reference with all options |
| references/snapshot-refs.md | Ref lifecycle, invalidation rules, troubleshooting |
| references/session-management.md | Parallel sessions, state persistence |
| references/authentication.md | Login flows, OAuth, 2FA handling |
| references/video-recording.md | Recording workflows for debugging |
| references/proxy-support.md | Proxy configuration, geo-testing |
| Template | Description |
|----------|-------------|
| templates/form-automation.sh | Form filling with validation |
| templates/authenticated-session.sh | Login once, reuse state |
| templates/capture-workflow.sh | Content extraction with screenshots |
Take coco-research/browser-automation from the repository into ~/.claude/skills for personal
use, or into .claude/skills inside a project.
The agent identifies a skill by the name field in its header. Two skills with the
same name cannot sit side by side — one of them will be ignored.
The instructions reference npm.
Without those the skill loads but fails at the first command.