fuzzy-browser-automation（模糊浏览器自动化）

Browser Automation

Control a Chromium browser directly from OpenClaw — navigate, click, type, snapshot, screenshot, extract data. Works with both the sandboxed OpenClaw-managed browser and your logged-in user browser (with profile="user").

Browser Selection

Core Actions

snapshot — Inspect the Page

browser(action="snapshot", target="sandbox")

Returns the full page DOM as a structured tree. Use refs="aria" for screen-reader-friendly selectors, refs="role" (default) for role+name based refs.

browser(
  action="snapshot",
  target="sandbox",
  refs="aria"
)

screenshot — Capture the Page

browser(action="screenshot", target="sandbox")

For full-page screenshots:

browser(
  action="screenshot",
  target="sandbox",
  fullPage=true
)

navigate — Open a URL

browser(action="navigate", target="sandbox", url="https://news.ycombinator.com")

act — Interact with Elements

The act action is the workhorse. It combines ref (what to target) + kind (action type) + request (action details).

Click:

browser(
  action="act",
  target="sandbox",
  ref="aria:Submit",
  request={"kind": "click"}
)

Type:

browser(
  action="act",
  target="sandbox",
  ref="id:search-box",
  request={"kind": "type", "text": "openclaw browser automation"}
)

Press a key:

browser(
  action="act",
  target="sandbox",
  ref="id:search-box",
  request={"kind": "press", "key": "Enter"}
)

Hover:

browser(
  action="act",
  target="sandbox",
  ref="css:.dropdown-menu",
  request={"kind": "hover"}
)

Select from dropdown:

browser(
  action="act",
  target="sandbox",
  ref="id:country-select",
  request={"kind": "select", "values": ["South Africa"]}
)

Wait for element:

browser(
  action="act",
  target="sandbox",
  ref="aria:Loading",
  request={"kind": "wait", "timeMs": 5000}
)

Locator Reference (ref types)

For stable refs across calls, prefer refs="aria" in snapshots — these use ARIA labels that rarely change.

Recipes

Recipe 1: Scrape a Dynamic Page

// 1. Navigate
browser(action="navigate", target="sandbox", url="https://news.ycombinator.com/news")

// 2. Wait for content to load
browser(
  action="act",
  target="sandbox",
  loadState="networkidle",
  ref="css:.itemlist",
  request={"kind": "wait", "timeMs": 3000}
)

// 3. Snapshot to extract structured data
browser(action="snapshot", target="sandbox", refs="aria")

Recipe 2: Fill and Submit a Form

// 1. Navigate to form
browser(action="navigate", target="sandbox", url="https://example.com/contact")

// 2. Fill inputs
browser(action="act", target="sandbox", ref="id:name",    request={"kind": "fill", "text": "Alice Smith"})
browser(action="act", target="sandbox", ref="id:email",   request={"kind": "fill", "text": "alice@example.com"})
browser(action="act", target="sandbox", ref="id:message", request={"kind": "fill", "text": "Hi, I'd like to know more..."})

// 3. Click submit
browser(action="act", target="sandbox", ref="aria:Submit", request={"kind": "click"})

// 4. Wait for confirmation
browser(
  action="act",
  target="sandbox",
  ref="aria:Thank you",
  request={"kind": "wait", "timeMs": 2000}
)

Recipe 3: Login to a Service (User Browser)

// Requires you to be present at the machine — uses your actual browser session
browser(action="navigate", target="host", url="https://github.com/login")

browser(action="act", target="host", ref="id:login_field", request={"kind": "fill", "text": "myuser"})
browser(action="act", target="host", ref="id:password",    request={"kind": "fill", "text": "mypassword"})
browser(action="act", target="host", ref="css:[type=submit]", request={"kind": "click"})

Recipe 4: Monitor Price / Availability

// Navigate and wait for price to update
browser(action="navigate", target="sandbox", url="https://example.com/product/123")

browser(
  action="act",
  target="sandbox",
  ref="css:.price",
  request={"kind": "wait", "timeMs": 10000}
)

// Capture screenshot
browser(action="screenshot", target="sandbox")

// Evaluate for price text
browser(
  action="act",
  target="sandbox",
  request={
    "kind": "evaluate",
    "fn": "() => document.querySelector('.price').innerText"
  }
)

Recipe 5: Multi-Tab Workflow

// Open new tab
browser(action="navigate", target="sandbox", url="https://mail.google.com")

// Switch tabs
browser(action="act", target="sandbox", request={"kind": "press", "key": "Control+Tab"})

// Close current tab
browser(action="act", target="sandbox", request={"kind": "press", "key": "Control+W"})

Recipe 6: Scroll and Load Lazy Content

// Scroll by a pixel amount
browser(
  action="act",
  target="sandbox",
  request={
    "kind": "evaluate",
    "fn": "() => window.scrollBy(0, 800)"
  }
)

// Scroll to bottom (infinite scroll pages)
browser(
  action="act",
  target="sandbox",
  request={
    "kind": "evaluate",
    "fn": "() => window.scrollTo(0, document.body.scrollHeight)"
  }
)

Recipe 7: Extract Table Data

browser(action="navigate", target="sandbox", url="https://example.com/sales-report")

browser(
  action="act",
  target="sandbox",
  ref="css:table",
  request={"kind": "wait", "timeMs": 2000}
)

browser(
  action="act",
  target="sandbox",
  request={
    "kind": "evaluate",
    "fn": "() => Array.from(document.querySelectorAll('table tr')).map(row => Array.from(row.querySelectorAll('td')).map(cell => cell.innerText))"
  }
)

Recipe 8: Download a File

browser(action="navigate", target="sandbox", url="https://example.com/export.csv")

browser(
  action="act",
  target="sandbox",
  request={
    "kind": "evaluate",
    "fn": "() => { const link = document.querySelector('a[href$=\".csv\"]'); return link ? link.href : null; }"
  }
)

Action Reference

act kind Reference

Anti-Patterns

• Don't click before the page loads — always navigate then wait for loadState="networkidle" or an explicit element wait
• Don't use hard pixel waits — prefer waiting for a specific element or networkidle state
• Don't scrape without rate limiting — add timeMs waits between actions to avoid IP blocks
• Don't use profile="user" for automated workflows — it's meant for attended use; automated flows should use the sandbox browser
• Don't use xpath unless nothing else works — xpath selectors break easily when the page changes

Troubleshooting

See Also

• webhook-automation skill — combining browser-extracted data with outgoing webhooks
• rss-aggregator skill — using browser scraping as a fallback when feeds aren't available
• cron-scheduler skill — scheduling browser-based monitoring tasks