feat(sparsekernel): tighten browser actionability checks

Author: AbrahamGreenman

docs/architecture/browser-broker.md

@@ -21,7 +21,7 @@ Set `OPENCLAW_RUNTIME_BROWSER_BROKER=native` to let SparseKernel launch and supe
 
 Set `OPENCLAW_RUNTIME_BROWSER_REQUIRE_PROXY=1` when a trust zone must use a proxy-backed browser egress path. The trust zone's network policy must contain a loopback `proxy_ref`, and native browser pools launch Chromium with `--proxy-server=<proxy_ref>`. Static or externally managed CDP endpoints are rejected in this mode unless `OPENCLAW_RUNTIME_BROWSER_EXTERNAL_PROXY_OK=1` asserts that the external browser process is already proxy-controlled. This protects the SparseKernel-owned browser process path; it is not host-level egress enforcement for arbitrary host processes.
 
-Supported v0 actions (`status`, `doctor`, `profiles`, `tabs`, `open`, `navigate`, `focus`, `close`, `snapshot`, `console`, `screenshot`, `pdf`, direct file-input `upload`, `dialog`, and brokered `act`) operate against broker-owned targets inside the leased CDP context. Brokered `act` covers the OpenClaw action contract for click, coordinate click, type, press, hover, scroll, drag, select, fill, resize, wait, evaluate, close, and batch using CDP input events plus bounded DOM evaluation. Selector-backed actions retry inside the leased page until their action timeout, and `wait --load networkidle` uses per-target CDP Network events plus a quiet window rather than only checking `document.readyState`. Actions that can change page state are followed by a broker-side navigation check: same-target navigations are accepted only when the resulting URL stays inside the context's allowed-origin policy, same-policy popups are attached as broker-owned targets, and disallowed popups are closed. When an allowed-origin policy is configured, the broker also enables CDP Fetch interception and fails requests outside that policy while recording `browser_network.blocked` observations; this is request control, not host isolation. Before opening or navigating, the ToolBroker checks the trust-zone network policy and denies unsupported schemes, private-network destinations when disallowed, literal denied CIDRs, and, when runtime policy enforcement is enabled, hostnames that resolve to denied/private addresses. Proxy-backed egress control remains future work. Snapshots use a bounded CDP `Runtime.evaluate` DOM read, actions resolve refs from the latest brokered snapshot where needed, console output is captured from CDP runtime/log events per target, and screenshot/PDF output is captured as SparseKernel artifacts, read back through artifact access, and converted to existing tool result formats for compatibility. Closing a broker-owned target now closes that target; the full browser context is released only when the last target closes or broker cleanup runs.
+Supported v0 actions (`status`, `doctor`, `profiles`, `tabs`, `open`, `navigate`, `focus`, `close`, `snapshot`, `console`, `screenshot`, `pdf`, direct file-input `upload`, `dialog`, and brokered `act`) operate against broker-owned targets inside the leased CDP context. Brokered `act` covers the OpenClaw action contract for click, coordinate click, type, press, hover, scroll, drag, select, fill, resize, wait, evaluate, close, and batch using CDP input events plus bounded DOM evaluation. Selector-backed actions retry inside the leased page until their action timeout and now require basic actionability before dispatch: visible connected target, stable bounding box, enabled form state where relevant, editable target for typing, and center-point hit testing. `wait --load networkidle` uses per-target CDP Network events plus a quiet window rather than only checking `document.readyState`. Actions that can change page state are followed by a broker-side navigation check: same-target navigations are accepted only when the resulting URL stays inside the context's allowed-origin policy, same-policy popups are attached as broker-owned targets, and disallowed popups are closed. When an allowed-origin policy is configured, the broker also enables CDP Fetch interception and fails requests outside that policy while recording `browser_network.blocked` observations; this is request control, not host isolation. Before opening or navigating, the ToolBroker checks the trust-zone network policy and denies unsupported schemes, private-network destinations when disallowed, literal denied CIDRs, and, when runtime policy enforcement is enabled, hostnames that resolve to denied/private addresses. Proxy-backed egress control remains future work. Snapshots use a bounded CDP `Runtime.evaluate` DOM read, actions resolve refs from the latest brokered snapshot where needed, console output is captured from CDP runtime/log events per target, and screenshot/PDF output is captured as SparseKernel artifacts, read back through artifact access, and converted to existing tool result formats for compatibility. Closing a broker-owned target now closes that target; the full browser context is released only when the last target closes or broker cleanup runs.
 
 Use `openclaw sparsekernel browser-pools` to inspect durable ledger pools and currently materialized native browser process pools. Native pool snapshots include trust zone, profile, active refs, max context slots, idle timeout, endpoint, PID when available, last activity, start count, clean stop count, and crash count.
 

docs/architecture/local-agent-kernel.md

@@ -106,7 +106,7 @@ The browser broker model is:
 
 Important boundary: BrowserContext isolation is session isolation, not host isolation. Playwright route blocking and SSRF guards are useful controls, but they are not hard security boundaries.
 
-The broker applies configured trust-zone network policy to explicit allowed origins before allocating a context. It also denies unsupported URL schemes, private-network destinations when the policy disallows them, literal IPs matching denied CIDRs, and, when `OPENCLAW_RUNTIME_BROWSER_POLICY_ENFORCE=1` is set, hostnames that resolve to denied/private addresses. Set `OPENCLAW_RUNTIME_BROWSER_POLICY_DNS=0` only when a caller intentionally wants literal-host checks without DNS resolution. Set `OPENCLAW_RUNTIME_BROWSER_REQUIRE_PROXY=1` to require a valid loopback `network_policies.proxy_ref`; native browser pools then launch Chromium with `--proxy-server=<proxy_ref>` and reject static or externally managed CDP endpoints unless `OPENCLAW_RUNTIME_BROWSER_EXTERNAL_PROXY_OK=1` accepts that external process as already proxy-controlled. This is a concrete proxy-backed browser egress path, but it is still process configuration, not a kernel or VM boundary. Set `OPENCLAW_RUNTIME_BROWSER_BROKER=cdp` and `OPENCLAW_SPARSEKERNEL_BROWSER_CDP_ENDPOINT=<loopback endpoint>` to make the OpenClaw browser tool acquire a real SparseKernel CDP context for the active run. Set `OPENCLAW_RUNTIME_BROWSER_BROKER=managed` to use the existing OpenClaw browser control service as the managed process owner and let SparseKernel lease CDP contexts from its reported endpoint. Set `OPENCLAW_RUNTIME_BROWSER_BROKER=native` to let SparseKernel launch and supervise a local Chromium-compatible process pool keyed by trust zone/profile, with process lifetime tied to brokered context leases and idle timeout. `OPENCLAW_SPARSEKERNEL_BROWSER_MAX_CONTEXTS` caps active contexts per native pool. Use `openclaw sparsekernel browser-pools` to inspect ledger pools plus in-process native pool refcounts, limits, start counts, clean stops, and crash counts. The runtime injects an internal browser proxy for supported navigation, tab, snapshot, console, screenshot, PDF, direct file-input upload, dialog, and action routes instead of exposing raw CDP to the agent. Brokered actions cover the OpenClaw action contract with CDP input events, bounded DOM evaluation, selector retry, per-target CDP-backed network-idle waiting, and post-action navigation checks. Same-target action navigations must stay inside the context's allowed origins when a policy is configured; same-policy popups are attached as broker-owned targets and disallowed popups are closed. Broker-owned targets and per-target console/network/artifact observations are persisted in first-class ledger tables and mirrored to audit events. CDP Fetch interception blocks out-of-policy requests when an allowed-origin policy is configured, but this remains request control rather than a hard security boundary. Closing a target releases the whole context only when no broker-owned targets remain. Screenshot and PDF outputs go through the artifact store.
+The broker applies configured trust-zone network policy to explicit allowed origins before allocating a context. It also denies unsupported URL schemes, private-network destinations when the policy disallows them, literal IPs matching denied CIDRs, and, when `OPENCLAW_RUNTIME_BROWSER_POLICY_ENFORCE=1` is set, hostnames that resolve to denied/private addresses. Set `OPENCLAW_RUNTIME_BROWSER_POLICY_DNS=0` only when a caller intentionally wants literal-host checks without DNS resolution. Set `OPENCLAW_RUNTIME_BROWSER_REQUIRE_PROXY=1` to require a valid loopback `network_policies.proxy_ref`; native browser pools then launch Chromium with `--proxy-server=<proxy_ref>` and reject static or externally managed CDP endpoints unless `OPENCLAW_RUNTIME_BROWSER_EXTERNAL_PROXY_OK=1` accepts that external process as already proxy-controlled. This is a concrete proxy-backed browser egress path, but it is still process configuration, not a kernel or VM boundary. Set `OPENCLAW_RUNTIME_BROWSER_BROKER=cdp` and `OPENCLAW_SPARSEKERNEL_BROWSER_CDP_ENDPOINT=<loopback endpoint>` to make the OpenClaw browser tool acquire a real SparseKernel CDP context for the active run. Set `OPENCLAW_RUNTIME_BROWSER_BROKER=managed` to use the existing OpenClaw browser control service as the managed process owner and let SparseKernel lease CDP contexts from its reported endpoint. Set `OPENCLAW_RUNTIME_BROWSER_BROKER=native` to let SparseKernel launch and supervise a local Chromium-compatible process pool keyed by trust zone/profile, with process lifetime tied to brokered context leases and idle timeout. `OPENCLAW_SPARSEKERNEL_BROWSER_MAX_CONTEXTS` caps active contexts per native pool. Use `openclaw sparsekernel browser-pools` to inspect ledger pools plus in-process native pool refcounts, limits, start counts, clean stops, and crash counts. The runtime injects an internal browser proxy for supported navigation, tab, snapshot, console, screenshot, PDF, direct file-input upload, dialog, and action routes instead of exposing raw CDP to the agent. Brokered actions cover the OpenClaw action contract with CDP input events, bounded DOM evaluation, selector retry with basic actionability checks, per-target CDP-backed network-idle waiting, and post-action navigation checks. Same-target action navigations must stay inside the context's allowed origins when a policy is configured; same-policy popups are attached as broker-owned targets and disallowed popups are closed. Broker-owned targets and per-target console/network/artifact observations are persisted in first-class ledger tables and mirrored to audit events. CDP Fetch interception blocks out-of-policy requests when an allowed-origin policy is configured, but this remains request control rather than a hard security boundary. Closing a target releases the whole context only when no broker-owned targets remain. Screenshot and PDF outputs go through the artifact store.
 
 ## Sandbox broker
 

packages/browser-broker/src/index.test.ts

@@ -1234,6 +1234,9 @@ describe("@openclaw/sparsekernel-browser-broker", () => {
     expect(actionExpression).toContain("waitForActionTarget");
     expect(actionExpression).toContain("const timeoutMs = 1234");
     expect(actionExpression).toContain("document.querySelector(selector)");
+    expect(actionExpression).toContain("getBoundingClientRect");
+    expect(actionExpression).toContain("aria-disabled");
+    expect(actionExpression).toContain("elementFromPoint");
   });
 
   it("waits for CDP network idle instead of treating document load as enough", async () => {

packages/browser-broker/src/index.ts

@@ -2404,7 +2404,7 @@ function buildActionExpression(
       normalizeMouseButton(request.kind === "click" ? request.button : undefined),
     );
     return `(async () => {
-  ${buildActionTargetHelpers(selectorJson, timeoutJson)}
+  ${buildActionTargetHelpers(selectorJson, timeoutJson, request.kind)}
   const node = await waitForActionTarget();
   node.scrollIntoView({ block: "center", inline: "center" });
   if (${JSON.stringify(request.kind)} === "scrollIntoView") return { ok: true };
@@ -2426,7 +2426,7 @@ function buildActionExpression(
     const submit = request.submit === true;
     const slowly = request.slowly === true;
     return `(async () => {
-  ${buildActionTargetHelpers(selectorJson, timeoutJson)}
+  ${buildActionTargetHelpers(selectorJson, timeoutJson, request.kind)}
   const node = await waitForActionTarget();
   node.scrollIntoView({ block: "center", inline: "center" });
   node.focus?.();
@@ -2460,7 +2460,7 @@ function buildActionExpression(
   if (request.kind === "select") {
     const values = JSON.stringify(request.values);
     return `(async () => {
-  ${buildActionTargetHelpers(selectorJson, timeoutJson)}
+  ${buildActionTargetHelpers(selectorJson, timeoutJson, request.kind)}
   const node = await waitForActionTarget();
   node.scrollIntoView({ block: "center", inline: "center" });
   const values = ${values};
@@ -2574,24 +2574,68 @@ function buildEvaluateExpression(
 })()`;
 }
 
-function buildActionTargetHelpers(selectorJson: string, timeoutJson: string): string {
+function buildActionTargetHelpers(selectorJson: string, timeoutJson: string, kind: string): string {
   return `const selector = ${selectorJson};
   const timeoutMs = ${timeoutJson};
+  const actionKind = ${JSON.stringify(kind)};
   const delay = (ms) => new Promise((resolve) => setTimeout(resolve, ms));
+  const rectSnapshot = (node) => {
+    const rect = node.getBoundingClientRect();
+    return { x: rect.x, y: rect.y, width: rect.width, height: rect.height };
+  };
   const isVisible = (node) => {
     if (!node?.isConnected) return false;
     const style = getComputedStyle(node);
-    if (style.visibility === "hidden" || style.display === "none") return false;
-    return node.getClientRects().length > 0;
+    if (style.visibility === "hidden" || style.display === "none" || style.pointerEvents === "none") return false;
+    const rect = rectSnapshot(node);
+    return rect.width > 0 && rect.height > 0 && node.getClientRects().length > 0;
+  };
+  const isEnabled = (node) => {
+    if (node.disabled === true) return false;
+    if (node.getAttribute?.("aria-disabled") === "true") return false;
+    return true;
+  };
+  const isEditable = (node) => {
+    if (node.isContentEditable) return true;
+    if (node instanceof HTMLTextAreaElement) return !node.readOnly && !node.disabled;
+    if (node instanceof HTMLInputElement) {
+      const type = String(node.type || "text").toLowerCase();
+      return !node.readOnly && !node.disabled && !["button", "checkbox", "color", "file", "hidden", "image", "radio", "range", "reset", "submit"].includes(type);
+    }
+    return false;
+  };
+  const receivesCenterHit = (node) => {
+    const rect = node.getBoundingClientRect();
+    const x = Math.min(Math.max(rect.left + rect.width / 2, 0), Math.max(window.innerWidth - 1, 0));
+    const y = Math.min(Math.max(rect.top + rect.height / 2, 0), Math.max(window.innerHeight - 1, 0));
+    const hit = document.elementFromPoint(x, y);
+    return !hit || hit === node || node.contains(hit);
+  };
+  const isStable = async (node) => {
+    const first = rectSnapshot(node);
+    await delay(50);
+    const second = rectSnapshot(node);
+    return Math.abs(first.x - second.x) < 0.5 &&
+      Math.abs(first.y - second.y) < 0.5 &&
+      Math.abs(first.width - second.width) < 0.5 &&
+      Math.abs(first.height - second.height) < 0.5;
+  };
+  const isActionable = async (node) => {
+    if (!isVisible(node)) return false;
+    if ((actionKind === "click" || actionKind === "type" || actionKind === "select") && !isEnabled(node)) return false;
+    if (actionKind === "type" && !isEditable(node)) return false;
+    if (!(await isStable(node))) return false;
+    if (actionKind !== "scrollIntoView" && !receivesCenterHit(node)) return false;
+    return true;
   };
   const waitForActionTarget = async () => {
     const deadline = Date.now() + timeoutMs;
     while (Date.now() <= deadline) {
       const node = document.querySelector(selector);
-      if (node && isVisible(node)) return node;
+      if (node && await isActionable(node)) return node;
       await delay(100);
     }
-    throw new Error("SparseKernel browser action target not found");
+    throw new Error("SparseKernel browser action target not actionable");
   };`;
 }