feat(sparsekernel): avoid base64 artifact helpers

Author: AbrahamGreenman

docs/architecture/artifact-store.md

@@ -26,6 +26,6 @@ Retention policies:
 
 Artifact reads and writes are capability-mediated where the caller is not the trusted runtime. Access grants are recorded in `artifact_access`.
 
-The local artifact store accepts bytes, streams, or file paths. File imports stream into a private temporary blob while computing sha256, then move into the content-addressed path, so downloads and snapshots do not need to be loaded fully into memory. The v0 `sparsekerneld` API exposes artifact create/read/metadata endpoints over local JSON. For small compatibility payloads, binary content can still be transported as base64. For large local payloads, use `/artifacts/import-file` with a file staged under the daemon-owned staging directory and `/artifacts/export-file` to copy content into the daemon-owned export directory without moving bytes through JSON.
+The local artifact store accepts bytes, streams, or file paths. File imports stream into a private temporary blob while computing sha256, then move into the content-addressed path, so downloads and snapshots do not need to be loaded fully into memory. The v0 `sparsekerneld` API exposes artifact create/read/metadata endpoints over local JSON. For small compatibility payloads, binary content can still be transported as base64. For large local payloads, use `/artifacts/import-file` with a file staged under the daemon-owned staging directory and `/artifacts/export-file` to copy content into the daemon-owned export directory without moving bytes through JSON. Node clients can use `@openclaw/sparsekernel-client/node-artifacts` to resolve the daemon-compatible staging directory, copy local files into it, and copy exported files to a caller destination without touching the base64 compatibility path.
 
 Browser broker adapters must route screenshots and downloads through this API. Agents should receive artifact ids and metadata, not raw browser download paths or large binary payloads.

docs/architecture/browser-broker.md

@@ -21,7 +21,7 @@ Set `OPENCLAW_RUNTIME_BROWSER_BROKER=native` to let SparseKernel launch and supe
 
 Set `OPENCLAW_RUNTIME_BROWSER_REQUIRE_PROXY=1` when a trust zone must use a proxy-backed browser egress path. The trust zone's network policy must contain a loopback `proxy_ref`, and native browser pools launch Chromium with `--proxy-server=<proxy_ref>`. Static or externally managed CDP endpoints are rejected in this mode unless `OPENCLAW_RUNTIME_BROWSER_EXTERNAL_PROXY_OK=1` asserts that the external browser process is already proxy-controlled. This protects the SparseKernel-owned browser process path; it is not host-level egress enforcement for arbitrary host processes.
 
-Supported v0 actions (`status`, `doctor`, `profiles`, `tabs`, `open`, `navigate`, `focus`, `close`, `snapshot`, `console`, `screenshot`, `pdf`, direct file-input `upload`, `dialog`, and brokered `act`) operate against broker-owned targets inside the leased CDP context. Brokered `act` covers the OpenClaw action contract for click, coordinate click, type, press, hover, scroll, drag, select, fill, resize, wait, evaluate, close, and batch using CDP input events plus bounded DOM evaluation. Selector-backed actions retry inside the leased page until their action timeout and now require basic actionability before dispatch: visible connected target, stable bounding box, enabled form state where relevant, editable target for typing, and center-point hit testing. `wait --load networkidle` uses per-target CDP Network events plus a quiet window rather than only checking `document.readyState`. Actions that can change page state are followed by a broker-side navigation check: same-target navigations are accepted only when the resulting URL stays inside the context's allowed-origin policy, same-policy popups are attached as broker-owned targets, and disallowed popups are closed. When an allowed-origin policy is configured, the broker also enables CDP Fetch interception and fails requests outside that policy while recording `browser_network.blocked` observations; this is request control, not host isolation. Before opening or navigating, the ToolBroker checks the trust-zone network policy and denies unsupported schemes, private-network destinations when disallowed, literal denied CIDRs, and, when runtime policy enforcement is enabled, hostnames that resolve to denied/private addresses. Proxy-backed egress control remains future work. Snapshots use a bounded CDP `Runtime.evaluate` DOM read, actions resolve refs from the latest brokered snapshot where needed, console output is captured from CDP runtime/log events per target, and screenshot/PDF output is captured as SparseKernel artifacts, read back through artifact access, and converted to existing tool result formats for compatibility. Closing a broker-owned target now closes that target; the full browser context is released only when the last target closes or broker cleanup runs.
+Supported v0 actions (`status`, `doctor`, `profiles`, `tabs`, `open`, `navigate`, `focus`, `close`, `snapshot`, `console`, `screenshot`, `pdf`, direct file-input `upload`, `dialog`, and brokered `act`) operate against broker-owned targets inside the leased CDP context. Brokered `act` covers the OpenClaw action contract for click, coordinate click, type, press, hover, scroll, drag, select, fill, resize, wait, evaluate, close, and batch using CDP input events plus bounded DOM evaluation. Selector-backed actions retry inside the leased page until their action timeout and now require basic actionability before dispatch: visible connected target, stable bounding box, enabled form state where relevant, editable target for typing, and center-point hit testing. Selector click and hover resolve an actionable center point in the leased page and dispatch real CDP mouse events rather than handing raw DOM click events to page code. `wait --load networkidle` uses per-target CDP Network events plus a quiet window rather than only checking `document.readyState`. Actions that can change page state are followed by a broker-side navigation check: same-target navigations are accepted only when the resulting URL stays inside the context's allowed-origin policy, same-policy popups are attached as broker-owned targets, and disallowed popups are closed. When an allowed-origin policy is configured, the broker also enables CDP Fetch interception and fails requests outside that policy while recording `browser_network.blocked` observations; this is request control, not host isolation. Before opening or navigating, the ToolBroker checks the trust-zone network policy and denies unsupported schemes, private-network destinations when disallowed, literal denied CIDRs, and, when runtime policy enforcement is enabled, hostnames that resolve to denied/private addresses. Proxy-backed egress control remains future work. Snapshots use a bounded CDP `Runtime.evaluate` DOM read, actions resolve refs from the latest brokered snapshot where needed, console output is captured from CDP runtime/log events per target, and screenshot/PDF output is captured as SparseKernel artifacts, read back through artifact access, and converted to existing tool result formats for compatibility. Closing a broker-owned target now closes that target; the full browser context is released only when the last target closes or broker cleanup runs.
 
 Use `openclaw sparsekernel browser-pools` to inspect durable ledger pools and currently materialized native browser process pools. Native pool snapshots include trust zone, profile, active refs, max context slots, idle timeout, endpoint, PID when available, last activity, start count, clean stop count, and crash count.
 

packages/browser-broker/src/index.test.ts

@@ -327,7 +327,7 @@ class FakeCdpTransport implements CdpTransport {
       this.nextActionNavigationUrl = undefined;
       this.nextActionNewTarget = undefined;
       this.respond(message.id, {
-        result: { value: { ok: true } },
+        result: { value: { ok: true, x: 42, y: 24 } },
       });
       if (navigationUrl) {
         setTimeout(() => this.emitFrameNavigation(navigationUrl, message.sessionId), 0);

packages/browser-broker/src/index.ts

@@ -329,6 +329,11 @@ type PostActionNavigationObservation =
   | { kind: "same-target"; url?: string }
   | { kind: "new-target"; targetId: string; url?: string };
 
+type CdpActionPoint = {
+  x: number;
+  y: number;
+};
+
 export class SparseKernelCdpBrowserBroker {
   private readonly kernel: SparseKernelBrowserKernelClient;
   private readonly fetchImpl: typeof fetch;
@@ -802,8 +807,34 @@ export class SparseKernelCdpBrowserBroker {
     }
     switch (request.kind) {
       case "click":
+      case "hover": {
+        return await this.withPostActionNavigationGuard(context, request.timeoutMs, async () => {
+          const selector = this.resolveActionSelector(context, request);
+          const evaluated = await context.connection.command<{ result?: { value?: unknown } }>(
+            "Runtime.evaluate",
+            {
+              expression: buildActionPointExpression(
+                request.kind,
+                selector,
+                resolveCdpActionTimeoutMs(request.timeoutMs),
+              ),
+              returnByValue: true,
+              awaitPromise: true,
+            },
+            context.page_session_id,
+            resolveCdpCommandTimeoutMs(request.timeoutMs),
+          );
+          const point = parseCdpActionPoint(evaluated.result?.value);
+          await dispatchCdpMouseAction(context, request, point);
+          return {
+            ok: true,
+            targetId: context.target_id,
+            kind: request.kind,
+            value: evaluated.result?.value,
+          };
+        });
+      }
       case "type":
-      case "hover":
       case "scrollIntoView":
       case "select": {
         return await this.withPostActionNavigationGuard(context, request.timeoutMs, async () => {
@@ -832,43 +863,7 @@ export class SparseKernelCdpBrowserBroker {
       }
       case "clickCoords": {
         return await this.withPostActionNavigationGuard(context, undefined, async () => {
-          const button = normalizeMouseButton(request.button);
-          const clickCount = request.doubleClick ? 2 : 1;
-          await context.connection.command(
-            "Input.dispatchMouseEvent",
-            {
-              type: "mouseMoved",
-              x: request.x,
-              y: request.y,
-              button: "none",
-            },
-            context.page_session_id,
-          );
-          await context.connection.command(
-            "Input.dispatchMouseEvent",
-            {
-              type: "mousePressed",
-              x: request.x,
-              y: request.y,
-              button,
-              clickCount,
-            },
-            context.page_session_id,
-          );
-          if (request.delayMs && request.delayMs > 0) {
-            await delay(Math.min(5_000, Math.floor(request.delayMs)));
-          }
-          await context.connection.command(
-            "Input.dispatchMouseEvent",
-            {
-              type: "mouseReleased",
-              x: request.x,
-              y: request.y,
-              button,
-              clickCount,
-            },
-            context.page_session_id,
-          );
+          await dispatchCdpMouseAction(context, request, { x: request.x, y: request.y });
           return { ok: true, targetId: context.target_id, kind: request.kind };
         });
       }
@@ -1874,6 +1869,53 @@ function normalizeMouseButton(button: string | undefined): "left" | "right" | "m
   return normalized === "right" || normalized === "middle" ? normalized : "left";
 }
 
+async function dispatchCdpMouseAction(
+  context: LiveBrowserContext,
+  request: Extract<SparseKernelBrowserActRequest, { kind: "click" | "clickCoords" | "hover" }>,
+  point: CdpActionPoint,
+): Promise<void> {
+  await context.connection.command(
+    "Input.dispatchMouseEvent",
+    {
+      type: "mouseMoved",
+      x: point.x,
+      y: point.y,
+      button: "none",
+    },
+    context.page_session_id,
+  );
+  if (request.kind === "hover") {
+    return;
+  }
+  const button = normalizeMouseButton(request.button);
+  const clickCount = request.doubleClick ? 2 : 1;
+  await context.connection.command(
+    "Input.dispatchMouseEvent",
+    {
+      type: "mousePressed",
+      x: point.x,
+      y: point.y,
+      button,
+      clickCount,
+    },
+    context.page_session_id,
+  );
+  if (request.delayMs && request.delayMs > 0) {
+    await delay(Math.min(5_000, Math.floor(request.delayMs)));
+  }
+  await context.connection.command(
+    "Input.dispatchMouseEvent",
+    {
+      type: "mouseReleased",
+      x: point.x,
+      y: point.y,
+      button,
+      clickCount,
+    },
+    context.page_session_id,
+  );
+}
+
 function normalizeAllowedOrigins(value: unknown): string[] {
   if (!Array.isArray(value)) {
     return [];
@@ -2482,6 +2524,37 @@ function buildActionExpression(
   throw new Error(`SparseKernel CDP browser action does not support ${request.kind} yet.`);
 }
 
+function buildActionPointExpression(
+  kind: "click" | "hover",
+  selector: string,
+  timeoutMs: number,
+): string {
+  return `(async () => {
+  ${buildActionTargetHelpers(JSON.stringify(selector), JSON.stringify(timeoutMs), kind)}
+  const node = await waitForActionTarget();
+  node.scrollIntoView({ block: "center", inline: "center" });
+  await delay(0);
+  const target = await waitForActionTarget();
+  const rect = target.getBoundingClientRect();
+  const x = Math.min(Math.max(rect.left + rect.width / 2, 0), Math.max(window.innerWidth - 1, 0));
+  const y = Math.min(Math.max(rect.top + rect.height / 2, 0), Math.max(window.innerHeight - 1, 0));
+  return { ok: true, x, y, rect: { left: rect.left, top: rect.top, width: rect.width, height: rect.height } };
+})()`;
+}
+
+function parseCdpActionPoint(value: unknown): CdpActionPoint {
+  if (!value || typeof value !== "object") {
+    throw new Error("SparseKernel browser action target did not return coordinates");
+  }
+  const record = value as { x?: unknown; y?: unknown };
+  const x = Number(record.x);
+  const y = Number(record.y);
+  if (!Number.isFinite(x) || !Number.isFinite(y)) {
+    throw new Error("SparseKernel browser action target returned invalid coordinates");
+  }
+  return { x, y };
+}
+
 function buildDragExpression(
   startSelector: string,
   endSelector: string,
@@ -2632,7 +2705,12 @@ function buildActionTargetHelpers(selectorJson: string, timeoutJson: string, kin
     const deadline = Date.now() + timeoutMs;
     while (Date.now() <= deadline) {
       const node = document.querySelector(selector);
-      if (node && await isActionable(node)) return node;
+      if (node) {
+        if (actionKind !== "scrollIntoView") {
+          node.scrollIntoView({ block: "center", inline: "center" });
+        }
+        if (await isActionable(node)) return node;
+      }
       await delay(100);
     }
     throw new Error("SparseKernel browser action target not actionable");

packages/sparsekernel-client/package.json

@@ -4,6 +4,7 @@
   "private": true,
   "type": "module",
   "exports": {
-    ".": "./src/index.ts"
+    ".": "./src/index.ts",
+    "./node-artifacts": "./src/node-artifacts.ts"
   }
 }

packages/sparsekernel-client/src/node-artifacts.test.ts

@@ -0,0 +1,129 @@
+import { access, mkdir, mkdtemp, readFile, rm, writeFile } from "node:fs/promises";
+import { tmpdir } from "node:os";
+import path from "node:path";
+import { describe, expect, it } from "vitest";
+import type {
+  SparseKernelArtifact,
+  SparseKernelExportArtifactFileResult,
+  SparseKernelImportArtifactFileInput,
+} from "./index.js";
+import {
+  createArtifactFromLocalFile,
+  defaultSparseKernelArtifactStagingDir,
+  exportArtifactToLocalFile,
+} from "./node-artifacts.js";
+
+describe("SparseKernel Node artifact helpers", () => {
+  it("stages local file imports without base64 transport and cleans the stage", async () => {
+    const root = await mkdtemp(path.join(tmpdir(), "sparsekernel-client-test-"));
+    try {
+      const source = path.join(root, "source.bin");
+      const stagingDir = path.join(root, "staging");
+      await writeFile(source, "large artifact");
+      let imported: SparseKernelImportArtifactFileInput | undefined;
+      let stagedBody = "";
+      const client = {
+        async importArtifactFile(
+          input: SparseKernelImportArtifactFileInput,
+        ): Promise<SparseKernelArtifact> {
+          imported = input;
+          stagedBody = await readFile(input.staged_path, "utf8");
+          return {
+            id: "artifact-1",
+            sha256: "a".repeat(64),
+            size_bytes: stagedBody.length,
+            storage_ref: "sha256/aa/aa/hash",
+            mime_type: input.mime_type,
+            retention_policy: input.retention_policy,
+            created_at: "2026-04-29T00:00:00Z",
+          };
+        },
+      };
+
+      const artifact = await createArtifactFromLocalFile(client, {
+        filePath: source,
+        stagingDir,
+        stagedName: "unsafe/name?.bin",
+        mime_type: "application/octet-stream",
+        retention_policy: "durable",
+      });
+
+      expect(artifact).toMatchObject({ id: "artifact-1", size_bytes: 14 });
+      expect(stagedBody).toBe("large artifact");
+      expect(imported?.staged_path.startsWith(stagingDir)).toBe(true);
+      expect(path.basename(imported?.staged_path ?? "")).toBe("name_.bin");
+      await expect(access(path.dirname(imported?.staged_path ?? ""))).rejects.toThrow();
+    } finally {
+      await rm(root, { recursive: true, force: true });
+    }
+  });
+
+  it("copies daemon staged exports to a caller destination and cleans the exported file", async () => {
+    const root = await mkdtemp(path.join(tmpdir(), "sparsekernel-client-test-"));
+    try {
+      const exportDir = path.join(root, "exports");
+      const stagedPath = path.join(exportDir, "artifact.bin");
+      const destination = path.join(root, "downloads", "artifact.bin");
+      await mkdir(exportDir, { recursive: true });
+      await writeFile(stagedPath, "exported artifact");
+      const client = {
+        async exportArtifactFile(input: {
+          id: string;
+          file_name?: string | null;
+        }): Promise<SparseKernelExportArtifactFileResult> {
+          expect(input).toMatchObject({ id: "artifact-1", file_name: "artifact.bin" });
+          return {
+            artifact: {
+              id: "artifact-1",
+              sha256: "b".repeat(64),
+              size_bytes: 17,
+              storage_ref: "sha256/bb/bb/hash",
+              created_at: "2026-04-29T00:00:00Z",
+            },
+            staged_path: stagedPath,
+          };
+        },
+      };
+
+      const exported = await exportArtifactToLocalFile(client, {
+        id: "artifact-1",
+        destinationPath: destination,
+      });
+
+      expect(exported.artifact.id).toBe("artifact-1");
+      await expect(readFile(destination, "utf8")).resolves.toBe("exported artifact");
+      await expect(access(stagedPath)).rejects.toThrow();
+    } finally {
+      await rm(root, { recursive: true, force: true });
+    }
+  });
+
+  it("exposes the staging directory used by local Node clients", () => {
+    const previousHome = process.env.SPARSEKERNEL_HOME;
+    const previousState = process.env.OPENCLAW_STATE_DIR;
+    const previousStaging = process.env.SPARSEKERNEL_ARTIFACT_STAGING_DIR;
+    try {
+      process.env.SPARSEKERNEL_HOME = path.join("tmp", "sparsekernel-home");
+      delete process.env.OPENCLAW_STATE_DIR;
+      delete process.env.SPARSEKERNEL_ARTIFACT_STAGING_DIR;
+      expect(defaultSparseKernelArtifactStagingDir()).toBe(
+        path.join("tmp", "sparsekernel-home", "artifacts", ".staging"),
+      );
+
+      process.env.SPARSEKERNEL_ARTIFACT_STAGING_DIR = path.join("tmp", "custom-staging");
+      expect(defaultSparseKernelArtifactStagingDir()).toBe(path.join("tmp", "custom-staging"));
+    } finally {
+      restoreEnv("SPARSEKERNEL_HOME", previousHome);
+      restoreEnv("OPENCLAW_STATE_DIR", previousState);
+      restoreEnv("SPARSEKERNEL_ARTIFACT_STAGING_DIR", previousStaging);
+    }
+  });
+});
+
+function restoreEnv(name: string, value: string | undefined): void {
+  if (value === undefined) {
+    delete process.env[name];
+    return;
+  }
+  process.env[name] = value;
+}