Trailblaze

Natural-language device control for your coding agent — across iOS, Android, and web. Every session is a replayable trail you can run as a test.

Point your coding agent — Claude Code, Cursor, Codex, Goose — at the trailblaze CLI and it can read any screen and act on it, on a real device. Save the session and you have a trail — one trail.yaml holding natural-language steps your team reads and recorded actions your CI replays deterministically — no LLM at replay time.

See it run

Real runs, generated by CI straight from the trails in this repo — click any timeline for the full interactive report (per-step screenshots, view hierarchy, video):

Android	iOS	Web

Install

brew install block/tap/trailblaze

That's it — bun (used for TypeScript custom tools) comes bundled with the formula. Or install from the GitHub release (requires Java 17+; the script installs bun for you if it's missing):

curl -fsSL https://raw.githubusercontent.com/block/trailblaze/main/install.sh | bash

Optional extras: ffmpeg for video capture in reports, esbuild for bundling scripted tools (brew install ffmpeg esbuild).

Drive a device in 60 seconds

# List connected devices (Android emulator, iOS simulator, or web browser)
trailblaze device list

# Pin this terminal to a device — subsequent calls inherit it
trailblaze device connect android

# Read the screen — returns a UI tree with refs (e.g. ab42) an agent can target
trailblaze snapshot

# Act on a referenced element. --step records intent, which powers self-heal later.
trailblaze tool tap ref=<ref-from-snapshot> -s "Tap sign in"

Paste those into any coding agent and it's already driving the device. When the flow works, keep it:

trailblaze session save -t "Login"    # saves one unified trail file: trails/login/trail.yaml
trailblaze run trails/login           # deterministic replay — no LLM, CI-ready

Teach your coding agent Trailblaze

Trailblaze ships an agent skill — skills/trailblaze/SKILL.md plus deep-dive references — that teaches a coding agent the CLI's conventions: driving a device, saving and replaying trails, and composing custom tools. It's written in the portable SKILL.md format that Claude Code, Codex, Cursor, Gemini CLI, Goose, and other agents all read. The skill is bundled into the CLI and versioned with the binary, so an installed binary (Homebrew or install.sh) hands your agent the exact instructions that match it — no repo checkout, no drift:

trailblaze skill show                   # print SKILL.md to stdout — always matches your CLI
trailblaze skill install                # Claude Code:        .claude/skills/trailblaze/
trailblaze skill install --agent agents # Codex/Cursor/Gemini CLI/Goose/…: .agents/skills/trailblaze/
trailblaze skill install --all          # both locations
trailblaze skill install --dir <path>   # any other layout

Two locations cover the field: Claude Code reads .claude/skills/, and every other standard-compliant agent (Codex, Cursor, Gemini CLI, Goose, OpenCode, …) reads the shared .agents/skills/. --agent also accepts codex/cursor/gemini/goose as aliases for that shared location.

For an agent, the zero-setup path is trailblaze skill show at the start of a task — it always reflects the CLI you're running, so upgrading the CLI (brew upgrade trailblaze) upgrades the skill. An installed copy is a committed snapshot instead; trailblaze skill status reports whether yours still matches the CLI, and re-running install refreshes it.

Working from a checkout instead? Copy skills/trailblaze/ into your project's skill directory (.claude/skills/, .agents/skills/, or wherever your agent discovers skills).

What you get

• Device control any agent can drive. snapshot to read, tool to act, on iOS, Android, and web — built-in primitives plus first-class custom commands like login or addToCart that you define in TypeScript with typed inputs and IDE support.
• Tests as a by-product. Any session becomes a trail — one trail.yaml holding the natural-language steps (what the flow does) plus a recording slot per platform (how it runs). Record the same flow on another platform and it merges into the same file. Commit it, and CI replays it deterministically.
• A cross-platform Trace Viewer. Every run — local or CI — produces a rich report: per-step screenshots, view hierarchy, video, platform logs, and the LLM transcript when an agent was involved. Same viewer for all three platforms; when a selector needs changing, pick from generated alternatives without re-recording.
• Self-heal instead of brittle failures. Replay is fail-loud by default. Opt in with --self-heal and the built-in agent patches small drift (text changes, a popup) against the live screen; bigger repairs go through your coding agent reading the trace, because every step recorded its natural-language intent.

Native fidelity on every platform

Most cross-platform tools expose the intersection of what iOS, Android, and web can do. Trailblaze exposes each platform's full native surface — the accessibility tree on Android, native UI semantics on iOS, the DOM on web — and lets the agent pick elements semantically ("the Sign in button") while it computes the platform-specific selector behind the scenes.

This only works because an agent is driving. Twenty selector strategies per element is no one's idea of a good testing SDK for humans; exposing them to an LLM is the point. You get native automation's expressive power with a single natural-language test.

How Trailblaze grows with you

You can stop at any rung:

1. Drive a device. Natural-language device control from your coding agent, through the CLI.
2. Save and replay. Sessions become committed trails — regression tests that are the same artifact ad-hoc, in CI, and in the Trace Viewer.
3. Compose your own agent surface. Ship trailmaps: typed custom commands, named waypoints for your screens, curated toolsets. Surface your login, hide the low-level taps — your agent gets more capable on every task that uses the composition. Share a trailmap across repos today; npm packaging for community distribution is in active development.

Examples — copy one to start

Runnable, standalone workspaces under examples/ — each a complete trailmap template with typed custom tools you can copy:

Example	Platform	What it teaches
examples/ios-contacts	iOS	Canonical mobile reference — typed scripted tools with unit tests (*.test.ts).
examples/wikipedia	Web	Canonical web reference — typed tools driving live en.wikipedia.org.
examples/playwright-native	Web	Smallest end-to-end scripted-tool setup, with a bundled sample app.

Desktop app

trailblaze app    # visual trail authoring, session browsing, report viewing

Active prototype: Waypoints

Waypoints — named, assertable app locations defined structurally, never by content — are landing now. Agents can ask "am I on the Inbox?", land on a waypoint after a step, or use waypoints as trail checkpoints. (devlog)

Documentation

Full docs at block.github.io/trailblaze:

• Getting Started — install to first trail
• CLI Reference — every command and flag
• Your First Trailmap — custom tools from an empty directory to a passing run
• Configuration — providers, devices, target apps

What Trailblaze is not

Trailblaze is not a coding agent — it ships a focused built-in agent (fine for many flows), but for serious authoring you want a full coding agent (Claude Code and Codex are popular examples) driving it with your codebase in context. And it's not a SaaS test platform: the trail YAML lives in your repo, you own it, you can read and edit it.

License

Apache 2.0