Skip to content

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

89 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

CostMarshal — cost-aware multi-agent orchestration for Codex

CostMarshal

Give every task the right model—not the most expensive model.

Two-level Codex-native orchestration: CostMarshal routes across APIs, while native Codex child agents parallelize each bounded attempt.

CI Version 5.1.2 Python 3.11+ MIT License

Install · Use it · How it works · Providers · Documentation


Install with one Codex prompt

Open Codex, paste this, and let Codex handle the installation and validation:

Install CostMarshal from https://github.com/yptang98/CostMarshal.
Follow INSTALL_PROMPT.md exactly: pin the reviewed commit, preserve my existing
runtime state and secrets, validate the plugin, then tell me how to use it from
Codex without requiring Python or CostMarshal CLI commands.

That prompt covers both a first install and an existing pinned installation. Start a new Codex task afterward so the plugin Skill is discovered.

Note

The hidden runtime requires Python 3.11+. Git is required for writable worker worktrees. Production worker isolation additionally requires Docker or Podman with Linux containers.

Prefer to install manually?

The recommended path is the prompt above. For a reviewed, commit-pinned first install:

codex plugin marketplace add yptang98/CostMarshal --ref <reviewed-40-character-commit>
codex plugin add costmarshal@costmarshal
codex plugin list --json

For updates, follow INSTALL_PROMPT.md. It preserves both current and legacy runtime roots and replaces only the pinned plugin snapshot.

Use from Codex

CostMarshal is designed to be operated in natural language:

Use CostMarshal to complete this task with the best cost-quality tradeoff
across low, medium, and high APIs.

You can also invoke the Skill explicitly:

$orchestrate-cost-aware-agents plan and execute this task within a 30 CNY budget.

No Python commands are required for normal use. The CLI is retained for runtime, recovery, automation, and diagnostics—not as a requirement for ordinary users.

Why CostMarshal

Capability What it gives you
💸 Cost-aware routing Chooses the safest economical provider chain from reviewed prices, token forecasts, and acceptance history.
🧭 Three routing tiers Routes bounded work across low, medium, and high cost/safety tiers without treating price as model intelligence.
🛡️ Safety floors Risk, difficulty, task type, and required capabilities can raise the minimum tier; cost never lowers it.
Leader-owned acceptance Workers report results, but only the Codex leader can accept, reject, continue, or apply changes.
♻️ Durable recovery Actors, attempts, mailboxes, budgets, reports, and recovery state survive interrupted sessions.
🔒 Bounded execution Write claims, sealed routes, generation fencing, and optional OCI isolation constrain worker scope.
🧠 Evidence-backed evolution Work graphs, artifact gates, six-dimensional scoring, model memory, and staged policy promotion improve later routing without self-authorizing changes.
📦 Project continuity Transcript-free Leader Snapshots, structured handoffs, atomic batch acceptance, and immutable artifact lineage keep long projects moving.
📚 Accepted knowledge Charter, architecture, ADR, interface, fact, risk, and milestone indexes can only come from accepted evidence or an explicit Leader decision.
🏗️ Large-project coordination Binds tasks to immutable repository identities and bounded Workstreams, then admits milestones through staged integration Gates.

Total-cost reports complement routing estimates with observable project economics: known monetary cost per accepted Artifact, execution and review tokens/time, retries, handoffs, Leader attention, and failure/recovery counts. Unknown costs remain explicit; CostMarshal never assigns invented prices to time, tokens, or context.

How it works

flowchart LR
    A[Your task in Codex] --> B[CostMarshal Skill]
    B --> C{Safety + cost routing}
    C -->|provider-backed agent| D[API Codex attempt]
    C -->|signed-in execution| E[Codex attempt]
    C -->|report-only| J[Proposal API worker]
    D --> H[Native child agents]
    E --> I[Native child agents]
    D --> F[Codex Leader review]
    E --> F
    J --> F
    F -->|accept| G[Verified result]
    F -->|reject + admitted successor| C
Loading
  1. Plan — Codex turns the request into bounded tasks, write scopes, budgets, and acceptance criteria.
  2. Route — CostMarshal applies a fail-closed safety floor, then compares valid non-decreasing provider chains.
  3. Execute — A task-scoped actor receives only its bound prompt, provider profile, and allowed paths. After the compatibility handshake, it may use a bounded attempt-local native child team.
  4. Review — The Codex leader inspects sealed evidence and explicitly accepts or rejects the attempt.
  5. Recover — Durable on-disk state allows the scheduler to resume without relying on chat memory.
  6. Learn — Accepted and rejected attempts become auditable evaluations; aggregate model profiles inform later routing and teaching decisions.

Project continuity does not take over your global workspace. CostMarshal keeps only current-project metadata: small local artifacts remain in place and are referenced by hash; large outputs remain on external storage; summaries and Skill candidates retain explicit lineage. CostMarshal never installs global Skills or moves source project files.

Large projects can register multiple Git repository roots and group their work into Workstreams with dependency, concurrency, and CNY allocation limits. Repository registration is metadata-only: CostMarshal does not move, adopt, or reorganize source projects. Cross-repository integration is deliberately staged rather than described as atomic. A Leader-approved integration Gate checks the complete Workstream task set, accepted interface Artifacts, exact repository heads, and real rollback commits before it unlocks dependent work.

Repeated successes can become a project-local Skill Candidate with explicit applicability, inputs, steps, verification, failure boundaries, and evidence. Export is a separate user-triggered preview/apply action that materializes only inside the CostMarshal project; installation remains the responsibility of an external reviewed Skill-management workflow.

Routing at a glance

Work profile Minimum tier
Low-risk bounded analysis, extraction, docs, tests, verification, or small edits Low
Medium risk, implementation, review, or code review Medium
High risk, hard difficulty, or an explicit --major-decision High
Unknown or judgment-heavy work Medium

New projects default to completion-first: the admitted route retains a strongest-compatible terminal fallback, while acceptance at an earlier step stops further spend. Provider repetition and tier downgrade are always rejected.

Leader and expert tier

The main leader does not have to be the strongest model. init accepts --leader-provider, --leader-model, and --leader-profile, and any project can change the leader later with configure-leader (preview with --dry-run). The default remains the signed-in Codex model; a user can point the leader at a relatively strong provider such as DeepSeek through a named Codex config profile:

python scripts/costmarshal.py init ... --leader-provider deepseek --leader-model deepseek-v4-pro --leader-profile deepseek
python scripts/costmarshal.py configure-leader --project <project-dir> --provider deepseek --model deepseek-v4-pro --profile deepseek
python scripts/costmarshal.py start-leader --project <project-dir> --dry-run

A non-Codex leader keeps the same workspace tools, sandbox, budget reservation, and evidence contract because it executes through Codex CLI with that provider's profile; only the model changes. The persisted policy is shown in the leader actor prompt and in status. Real Codex profiles are accepted as-is: standard non-secret fields such as model_context_window and model_catalog_json are allowed, and a profile may inherit its provider endpoint/key contract from the shared config.toml. Unknown or credential- bearing settings still fail closed.

Codex's built-in strongest models are advanced experts, not default execution. The default catalog marks the Codex provider expert_only: automatic routing uses it as the first step only when the safe floor is high (risk high, difficulty hard, or an explicit --major-decision on new-task/route), or when no non-expert provider can serve the task. Later chain steps are reached only through explicit leader-authorized escalation. This keeps the strongest Codex calls reserved for high-difficulty work and genuinely major decisions (final acceptance, integration Gates, architecture, security review).

Token accounting for leader and agent executions uses the Codex CLI's own reported usage from its JSON events; CostMarshal does not guess token counts on those paths. Provider-reported usage remains authoritative only for report-only gateway executions where Codex is not the executing agent.

Routing and budget model

When enabled providers have reviewed prices and the task includes non-zero token estimates, CostMarshal evaluates every valid non-decreasing chain of one to three distinct providers:

expected_chain_cost = C1 + (1-P1)C2 + (1-P1)(1-P2)C3
success_probability = 1 - product(1-Pi)
objective = expected_chain_cost / success_probability

Pi comes only from audited leader result records. New records use a stricter quality-aware routing outcome: acceptance, gate passage, quality, and error severity must all agree. Missing pricing or token estimates never produce an invented cost; routing falls back to the minimum safe tier, and budgeted dispatch fails closed if it cannot form an eligible estimate.

CostMarshal reserves the full admitted chain estimate before first dispatch. Every step binds its own token forecast, reviewed price snapshot, provider identity, profile hash, and acceptance evidence. A rejected result can continue only to the exact next provider in the sealed route, and only after explicit leader authorization.

For the complete routing and accounting contract, read the repository-level SKILL.md and inspect the CLI help.

Providers

CostMarshal includes reviewed API presets for common providers. A preset knows the endpoint, protocol, key variable, current model, and capabilities—but never contains a key or an unreviewed price.

Provider Included models API input CostMarshal input
DeepSeek V4 Flash / Pro Text Text through the production Chat adapter
Kimi K3 / K2.6 Text, image; K2.6 also video Text/image through the production Chat adapter
LongCat 2.0 Text only (documented and live-probed) Text
Xiaomi MiMo 2.5 / 2.5 Pro 2.5: text, image, audio, video Text/image Agent; image/audio/video report-only API
Doubao Ark Seed 2.0 Lite Text, image, audio, video Text/image Agent; image/audio/video report-only API
Codex Native signed-in model Model-dependent Text

low, medium, and high are routing authority and cost/safety tiers, not intelligence labels. LongCat can be placed in any reviewed tier. CostMarshal then learns its actual acceptance rate, quality, efficiency, errors, and task fit separately for each exact model/profile/task/role scope.

For bounded text analysis, debugging, implementation proposals, and first-pass review, the LongCat proposal-api path sends only explicitly allowlisted blobs from the repository's committed HEAD, gives the model no tools or write scope, records authoritative Chat usage even when output is truncated, and requires Codex Leader or Codex Worker review before anything is applied. The normal Codex Worker remains the strong execution path and reuses the user's existing signed-in Codex login without requiring a misleading CODEX_API_KEY.

Ask Codex to configure the providers and assign tiers without exposing keys:

Configure CostMarshal with DeepSeek, Kimi, LongCat, MiMo, and Doubao as
appropriate. Inspect the built-in provider presets, assign reviewed low/medium/
high tiers, keep credentials outside actor workspaces, and require an exact
input:image capability for tasks that include images.

Modern Codex workers use the OpenAI Responses protocol. MiMo and Doubao expose it officially, and the current LongCat deployment uses it. For a reviewed DeepSeek or Kimi Chat endpoint, the v4.3 production Proxy can translate bounded Responses text/image/audio messages, function tools, completed JSON, and SSE events by setting wire_api: chat-completions. The adapter buffers the bounded upstream completion before emitting Responses SSE, so it is compatible but not token-by-token realtime. Video and document input require a native Responses provider and are rejected by the Chat adapter.

Codex accepting image attachments does not make every configured provider visual.

Codex v5 compatibility

CostMarshal v5 deliberately separates the two orchestration layers:

  • CostMarshal owns provider selection, credential and profile binding, budget reservation, leases, recovery, evidence, and final Leader acceptance.
  • Native Codex child agents are optional and live only inside one admitted attempt. They inherit that attempt's provider, model, sandbox, context, deadline, and budget; they cannot switch to another API or create a nested CostMarshal route.

The production worker pins @openai/codex 0.145.0 and performs a fail-closed version handshake before enabling native children. Diagnose a host installation without making a provider call:

python scripts/costmarshal.py codex-native-status --require-app-server

native_exec_ready is the execution requirement. app_server_ready reports whether the same installation can also be embedded through Codex App Server; CostMarshal's deterministic scheduler remains the authority in either case. LongCat's current Chat API documentation specifies text-only input. A 2026-07-27 live probe sent both a local PNG data URI and a public image URL through LongCat-2.0 Responses, plus a public image URL through Chat Completions. All requests returned HTTP 200, but the model reported that no image was available. CostMarshal therefore keeps LongCat text-only. An HTTP success, accepted JSON field, or advertised Responses transport must never auto-grant input:image; a reviewed semantic probe must prove that the model actually perceived a challenge image.

Multimodal is enforced end to end: routing uses the intersection of the model's documented API capabilities and the selected execution adapter. Agent mode supports committed local images. Gateway-bound native Responses providers can instead use report-only multimodal-api mode for committed image, audio, video, and provider-supported document inputs. It runs only in strongly isolated OCI, has a 2 MiB total attachment envelope, cannot claim write paths or invoke tools, and accepts a result only when the hard-budget Proxy returns authoritative usage plus a settled receipt. The Chat adapter still excludes video and document. See references/providers.md for exact models and commands.

Provider metadata also has a fail-closed lifecycle. Bounded schema, pricing, capability, and behavior observations may automatically block or de-prioritize a route, but never grant new authority. A provider is restored or updated only through an expiring, human-reviewed catalog row bound to those observations.

Provider profile and catalog setup

The internal CLI can inspect presets and create profiles without storing API keys:

python scripts/costmarshal.py provider-presets
python scripts/costmarshal.py provider-presets --preset mimo

python scripts/costmarshal.py configure-provider `
  --preset mimo-v2.5 `
  --profile mimo `
  --tier medium

# Chat-only providers are bound to the enforced CostMarshal gateway:
python scripts/costmarshal.py configure-provider `
  --preset kimi-k2.6 `
  --profile kimi-gateway `
  --tier high `
  --via-production-gateway

Budgeted routing requires a hash-bound pricing snapshot for each enabled provider. Use costmarshal_v2.routing.build_pricing_snapshot(...) to canonicalize reviewed CNY rates and timestamps; do not hand-edit snapshot hashes. Expired, future-effective, mixed-currency, malformed, or incomplete pricing fails closed.

Provider identity is separate from capability tier, so the catalog can be replaced without changing routing policy. Native Codex execution reuses an actor-private copy of the existing Codex login. Because an isolated OCI worker must not inherit the host session, its explicit OpenAI API path uses the standard OPENAI_API_KEY. Direct-provider credentials are supplied through the process environment or an external secrets file. Gateway-bound catalog rows carry runtime_adapter: costmarshal-gateway-v1; dispatch fails closed unless the project has a ready, enforced, externally certified production boundary. Workers then receive only a scoped lease. No credential is written into profiles, prompts, reports, or repository files.

The home directory resolution order is an explicit --codex-home, then non-empty CODEX_HOME, then ~/.codex. Service and container launches should use an absolute CODEX_HOME.

Safety and trust boundary

Important

In production gateway mode, Workers receive only a short-lived provider/model/token/budget-scoped lease; the Provider Proxy alone holds the real provider key. Legacy raw-key OCI mode remains available only outside an enforced production boundary.

  • Production workers require an attested Docker/Podman Linux-container boundary and never silently fall back to a native process.
  • Workers cannot accept results, authorize additional provider spend, broaden their write scope, or apply their own changes.
  • Writable changes are previewed in a detached Git worktree and verified by path, blob, and executable mode before explicit application.
  • The production proxy atomically reserves a conservative request envelope before every upstream call, requires explicit token caps, treats unknown usage as fully spent, and disables an overrun lease. Legacy scheduler-only estimates are not a hard external cap.
  • Real-provider backtests and live malicious-container evidence are still required for deployment-specific production certification. Local and mocked tests are not treated as that proof.
  • v4.1 includes an mTLS SPIFFE Credential Broker, signed short-lived leases, a provider-key-isolating hard-budget Proxy, live policy-hash health probes, and an OCI Actor adapter.
  • v4.2 separates runtime readiness from certification. Enforced dispatch also requires a short-lived OpenSSH-signed canonical claim bound to the exact commit, release, boundary, policy, Worker/gateway image digests, reviewed trust-root hash, and all five accepted production-evidence report hashes. Artifact IDs or self-asserted JSON can no longer certify a deployment.
  • v4.3 adds immutable non-image inputs, safety-only Provider drift guardrails, and a preview-first production deployment preflight. These controls do not replace deployment-specific real-provider, OCI, and external-signature evidence.
  • v4.3.1 makes v4 a CI-protected branch, provides a manual GHCR build that publishes only digest-addressed Worker/Gateway images with SBOM and provenance, and requires both production services to become healthy before deployment can succeed.
  • v4.3.2 binds the Compose Proxy network to the OCI Worker trust contract and makes deployment success require certificate/key validation, immutable network attestation, and live policy-hash TLS health from both services.
  • v4.3.3 commits an expiring production-build review, pins the complete Codex npm dependency graph, verifies the selected linux/amd64 base manifests, and makes CI build and exercise the actual hardened Gateway and Worker images.
  • CostMarshal records an idempotent local evolution cycle after each accepted Leader result, creates transcript-free Leader Snapshots from decision events, and gives Leader startup a bounded Hot/Warm context view while Cold references remain indexed but unloaded. These actions create no provider calls, tasks, or active policy changes.

Read SECURITY.md before production use.

Architecture

Component Responsibility Boundary
Codex Skill Converts natural-language intent into bounded orchestration Normal user-facing product surface
Scheduler Relays messages, enforces locks, records state, and launches fenced effects Never plans, reviews, or calls a model itself
Codex Leader Plans, reviews, integrates, and accepts at explicit gates The current Codex agent retains final authority
Codex Worker Performs strong independent implementation or review Uses the built-in signed-in Codex provider; no invented API-key requirement
Provider Worker Produces one bounded proposal with LongCat or another reviewed API Cannot broaden context, mutate control state, apply its own output, or self-authorize continuation
Work Graph Tracks dependencies, roles, readiness, and accepted joins A blocked package cannot dispatch
Artifact & Gate Engine Registers content-addressed outputs and evaluates deterministic acceptance policy Leader acceptance cannot override a failed configured gate
Evolution Engine Records scores/errors, rebuilds cross-project model profiles, chooses teaching policy, and proposes candidates Observations never activate policy directly
Cost Engine Builds evidence-bound total-cost snapshots per accepted Artifact Unknown monetary observations remain explicit; time and tokens are never assigned invented prices
Repository & Workstream Registry Binds task ownership, dependencies, concurrency, and budget allocations across project repositories Registration never mutates or adopts source repositories
Integration Gate Freezes staged per-repository plans, interface evidence, exact heads, and rollback commits Never claims atomic cross-repository commit

How self-evolution stays safe

Each completed attempt records quality, efficiency, instruction following, handoff quality, reliability, routing fit, token/cost variance, and an explicit error attribution. Cross-project model memory is an aggregate, rebuildable view over those immutable ledgers; it contains no prompts, reports, summaries, or raw artifacts. Profiles are isolated by exact provider/model/profile hash and task scope, publish 95% Wilson intervals, and reduce stale evidence with a 90-day half-life. Failures attributed to environment, tools, dependencies, budget, context, routing, or human review remain auditable but do not count as negative model-capability evidence.

Teaching is selected for cold-start scopes, high-risk work, repeated weak outcomes, or low-confidence evidence. Automatic teaching is advisory; an explicit review, paired, or replay policy creates a fixed execution graph and requires a validated teaching run before enforced acceptance. Review binds a separate reviewer result, paired mode binds two model identities plus a comparison, and replay holds model identity and task scope fixed. Learned recommendations move through candidate → replayed → shadow → canary → active, with explicit review at every transition. One successful or failed task can never rewrite active routing policy by itself.

After every recorded result, CostMarshal also appends one evidence-hash-deduplicated local evolution cycle. The cycle summarizes quality, efficiency, cost, error attribution, model-memory confidence, and the next matching task's teaching recommendation. It never starts paired/replay work, calls a provider, or activates a policy. evolution-status exposes the latest recorded cycle and a current rebuild.

Decision events also refresh a transcript-free Leader Snapshot. Leader startup uses context-view semantics to include only bounded Hot/Warm metadata references; Cold knowledge is indexed by ID and title but its content is not loaded. This keeps long-project context inspectable without silently widening the model prompt.

CostMarshal stores project state under $CODEX_HOME/costmarshal-v2 when CODEX_HOME is set, otherwise under ~/.codex/costmarshal-v2. The plugin snapshot is curated from an explicit allowlist and excludes repository metadata, development tests, generated artifacts, legacy interfaces, and secret-bearing files.

Documentation

Document Use it for
INSTALL_PROMPT.md Commit-pinned install or update through Codex
SKILL.md Canonical orchestration policy and operating contract
SECURITY.md Threat model, isolation guarantees, and limitations
references/migration-v3.md Migrating v2/v3 projects and standalone Skill installs
references/protocol.md Actor, mailbox, task, and acceptance protocol
references/providers.md Provider presets, API/runtime capabilities, and multimodal input
references/storage.md Durable state layout and storage semantics
references/evolution.md Work graph, evaluation memory, teaching triggers, and policy promotion
references/large-projects.md Repository identities, Workstreams, staged integration, and production boundary
references/backtest.md Blind real-provider evaluation format and gates
container/worker/README.md Building the digest-pinned worker image
deploy/production/README.md Deploying the mTLS Broker and hard-budget Provider Proxy
scripts/costmarshal_production_certification.py Creating and verifying the exact short-lived production claim
scripts/costmarshal_production_deploy.py Previewing and explicitly applying a fail-closed single-host deployment
CHANGELOG.md Release history
Internal CLI, recovery, and validation

The Python CLI is an internal runtime, automation, recovery, and diagnostic surface:

python scripts/costmarshal.py --help
python scripts/costmarshal.py route --help
python scripts/costmarshal.py dashboard --help
python scripts/costmarshal.py work-graph --help
python scripts/costmarshal.py workstreams --help
python scripts/costmarshal.py create-integration-plan --help
python scripts/costmarshal.py production-status --help
python scripts/costmarshal.py model-memory --help
python scripts/costmarshal.py record-teaching-run --help
python scripts/costmarshal.py cost-report --help
python scripts/costmarshal.py policy-status --help
python scripts/costmarshal.py recover --help

Existing projects remain on legacy JSON/JSONL authority until an explicit offline SQLite WAL cutover. Preview migration first, stop live actors, preserve the generated backup, and apply only after validation:

python scripts/costmarshal.py migrate-state --project <project-dir>
python scripts/costmarshal.py migrate-state --project <project-dir> --apply
python scripts/costmarshal.py state-store --project <project-dir>

Required OCI actors must cut over before dispatch --start, ensuring every production container start has a recoverable STOP-effect path.

Development verification

CI runs the complete SHA-bound local evidence suite on Windows and Linux with Python 3.11 and 3.13. A separate Linux job resolves the reviewed OCI manifests, builds both production Dockerfiles, validates the Gateway policy inside the image, and runs the Worker isolation canary under the production security options:

python scripts/sync_plugin_package.py
python tests/release/run_local_test_evidence.py

Non-beta release evidence additionally requires preregistered trust roots, an attested real-provider blind dataset, and a reviewed live OCI/provider-proxy topology. Without those external inputs, a blocked report is the expected safe outcome.

Compatibility

CostMarshal can work alongside ArchMarshal through explicit, read-only governance binding checks. It never adopts a workspace, applies an ArchMarshal plan, starts or ends a managed session, or edits ArchMarshal automatically.

Legacy v2 state remains auditable and is never silently rewritten. See the v3 migration guide.

License

MIT © yptang98

About

Cost-aware multi-model orchestration for Codex CLI: route hard work to strong agents, replay proven paths with cheaper agents, and track cost, memory, and project evolution.

Topics

Resources

Security policy

Stars

4 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages