diff --git a/BUGS.md b/BUGS.md index 766e5aa..6ced6e3 100644 --- a/BUGS.md +++ b/BUGS.md @@ -2,10 +2,10 @@ These are bugs (or missing features) I've observed while working with `multi checks`. -- [ ] Remove the `Claude -p` executor. - - [ ] No use of Cersei workflows to chain multiple prompts together. +- [ ] No GitHub Action available. + - [ ] Logs no longer report the id of the check that failed (or the number of attempted retries) - [ ] Assemble_instructions is hard-coded: src/checks/executor/mod.rs:98 (definition), called from src/checks/executor/cersei.rs:110 @@ -18,10 +18,6 @@ These are bugs (or missing features) I've observed while working with `multi che - [ ] No limit on max turns. -- [ ] No support for Fireworks AI. - -- [ ] No GitHub Action available. - - [ ] No loading of skills. - [ ] No loading of RULES.md files from the .claude directory. @@ -43,6 +39,10 @@ guaranteeing the terminal is restored on the way out. ## Fixes +- [x] No support for Fireworks AI. + +- [x] Remove the `Claude -p` executor. + - [x] No trace capture. We need a way to record all session traces so that we can analyze why they failed. - [x] Output is now hanging. I suspect this is recent (within the last few commits) and it started diff --git a/CHECKS.md b/CHECKS.md index a3075f7..8f39e50 100644 --- a/CHECKS.md +++ b/CHECKS.md @@ -2,19 +2,19 @@ # Requirement Provider-Agnostic Execution -The agent executor is swappable behind a trait so execution can run over different engines (the in-process `cersei-agent` executor, the legacy `claude -p` fallback, or a test fake) without rewriting the execution phase. This extensibility is the whole point of the configuration/executor seam. +The agent executor is defined behind a trait so execution can run over the in-process `cersei-agent` executor or a test fake without rewriting the execution phase. This extensibility is the whole point of the configuration/executor seam. ## Check Boxed Executor Trait -Inspect the Rust sources for the `multi check` feature. Confirm that agent execution is defined behind a trait (for example `CheckExecutor`) that is consumed as a boxed or `Arc`-wrapped, dynamically-dispatched trait object — for example a type alias of the form `Box`, mirroring the existing `BoxedIngress`, `BoxedMonitor`, and `BoxedPlatform` aliases. The check passes only if the execution phase depends on this trait rather than naming a concrete executor type. Report a failure if the execution path references a concrete executor struct (such as the cersei or claude executor) directly instead of the trait object. +Inspect the Rust sources for the `multi check` feature. Confirm that agent execution is defined behind a trait (for example `CheckExecutor`) that is consumed as a boxed or `Arc`-wrapped, dynamically-dispatched trait object — for example a type alias of the form `Box`, mirroring the existing `BoxedIngress`, `BoxedMonitor`, and `BoxedPlatform` aliases. The check passes only if the execution phase depends on this trait rather than naming a concrete executor type. Report a failure if the execution path references a concrete executor struct (such as the cersei executor) directly instead of the trait object. # Requirement In-Process Agent Execution -The default executor runs each check's agent **inside the CLI process** via the `cersei-agent` library — there is no longer any requirement to shell out to the `claude` CLI for a check to run. A `claude -p` executor is retained only as an optional, explicitly-selected migration fallback. +The executor runs each check's agent **inside the CLI process** via the `cersei-agent` library — there is no requirement to shell out to any external CLI for a check to run. ## Check Default Executor Runs In-Process -Inspect the executor implementations and how the default executor is selected from configuration. Confirm that the default execution engine builds and runs a `cersei_agent::Agent` in-process (calling its `run`/`run_stream` method) rather than spawning the `claude` CLI. Confirm that any use of `std::process::Command` / `tokio::process` to launch `claude -p` lives only in the non-default fallback executor, gated behind an explicit `executor` selection. The check fails if running a check with the default configuration requires spawning an external `claude` process. +Inspect the executor implementation and how it is constructed from configuration. Confirm that the execution engine builds and runs a `cersei_agent::Agent` in-process (calling its `run`/`run_stream` method) rather than spawning an external process. The check fails if running a check requires spawning an external `claude` process. # Requirement Trustworthy In-Process Reporting @@ -66,7 +66,7 @@ Inspect the execution phase. Confirm that checks are dispatched concurrently — # Requirement Concurrency Defaults To The Host's Core Count -The concurrency cap is user-configurable (a `--concurrency` flag, layered the same way as `--provider`/`--model`/`--effort`/`--executor`), but absent any override its default must equal the number of CPU cores available on the machine running `multi check` — not a hardcoded constant. A fixed default either strands cores on big machines or overcommits small ones. +The concurrency cap is user-configurable (a `--concurrency` flag, layered the same way as `--provider`/`--model`/`--effort`), but absent any override its default must equal the number of CPU cores available on the machine running `multi check` — not a hardcoded constant. A fixed default either strands cores on big machines or overcommits small ones. ## Check Default Concurrency Equals Available Parallelism @@ -90,11 +90,11 @@ Confirm that the command exits with status code 0 when every requirement is sati # Requirement Layered Configuration -The model, provider, effort, and executor are resolved from three sources with standard CLI precedence — flag, then environment, then config file (flag wins) — while credentials are read only from each provider's native environment variable, never from the config file. +The model, provider, and effort are resolved from three sources with standard CLI precedence — flag, then environment, then config file (flag wins) — while credentials are read only from each provider's native environment variable, never from the config file. ## Check Precedence And Validation -Inspect the configuration phase of `multi check`. Confirm that provider/model/effort/executor are merged from a config file, `MULTI_`-prefixed environment variables, and CLI flags, with flags overriding environment overriding file. Confirm the selected model is validated against a hardcoded allowlist of known IDs for the provider (an unknown ID is a clear error). The check fails if any of these values cannot be set from configuration, or if the merge precedence is not flag > env > file. +Inspect the configuration phase of `multi check`. Confirm that provider/model/effort are merged from a config file, `MULTI_`-prefixed environment variables, and CLI flags, with flags overriding environment overriding file. Confirm the selected model is validated against a hardcoded allowlist of known IDs for the provider (an unknown ID is a clear error). The check fails if any of these values cannot be set from configuration, or if the merge precedence is not flag > env > file. ## Check Credentials Are Environment-Only diff --git a/Cargo.lock b/Cargo.lock index 25039e8..ccd882a 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -1005,7 +1005,7 @@ checksum = "4f4c707c6a209cbe82d10abd08e1ea8995e9ea937d2550646e02798948992be0" [[package]] name = "cersei-agent" version = "0.2.6" -source = "git+https://github.com/wack/cersei.git?branch=trunk#521ea38bd1c178967e6d5fe564f8fcc911780bcb" +source = "git+https://github.com/wack/cersei.git?branch=trunk#1045e58a0a127ef798453adcc87c2a6f667170b9" dependencies = [ "anyhow", "async-trait", @@ -1032,7 +1032,7 @@ dependencies = [ [[package]] name = "cersei-compression" version = "0.2.6" -source = "git+https://github.com/wack/cersei.git?branch=trunk#521ea38bd1c178967e6d5fe564f8fcc911780bcb" +source = "git+https://github.com/wack/cersei.git?branch=trunk#1045e58a0a127ef798453adcc87c2a6f667170b9" dependencies = [ "anyhow", "once_cell", @@ -1046,7 +1046,7 @@ dependencies = [ [[package]] name = "cersei-embeddings" version = "0.2.6" -source = "git+https://github.com/wack/cersei.git?branch=trunk#521ea38bd1c178967e6d5fe564f8fcc911780bcb" +source = "git+https://github.com/wack/cersei.git?branch=trunk#1045e58a0a127ef798453adcc87c2a6f667170b9" dependencies = [ "async-trait", "futures", @@ -1062,7 +1062,7 @@ dependencies = [ [[package]] name = "cersei-hooks" version = "0.2.6" -source = "git+https://github.com/wack/cersei.git?branch=trunk#521ea38bd1c178967e6d5fe564f8fcc911780bcb" +source = "git+https://github.com/wack/cersei.git?branch=trunk#1045e58a0a127ef798453adcc87c2a6f667170b9" dependencies = [ "async-trait", "cersei-types", @@ -1074,7 +1074,7 @@ dependencies = [ [[package]] name = "cersei-lsp" version = "0.2.6" -source = "git+https://github.com/wack/cersei.git?branch=trunk#521ea38bd1c178967e6d5fe564f8fcc911780bcb" +source = "git+https://github.com/wack/cersei.git?branch=trunk#1045e58a0a127ef798453adcc87c2a6f667170b9" dependencies = [ "dashmap", "serde", @@ -1088,7 +1088,7 @@ dependencies = [ [[package]] name = "cersei-mcp" version = "0.2.6" -source = "git+https://github.com/wack/cersei.git?branch=trunk#521ea38bd1c178967e6d5fe564f8fcc911780bcb" +source = "git+https://github.com/wack/cersei.git?branch=trunk#1045e58a0a127ef798453adcc87c2a6f667170b9" dependencies = [ "async-trait", "cersei-types", @@ -1102,7 +1102,7 @@ dependencies = [ [[package]] name = "cersei-memory" version = "0.2.6" -source = "git+https://github.com/wack/cersei.git?branch=trunk#521ea38bd1c178967e6d5fe564f8fcc911780bcb" +source = "git+https://github.com/wack/cersei.git?branch=trunk#1045e58a0a127ef798453adcc87c2a6f667170b9" dependencies = [ "async-trait", "cersei-types", @@ -1120,7 +1120,7 @@ dependencies = [ [[package]] name = "cersei-provider" version = "0.2.6" -source = "git+https://github.com/wack/cersei.git?branch=trunk#521ea38bd1c178967e6d5fe564f8fcc911780bcb" +source = "git+https://github.com/wack/cersei.git?branch=trunk#1045e58a0a127ef798453adcc87c2a6f667170b9" dependencies = [ "async-trait", "base64 0.22.1", @@ -1140,7 +1140,7 @@ dependencies = [ [[package]] name = "cersei-tools" version = "0.2.6" -source = "git+https://github.com/wack/cersei.git?branch=trunk#521ea38bd1c178967e6d5fe564f8fcc911780bcb" +source = "git+https://github.com/wack/cersei.git?branch=trunk#1045e58a0a127ef798453adcc87c2a6f667170b9" dependencies = [ "async-trait", "base64 0.22.1", @@ -1184,7 +1184,7 @@ dependencies = [ [[package]] name = "cersei-types" version = "0.2.6" -source = "git+https://github.com/wack/cersei.git?branch=trunk#521ea38bd1c178967e6d5fe564f8fcc911780bcb" +source = "git+https://github.com/wack/cersei.git?branch=trunk#1045e58a0a127ef798453adcc87c2a6f667170b9" dependencies = [ "anyhow", "base64 0.22.1", diff --git a/guides/checks.md b/guides/checks.md index 56fdfd4..f19b0d2 100644 --- a/guides/checks.md +++ b/guides/checks.md @@ -15,11 +15,9 @@ requirement is satisfied only if **all** of its checks pass (logical AND). ## ✅ Prerequisites - [ ] An API key for your chosen provider in the environment (e.g. - `ANTHROPIC_API_KEY`) — see [Configuration](#️-configuration). The default + `ANTHROPIC_API_KEY`) — see [Configuration](#️-configuration). The in-process executor talks to the provider directly; **no `claude` CLI is - required**. (The optional `claude -p` fallback — `--executor claude` — does - need the [`claude`](https://docs.claude.com/en/docs/claude-code) CLI on your - `PATH`.) + required**. - [ ] **macOS** — the MVP sandboxes each check with an APFS copy-on-write clone. Other operating systems are not yet supported. @@ -159,15 +157,14 @@ Glob, plus the judge tool) — a verification agent observes, it does not mutate ## ⚙️ Configuration -The default **provider**, **model**, **effort**, **executor**, and -**concurrency** are resolved from three sources, in order of precedence -(highest wins): +The default **provider**, **model**, **effort**, and **concurrency** are +resolved from three sources, in order of precedence (highest wins): -1. **Flags** — `--provider`, `--model`, `--effort`, `--executor`, - `--concurrency` on `multi check`. +1. **Flags** — `--provider`, `--model`, `--effort`, `--concurrency` on + `multi check`. 2. **Environment** — `MULTI_`-prefixed vars mapped into the `checks` namespace, e.g. `MULTI_CHECKS_MODEL`, `MULTI_CHECKS_PROVIDER`, `MULTI_CHECKS_EFFORT`, - `MULTI_CHECKS_EXECUTOR`, `MULTI_CHECKS_CONCURRENCY`. + `MULTI_CHECKS_CONCURRENCY`. 3. **Config file** — the `[checks]` table of `MultiTool.toml` (or `.json` / `.jsonc`), discovered up the directory tree like any MultiTool manifest. @@ -176,7 +173,6 @@ The default **provider**, **model**, **effort**, **executor**, and provider = "anthropic" # anthropic | openai | gemini model = "claude-sonnet-4-6" # must be a known model ID for the provider effort = "low" # low | medium | high → thinking-token budget -executor = "cersei" # cersei (in-process, default) | claude (fallback) concurrency = 8 # checks run at once; must be > 0 (default: CPU core count) # optional, non-secret base-URL overrides per provider @@ -192,11 +188,8 @@ enable extended thinking (4096- and 8192-token budgets respectively), while `low` — the default — keeps thinking off for speed and cost, running the agent deterministically instead. -The **`executor`** selects the execution engine. The default `cersei` runs each -check as an in-process agent (native multi-provider model swapping, no external -CLI). `claude` is the legacy `claude -p` shell-out fallback, kept selectable for -migration while the in-process path is validated; it requires the `claude` CLI on -your `PATH` and will be removed once cersei is proven out. +Each check runs as an in-process agent (native multi-provider model swapping, +no external CLI or subprocess). The **`concurrency`** flag caps how many checks run at once; it must be a positive integer (`0` is rejected with a clear error). Its default matches the diff --git a/src/checks/config/mod.rs b/src/checks/config/mod.rs index 9b9aded..194f52d 100644 --- a/src/checks/config/mod.rs +++ b/src/checks/config/mod.rs @@ -1,16 +1,15 @@ //! The configuration phase (M2 #1341, global config #1359, executor wiring #1367). //! -//! Resolves the global default **provider**, **model**, **effort**, and -//! **executor** from three sources with standard CLI precedence — -//! `flag > env var > config file` — merged with [`figment`], constructs a -//! registry of ready-to-use model providers, and builds the selected -//! [`BoxedExecutor`] from a per-provider [`ProviderFactory`]. +//! Resolves the global default **provider**, **model**, and **effort** from +//! three sources with standard CLI precedence — `flag > env var > config file` +//! — merged with [`figment`], constructs a registry of ready-to-use model +//! providers, and builds the [`BoxedExecutor`] from a per-provider +//! [`ProviderFactory`]. //! //! The resolved [`Config`] is **dependency-injected** forward: execution //! receives a [`BoxedExecutor`] (see [`Resolved::build_executor`]) rather than -//! reading provider details at point of use. The default executor is the -//! in-process [`CerseiExecutor`]; the legacy [`ClaudeExecutor`] remains -//! selectable as a migration fallback. +//! reading provider details at point of use. The executor is the in-process +//! [`CerseiExecutor`]. mod file; mod models; @@ -29,10 +28,9 @@ use miette::{Result, miette}; use crate::checks::executor::BoxedExecutor; use crate::checks::executor::cersei::CerseiExecutor; -use crate::checks::executor::claude::ClaudeExecutor; pub use providers::{ProviderFactory, ProviderRegistry}; -pub use schema::{CliOverrides, Effort, ExecutorKind, ProviderKind}; +pub use schema::{CliOverrides, Effort, ProviderKind}; /// Per-agent wall-clock timeout. Generous: the heaviest reasoning checks can /// take a few minutes under contention before they report. @@ -55,16 +53,14 @@ fn default_concurrency() -> usize { pub struct Config { /// The selected provider. pub provider: ProviderKind, - /// The selected provider's optional base-URL override, if configured. Used - /// by the `claude -p` fallback as `ANTHROPIC_BASE_URL` (the in-process - /// executor applies it via the provider factory); `None` uses the default. + /// The selected provider's optional base-URL override, if configured + /// (applied by the in-process executor via the provider factory); `None` + /// uses the default. pub provider_url: Option, /// The concrete model ID to run (validated against the hardcoded allowlist). pub model: String, /// The effort level. pub effort: Effort, - /// Which execution engine runs each check (default: in-process cersei). - pub executor: ExecutorKind, /// Maximum number of checks executed concurrently (default: the number of /// available CPU cores; see [`default_concurrency`]). pub concurrency: usize, @@ -80,21 +76,6 @@ pub struct Config { pub trace_archive: Option, } -impl Config { - /// Construct the legacy `claude -p` fallback executor from this - /// configuration. The in-process cersei executor needs the resolved provider - /// factory and so is built from [`Resolved`]; this builder only covers the - /// fallback, which needs nothing beyond [`Config`]. - pub fn build_claude_executor(&self) -> BoxedExecutor { - Box::new(ClaudeExecutor::new( - self.model.clone(), - self.provider_url.clone(), - self.effort, - self.agent_timeout, - )) - } -} - /// Merge the three config layers and extract the resolved `[checks]` table. /// /// Merge order (low → high) is `file → MULTI_-prefixed env → flags`, giving @@ -128,24 +109,20 @@ pub struct Resolved { } impl Resolved { - /// Construct the selected [`BoxedExecutor`]. This is the injection point and - /// the migration lever: `cersei` (default) runs the in-process agent; - /// `claude` runs the legacy `claude -p` fallback over the same checks. + /// Construct the [`BoxedExecutor`]: the in-process cersei agent. This is the + /// injection point — execution depends on the trait object, not this + /// concrete type. pub fn build_executor(&self) -> Result { let cfg = &self.config; - let executor: BoxedExecutor = match cfg.executor { - ExecutorKind::Cersei => Box::new(CerseiExecutor::new( - self.factory.clone(), - cfg.model.clone(), - cfg.effort, - cfg.agent_timeout, - // The archive path lives at the orchestration layer; the executor - // only needs to know whether to capture a per-execution trace. - cfg.trace_archive.is_some(), - )), - ExecutorKind::Claude => cfg.build_claude_executor(), - }; - Ok(executor) + Ok(Box::new(CerseiExecutor::new( + self.factory.clone(), + cfg.model.clone(), + cfg.effort, + cfg.agent_timeout, + // The archive path lives at the orchestration layer; the executor + // only needs to know whether to capture a per-execution trace. + cfg.trace_archive.is_some(), + ))) } } @@ -163,7 +140,6 @@ pub fn load(overrides: CliOverrides) -> Result { .model .unwrap_or_else(|| models::default_model(provider).to_string()); let effort = checks.effort.unwrap_or(Effort::Low); - let executor = checks.executor.unwrap_or(ExecutorKind::Cersei); let concurrency = checks.concurrency.unwrap_or_else(default_concurrency); if !models::is_valid_model(provider, &model) { @@ -201,7 +177,6 @@ pub fn load(overrides: CliOverrides) -> Result { provider_url, model, effort, - executor, concurrency, agent_timeout: DEFAULT_AGENT_TIMEOUT, max_attempts: DEFAULT_MAX_ATTEMPTS, @@ -228,7 +203,6 @@ pub fn configuration() -> Config { // sonnet reasons efficiently and reports in well under a minute.) model: models::default_model(provider).to_string(), effort: Effort::Low, - executor: ExecutorKind::Cersei, concurrency: default_concurrency(), agent_timeout: DEFAULT_AGENT_TIMEOUT, max_attempts: DEFAULT_MAX_ATTEMPTS, @@ -252,7 +226,6 @@ mod tests { provider: Some(provider), model: Some(model.to_string()), effort: Some(Effort::Low), - executor: None, concurrency: None, trace_archive: None, providers: ProvidersSection::default(), @@ -265,12 +238,9 @@ mod tests { let cfg = configuration(); assert_eq!(cfg.provider, ProviderKind::Anthropic); assert_eq!(cfg.model, "claude-sonnet-4-6"); - assert_eq!(cfg.executor, ExecutorKind::Cersei); assert!(cfg.concurrency >= 1); // The default must track the machine's core count, not a hardcoded value. assert_eq!(cfg.concurrency, default_concurrency()); - // The fallback executor is constructible from config alone (DI seam works). - let _exec = cfg.build_claude_executor(); } #[test] @@ -283,7 +253,6 @@ mod tests { None, None, None, - None, ); let checks = resolve_layers(file, overrides).unwrap(); assert_eq!(checks.provider, Some(ProviderKind::OpenAi)); @@ -318,7 +287,7 @@ mod tests { // ...and a flag outranks env. let overrides = - CliOverrides::new(None, Some("claude-opus-4-8".into()), None, None, None, None); + CliOverrides::new(None, Some("claude-opus-4-8".into()), None, None, None); let checks = resolve_layers(file, overrides).unwrap(); assert_eq!(checks.model.as_deref(), Some("claude-opus-4-8")); Ok(()) diff --git a/src/checks/config/schema.rs b/src/checks/config/schema.rs index 7a67bfb..f83180f 100644 --- a/src/checks/config/schema.rs +++ b/src/checks/config/schema.rs @@ -43,20 +43,6 @@ pub enum Effort { High, } -/// Which execution engine runs each check. The default is the in-process -/// [`cersei`](crate::checks::executor::cersei) agent; `claude` selects the -/// legacy `claude -p` shell-out fallback, kept selectable during the migration -/// (MULTI-1367) so verdicts from both can be compared before the fallback is -/// retired. -#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize, ValueEnum)] -#[serde(rename_all = "lowercase")] -pub enum ExecutorKind { - /// The in-process `cersei-agent` executor (default). - Cersei, - /// The legacy `claude -p` shell-out fallback. - Claude, -} - /// The whole config file, of which only the `[checks]` table concerns us. Other /// top-level keys (the legacy manifest's `workspace`/`application`/`config`) are /// ignored rather than rejected, so a single `MultiTool.toml` can carry both. @@ -76,9 +62,6 @@ pub struct ChecksSection { pub model: Option, #[serde(default, skip_serializing_if = "Option::is_none")] pub effort: Option, - /// Which execution engine runs each check (`cersei` by default). - #[serde(default, skip_serializing_if = "Option::is_none")] - pub executor: Option, /// Maximum number of checks executed concurrently (default: the number of /// available CPU cores). #[serde(default, skip_serializing_if = "Option::is_none")] @@ -145,8 +128,6 @@ pub struct CliChecksOverrides { #[serde(skip_serializing_if = "Option::is_none")] pub effort: Option, #[serde(skip_serializing_if = "Option::is_none")] - pub executor: Option, - #[serde(skip_serializing_if = "Option::is_none")] pub concurrency: Option, #[serde(skip_serializing_if = "Option::is_none")] pub trace_archive: Option, @@ -158,7 +139,6 @@ impl CliOverrides { provider: Option, model: Option, effort: Option, - executor: Option, concurrency: Option, trace_archive: Option, ) -> Self { @@ -167,7 +147,6 @@ impl CliOverrides { provider, model, effort, - executor, concurrency, trace_archive, }, diff --git a/src/checks/executor/cersei.rs b/src/checks/executor/cersei.rs index 3c4450b..1c97046 100644 --- a/src/checks/executor/cersei.rs +++ b/src/checks/executor/cersei.rs @@ -118,9 +118,19 @@ fn project_instructions(project_root: &Path) -> Option { /// the provider fixes for round-tripping thinking blocks (`signature_delta` /// accumulation in 94f18b2, `redacted_thinking` preservation in 5bd06db), so /// the temperature-as-effort workaround that previously lived here is retired. -/// Low effort — the default — keeps thinking off to stay fast and cheap, and +/// Low effort — the default — turns thinking off to stay fast and cheap, and /// steers with temperature instead (see [`attempt_temperature`]). /// +/// "Off" must be *explicit*: `None` here maps to cersei's `disable_thinking()` +/// (an explicit `thinking: {"type": "disabled"}` in the request), not to +/// omitting the field. Omission means "model default", and for +/// hybrid-reasoning models behind Fireworks' Anthropic-compatible endpoint +/// (GLM 5.1) the default is reasoning ON — the 2026-07-02 trace archive shows +/// low-effort GLM runs thinking on every turn with the field omitted. Every +/// model in our allowlist ([`super::super::config::models::ANTHROPIC_MODELS`]) +/// accepts an explicit disable; models that reject it (Claude Fable 5) are +/// not selectable here. +/// /// Budgets follow cersei's own `EffortLevel` scale (medium 4096, high 8192) /// and sit comfortably under the agent's default 16k `max_tokens` (the API /// requires `budget_tokens < max_tokens`). @@ -190,10 +200,14 @@ impl CheckExecutor for CerseiExecutor { .cancel_token(cancel.clone()); // Exactly one reasoning control applies: the API rejects a temperature - // when extended thinking is enabled (see [`thinking_budget`]). + // when extended thinking is enabled (see [`thinking_budget`]). The + // no-budget arm disables thinking *explicitly* — omitting the field + // would leave gateway-served hybrid models (GLM) reasoning by default. agent_builder = match thinking_budget(self.effort) { Some(budget) => agent_builder.thinking_budget(budget), - None => agent_builder.temperature(attempt_temperature(req.attempt)), + None => agent_builder + .disable_thinking() + .temperature(attempt_temperature(req.attempt)), }; // `.system_prompt()`, not `.append_system_prompt()`: cersei's agent diff --git a/src/checks/executor/claude.rs b/src/checks/executor/claude.rs deleted file mode 100644 index 7503ae9..0000000 --- a/src/checks/executor/claude.rs +++ /dev/null @@ -1,172 +0,0 @@ -//! The legacy shell-out [`CheckExecutor`]: invoke the Claude Code CLI via -//! `claude -p`. Retained only as a **migration fallback** (selectable with -//! `--executor claude`) so its verdicts can be compared against the in-process -//! [`super::cersei::CerseiExecutor`] until cersei is validated, then retired. -//! -//! With the in-process MCP result server removed (MULTI-1367), this path can no -//! longer report through a localhost tool endpoint. Instead the agent is told to -//! write its verdict as JSON to a per-check sentinel file in the sandbox, which -//! the executor reads after the process exits. This is a deliberately simpler, -//! less-trustworthy channel than the in-process judge tool — acceptable for a -//! soon-to-be-removed fallback. - -use std::path::Path; -use std::time::Duration; - -use async_trait::async_trait; -use miette::{IntoDiagnostic, Result}; -use tokio::process::Command; - -use super::judge::CheckReport; -use super::{AgentOutcome, AgentRunRequest, CheckExecutor, assemble_instructions}; -use crate::checks::config::Effort; - -/// The sandbox-relative filename the fallback agent writes its verdict to. -const REPORT_FILE: &str = ".multitool-check-report.json"; - -/// Runs each check by invoking `claude -p` non-interactively. Model/provider/ -/// effort come from injected configuration (see [`crate::checks::config::Config`]), -/// never hardcoded here. -pub struct ClaudeExecutor { - /// The model to run (a concrete ID, e.g. `claude-sonnet-4-6`). - model: String, - /// Optional model-provider base URL; when set, passed as `ANTHROPIC_BASE_URL`. - provider_url: Option, - /// The effort level (logged for diagnostics; `claude -p` has no effort flag). - effort: Effort, - /// Per-agent wall-clock timeout; on expiry the child is killed and the check - /// resolves as errored (no verdict). - timeout: Duration, - /// The CLI program to invoke (`claude`). - program: String, -} - -impl ClaudeExecutor { - pub fn new( - model: String, - provider_url: Option, - effort: Effort, - timeout: Duration, - ) -> Self { - Self { - model, - provider_url, - effort, - timeout, - program: "claude".to_string(), - } - } -} - -/// The reporting directive for the file-based fallback channel. -fn file_report_directive() -> String { - format!( - "Carry out the check described below. When — and only when — you have reached a conclusion, \ -write your verdict as a single JSON object to the file `{REPORT_FILE}` in your current working \ -directory, with this exact shape:\n\ - {{\"success\": true|false, \"evidence\": \"a short explanation\"}}\n\ -Set `success` to true if the check passes, or false if it fails. Write the file EXACTLY ONCE, then \ -stop. If you finish without writing `{REPORT_FILE}`, the check is treated as a FAILURE.", - ) -} - -/// Read and parse the sentinel verdict file, if the agent wrote one. -fn read_report(working_dir: &Path) -> Option { - #[derive(serde::Deserialize)] - struct Wire { - success: bool, - #[serde(default)] - evidence: Option, - } - let path = working_dir.join(REPORT_FILE); - let contents = std::fs::read_to_string(path).ok()?; - let wire: Wire = serde_json::from_str(&contents).ok()?; - Some(CheckReport { - success: wire.success, - evidence: wire.evidence, - }) -} - -#[async_trait] -impl CheckExecutor for ClaudeExecutor { - async fn run_check(&self, req: AgentRunRequest) -> Result { - tracing::debug!( - check_id = req.check_id, - model = %self.model, - effort = ?self.effort, - "dispatching claude -p check (fallback)", - ); - - let instructions = assemble_instructions( - &req.check, - &file_report_directive(), - &req.working_dir, - req.attempt, - ); - - let mut cmd = Command::new(&self.program); - cmd.arg("-p") - .arg(&instructions) - .arg("--model") - .arg(&self.model) - // The sandbox is a throwaway CoW clone, so skip interactive - // permission prompts (the agent runs non-interactively and must be - // able to write its verdict file). - .arg("--dangerously-skip-permissions") - .current_dir(&req.working_dir) - .stdin(std::process::Stdio::null()) - .stdout(std::process::Stdio::piped()) - .stderr(std::process::Stdio::piped()) - // Ensure the child is reaped if we drop it on timeout. - .kill_on_drop(true); - if let Some(url) = &self.provider_url { - cmd.env("ANTHROPIC_BASE_URL", url); - } - - let child = cmd.spawn().into_diagnostic()?; - match tokio::time::timeout(self.timeout, child.wait_with_output()).await { - Ok(result) => { - let output = result.into_diagnostic()?; - let verdict = read_report(&req.working_dir); - let error = if verdict.is_none() && !output.status.success() { - let stderr = String::from_utf8_lossy(&output.stderr); - Some(format!( - "claude -p exited without a verdict (exit {:?}){}", - output.status.code(), - stderr_suffix(&stderr), - )) - } else { - None - }; - Ok(AgentOutcome { - verdict, - stop_reason: Some(format!("exit {:?}", output.status.code())), - turns: 0, - error, - trace_jsonl: None, - }) - } - Err(_elapsed) => { - // The wait future is dropped here; `kill_on_drop` reaps the child. - Ok(AgentOutcome { - // A verdict file may have been written just before the timeout. - verdict: read_report(&req.working_dir), - stop_reason: None, - turns: 0, - error: Some(format!("agent timed out after {:?}", self.timeout)), - trace_jsonl: None, - }) - } - } - } -} - -fn stderr_suffix(stderr: &str) -> String { - let trimmed = stderr.trim(); - if trimmed.is_empty() { - String::new() - } else { - let snippet: String = trimmed.chars().take(200).collect(); - format!(": {snippet}") - } -} diff --git a/src/checks/executor/mod.rs b/src/checks/executor/mod.rs index 3633b7b..33202b0 100644 --- a/src/checks/executor/mod.rs +++ b/src/checks/executor/mod.rs @@ -2,15 +2,12 @@ //! abstracts "run one check's agent → verdict/outcome". cersei-agent absorbs the //! *provider-abstraction* rationale the seam originally carried, but not its //! *test-seam* rationale: `cersei_agent::Agent` is a concrete struct, so the -//! execution-phase tests still need a fake. The trait keeps one method with three -//! impls — the real in-process [`cersei::CerseiExecutor`], the soon-to-retire -//! shell-out [`claude::ClaudeExecutor`] fallback (selectable for migration), and -//! the test [`FakeExecutor`]. It is a boxed trait object for dynamic dispatch, -//! mirroring the repo's `BoxedIngress` / `BoxedMonitor` / `BoxedPlatform` -//! convention. +//! execution-phase tests still need a fake. The trait keeps one method with two +//! impls — the real in-process [`cersei::CerseiExecutor`] and the test +//! [`FakeExecutor`]. It is a boxed trait object for dynamic dispatch, mirroring +//! the repo's `BoxedIngress` / `BoxedMonitor` / `BoxedPlatform` convention. pub mod cersei; -pub mod claude; #[cfg(test)] mod fake; mod jail; @@ -69,9 +66,9 @@ pub struct AgentOutcome { pub error: Option, /// The self-contained NDJSON session trace for this one execution, when /// trace capture is enabled (`multi check --trace-archive`). `None` when - /// capture is off and for executors that don't produce traces (the `claude` - /// fallback and the test fake). The execution layer moves these into the - /// per-run [`crate::checks::trace_archive`] bundle. + /// capture is off and for executors that don't produce traces (the test + /// fake). The execution layer moves these into the per-run + /// [`crate::checks::trace_archive`] bundle. pub trace_jsonl: Option>, } @@ -93,8 +90,8 @@ pub trait CheckExecutor: Send + Sync { pub type BoxedExecutor = Box; /// The reporting directive for the default (in-process) executor: call the judge -/// tool exactly once. Kept separate from [`assemble_instructions`] so the legacy -/// shell-out fallback can substitute its own reporting channel. +/// tool exactly once. Kept separate from [`assemble_instructions`] so other +/// executors can substitute their own reporting channel. pub fn judge_tool_directive() -> String { format!( "Carry out the check described below. When — and only when — you have reached a conclusion, \ diff --git a/src/checks/mod.rs b/src/checks/mod.rs index c937064..e3d9c5b 100644 --- a/src/checks/mod.rs +++ b/src/checks/mod.rs @@ -67,14 +67,13 @@ use crate::checks::sandbox::Sandbox; /// Operational errors (e.g. an invalid `CHECKS.md`) surface as `Err` diagnostics /// rather than an exit code, so CI can tell "checks failed" from "tool errored". pub async fn run(terminal: &Terminal, working_dir: &Path, overrides: CliOverrides) -> Result { - // Phase 1: configuration — resolve provider/model/effort/executor + // Phase 1: configuration — resolve provider/model/effort // (flag > env > file) and construct the provider registry, injected forward. let resolved = config::load(overrides)?; tracing::debug!( provider = resolved.config.provider.as_str(), model = %resolved.config.model, - executor = ?resolved.config.executor, concurrency = resolved.config.concurrency, available_providers = ?resolved.providers.keys().collect::>(), "resolved checks configuration and provider registry", diff --git a/src/config/check/mod.rs b/src/config/check/mod.rs index 169b3fd..560c1ed 100644 --- a/src/config/check/mod.rs +++ b/src/config/check/mod.rs @@ -3,7 +3,7 @@ use std::path::{Path, PathBuf}; use clap::Args; -use crate::checks::config::{CliOverrides, Effort, ExecutorKind, ProviderKind}; +use crate::checks::config::{CliOverrides, Effort, ProviderKind}; /// `multi check`: validate the requirements declared in `CHECKS.md` files. /// @@ -29,11 +29,6 @@ pub struct CheckSubcommand { #[arg(long, value_enum)] effort: Option, - /// The execution engine: `cersei` (in-process, default) or `claude` (the - /// legacy `claude -p` fallback). Overrides `checks.executor` from env/file. - #[arg(long, value_enum)] - executor: Option, - /// Maximum number of checks to run concurrently. Must be greater than 0. /// Overrides `checks.concurrency` from env/file. Defaults to the number of /// available CPU cores. @@ -61,7 +56,6 @@ impl CheckSubcommand { self.provider, self.model.clone(), self.effort, - self.executor, self.concurrency.map(NonZeroUsize::get), self.trace_archive.clone(), )