Skip to content

[Feature]: Enable the stall watchdog by default (stall_timeout_secs)Β #10168

Description

@JordanTheJet

Summary

Give stall_timeout_secs a conservative non-zero default so a stalled turn aborts and retries instead of hanging indefinitely.

Problem statement

stall_timeout_secs documents 0 = disabled, and 0 is the default β€” so the stall watchdog ships off.

A turn that stops making progress (provider stops streaming, upstream connection wedges) hangs with no abort and no retry. The operator's only signal is a bot that never replies, which is indistinguishable from a dozen other failure modes. The recovery mechanism already exists and is simply not enabled.

Proposed solution

Default stall_timeout_secs to a conservative non-zero value. 300 is a starting suggestion; the specific value is a maintainer call. 0 remains available to disable the watchdog explicitly.

Suggested sequencing:

  1. Agree a default generous enough not to abort legitimately slow turns.
  2. Change the default; document it and note it in the changelog.
  3. Add a regression test covering the abort-and-retry path at the default.

Non-goals / out of scope

  • No change to retry counts or backoff strategy.
  • No new watchdog behaviour β€” this only enables what already exists.
  • No change to approval_timeout_secs or other channel timeouts.

Alternatives considered

  • Keep 0 and document the recommendation. Leaves every install one wedged connection away from a silent hang, and the failure is hard to attribute.
  • Enable only for specific channels. Arbitrary: the failure mode is transport-independent.
  • Derive it from an existing timeout. Couples two unrelated tunables and makes the effective value harder to reason about.

Acceptance criteria

  • Default is non-zero and documented.
  • A stalled turn aborts and retries within the window.
  • 0 still disables the watchdog.
  • A regression test covers the abort path.

Architecture impact

channels/, docs/. No security surface: this changes failure recovery only, not what the agent may read, write, or execute.

Risk and rollback

Risk: a legitimately slow turn is aborted mid-flight if the default is too low β€” the main argument for a generous value rather than an aggressive one. Worth confirming during implementation whether abort-and-retry is safe for a turn that had already acted before stalling; that concern exists today for anyone who enabled the watchdog manually.

Rollback: set stall_timeout_secs = 0, restoring today's behaviour exactly.

Expected routing

Ordinary feature triage

Next decision surface

Ordinary triage is enough; the only genuinely open question is the specific value. Companion to #10166; the two are independent and can land separately.

Breaking change?

Yes

Data hygiene checks

  • I removed personal/sensitive data from examples, payloads, and logs.
  • I used neutral, project-focused wording and placeholders.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    channelAuto scope: src/channels/** changed.configAuto scope: src/config/** changed.enhancementNew feature or requestpriority:p2Medium priorityrisk:mediumOrdinary behavioral change without a high-impact boundary.runtimeAuto scope: src/runtime/** changed.status:acceptedAccepted RFC/work item; not stale-exempt by itself.status:in-progressOpen PR actively targets this issue; recheck when PR closes.

    Type

    No type

    Projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions