Skip to content

Latest commit

 

History

History
113 lines (71 loc) · 6.92 KB

File metadata and controls

113 lines (71 loc) · 6.92 KB

ipman v2.4.1 — orphan-doc eviction with refuse-on-limit

ipman v2.4.1 is a point release to v2.4 that closes a latent drift gap in the auto-doc generator. The JSON protocol stays at protocol_version: 2; no operations were added, renamed, or removed; existing clients are unaffected.


The gap

workspace.refresh_agent_docs regenerates the .ipman/ agent knowledge pack from the live source: per-op schemas, JSON request schemas, examples, indexes, the manifest, and so on. Before v2.4.1 the generator only wrote — it never deleted. Files emitted by an older binary version (after a future op rename or removal, an entity removal, a workflow rename, …) would stay on disk indefinitely, growing the workspace and confusing agents that read .ipman/ directly.

The gap was latent in v2.4: every transition from v2.0 to v2.4 was purely additive (new ops, never renames or removals), so no orphans were ever produced. But the failure mode was a single rename away: a v2.5 that renames plan.deactivate to plan.unset_active would have left .ipman/operations/ipman.op.plan.deactivate.schema.md on disk forever, and an agent reading it would call a non-existent op.

The bug surfaced during a post-merge cross-layer consistency review of v2.4 — exactly the kind of audit the user does when they're worried about coherence across capas (source, on-disk artifacts, MCP, manifest).


What's new at a glance

Change Detail
Eviction (not deletion) Orphans go to .ipman/.attic/<microsec-iso-ts>/<rel-path>. The data is recoverable until the operator removes .attic explicitly.
Configurable cap IPMAN_ATTIC_LIMIT (default 100) bounds attic growth.
Progressive enforcement At ≥ half the limit: WARN log on the explicit refresh. At ≥ the limit: refuse with new error code attic_full.
Bricking-safe The implicit refresh during open_workspace deliberately bypasses enforcement. Other ops (task.list, plan.activate, …) keep working when attic is over the limit; only the explicit workspace.refresh_agent_docs op fails until you clean.
New result fields files_removed, attic_files_total, attic_dir on the refresh response.
New error code attic_full (semantic, non-fatal). Listed alongside validation_failed, not_found, etc. in the envelope doc.

Surface stays at 76 ops / 13 entities. No DB migration. No protocol bump.


Why eviction, not deletion

Three reasons surfaced during design:

  1. First release of any sweep is the wrong moment for unlink. A bug in the sweep heuristic — a subdir not contemplated, a path encoding edge case, an off-by-one — is recoverable when files are moved, not when they are gone.
  2. Recoverability gives operator confidence. "Did the upgrade just delete my custom-edited doc?" is a question with a one-second answer (ls .attic/) instead of "git diff against the previous binary's checkout".
  3. The .attic/<timestamp>/ layout is its own audit trail. One sweep = one timestamp directory. Comparing two timestamps shows what migrated when.

The microsecond precision in the timestamp (2026-05-05T02-38-53.463294Z) avoids same-second collisions in fast CI loops where multiple refresh calls fire in well under a second. The run directory is created lazily — a refresh that finds no orphans does not pollute .attic/ with empty subdirs.


Why progressive warn, then refuse

Eviction without a cap is the same monotonic-growth problem the sweep was supposed to solve, just relocated. Three retention policies were considered:

  • Manual cleanup. Operator runs rm -rf .ipman/.attic on their own clock. Simple, but invites forgetting.
  • TTL auto-prune. Refresh deletes attic subdirs older than N days. Hands-off, but silently destroys data the operator might still want.
  • Refuse at limit. Explicit refresh refuses once attic crosses a threshold. Loud, never silently destructive, requires acknowledgment.

v2.4.1 chose refuse at limit with progressive warn. It matches the project's -Werror philosophy: failures should be visible, not papered over. Half-limit warn gives CI and dev VMs a heads-up before the hard fail.


Configuring the limit

# Default (100 files in .attic before refresh refuses).
ipman <... refresh ...>

# Higher limit for a release cycle that renames many ops.
IPMAN_ATTIC_LIMIT=500 ipman <... refresh ...>

# Tighter limit for tests / CI.
IPMAN_ATTIC_LIMIT=10 ipman <... refresh ...>

The variable is parsed once per invocation. Invalid values (negative, non-numeric) fall back to the default with a one-line WARN log.


Recovery flow

When you see attic_full:

# 1. See what's been evicted across all past sweeps.
ls .ipman/.attic/

# 2. (Optional) Inspect a specific run.
ls .ipman/.attic/2026-05-05T02-38-53.463294Z/

# 3. (Optional) Recover anything you actually need.
cp .ipman/.attic/2026-05-05T02-38-53.463294Z/operations/ipman.op.something.schema.md /somewhere/

# 4. Acknowledge by removing .attic.
rm -rf .ipman/.attic

# 5. Retry the refresh.
echo '{"protocol_version":2,"request_id":"r","actor":"agent","op":"workspace.refresh_agent_docs","params":{}}' | ipman

Or, if you don't care about the contents and just want to keep moving, the one-line acknowledgment is rm -rf .ipman/.attic.


Implementation notes

The sweep is deliberately scoped to the 9 directories ensure_subdirs creates: protocol, concepts, entities, operations, examples, workflows, indexes, schemas, guides. The home root itself is never enumerated — that is where ipman.db, keysalt, and .init.lock live alongside the artifacts. Top-level generated files (manifest.json, START-HERE.md) are not sweep targets either; their names are stable.

The implicit refresh fires during open_workspace for every ipman invocation, including non-refresh ops. It always does the work (writes + evicts) without enforcing the limit, so the attic count can grow past the limit during normal usage. The limit only fires when the user explicitly invokes workspace.refresh_agent_docs — the one entry point dedicated to refresh policy. This keeps the user out of a bricked state where attic policy blocks every other op.


See also

  • docs/v2.4-slug-import-docs.md — the v2.4 release this point release polishes.
  • docs/v2.4.2-workspace-discovery.md — next point release that closed the silent-fork hazard for agents running ipman from subdirs or git worktrees.
  • docs/v2.3-tools-and-env-vars.md — project / tools / env_vars registry.
  • src/agent_docs.cevict_to_attic, count_attic_files, compact_iso_now, resolve_attic_limit.
  • tests/integration/001_doc_orphan_sweep.sh — eviction semantics, timestamp dir format, lazy run-dir creation.
  • tests/integration/002_doc_attic_limit.sh — warn-at-half, refuse-at-limit, recovery flow, env var override, bricking-safety.