feat: Time Travel shadow-repo retention (orphan cleanup, optional age-out, stale-lock repair)#1775
Open
King0James0 wants to merge 1 commit into
Open
feat: Time Travel shadow-repo retention (orphan cleanup, optional age-out, stale-lock repair)#1775King0James0 wants to merge 1 commit into
King0James0 wants to merge 1 commit into
Conversation
…-out, stale-lock repair) Time Travel keeps a hidden git repository per workspace under /a0/usr/.time_travel/workspaces/<id>/repo.git and snapshots on every file change, but ships no retention: nothing deletes a shadow repository when its chat or project is removed (chat_remove never touches .time_travel), there is no delete/prune endpoint in the API or web UI, and a workspace whose 'git add' ever exceeds GIT_TIMEOUT_SECONDS strands repo.git/index.lock, failing every later snapshot with 'index.lock: File exists'. Observed on a live instance: 518 shadow repositories / 12 GB, most belonging to long-deleted chats, plus a permanently wedged workspace. This adds a throttled retention sweep driven from job_loop: - Orphans: live workspace paths (project folders, the configured workdir, per-chat workdirs) are forward-enumerated and hashed with the existing workspace_id_for derivation; a shadow directory matching none of them is unreachable from the UI forever and is removed after a grace window. - Optional age-out: retention_max_age_days (default 0 = keep forever) removes a live workspace's history when it has had no snapshot in N days. Time Travel lazily re-initializes an empty history on the next snapshot, so this is always safe for the feature. - Stale locks: repo.git/index.lock older than retention_stale_lock_minutes is removed (git subprocesses are killed at GIT_TIMEOUT_SECONDS, so no legitimate lock lives that long), un-wedging future snapshots. - Corrupt-repo set-asides (repo.git.invalid*) past the grace window. Deletion is refused for any path outside the shadow root. Settings render in the existing plugin-config pattern (default_config.yaml + webui/config.html, settings_sections: agent): enable toggle, sweep interval, age-out days, orphan grace, stale-lock age. Durable evidence lives next to the workspaces dir: retention.json (running totals + last sweep stamp) and retention.log (one JSON line per sweep naming everything removed, tail-capped at 1000). Sweeps run in a worker thread off the event loop, at most one in flight, default every 6 hours. Tests: tests/test_time_travel_retention.py (config defaults/clamps, the full sweep matrix, keep-forever default, disabled no-op, deletion guard, marker accrual, per-sweep history + cap, throttle, and workspace_id_for parity).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
The
_time_travelplugin keeps a hidden git repository per workspace under/a0/usr/.time_travel/workspaces/<id>/repo.gitand snapshots it on every file change — but ships no retention of any kind:chat_removenever touches.time_travel, and there is no delete/prune endpoint in the plugin's API or web UI. Every removed chat/project leaves its full history behind forever, unreachable from the UI.index.lockpermanently wedges snapshots. Any workspace whosegit addever exceedsGIT_TIMEOUT_SECONDS(large/busy work-trees do this reliably) strandsrepo.git/index.lock, and every later snapshot fails withindex.lock: File existsuntil someone deletes it by hand.Observed on a live instance: 518 shadow repositories / 12 GB, most belonging to long-deleted chats, plus one workspace whose snapshots had been failing for days on a stranded lock.
Fix
A throttled retention sweep inside the plugin, driven from
job_loop(worker thread, at most one in flight, default every 6h):workspace_id_forderivation; a shadow directory matching none of them has no owner and is removed after a grace window (default 24h).retention_max_age_days(default 0 = keep forever) removes a live workspace's history when it has had no snapshot in N days. Always safe for the feature: Time Travel lazily re-initializes an empty history on the next snapshot.repo.git/index.lockolder thanretention_stale_lock_minutes(default 30; git subprocesses are killed atGIT_TIMEOUT_SECONDS, so no legitimate lock lives that long) is removed, un-wedging future snapshots.repo.git.invalid*backups past the grace window.Deletion is refused for any path outside the shadow root.
Settings
Standard plugin-config surface (
default_config.yaml+webui/config.html,settings_sections: [agent]): enable toggle, sweep interval, age-out days (0 = keep forever), orphan grace, stale-lock age. Defaults are conservative — only unreachable garbage is removed unless the operator opts into age-out.Observability
Durable state next to the workspaces dir:
retention.json(running totals + last sweep stamp) andretention.log(one JSON line per sweep naming exactly what was removed, tail-capped at 1000 lines).Testing
tests/test_time_travel_retention.py: config defaults/clamps, full sweep matrix (orphan past/inside grace, aged, keep-forever default, stale vs fresh lock, invalid backup), disabled no-op, shadow-root deletion guard, marker accrual, per-sweep history + cap, throttle,workspace_id_forparity — 9/9.agent0ai/agent-zero:latestcontainer: boot clean, the job_loop sweep fired live (orphan removed, young orphan + live workspace kept, stale lock cleared), marker + history written,retention_enabled: falsevia saved plugin config correctly disables it, settings page renders.