Add entire doctor migrate-checkpoints (git-branch → git-refs)#1611
Merged
Conversation
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 2 potential issues.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Want fixes drafted automatically? Bugbot Autofix can create code changes for findings. A team admin can enable Autofix in the Cursor dashboard.
Comment @cursor review or bugbot run to trigger another review on this PR
Reviewed by Cursor Bugbot for commit 856c7ca. Configure here.
Contributor
There was a problem hiding this comment.
Pull request overview
This PR adds a doctor-time migration path to move existing checkpoints from the legacy single branch store (entire/checkpoints/v1) into the per-checkpoint ref layout used by the git-refs checkpoint store (refs/entire/checkpoints/<shard>/<id>), enabling repos to transition storage backends without rewriting branch history.
Changes:
- Add
entire doctor migrate-checkpointsto migrate v1-branch checkpoint subtrees into per-checkpoint ref commits, with--dry-runand optional interactive “push now”. - Implement
checkpoint.MigrateBranchToRefsplus unit tests validating byte-identical trees, idempotency, fast-forward advancement, and dry-run behavior. - Extract/refactor checkpoint-ref queue flushing in
manual_commit_push.goto share logic between pre-push (fail-soft) and migration “push now” (error-surfacing).
Reviewed changes
Copilot reviewed 5 out of 5 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
| cmd/entire/cli/strategy/manual_commit_push.go | Extracts queued checkpoint-ref flush into a shared helper and adds an error-surfacing push entrypoint for migration. |
| cmd/entire/cli/doctor.go | Registers the new doctor migrate-checkpoints subcommand. |
| cmd/entire/cli/doctor_migrate.go | Implements the doctor migrate-checkpoints command, including interactive “push now” prompt and dry-run flag. |
| cmd/entire/cli/checkpoint/migrate.go | Adds the core migration routine to convert v1-branch checkpoints into per-checkpoint refs. |
| cmd/entire/cli/checkpoint/migrate_test.go | Adds tests covering migration correctness, idempotency, advancement, dry-run, and no-branch no-op. |
pfleidi
reviewed
Jul 2, 2026
Convert checkpoints on the entire/checkpoints/v1 branch into per-checkpoint refs (refs/entire/checkpoints/<shard>/<id>), the git-refs store's layout. checkpoint.MigrateBranchToRefs walks the v1 branch tip and, for each checkpoint, wraps its CURRENT subtree object in a fresh commit and points the ref at it — existing branch commits are not remapped. Since the git-refs ref tree is the branch's <shard>/<id> subtree byte-for-byte, a migrated checkpoint reads identically under either backend. It is idempotent: a checkpoint whose ref already carries the same tree is skipped, and a changed checkpoint re-migrates by fast-forward (new commit parents on the existing ref, so no history is lost). New/advanced refs are enqueued for push; the function itself never pushes. The `doctor migrate-checkpoints` command reports migrated/skipped/total and, per the requested policy, only pushes when it can prompt: interactively it asks whether to push now; non-interactively it never pushes (refs stay queued and flush on the next push once git-refs is primary). `--dry-run` reports what would change without writing refs. Push reuse: the git-refs pre-push queue-flush is extracted into strategy.flushCheckpointRefsQueue (shared by the fail-soft pre-push path and the new error-surfacing strategy.PushMigratedCheckpointRefs), so the "push now" option goes through the exact same fast-forward + fetch/replay logic. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Entire-Checkpoint: ceed5d17e529
…ntRefs - doctor migrate-checkpoints now reuses the existing confirmDoctorFix helper instead of a near-duplicate confirm form — picking up its context-cancellation guard (huh opens the TTY during startup regardless of ctx state) and dropping the huh/errors/context imports. - PushMigratedCheckpointRefs takes the *git.Repository the command already opened rather than re-opening one, matching the strategy entry-point pattern and removing a redundant repo open per invocation. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Entire-Checkpoint: 01f324c13852
…remote, parent validation) - Map context.Canceled to NewSilentError so Ctrl-C exits quietly instead of printing a noisy Cobra error (matches the clean.go convention). [cursor] - When the user confirms push but checkpoint pushing is disabled in settings, report that the refs stay queued instead of "Pushed 0 checkpoint ref(s)". [cursor] - Add a --remote flag (default origin) instead of hardcoding "origin", so repos that push to a different remote work. [copilot] - MigrateBranchToRefs now only uses an existing ref as the new commit's parent when it resolves to a real commit; a ref at an unreadable/non-commit object is treated as absent (orphan) rather than parenting on a bad hash, which would corrupt the commit graph for fetch+replay. [copilot] Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Entire-Checkpoint: 5d352fa1b1bb
Soph
force-pushed
the
feat/checkpoint-migrate
branch
2 times, most recently
from
06b3c99 to
96537c1
Compare
PushMigratedCheckpointRefs flushed the queue without the policy check both pre-push paths enforce, so a policy that blocks checkpoint pushes (diverged or requiring a newer CLI) could be bypassed via the doctor's opt-in push. Apply the same sync-and-gate, returning an error with the refs left queued. Cover the push entry point: success returns the pushed count and clears the queue; a blocked policy and a failed push both leave refs queued. Entire-Checkpoint: 0c43e61ea113
Migrated checkpoints carried their v1-branch metadata verbatim: the
legacy checkpoint_version stamp ("branch-v1") and sessions[] file
paths relative to the sharded branch root, both wrong for a
per-checkpoint ref whose tree root is the checkpoint itself.
The migration now rewrites the root metadata.json per checkpoint:
checkpoint_version is dropped and any session string value under the
branch prefix is rebased to the ref's tree root, matching native
git-refs writes without hardcoding the SessionFilePaths field names.
The JSON object is edited in place so fields this CLI doesn't model
survive. Session subtrees carry over byte-identical (the metadata.json
swap goes through ApplyTreeChanges, so sibling subtree hashes are
untouched structurally), and idempotency compares against the
normalized tree.
See #1611 (comment)
Entire-Checkpoint: 4e6e24afb3e3
Resolve the existing ref through the refs store's refBase instead of an inline Reference/CommitObject chain, keeping the same semantics: a ref that doesn't resolve to a readable commit is treated as absent and the checkpoint is re-migrated as an orphan (pinned by a new test). Drop the unreachable malformed-id skip — WalkCheckpointShards only yields validated IDs, and refBase would surface the error anyway. Rename PushMigratedCheckpointRefs to PushQueuedCheckpointRefs: it flushes the whole queue, not just migrated refs. Entire-Checkpoint: 57d756e084fd
# Conflicts: # cmd/entire/cli/strategy/manual_commit_push.go
A re-run compared only the ref tip tree, so a ref that advanced past the migration snapshot through refs-store writes (summary/transcript backfills) would be "re-migrated": a new commit wrapping the stale branch tree, fast-forwarding the tip backwards content-wise and propagating on the next push. Idempotency now walks the ref's first-parent chain — a snapshot found anywhere in history is skipped. The doctor command additionally refuses to run when git-refs is already the primary store, where the v1 branch may lag the refs and re-importing could only regress them. Entire-Checkpoint: 01KWX0X0VSAQD8ZCC4Q8QRDAP6
MigrateBranchToRefs promised refs are queued for push, but setRef's enqueue is best-effort — a failed enqueue left migrated refs written locally and silently never pushed. The migration now enqueues explicitly and fails on error; duplicates collapse on Drain. PushQueuedCheckpointRefs now runs post-push shadow-branch cleanup after a successful flush, matching the pre-push paths. Entire-Checkpoint: 01KWX19DEHT2AEBYYGZQY33HV7
Contributor
|
Quick summary of the recent changes: Changes on
All seven open review threads on #1611 are replied to and resolved. Known deliberate calls: OPF is not re-applied during migration (branch content copies as-is), and a corrupt/unreadable checkpoint still fails the whole run as a readability signal. |
MigrateBranchToRefs collapsed *every* refBase error to parent=ZeroHash, so a momentary read failure — a concurrent repack, lock contention — made the migration treat a still-valid checkpoint ref as absent, wrap the branch snapshot in a fresh orphan commit, and setRef over the good ref (discarding its history; the follow-up fast-forward-only push then rejects it). Split the cases: an absent ref stays an orphan, a ref whose commit object is genuinely missing (ErrObjectNotFound: corrupt/pruned) is still re-imported as an orphan, but any other read error now aborts the checkpoint so an idempotent re-run retries rather than rewriting a ref that was only transiently unreadable. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Entire-Checkpoint: 01KWY6NRP1P3QH7DA54EYCM61W
The idempotency skip keyed only on "snapshot already on the ref," so a ref written by a prior run that then failed to enqueue it — an Enqueue error, or a crash between setRef and Enqueue — was skipped on every later run and never queued for push. The migrated ref then existed locally but its checkpoint data never reached the remote, silently defeating the guaranteed-enqueue contract on exactly the retry it was meant to enable. Compute the ref name up front and enqueue on the skip path too (never in dry-run). Enqueue is idempotent — duplicates collapse on Drain and an already-pushed ref is a no-op on the next push — so re-enqueuing a skipped checkpoint is safe and closes the gap. Dry-run now accounts an already-imported checkpoint as a skip rather than reaching the old would-migrate branch, and writes/enqueues nothing. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Entire-Checkpoint: 01KWY6VAT1V2Z6NHCKZ3BNNJY2
migratedCheckpointTree ran before the dry-run guard and, for any checkpoint whose metadata needs normalizing, called CreateBlobFromContent + ApplyTreeChanges — both persist via SetEncodedObject. So a "preview" that promises to report "without writing" left a dangling blob + tree per checkpoint in the object store (thousands on a large v1 branch, until a future git gc). Thread the dry-run flag into migratedCheckpointTree: when not persisting, compute the normalized blob and tree hashes in memory (git hashes are content-addressed, so an object encoded but not stored hashes identically to a persisted one) and return without touching the object store. The persisting path is unchanged. A new test asserts the object count is unchanged across a dry-run, and another migrates for real then dry-runs again and asserts the checkpoint is recognized as already-migrated — pinning the in-memory hash to the persisted tree hash. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Entire-Checkpoint: 01KWY7CS85ZMCCM8RQEGKHPQAF
normalizeMigratedMetadata rebases session paths by string-stripping the "/<shard>/<id>" prefix rather than round-tripping through the metadata model the git-refs writer uses, and nothing tied the two code paths together. The rebasing is correct for today's layout, but if the native layout ever drifts — a renamed session dir, a new path field, a field that isn't simply prefix-relative — the migration would silently ship checkpoints whose paths a native reader can't resolve. Add a regression test that writes the same checkpoint content two ways — natively via the git-refs store and migrated from the git-branch store — and asserts their root-metadata session paths are identical. Any future divergence between the writer and the migration now fails loudly. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Entire-Checkpoint: 01KWY7PAG1KG98TP2RHZX9RP6K
PushQueuedCheckpointRefs returned before cleanupPushedShadowBranches whenever flush returned an error — but the diverged path returns an error even when some refs pushed successfully. So a "push now" that partially succeeds left the landed refs' shadow branches uncleaned, unlike the pre-push path, which always runs cleanup after flush. Run cleanup unconditionally after flush for parity; it is best-effort and a no-op when nothing pushed. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Entire-Checkpoint: 01KWY7RMYDFVJART4FKV3FT0KS
…bled" PushQueuedCheckpointRefs returned (0, nil) both when push_sessions is disabled and when the queue is empty, so the doctor command reported "Checkpoint pushing is disabled in settings" even when pushing was enabled and the refs had simply already been pushed — e.g. a concurrent git push flushed the just-migrated refs while the confirm prompt blocked. Return an explicit pushDisabled flag alongside the pushed count so the two cases are distinguishable. The command now prints the disabled message only when pushing is actually disabled, and a distinct "no queued refs to push" message otherwise. Adds a push-disabled test and updates the existing entry-point tests for the new signature. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Entire-Checkpoint: 01KWY7XFF3PTNHZTXKW66FGRF1
The new dry-run tests added a third "branch-v1" literal, tripping goconst. Name it once as legacyCheckpointVersion. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Entire-Checkpoint: 01KWY80X77KHP4HV0Z9EJHFK16
pfleidi
approved these changes
Jul 7, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
https://entire.io/gh/entireio/cli/trails/735
Adds a migration that converts checkpoints on the
entire/checkpoints/v1branch into per-checkpoint refs underrefs/entire/checkpoints/<shard>/<id>— the layout the git-refs store uses — so a repo on the git-branch store can move to git-refs.How it works
checkpoint.MigrateBranchToRefs(ctx, repo, dryRun)walks the v1 branch tip and, for each checkpoint:<shard>/<id>subtree byte-for-byte, a migrated checkpoint reads identically under either backend.Idempotent: a checkpoint whose ref already carries the same tree is skipped; a changed checkpoint re-migrates by fast-forward (the new commit parents on the existing ref, so nothing is lost from history). Safe to re-run after more branch activity.
The command
entire doctor migrate-checkpoints— reportsmigrated / already-up-to-date / total.Push behavior (as requested): the function only enqueues; the command decides whether to push:
git pushonce git-refs is the primary store.--dry-runreports what would migrate without writing refs.Push reuse
The git-refs pre-push queue-flush is extracted into
strategy.flushCheckpointRefsQueue, shared by the fail-soft pre-push path and the new error-surfacingstrategy.PushMigratedCheckpointRefs. So "push now" goes through the exact same fast-forward + fetch/replay recovery logic (no force). No behavior change to pre-push.Testing
migrate_test.go: basic migration + byte-identical ref trees + read-back via the git-refs store; idempotent re-run (all skipped, refs unmoved); advance-on-branch-change (fast-forward parenting); dry-run writes nothing; no-branch no-op.duplquiet). Checkpoint + strategy unit suites pass; integration 392; git-refs canary 58/58 +1 skip, roger 4/4.Not included (follow-ups)
git-branchmirror for downgrade safety — unchanged from the git-refs store's existing scope.Refs: #1471
🤖 Generated with Claude Code
Note
Medium Risk
Migration rewrites local git refs and checkpoint data layout; push behavior differs between interactive opt-in and fail-soft pre-push, though logic is shared and tested.
Overview
Adds
entire doctor migrate-checkpointsso repos on the git-branch checkpoint store can move to the git-refs layout (refs/entire/checkpoints/<shard>/<id>).checkpoint.MigrateBranchToRefswalks checkpoints onentire/checkpoints/v1, wraps each checkpoint’s current subtree in a new commit, and points the per-checkpoint ref at it (enqueueing push like normal git-refs writes). It is idempotent (same tree → skip; changed tree → fast-forward parent) and supports--dry-run. Unmappable IDs are skipped with a warning.The doctor subcommand reports migrated / skipped / total counts. Interactive runs can opt into an immediate push; non-interactive runs leave refs queued for the next
git push.strategy.flushCheckpointRefsQueueextracts the git-refs push-queue drain logic from pre-push; pre-push still fail-soft on errors, whilePushMigratedCheckpointRefssurfaces errors for the migration “push now” path.migrate_test.gocovers migration fidelity, idempotency, branch updates, dry-run, and no-branch no-op.Reviewed by Cursor Bugbot for commit 856c7ca. Configure here.