turbo-tasks-backend: add persistence delete/tombstone plumbing for GC#95975
Draft
lukesandberg wants to merge 1 commit into
Draft
turbo-tasks-backend: add persistence delete/tombstone plumbing for GC#95975lukesandberg wants to merge 1 commit into
lukesandberg wants to merge 1 commit into
Conversation
Contributor
Author
|
Warning This pull request is not mergeable via GitHub because a downstack PR is open. Once all requirements are satisfied, merge this PR as a stack on Graphite.
This stack of pull requests is managed by Graphite. Learn more about stacking. |
Contributor
Stats from current PR🟢 1 improvement
📊 All Metrics📖 Metrics GlossaryDev Server Metrics:
Build Metrics:
Change Thresholds:
⚡ Dev Server
📦 Dev Server (Webpack) (Legacy)📦 Dev Server (Webpack)
⚡ Production Builds
📦 Production Builds (Webpack) (Legacy)📦 Production Builds (Webpack)
📦 Bundle SizesBundle Sizes⚡ TurbopackClient Main Bundles
Server Middleware
Build DetailsBuild Manifests
📦 WebpackClient Main Bundles
Polyfills
Pages
Server Edge SSR
Middleware
Build DetailsBuild Manifests
Build Cache
🔄 Shared (bundler-independent)Runtimes
📝 Changed Files (29 files)Files with changes:
View diffsapp-page-exp..ntime.dev.jsfailed to diffapp-page-exp..time.prod.jsfailed to diffapp-page-tur..ntime.dev.jsfailed to diffapp-page-tur..time.prod.jsfailed to diffapp-page-tur..ntime.dev.jsfailed to diffapp-page-tur..time.prod.jsfailed to diffapp-page.runtime.dev.jsfailed to diffapp-page.runtime.prod.jsfailed to diffapp-route-ex..ntime.dev.jsDiff too large to display app-route-ex..time.prod.jsDiff too large to display app-route-tu..ntime.dev.jsDiff too large to display app-route-tu..time.prod.jsDiff too large to display app-route-tu..ntime.dev.jsDiff too large to display app-route-tu..time.prod.jsDiff too large to display app-route.runtime.dev.jsDiff too large to display app-route.ru..time.prod.jsDiff too large to display pages-api-tu..ntime.dev.jsDiff too large to display pages-api-tu..time.prod.jsDiff too large to display pages-api.runtime.dev.jsDiff too large to display pages-api.ru..time.prod.jsDiff too large to display pages-turbo...ntime.dev.jsDiff too large to display pages-turbo...time.prod.jsDiff too large to display pages.runtime.dev.jsDiff too large to display pages.runtime.prod.jsDiff too large to display server.runtime.prod.jsDiff too large to display use-cache-pr..ntime.dev.jsDiff too large to display use-cache-pr..ntime.dev.jsDiff too large to display use-cache-pr..ntime.dev.jsDiff too large to display use-cache-pr..ntime.dev.jsDiff too large to display 📎 Tarball URLCommit: 37de3fe |
Contributor
Tests PassedCommit: 37de3fe |
lukesandberg
force-pushed
the
lukesandberg/gc-01-scope-self-feeding
branch
from
July 20, 2026 22:43
8e95270 to
ed85eb4
Compare
lukesandberg
force-pushed
the
lukesandberg/gc-02-tombstone-plumbing
branch
from
July 20, 2026 22:43
b80ac91 to
14080b5
Compare
lukesandberg
force-pushed
the
lukesandberg/gc-01-scope-self-feeding
branch
from
July 20, 2026 23:17
ed85eb4 to
f43d126
Compare
lukesandberg
force-pushed
the
lukesandberg/gc-02-tombstone-plumbing
branch
2 times, most recently
from
July 21, 2026 16:08
5dbf6f4 to
73dc675
Compare
Contributor
Author
|
Reworked per the 'do we still need the delete list?' question — good catch. Dropped the |
lukesandberg
force-pushed
the
lukesandberg/gc-02-tombstone-plumbing
branch
from
July 21, 2026 16:29
73dc675 to
b5ba401
Compare
lukesandberg
force-pushed
the
lukesandberg/gc-01-scope-self-feeding
branch
2 times, most recently
from
July 21, 2026 16:41
040c21d to
6d9359b
Compare
lukesandberg
force-pushed
the
lukesandberg/gc-02-tombstone-plumbing
branch
2 times, most recently
from
July 21, 2026 16:46
011b444 to
d903753
Compare
lukesandberg
force-pushed
the
lukesandberg/gc-01-scope-self-feeding
branch
from
July 21, 2026 16:46
6d9359b to
d3ffbd3
Compare
Stage 0 of the mark-and-sweep GC work: the persistence-layer plumbing to tombstone unreachable tasks, landed independently of the mark/sweep logic so its semantics can be verified on their own. - Add `TurboWriteBatch::delete` passthrough to `turbo_persistence`. - Add a `TaskDeletion` struct and thread a `deletes: Vec<TaskDeletion>` list through `save_snapshot`. Tombstones are applied in the same commit as the snapshot, after the parallel put phase and before flush. - `TaskMeta`/`TaskData` (SingleValue) are tombstoned by task-id key. The MultiValue `TaskCache` is tombstoned by whole hash bucket and any colliding live survivors are re-put in the same batch (collisions are astronomically rare; a composite key was rejected because get_multiple is exact-hash and there is no prefix-scan primitive). - The production snapshot call passes an empty delete list for now; GC wiring comes in a later stage. Tests verify the tombstone path across a reopen, and that deleting one entry from a colliding TaskCache bucket + re-putting the survivor leaves exactly the survivor — confirming that a delete and a put for the same key compose correctly in one commit regardless of insertion order (the collector sorts tombstones last within a key group). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
lukesandberg
force-pushed
the
lukesandberg/gc-01-scope-self-feeding
branch
from
July 21, 2026 17:02
d3ffbd3 to
dde6bcf
Compare
lukesandberg
force-pushed
the
lukesandberg/gc-02-tombstone-plumbing
branch
from
July 21, 2026 17:02
d903753 to
37de3fe
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
The persistence-layer mechanism to tombstone (delete) tasks from the on-disk cache. A GC-collected task is represented as a
SnapshotItem::Deletethat rides the same streaming iteratorsave_snapshotalready consumes for puts, so tombstones are applied in the same commit/batch as the snapshot with no side-channel.SnapshotItembecomes an enum:Put { … }(the existing path) andDelete(TaskDeletion).TurboWriteBatch::deletepassthrough toturbo_persistence.save_snapshotapplies deletes inline in the parallel put phase:TaskMeta/TaskData(SingleValue) are tombstoned by id key; the MultiValueTaskCachebucket is tombstoned wholesale and any live task that xxh3-collides in that bucket is re-inserted — those survivors are resolved by reading the authoritative on-disk bucket at apply time (not carried onTaskDeletion). The bucket-rewrite runs in parallel across distinct hashes.Why this shape (addressing "do we still need the delete list?")
Note
SnapshotItem::Deleteis#[allow(dead_code)]here: this PR lands the persistence-side handling and its tests, but nothing emits aDeleteyet. The GC pass that soft-deletes tasks and serializes them toDeletelands in the next PR in the stack. So this PR is a no-op at runtime.Tests
TaskMeta/TaskData/TaskCache.TaskCachebucket re-inserts the survivor (both add-orders), proving delete+put compose in one commit.save_snapshotre-inserts a disk-only colliding survivor when applying aSnapshotItem::Delete.