Conversation
Add StoragePayloadForSpaceDeriveV1 (deterministic v1 derive builder) and switch CreateSpace, DeriveId and DeriveSpace to the v1 builders. v1 headers embed the acl and settings payloads and validate via byte-compare (needCheckSpaceId=false), matching the one-to-one path already shipping in production. Note: v1 derive produces a different derived space id than v0 for the same keys (the header bytes, hence the CID, differ). Safe on greenfield; existing v0-derived spaces would need an id-continuity strategy before an in-place upgrade. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add the SpaceFileProtoVersion enum (Unspecified=0, V2=2) and a fileprotoVersion field to the SpaceHeader proto, and plumb it through SpaceCreatePayload and SpaceDerivePayload into every builder. The field lives in the signed, immutable header, so it is tamper-evident and recovered verbatim on cold restore. Gates the v2 file protocol (filenode v2 + S3-direct + networkSign) per SYN-15. Callers set the value (e.g. the SDK mints spaces at V2); unset stays Unspecified, so legacy any-sync consumers are unaffected. OneToOne spaces stay Unspecified. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Coverage provided by https://github.com/seriousben/go-patch-cover-action |
Add the NodeTypeFileV2 node type ("fileV2") and a second, independent chash
container built from the v2 file-node pool at RF=2 (FileV2ReplicationFactor).
New NodeConf methods:
- FileV2Peers() lists the v2 filenodes (mirrors FilePeers).
- FileV2NodeIds(spaceId) returns the RF=2 responsible v2 filenodes for a space.
Unlike NodeIds, the current account is NOT excluded, so the full responsible
pair is visible (e.g. to derive the leader).
The tree (sync) and v1-file containers are untouched. Leader selection and
per-space responsibility consumers are out of scope. Propagated to service,
testconf StubConf and the regenerated mock.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…ath (SYN-16) Add FileV2API=6 to the coordinator NodeType proto enum and map it to nodeconf.NodeTypeFileV2 in nodeconfsource. Without this the v2 file type only survived local YAML configs and was dropped on the coordinator-distributed path. The enum->string switch keeps its no-default behavior: an older client that does not know FileV2API (or any future type) silently drops the unknown value while keeping the node's known types, so it stays correctly routed in the tree/file pools and never crashes. New enum values are therefore backward-compatible. Note: the coordinator server (any-sync-coordinator) still needs the reverse string->enum mapping to actually serve fileV2 nodes. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Define the filenode v2 (FileV2) broker dRPC contract: a byteless broker that authorizes uploads, signs durable-custody receipts, and issues signed download URLs — never transferring bytes itself. All four RPCs are batch (Upload / RequestSign / RequestDownload / SpaceInfo): each request carries a shared spaceId + repeated items, and each response is a repeated per-item result keyed by the echoed rootCid/spaceId with a flat status enum, so one bad item never fails the whole batch. A two-layer error model keeps whole-RPC failures (auth/malformed/batch-too-large) on the drpc error while per-item business outcomes travel in-body. New proto package filesyncv2 / service FileV2 (route /filesyncv2.FileV2/*), distinct from v1 filesync/File, plus the fileprotov2err registry at error offset 800 (see docs/rpc-error-offsets.md). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Document the process-global rpcerr registry as a shared cross-repo namespace: the mechanism, allocation rules (offsets are multiples of 100, local codes 0-99, any-sync core < 1000 / downstream repos >= 1000), the globally reserved low codes, and a full table of offsets in use across the ecosystem (any-sync 100-800, any-sync-node 1000, anytype-push-server 1200, bobrik-clusterctl 1300). Add an ErrGroup doc comment pointing to it. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…lFilter (SYN-18) Two opt-in, backward-compatible hooks for participants that head-sync a whole space but materialize only selected tree types (filenode-v2 broker): - TreeFullSyncRequest.probe: the responder replies with the tree's root and current heads only, no change bodies. Old responders ignore the flag and stream the full tree. Plumbed via BuildTreeOpts.Probe. - treesyncer.PullFilter: optional extension of the registered TreeSyncer, consulted by objectsync on a head update for a locally-missing tree; head updates already carry the raw root change, so the filter can classify the tree (cleartext changeType) and decline the pull with no extra round trips. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
feat(sync): selective-sync hooks — tree probe requests + optional PullFilter (SYN-18)
…SYN-31) Spaces chash-shard across the NodeTypeFileV2 pool (RF=2), and a filenode must reject — and never activate/sync — spaces outside its responsibility set. Give that rejection a dedicated retryable routing signal instead of overloading Forbidden, so client SDKs can re-route to the responsible pair: * whole-RPC ErrCodes.NotResponsible = 4 (registered in fileprotov2err at offset 800+4) — for the single-space RPCs where the entire request is misrouted; * per-item ErrCode.ErrNotResponsible = 6 — for SpaceInfo, whose batch may legally mix spaces owned by different pairs. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Downloads start PUBLIC-read: chat-message images must render with zero
broker round-trips, so the client constructs durable object URLs straight
from the rootCid — {publicReadBaseUrl}/blob/{spaceId}/{rootCid} — and GETs
the CDN. The base URL is distributed by the filenode itself (not nodeconf)
via a new node-level Info RPC that clients cache: no spaceId, no
responsibility gate, no auth beyond the connection. InfoRequest is
deliberately empty so future node metadata (capabilities, limits) extends
it without a wire break; an empty publicReadBaseUrl means no public read —
use RequestDownload.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
feat(fileprotov2): NotResponsible error codes + node-level Info RPC (SYN-31)
mockConf did not implement the FileV2Peers/FileV2NodeIds methods added to nodeconf.NodeConf on this branch, causing a panic in TestInbox_Fetch. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Proto version 13 aligns with the v0.13.X release line and covers the files v2 work (v2 filenodes, chash routing, space header fileprotoVersion). Version 13 is already present in defaultCompatibleVersions. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Per docs/proto-versioning.md the default compatibility range is self ±1. v0.13.X accepts V-1=12, V=13, V+1=14. Drops bridge v9, which is no longer needed now that v0.12 has shipped. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…entity One shared account (signing) key signs fileV2 durability receipts for the whole NodeTypeFileV2 fleet (per-node signing was rejected: receipts live in CRDT rows for years and must survive fleet churn). Clients need its public identity — the fileNetworkId — from the network config, so: - nodeconf.Configuration gains FileNetworkId (yaml fileNetworkId, network string encoding, decode with crypto.DecodeNetworkId) - coordinatorproto NetworkConfigurationResponse gains fileNetworkId=5 - nodeconfsource maps it off the proto, so coordinator-refreshed configs carry it Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
New commonfile/fileproto/filep2p service filesyncp2p.FileP2P, served read-only by clients to LAN peers for direct file fetch: - FileCheck(spaceId, rootCids) -> None/Partial/Full availability - ObjectRead(spaceId, rootCid, offset, length) -> (data, totalSize) A client stores one immutable CARv2 object per file, byte-identical to the S3/public-read object, so a peer is just an ALTERNATE SOURCE of that same object: ObjectRead returns object byte ranges exactly like an HTTP Range GET, and the fetcher's existing seed/verify path is reused. Only full holders serve ObjectRead; FileCheck reports which. Requests are root-scoped (no global block index). Blocks are ciphertext; the fetcher verifies hash==cid. Whole-RPC errors in filep2perr at rpcerr offset 900 (v1=200, v2=800). Route prefix /filesyncp2p.FileP2P/ coexists on one mux. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017Hq5ube4stoD1XM5QAU931
filesyncp2p.FileP2P already declares ErrorOffset=900 (the next free core band); record it in docs/rpc-error-offsets.md, the source of truth for who owns which offset. Adds a filep2perr test that co-registers the neighbouring v1/v2 error groups so a colliding offset would panic at init — a green run proves the 900 band is clear. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017Hq5ube4stoD1XM5QAU931
filep2p: peer-to-peer file transfer protocol (SYN-48)
…storage limits (SYN-19) The v2 file-limits API: brokers pull absolute account-pool bounds per (spaceId, uploader identity) and push per-(space, identity) durable usage reports; the coordinator aggregates the identity total. In-flight never reaches the coordinator; values are absolute and idempotent. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AoLHZmQFxyhxDkkwJ7Y1jX
SpaceInfo now surfaces the requesting identity's shared account pool (accountLimitBytes / accountUsageBytes) next to the per-space view — files are charged to their uploader, all spaces share one pool. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AoLHZmQFxyhxDkkwJ7Y1jX
feat(coordinatorproto): files-v2 storage limits API — FileLimitsGet / FileUsageReport (SYN-19)
Replaces plaintext spaceId lists in LAN peer discovery with per-space HMAC tokens keyed by a discovery key derived (HKDF) from the space's first ACL read key. Peers learn only the intersection of their space sets; a valid token simultaneously hides the space from non-members and proves the sender's membership, so strangers on the LAN can no longer enumerate spaces, correlate devices across sessions, or poison the local peer table with spaces they don't hold. Request tokens are padded with random fillers to fixed buckets and sorted to hide the true space count; response tokens are keyed by the caller's nonce (challenge-response). Gated by ProtoVersion 13; v1 fallback must be version-gated, never error-triggered. Bench (1000 spaces/peer, 500 shared): ~1.5ms CPU for a full round trip, plus 0.66ms one-time discovery-key derivation. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
feat(clientspaceproto): SpaceExchangeV2 — token-based LAN space discovery
All connectable peers negotiate DiffType_V3 since ProtoVersion 7 and the minimum compatible version is 12, so the V2 paths are dead code: - drop app/olddiff and the ldiff DiffContainer (single diff now) - DiffManager keeps only the V3 diff; V2 range requests are rejected - statestorage keeps a single hash (old hash is no longer written) - keep the DiffType negotiation as the future upgrade mechanism, see the upgrade recipe comment on DiffManager
- statestorage: keep mirroring the hash into the legacy "oh" key so a previous release (rollback or mixed-fleet coldsync) doesn't misread NewHash as empty via its legacy-format fallback; transitional, remove once downgrades below this version are out of support - diffmanager: guard FillDiff against empty Heads entries, set all elements with a single variadic Set (one range recalculation instead of N), document why FillDiff and UpdateHeads deliberately use different empty-root conditions, and note the V4 fallback-on-error requirement in the upgrade recipe - add a hash-stability regression test pinned to the golden value produced by the pre-removal dual-diff DiffManager
refactor(headsync): remove old diffsync (DiffType_V2) support
cheggaaa
marked this pull request as ready for review
July 16, 2026 13:52
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Files v2 — umbrella
Tracking/umbrella branch for the Files v2 effort (Linear SYN-12): filenode v2, S3-direct broker, and
networkSign, per the SDK design doc07c-filenode-v2-networksign.md. Individual sub-tasks land here as separate commits; this PR stays draft until the epic is ready to merge.Landed so far
SYN-15 — space header
fileprotoVersionSpaceFileProtoVersionenum (Unspecified=0,V2=2) andfileprotoVersionfield (Bump github.com/ipfs/go-blockservice from 0.5.1 to 0.5.2 #9) toSpaceHeader; plumbFileProtoVersionthroughSpaceCreatePayload/SpaceDerivePayloadinto every builder. The field lives in the signed, immutable header → tamper-evident and recovered verbatim on cold restore. Callers/SDK set the value; unset staysUnspecifiedso legacy consumers are unaffected.Default new spaces to v1 header
StoragePayloadForSpaceDeriveV1(deterministic v1 derive builder) and switchCreateSpace/DeriveId/DeriveSpaceto the v1 builders (v1 embeds acl/settings, validates via byte-compare — the path one-to-one spaces already use in production).Switching derive to v1 changes the derived space id for the same keys (the header bytes, hence the CID, differ). Safe on this greenfield branch, but an in-place legacy/anytype-heart upgrade would need an id-continuity strategy (dual-derive lookup or id-alias) before shipping. A 3-repo audit (any-sync + coordinator + node) confirmed v0/v1 headers otherwise coexist correctly end-to-end.
Notes
any-sync-coordinator/any-sync-nodecurrently pinany-sync v0.11.20; they won't seefileprotoVersionuntil they bump the dependency (harmless — headers are stored/verified as raw bytes).Checklist
go build ./...commonspace,spacepayloads,spacestorage/migration,headsynctests pass🤖 Generated with Claude Code