CHANGELOG.md

Changelog

All notable changes to the VTX SDK will be documented in this file.

The format is based on Keep a Changelog, and this project adheres to Semantic Versioning.

[Unreleased]

[0.5.0] - 2026-07-27

Added

• tools/cli: the CLI now exposes the file's timing data, which was previously unreachable from the command surface:
  • new times [start] [end] command -- dumps the footer per-frame time table: counts, first_created_utc_* / last_created_utc_*, wall_duration_seconds, game_duration_seconds, the median wall-clock frame step, a discontinuity scan (anomalies = frames whose created_utc delta is negative or more than twice the median step, capped at 100 with anomaly_count / anomalies_truncated), raw gaps / segments, and -- with a frame range -- a per-frame frames slice (game_time_ticks, created_utc_ticks, created_utc_iso, delta_ticks)
  • frame now reports the current frame's game_time_ticks / game_time_seconds and created_utc_ticks / created_utc_iso (null when the file did not record them)
  • header adds recorded_utc_iso alongside the raw recorded_utc_timestamp; info adds recorded_utc_ticks / recorded_utc_iso
  • footer adds game_time_count / created_utc_count / gap_count / segment_count
  • chunks entries add checksum (xxHash64 of the on-disk chunk payload, 0 = not set)
  • events entries add derived utc_ticks / utc_iso (recording start + game_time)
  • absolute wall-clock values follow one convention everywhere: <name>_ticks (the stamp exactly as stored in the file, so raw deltas stay diagnosable) + <name>_iso (ISO-8601 UTC derived after unit normalization), both null when not recorded; events.utc_ticks is synthetic and always UE-tick-based
• common/time: TimeUtils::FormatUtcTicksIso8601(ticks) -- machine-readable ISO-8601 sibling of FormatUtcTicks
• common/time: TimeUtils::NormalizeUtcToUeTicks(value) -- maps an absolute UTC stamp of unknown legacy unit (unix seconds / unix milliseconds / unix-relative ticks / UE ticks) onto the UE tick timeline. Needed because the writer stamps the header's recorded_utc_timestamp in unix seconds while the footer's per-frame created_utc uses UE ticks -- the same file carries two units. The CLI derives every *_iso string and the event UTC through it while passing stored *_ticks values through raw. (The writer-side unit inconsistency itself is left as is in this change -- fixing it alters what new files contain and deserves its own decision.)
• writer/sinks: PRISM sink timing refactored to an injectable perf observer -- new IFileSinkPerfObserver (OnSerialize / OnCompress / OnDiskWrite, called synchronously from the writer thread) plus a default thread-safe collector FileSinkAtomicPerfObserver (Snapshot() -> FileSinkPerformanceStats {serialization_us, compression_us, disk_write_us}, Reset()). Attach via ChunkedFileSink::Config::perf_observer or the new WriterFacadeConfig::perf_observer; leave null for a zero-cost no-op. Covered by RoundtripTest.PerfObserverReceivesSinkTimings on both backends
• tools/inspector: File > Repair Replay -- repairs a crashed .vtx from its .recovery sidecar via VTX::RepairReplayFile, from inside the GUI. Floating window with pickers for the .vtx and the sidecar (auto-filled to the adjacent <file>.vtx.recovery; a sidecar picked from elsewhere is staged next to the .vtx safely, never clobbering an existing one), the repair runs on a worker thread so the UI stays responsive, and the panel reports live progress then success (recovered chunks/frames) or failure (error detail) -- or "nothing to repair" for an already-complete file. Always available, independent of the loaded replay; vtx_inspector now links vtx_writer for the repair entry point
• tools/inspector: File > Cut Replay -- write a sub-range of the loaded replay as a new .vtx (#46). ReplayCutService plans the cut and writes: header + kept chunks copied verbatim, footer rebuilt with frame numbering rebased to 0, the time table sliced (tick values stay absolute), gaps/segments filtered + rebased, duration recomputed from the sliced UTC stamps. The window accepts the range as elapsed time, frames, or absolute UTC (ISO-8601 or unix seconds/ms/100ns ticks), previews the resolved chunks/frames/time/size live, and runs the copy on a worker thread with a Save As dialog. Exact-frame cuts: when a bound falls inside a chunk, that edge chunk is re-serialized with only the kept frames (SaveChunk framing mirrored: length prefix, zstd-if-beneficial, xxHash64 checksum) and its seek-table entry regenerated; whole interior chunks still copy verbatim. A "Chunk (whole chunks)" mode keeps snap-to-chunk behavior. Timeline events are not carried over
• tools/inspector: recording-gap visualization + wall-clock navigation (#45) -- BuildDroppedFrameMap flags every frame whose wall-clock delta from the previous stamped frame (footer created_utc, game_time fallback) exceeds 1.5x the expected interval and estimates missing frames per gap. The timeline strip paints flagged frames red with a gap/missing summary and hover detail; the main slider paints the same gaps as red bands sized by wall-clock proportion, and now scrubs wall-clock time instead of frame index (grab at the current frame's footer-table time; dragging resolves the frame by binary search over the time table; inside a gap the grab snaps to the gap's start), so equal slider distance means equal recorded time. The expected rate is an editable "Drop FPS" input, derated by 0.75 (captures never sustain nominal rate; a 30 entry detects at 22.5 effective) with the factor shown in the label. Files without a time table keep the previous linear behavior

Fixed

• reader/flatbuffers: struct-array subarray boundaries lost on read -- the FlatBuffers reader's unpackStructArray restored a struct-array's data but silently dropped its offsets, so any struct-array field (Vector/Quat/Transform/FloatRange) with two or more subarrays lost its partitioning on load. Regression-tested by RoundtripTest.StructArraySubArrayOffsetsRoundtrip (three subarrays with mixed sizes including an empty one, on both backends)
• tools/inspector: timeline time drifted on captures with recording stalls (#45) -- frame time was derived as frame_index / avg_fps, so stalls skewed every displayed time by the total stalled duration (observed: a frame stamped 13:54 wall clock displayed as ~6:47 on a capture with 643s of stalls). Frame -> elapsed seconds now maps through the footer ReplayTimeData table (created_utc preferred, game_time fallback, walking back to the nearest stamped frame); applies to the current-time readout and the strip hover tooltip. Files without a time table keep the old linear mapping
• writer/sinks: disk_write_us measured only the buffered fwrite, not the durability flush -- sub-microsecond buffered writes round to 0 on a fast disk, so the metric was misleading (it excluded the fsync, i.e. the actual disk write) and PerfObserverReceivesSinkTimings flaked on fast CI runners. A new TimedSyncOrFlush() counts the fsync/fflush toward disk_write_us at the three sink sites (header / chunk / footer); same syncs as before, now timed
• repo: repaired a fused .gitignore entry (.DS_Storebuild-inspector/, from an append without a trailing newline) that left both .DS_Store and build-inspector/ effectively un-ignored

[0.4.0] - 2026-07-20

Added

• common/diagnostics: unified structured-diagnostics model -- new header sdk/include/vtx/common/vtx_diagnostics.h. Replaces the SDK's previous mix of nullptr / false / thrown / logged / silently-ignored failures with one model that automation (PRISM v2) can act on without parsing logs:

  • VtxDiagnostic -- one structured error/warning record carrying everything a strict failure should expose: code (VtxErrorCode), severity (Severity), message, and the location/context it concerns (frame_index, bucket, unique_id, entity_type, field_path, expected_type/expected_container, provided_type/provided_container, source_api). Aggregate type -- call sites fill only the fields they know. ToString() + operator<< for logging/gtest streaming.
  • VtxError / VtxWarning -- aliases of VtxDiagnostic (severity distinguishes).
  • VtxResult<T> -- a produced value plus a (possibly empty) error and warnings; VtxStatus = VtxResult<Unit> for status-only results. Factory helpers Success / Failure.
  • ValidationReport -- aggregated diagnostics from a validation pass (HasErrors / ok / ErrorCount / WarningCount / Errors / Warnings / ToString), warnings carried as structured data rather than only log lines.

• common/validation: independently-callable validation passes -- new header sdk/include/vtx/common/vtx_validation.h (impl in vtx_validation.cpp). ValidateSchema(json) (wraps the rule-based SchemaValidator; JSON parse failure -> SchemaParseError, rule violations -> SchemaInvalid), ValidateEntity(entity, schema) (entity type resolves to a schema struct; no per-type property array exceeds the schema-declared max), and ValidateFrame(frame, schema) (duplicate unique_id detection per bucket + per-entity validation with full frame/bucket/id context). Each takes the schema either as a resolved PropertyAddressCache or a SchemaRegistry (convenience overload)

• reader/validation: whole-replay validation -- new header sdk/include/vtx/reader/core/vtx_replay_validation.h. ValidateReplay(IVtxReaderFacade&) validates an already-open replay (no file I/O -- the entry point for callers that already hold a reader, e.g. PRISM); ValidateReplayFile(path) is a thin wrapper that opens then delegates. Validates the embedded schema document and runs ValidateFrame over every (or up to max_frames) frame

• common/accessor: strict accessors that return VtxResult alongside the existing tolerant Get/Set (which stay unchanged). FrameAccessor::TryResolve<T>(struct, field) resolves a scalar key or reports NotFound / ContainerMismatch / TypeMismatch (the last with expected/provided type populated); EntityView::TryGet<T>(key) (read) and EntityMutator::TrySet<T>(key, value) (write, respects the frame freeze) report FieldIndexOutOfRange / InvalidArgument

• writer/api: strict frame recording + finalization. ReplayWriter::TryRecordFrame and IVtxWriterFacade::TryRecordFrame return a RecordResult so a rejected frame is observable (a bad game-time registry previously rolled back and returned void silently); RecordFrame stays as the back-compatible fire-and-forget wrapper. A new private FinalizeFrame runs before a frame enters the chunk pipeline: it validates that every entity type resolves to a schema struct and recomputes each entity's content_hash AFTER all schema fields and post-processor overrides (a stale build-time hash would otherwise be persisted). The frame index is assigned monotonically by the writer. New outcome types in sdk/include/vtx/writer/core/vtx_writer_result.h (RecordResult, PipelineReport)

• writer/api: frame freeze on finalization -- once FinalizeFrame runs, every mutation handle derived from that frame's FrameMutationView (including ones a post-processor stashed) is revoked: mutating ops become no-ops and valid() returns false. Implemented via a shared frozen flag threaded through FrameMutationView / BucketMutator / EntityMutator (Freeze())

• writer/api: opt-in last-finalized snapshot -- ReplayWriter::Config::retain_finalized_snapshot (and WriterFacadeConfig / NetworkWriterFacadeConfig, default off). When enabled, the writer keeps the last finalized frame as a read-only snapshot queryable by GetLastFinalizedFrame() and FindEntity(bucket, unique_id). Off by default because it costs a full-frame copy per recorded frame

• writer/pipeline: RecordPipeline::Run now returns a PipelineReport (written / rejected / skipped frames, with rejections split into validation_errors vs timer_errors, plus the per-frame VtxErrors) instead of a bare bool

• common/schema: rule-based schema validator -- SchemaValidator (sdk/include/vtx/common/readers/schema_reader/schema_validator.h) runs a set of independent ISchemaValidationRules over a schema BEFORE the registry resolves it into indices, so malformed schemas are rejected up front instead of degrading at runtime. Produces a SchemaValidationResult (list of SchemaIssue). SchemaRegistry::LoadFromJson / LoadFromRawString reject schemas with validation errors

• common/schema: per-type pre-sizing -- on load, PropertyContainer arrays are pre-sized to the schema's per-type max index (type_max_indices), so field writes by index never reallocate or go out of bounds for schema-declared fields

• samples (arena): the arena pipeline now exercises the full container model end-to-end. Player gains scalar arrays (Abilities string[], AbilityCooldowns float[]), a nested struct (Loadout), a variable-length array-of-structs inventory (Inventory of InventoryItem -- the armor plate drops out while a player is dead and returns on respawn, so the array length changes frame-to-frame), and a map (AmmoByWeapon = map<weapon, AmmoEntry>). These all mutate over the match (ammo spent on fire + reload from reserve, ability cooldowns tick down, medkits consumed on respawn). All four data-source formats -- JSON, Protobuf, FlatBuffers and raw binary -- emit and ingest the new containers identically, verified by equal per-entity content_hash across formats. New schema structs Loadout / InventoryItem / AmmoEntry added to content/writer/arena/arena_schema.json, schemas/arena_data.proto, schemas/arena_data.fbs; arena_generated.h regenerated

• common/loaders: the Protobuf and native (struct-mapping) loaders now build map_properties automatically for containerType: Map struct fields, matching the FlatBuffers loader -- a vector<KVStruct> / repeated message mapped to a Map field lands in map_properties instead of any_struct_arrays. Shared via a new GenericLoaderBase::PushToMap helper (key = the entry's first non-empty string property, else first int32, else Key_N), which the FlatBuffers loader now uses too. Callers no longer need to hand-populate maps for these formats

• samples: new helper samples/arena_container_helpers.h -- ArenaHelpers::AppendMapEntry(loader, dest, schema, field, value) populates a map_properties field with the same key convention. Now used only by the raw-binary arena binding, whose GenericBinaryLoader hand-reads bytes and has no LoadArray auto path; the JSON / Protobuf / FlatBuffers bindings build the map through the loader directly

• tests (differ): DiffEdges.DiffWithMapProperties reactivated (was skipped pending a Map-field fixture) and a Protobuf counterpart DiffEdges.DiffWithMapPropertiesProtobuf added -- both run the same scenario (change a value under an existing key + add a new key) through DiffRawFrames on their respective backend. New dedicated fixture tests/fixtures/test_schema_map.json (the arena schema with the Map field) keeps the shared test_schema.json layout that the other diff/reader/writer tests depend on untouched

• common/accessor: typed map read API -- new VTX::MapView (read-only view over a MapContainer: Size / Empty / Keys / Contains(key) / At(key) -> EntityView / ordinal KeyAt / ValueAt), EntityView::GetMap(PropertyKey<MapView>), and FrameAccessor::GetMapKey(struct, field) (resolves Struct + Map fields). Completes the runtime read surface for maps alongside GetView (nested struct) and GetViewArray (struct array). Covered by tests/common/test_frame_accessor.cpp

• scripts/codegen: vtx_codegen.py now emits a real map accessor for containerType: Map fields -- XView / XMutator get VTX::MapView GetField() const (built on accessor.GetMapKey + EntityView::GetMap) instead of the previous incorrect single-EntityView getter (no setter -- read-only). arena_generated.h regenerated (Player::GetAmmoByWeapon() now returns VTX::MapView)

• writer/api: one-call WriteReplay pipeline (#30) -- new header sdk/include/vtx/writer/core/vtx_write_replay.h. WriteReplay(config, source, format) runs the whole recording pipeline in one call: creates the writer (schema missing/invalid -> SchemaInvalid), initializes the IFrameDataSource (failure -> Internal), drains every frame through TryRecordFrame, finalizes, and returns a WriteReplayResult (ok, error, warnings, frames_written, frames_dropped, total_frames, output_path, elapsed_seconds). Frames rejected by finalization/timer are counted in frames_dropped and surfaced one-per-VtxWarning; the call still produces a valid replay of the accepted frames. SerializationFormat selects FlatBuffers (default) or Protobuf

• writer/api: automatic output-directory creation + destructor finalize (#29) -- WriterFacadeConfig::create_output_dirs (default on) creates any missing parent directories of output_filepath before the sink opens the file (set false to require the directory to pre-exist and keep the previous throw-on-missing behavior). The writer facade destructor now finalizes the file as a best-effort fallback (Stop() wrapped in try/catch) when the caller forgot to call Stop(), so a dropped writer still yields a readable .vtx

• writer/api: in-memory schema sources -- WriterFacadeConfig / NetworkWriterFacadeConfig (and the internal ReplayWriter::Config) gain schema_json_content (a raw JSON string) and schema_registry (a pre-built std::shared_ptr<VTX::SchemaRegistry>, copied in with no re-parse) alongside schema_json_path. Source precedence is registry > content > path, applied consistently in the create-time schema probe and in ReplayWriter. The writer no longer requires the schema to live on disk

• common/schema: the schema's top-level "buckets" array is now parsed and is the source of truth for the frame bucket layout -- SchemaRegistry::GetBucketNames() returns the declared names in order ("buckets"[i] names Frame bucket index i), and PropertyAddressCache carries them (bucket_names) so the reader can use them too. A new Buckets validation rule rejects a malformed key when present (non-array, non-string entries, empty or duplicate names); schemas without the key stay valid (legacy)

• writer/api: schema-driven bucket normalization -- before finalization the writer rearranges every frame's buckets into the schema-declared layout: buckets are reordered to schema order, declared buckets missing from the frame are created empty, and a bucket the schema does not declare rejects the frame with the new VtxErrorCode::BucketUnresolved (observable via TryRecordFrame). Frames built positionally (no bucket_map) adopt the schema layout as long as they do not exceed the declared bucket count. Schemas without a "buckets" array skip the normalization entirely

• reader/api: bucket names restored on read -- bucket names never hit the wire (both formats serialize buckets positionally), so deserialized frames used to come back with an empty bucket_map. The reader now stamps bucket_map from the embedded schema's "buckets" array when chunks are deserialized, so by-name lookups (Frame::GetBucket(name) const) and bucket_map iteration work on read frames. Replays written without a "buckets" array keep the old positional-only behavior

• common/schema: array & map pre-sizing -- extends the existing scalar pre-sizing (type_max_indices) so declared array and map fields are also materialized on load. SchemaStruct / StructSchemaCache gain array_max_indices (max Array-container index per FieldType) and map_max_index (count of Struct-valued map fields); Helpers::ResizeContainerToMaxIndices now pre-creates one empty subarray per declared array field in the matching FlatArray (via new FlatArray::EnsureSubArrayCount) and pre-sizes map_properties. A declared-but-unpopulated array/map field is now present-and-empty instead of absent, symmetric with scalars. Applies wherever scalar pre-sizing already ran (the four frame loaders + schema_dynamic_loader); the loader CRTP hook GetTypeMaxIndices became GetStructSizing (returns the StructSchemaCache)

• reader/api: declared-empty arrays restored on read -- an array with no data is not serialized (nothing to store), so deserialized entities used to come back missing those subarrays even for schema-declared array fields (maps already round-trip their slot count). The reader now re-creates the declared-but-empty subarrays from the embedded schema (Helpers::EnsureDeclaredArrays, grow-only, recursing into nested structs / struct-array elements / map values) so a read frame mirrors the array layout of an ingest-loaded frame. Arrays that carry data round-trip unchanged (their offsets already encode empty subarrays); scalars and maps are untouched

• writer/durability: crash recovery for the file sink -- a recording that dies before Stop() (process crash or power loss, mid-chunk or mid-frame, before the body/footer are closed) can now be recovered up to the last recorded frame instead of losing the whole session. Three cooperating pieces, all file-sink only (the network sink accepts the per-frame hook as a no-op):

  • per-chunk integrity + durability -- ChunkedFileSink::Config gains durable_writes (default on: fsync/_commit each chunk and journal record to physical media via the new DurableFile FILE* wrapper; set off to only fflush to the OS -- survives a process crash but not power loss) and enable_recovery_journal (default on). Every chunk carries an xxHash64 checksum of its on-disk payload, added to ChunkIndexEntry / ChunkIndexData and to both the FlatBuffers (vtx_schema.fbs) and Protobuf (vtx_schema.proto, field 6) footer schemas, so a torn or corrupt chunk is detectable on recovery. The reader parses and exposes it; FlatBuffers header/footer parsing gained a flatbuffers::Verifier guard so a truncated footer is rejected cleanly instead of reading out of bounds
  • recovery journal -- a single write-ahead sidecar <file>.recovery (sdk/include/vtx/writer/policies/sinks/recovery_journal.h) holds a typed, self-validating record stream (each record [u8 type][u32 len][payload][u64 xxHash64], so a torn last record from a mid-write crash is detected and dropped): an S record (written once) carries the writer's timing parameters {fps, is_increasing}, C records commit a durable chunk's index entry, T records carry each committed frame's exact {game_time, created_utc}, and F records carry each in-flight (un-flushed) frame's times + serialized payload. The log is strictly append-only in the crash-critical path -- a frame appends an F record; a flush appends the chunk's C + T records -- so at every instant the file is a valid prefix and there is no window in which a fsync'd chunk is described by neither its F records nor its C record. Ordering is data-before-journal: a chunk is fsync'd into the .vtx first, then its C/T records are appended and fsync'd. A committed chunk's now-redundant F records (repair dedups them by frame index) are reclaimed by periodic compaction -- the log is rewritten (all C/T + only the still-pending F) into a temp file that atomically replaces the sidecar, so a crash during compaction leaves either the old or the new complete journal. The writer journals every frame as it is recorded (ReplayWriter::TryRecordFrame -> new SinkPolicy::JournalFrame hook; timing via SinkPolicy::JournalTiming), so a crash between flushes still recovers the pending batch. If the journal cannot be opened cleanly, journaling is disabled and the torn sidecar is removed (a half-written journal would otherwise block repair). On a clean Close() the footer is written and the .recovery is deleted -- its presence at open time signals an unclean shutdown
  • repair -- VTX::RepairReplayFile(path) (sdk/include/vtx/writer/core/vtx_replay_recovery.h) reconstructs a valid, readable file from a footerless .vtx + its journal: it verifies each committed chunk's checksum (dropping a torn/corrupt tail), re-appends the in-flight frames as chunks, rebuilds the seek table and the exact per-frame times, and synthesizes the footer -- including the derived time data (duration_seconds, timeline gaps, game segments), reconstructed from the journaled timing (S record) with the same expressions VTXGameTimes uses, so a recovered footer matches a clean Stop() exactly (manual segment marks are the one thing not journaled). Recovery is deliberately not automatic on open -- the user-driven flow is ReplayNeedsRecovery(path) (cheap sidecar check) / RecoveryJournalPath(path) (locate the sidecar) then RepairReplayFile(path). A file that already ends with a valid footer plus a leftover journal (a crash between the footer fsync and the journal delete) is detected and preserved untouched; a journal whose recorded format magic disagrees with the file, or whose header is unreadable, is refused rather than applied; a recording still being written is refused without touching it (on Windows the writer holds a deny-write handle, so repair's truncate fails cleanly). Returns a RepairResult (was_clean / repaired / recovered_chunks / recovered_frames / error). Both FlatBuffers and Protobuf files are supported

• tests: tests/writer/test_crash_recovery.cpp -- 65 cases covering the full crash-window matrix: recovery between chunks (single- and multi-frame), between frames (in-flight batch, and nothing-flushed-yet), a torn tail chunk, a partial chunk written before its journal record, a checksum-detected corrupt chunk, a torn pending-frame record, a torn chunk-commit record (the batch falls back to its still-present F records -- the append-only guarantee), a crash mid-footer-write, a leftover-journal-over-valid-footer, a stale compaction temp, a mismatched journal format, a clean file (no-op), a header-only crash (0-frame result), the lingering-F-record dedup guard, compaction reclaim, the recovery helpers, exact per-frame time preservation, a live-recording repair refusal (Windows), and the FlatBuffers + Protobuf repair paths. Crash states are fabricated byte-faithfully through the same RecoveryJournal API the sink uses, plus end-to-end tests through the real writer (a raw ReplayWriter dropped without Stop()) for both formats that require the recovered footer to match a cleanly-stopped control exactly: per-frame game_time + created_utc, duration_seconds, timeline gaps, game segments, and per-entity content_hash -- and require the recovered file to pass whole-replay ValidateReplayFile and cache-hostile out-of-order seeks across the committed-chunk/recovered-chunk boundary. Also covered: the sink's full config matrix (durable_writes x b_use_compression) with frames large enough that zstd compression genuinely engages (committed chunks and journaled F payloads take the compressed path), a repair interrupted mid-run and re-run (idempotent), a second repair on an already-repaired file (clean no-op, byte-identical), a journal-opted-out crash (repair reports clean and leaves the footerless file byte-identical; the reader rejects it gracefully), a session-start crash (torn S record + header-only .vtx -> valid 0-frame file), torn T records (frame times fall back to the still-present F records -- exact, not zeroed), a 0-byte journal (refused, main file untouched), compaction through the real sink under crash (new Config::journal_compact_threshold_bytes, 0 = default), map-container frames recovered intact from F records, a non-ASCII path through the full sidecar flow, hostile checksummed journal records (a C offset inside the header region, a C size at or below its own length prefix, an F with an empty payload -- each dropped safely, degrading to a valid file), a torn main-file header alongside a valid journal (refused, bytes untouched), and a decreasing-time recording (is_increasing = false -- the other branch of the segment reconstruction) matching its clean control. Scale + pipeline coverage: a stress run (2030 multi-entity frames, 40 committed chunks + a 30-frame in-flight batch -- every timestamp exact, whole-replay validation clean), rejected frames interleaved mid-recording (timer rejections leave no trace and do not desync the journal's frame indexing), and a post-processor recording (journaled F records capture the frame as it would hit disk -- post-mutation -- proven by a pending frame that only ever existed in the journal). Lifecycle + hygiene: a double-crash lifecycle over the same path (crash -> repair -> re-record -> crash -> repair, second recovery reflects only the second session), a stale sidecar under journaling opt-out (a session with enable_recovery_journal = false removes any leftover .recovery at start so it cannot masquerade as recovery state), a foreign .recovery file (no VTXR magic -- never deleted by repair, even on the clean-file path), a hostile out-of-range T record (ignored by the bounds check), the map crash-recovery path on Protobuf as well as FlatBuffers, a stale journal from a different recording over a replaced file (per-chunk checksums reject the foreign chunks -- graceful degradation to a valid empty file), a read-only crashed file (repair refuses without consuming the journal; succeeds untouched once the file is writable), a no-time-registry recording (EGameTimeType::None, fully FPS-synthesized timeline recovered exactly), and byte-budget chunk splitting (max_bytes, the other ThresholdChunkPolicy branch) through crash + recovery. The maximal guarantee is pinned by BoundaryCrashRecoversByteIdenticalFile (FlatBuffers and Protobuf, plus a 120-frame large-footer variant that exercises the compressed-footer path): a crash exactly at a chunk boundary recovers to a file that is bit-for-bit identical to a clean Stop() with the same inputs -- repair passes the footer's time vectors exactly as Stop() does (present-but-empty rather than absent) and compresses the synthesized footer exactly as the sink's Close() would (the S record now also journals {use_compression, compression_level}; journal version 4). A journal with an incompatible version field is refused outright, main file untouched. On top of the hand-picked windows, a brute-force sweep suite (CrashRecoverySweep) proves the "any crash point" claim literally: the journal truncated at every byte length, the main .vtx truncated at every byte length, and every journal byte flipped one at a time -- thousands of repair runs, none may crash, and every claimed repair must open and agree with its own reported frame count and footer times. ReadValid now bounds each record's payload allocation by the journal's actual file size (a corrupt length field can no longer trigger a giant transient allocation), pinned by HostileRecordLengthIsBoundedByFileSize; the truncation sweep also runs over the post-compaction journal layout. Finally, CrashRecoveryProcess (Windows) performs a genuine process kill: vtx_tests spawns itself as a child in a hidden endless-writer mode and TerminateProcesses it mid-recording -- no destructors, no fclose, handles abandoned to the kernel -- then repairs and verifies frame content and exact footer times, in both durability modes (this is the honest validation of the flush-only claim that data handed to the OS survives a process death) -- plus a kill under an aggressive compaction cadence (a full close/rewrite/atomic-rename cycle per commit), so real process death lands amid compaction traffic and the atomic-rename guarantee holds, and a varied-progress kill soak (kills landing just after the first journaled frames, around the first flush, amid compaction cycles, and several chunks deep -- alternating durability modes). The recovered output was also smoke-checked through the end-user toolchain: vtx_cli opens a repaired file and serves info / footer / frame normally. The whole suite (including the brute-force sweeps and real process kills) also ran clean under AddressSanitizer (MSVC /fsanitize=address) -- zero memory errors in the SDK; the one report is a known protobuf-internal DynamicMessage sized-delete artifact, documented in tests/sanitizer_suppressions/asan.supp (ASAN_OPTIONS=new_delete_type_mismatch=0). RecoveredFileTranscodesToCleanChunks pins the normalization workflow for salvaged files (open -> drain frames with footer times -> re-record): the one-frame recovery chunks become proper chunks, created_utc round-trips exactly, and game_time drifts at most 1 tick (100 ns) per frame through the float-seconds register -- the documented transcode caveat. BM_ReaderRecoveredTail (benchmarks) measures the salvage cost that motivates it: a 200-frame one-frame-chunk tail reads sequentially ~3x slower than clean chunking

• benchmarks: new benchmarks/bench_crash_recovery.cpp (BM_WriterDurabilityTier) quantifies the writer hot-path cost of the recovery defaults across the three durability tiers (journal off / journal + flush-only / journal + fsync). Reference numbers on an NVMe dev machine, 200 small frames per iteration: ~19 us/frame with no journal, ~30 us/frame flush-only (process-crash safe -- effectively free), ~289 us/frame with the default per-operation fsync (power-loss safe; ~1.7% of a 60 fps frame budget on NVMe -- on spinning disks prefer durable_writes = false). The byte-identity tests now compare from the end of the header on (the header embeds a second-granularity recording timestamp, so two separately recorded sessions may legitimately differ there; repair never rewrites the header)

• docs: docs/SDK_API.md "Writing Replays" gains a "Crash recovery" section -- the .recovery sidecar model, the user-driven ReplayNeedsRecovery / RepairReplayFile / RecoveryJournalPath flow with RepairResult semantics, the exact-times guarantee, the refusal cases, and the sink durability knob table (durable_writes / enable_recovery_journal / journal_compact_threshold_bytes). README.md feature list gains a "Crash-safe recording" bullet. DurableFile hardening: Seek/SeekEnd failures now latch Good() false (a write after an unnoticed seek failure would land at the wrong offset), and the dead Truncate primitive left over from the pre-append-only journal design was removed

• reader/api: side-effect-free frame peek -- IVtxReaderFacade::GetResidentFrame(frame_index) (ReplayReader::GetResidentFramePtr) returns the frame ONLY if its chunk is already resident, without moving the cache window, triggering loads, or cancelling in-flight ones. Intended for incidental reads (e.g. drawing a stale frame while the target frame streams in); GetFrame remains the sole window-driving read. The inspector's stale-frame fallback now uses it (see Fixed)

• reader/api: new ReplayReaderEvents::OnChunkLoadCancelled(int32_t) -- fired when a load that already emitted OnChunkLoadStarted ends without the chunk becoming resident (cancelled by a window shift, or the worker failed). ReaderChunkState handles it (clears the chunk from the loading set without marking it loaded) and OpenReplayFile wires it automatically, so the started/finished pair always balances

Changed

• reader/perf: chunk teardown moved off the caller thread. Evicting a chunk destroys up to ~1000 frames x hundreds of entities (measured hundreds of ms to >1s for real captures); UpdateCacheWindow now only unlinks evicted chunks and hands the owned data to a detached background task, so cross-range jumps no longer freeze the calling (UI) thread. The reader destructor does the same with the resident cache (closing a large replay returns at once) -- it still waits for in-flight load workers (they touch the reader), but not for the frees (chunk data is self-contained). Cancelled prefetch futures are parked and reaped via wait_for(0) instead of being destructed inline (a std::async future blocks in its destructor until the worker unwinds)
• sdk-wide: error/warning handling unified onto the diagnostics model -- there is now a single Severity enum, a single VtxErrorCode vocabulary, and a single diagnostic/result type family across the SDK:
  • The schema validator's SchemaIssueSeverity is now an alias of Severity, and SchemaValidationResult::ToReport() is the single canonical bridge from SchemaIssue to VtxDiagnostic (used by ValidateSchema).
  • The writer's frame-rejection reason (previously a writer-private FrameRejectReason enum) was removed; RecordResult now carries a VtxError.
  • The reader's ready-state and open-failure now use VtxError: ReplayReader/IVtxReaderFacade::GetReadyError() and ReaderContext::GetError() return VtxError (was std::string), and ReplayReaderEvents::OnReadyFailed delivers a const VtxError&. ValidateReplay surfaces these structured errors directly.
• writer/reader: strict cancel on invalid schema -- creating a writer or opening a reader against a schema that fails validation is refused up front rather than proceeding with a degraded schema
• writer/api (entity-type resolution): entity type names must resolve to schema structs. Unknown types now fail at entity creation (in the loader's EnsureResolvedType) and are rejected during finalization, instead of surfacing later during serialization
• writer/api (unique-id dedup): duplicate unique_ids within the same bucket are rejected. BucketMutator::AddEntity(unique_id) refuses a duplicate id, BucketMutator::SetUniqueId enforces uniqueness at assignment, and the loader's actor-append path deduplicates within a bucket
• build: module CMakeLists.txt files use PROJECT_SOURCE_DIR / VTX_SDK_SOURCE_DIR instead of CMAKE_SOURCE_DIR for source paths, so projects that consume VTX via FetchContent / add_subdirectory no longer need a patch
• ci: Windows matrix jobs pinned to windows-2022 (was windows-latest). The windows-latest image moved to a newer default CMake generator (Visual Studio 18 2026) that mismatches the Visual Studio 17 2022 generator baked into the cached build/_deps FetchContent subbuild trees, failing the flatbuffers populate step (CMake step for flatbuffers_src failed). Pinning restores a stable default generator (matching local dev) and changes the cache-key prefix so the mismatched cache is no longer restored

Fixed

• reader/async: cancelled or failed async chunk loads corrupted the chunk-state tracking. AsyncLoadTask fired OnChunkLoadFinished for ANY surviving worker -- including cancelled ones whose data was discarded -- so ReaderChunkState marked chunks "loaded" that were never resident; eviction (which iterates the cache) never fired for them and the loaded set grew without bound. A failed (non-cancelled) load also inserted an empty CachedChunk into the cache, permanently blocking retries for that chunk (reads returned null forever). Now a chunk is inserted only when the load actually succeeded, and non-resident outcomes emit the new OnChunkLoadCancelled. Regression test: AsyncRandomJumpsDoNotLeakLoadedSet
• reader/async: the async view path could wedge on "loading" forever. trigger(current_idx) ran subject to the max_concurrent_loads cap and AFTER the range-equality short-circuit -- a cross-range jump with the slots saturated by not-yet-reaped cancelled loads skipped the viewed chunk, committed the new range, and every subsequent (unchanged-range) call returned early, so the chunk was never queued. The viewed chunk now triggers first, uncapped (priority=true), and before the short-circuit, making the path self-healing. Regression test: AsyncViewPathResolvesAfterCrossRangeJump
• tools/inspector: scrubbing could ping-pong the cache window and never finish loading. While the target frame streamed in, the stale-frame fallback called GetFrame(last_drawn) -- a window-driving read that moved the window back to the stale chunk and cancelled the target's load; the next tick reversed it, indefinitely. The fallback now uses the side-effect-free GetResidentFrame. Regression tests: ResidentFramePeekDoesNotCancelTargetLoad, ResidentFramePeekNeverTriggersLoad
• common/hash: Helpers::CalculateContainerHash used a single shared thread_local XXH3 state while recursing into nested structs / struct-arrays / maps -- each recursive reset wiped the parent's in-progress accumulation, so once a container had any nested/map field its scalars / strings / vectors no longer affected the hash. Now each call uses its own state. Latent until the arena gained nested/map fields; it made distinct entities collide (e.g. two players sharing a content_hash), which in turn made the content-hash-based diff short-circuit treat changed frames as identical. Regression tests added in tests/common/test_content_hash_edges.cpp (NestedStruct/Map/StructArrayDoesNotMaskPreRecursionFields)
• tools/cli: the interactive diff <a> <b> command always reported "frames identical" (0 ops) -- CliSession::DiffFrames computed the patch but discarded it (returned {}) and passed the possibly-evicted span_a instead of the bytes_a copy it had made. Now returns the real patch computed over the copy (compounded by the content_hash bug above; both fixed)
• common/loaders: GenericFlatBufferLoader::LoadArray could not handle vector<string> -- the pointer-element branch assumed table structs and tried to Load a flatbuffers::String (no FlatBufferBinding, a hard compile error if instantiated). It now detects the string element (via s->str()) and fills the scalar string array, removing the manual workaround in the arena FlatBuffers binding
• reader/proto: the Protobuf reader silently dropped map data on read. proto_to_vtx.cpp deserialized every container except map_properties / map_arrays, so a map written into a .vtx came back empty from the Protobuf reader (FlatBuffers was unaffected). Added a FromProto(MapContainer) overload and wired both map_properties and map_arrays into FromProto(PropertyContainer). The Protobuf schema (vtx_schema.proto) and writer (vtx_to_proto.h) already serialized maps -- only the read path was missing
• differ: map fields were never diffed correctly on either backend. The FlatBuffers and Protobuf view adapters' GetMapSize / GetMapKey / GetMapValueAsStruct treated the outer map_properties vector (one MapContainer per map-field slot) as if each element were a single key/value pair -- so a multi-entry map reported size 1, and added / removed keys were never seen. They now iterate the entries within the field's MapContainer (slot 0, matching the writer/loader convention), so DefaultTreeDiff::DiffMapContainers emits correct add / remove / value-change operations. The Protobuf adapter's "Case 2" also looked for capitalised Keys / Values fields on the wrong message; it now descends into map_properties -> MapContainer.keys / .values. A single map field per struct is assumed (documented in both adapters)
• writer/reader (Protobuff -> Protobuf rename) (#28): the misspelled "Protobuff" was removed from the public API -- breaking, no aliases kept. SerializationFormat::Protobuffs -> SerializationFormat::Protobuf; CreateProtobuffWriterFacade -> CreateProtobufWriterFacade; CreateProtobuffNetworkWriterFacade -> CreateProtobufNetworkWriterFacade; reader CreateProtobuffFacade -> CreateProtobufFacade (internal ProtobuffFacadeImpl -> ProtobufFacadeImpl). Migrate call sites with a literal Protobuff -> Protobuf rename
• common/types: FlatArray::GetSubArray & FlatArray::GetMutableSubArray did not have a check for StartIndex being greater or equal to EndIndex. In exceptional cases, like an empty array this can cause a crash or undefined behaviour in the best case scenario.
• writer/flatbuffers: FlatBuffersVtxPolicy::FromNative hardcoded exactly two bucket slots -- buckets[0] was renamed to "data", buckets[1] to "bone_data", and any bucket at index >= 2 was silently dropped from the file. It now iterates all buckets generically (same shape as the Protobuf policy, shared Serialization::SortBucketByTypeId helper in bucket_type_sort.h), preserving every bucket and the frame's bucket_map. Bucket naming is schema-driven (see the "buckets" entries above), not serializer-driven
• reader/schema: the reader-side PropertyAddressCache built from the embedded schema (PopulateCacheFromJsonString) hand-copied the registry cache and omitted type_max_indices, so ValidateEntity on read frames treated every per-type max as 0 (any populated property would flag FieldIndexOutOfRange once frame validation ran). It now reuses SchemaRegistry::GetPropertyCache() verbatim. This also makes ValidateReplay's per-frame pass effective for replays whose schema declares a "buckets" array -- it iterates frame.bucket_map, which was always empty on read frames before bucket-name restoration. Replays whose schema has no "buckets" array still read back with an empty bucket_map, so their per-frame pass remains a no-op (unchanged from before)
• samples/benchmarks: basic_write.cpp and bench_writer.cpp created a "Players" bucket while writing against the arena schema, which declares "buckets": ["entity"] -- under schema-driven normalization those frames would now be rejected. Both use "entity"

Notes

• writer/sources -- shared-memory input is experimental / WIP and not functional yet. A set of shared-memory headers (SharedMemoryFrameDataSource, ISharedMemoryTransport, SpscRingTransport, detail/shm_ring.h) landed unintentionally as part of #22's squash (the commit was titled as a CMake fix). The shared-memory frame source is work in progress and not a supported input path -- do not depend on it. The two functional streaming sources remain pipe and WebSocket.

[0.3.0] - 2026-05-26

Added

• tests: integration coverage for both new streaming IFrameDataSource implementations -- runs in the existing GoogleTest suite and so on every GitHub Actions matrix entry (Windows + Linux). tests/writer/test_pipe_source.cpp drives PipeFrameDataSource<TestAdapter> in server mode with VTX on the receive end and a test-spawned client thread as the producer (Windows CreateFileA+WriteFile, POSIX open+write); covers the happy path (50 frames + sentinel round-trip via OpenReplayFile), sentinel-only empty stream, adapter-false-stops-stream after N valid frames, and the GetExpectedTotalFrames() == 0 streaming contract. tests/writer/test_websocket_source.cpp drives WebSocketFrameDataSource<TestWsAdapter> against an in-process ix::WebSocketServer loopback on an OS-assigned port -- covers the happy path (25 messages + parseable .vtx), Initialize() returns false on a refused connection, and the streaming-total contract. Both files share a DrainSourceIntoWriter helper that takes the writer by unique_ptr and destroys it before reading back the file -- the ChunkedFileSink only flushes its std::ofstream in its destructor, so an in-scope writer would race the reader. vtx_tests links $<BUILD_INTERFACE:VTX::deps::ixwebsocket> directly because vtx_writer keeps the dep PRIVATE behind its PIMPL boundary. WebSocket happy-path uses the per-client message callback (setOnClientMessageCallback, v12 path) which gives a WebSocket& directly and avoids the weak_ptr-vs-handshake race that the v11-style setOnConnectionCallback triggers; the bound port is pre-allocated with a raw bind+getsockname dance because ix::WebSocketServer::getPort() returns the constructor argument, not the OS-assigned port

• writer/sinks: ChunkedNetworkSink<Policy> -- second sink alongside ChunkedFileSink, streams the identical binary .vtx byte stream over a TCP socket instead of writing to disk. Receiver concatenates the bytes into a file and gets a valid .vtx; no custom wire framing beyond the on-disk format's own magic + size prefixes. New public header sdk/include/vtx/writer/policies/sinks/network_sink.h -- self-contained Win32/POSIX socket portability layer (getaddrinfo + socket + connect), throws std::runtime_error on connect failure (parallel to ChunkedFileSink throwing on open failure), SendAll retries on partial send, chunk index entries record file_offset via a running bytes_sent_ counter (the tellp() analogue for a stream). New facade entry points: NetworkWriterFacadeConfig (host + port + the usual replay/chunking/compression knobs, no output_filepath) plus CreateFlatBuffersNetworkWriterFacade / CreateProtobuffNetworkWriterFacade -- mirror of the file-sink factories, behind the same IVtxWriterFacade abstraction so existing user code is sink-agnostic. vtx_writer propagates ws2_32 PUBLIC on Windows so consumers that instantiate ChunkedNetworkSink get the linkage automatically. Six integration tests in tests/writer/test_network_sink.cpp spin up a loopback TCP server on a free ephemeral port (server thread signals readiness via std::promise so the client cannot connect before listen() returns), drain bytes into a vector, and prove the received stream parses cleanly via VTX::OpenReplayFile (FlatBuffers + Protobuf), the constructor throws on a refused connection, zero-frame sessions produce a valid stream, and chunked sessions produce the expected seek-table entries

• writer/sources: WebSocketFrameDataSource<Adapter> -- streaming IFrameDataSource over WebSocket (RFC 6455), connects as a client to ws://host:port/path or wss://host:port/path (TLS). Each WebSocket message is one serialized frame; ping/pong + close + fragment reassembly are handled transparently by the underlying transport. New public header sdk/include/vtx/writer/sources/websocket_frame_source.h plus a PIMPL facade in sdk/include/vtx/writer/sources/detail/websocket_client.h -- the implementation in sdk/src/vtx_writer/.../websocket_client.cpp wraps IXWebSocket and bridges its async callbacks to a blocking ReadMessage via a thread-safe queue (std::mutex + std::condition_variable), so the writer's pull-based GetNextFrame slots in cleanly. IXWebSocket stays fully hidden behind the PIMPL boundary -- the public SDK headers do not see it. wss:// verifies the server certificate against the OS trust store by default (tls.caFile = "SYSTEM"). Auto-reconnect is disabled: a dropped connection surfaces as end-of-stream so the consumer finalises the .vtx instead of silently resuming mid-file. Format-agnostic: the Adapter (constrained by the IFramePayloadAdapter concept) is the only piece that knows the on-wire payload format -- the sample's JsonWebSocketAdapter parses each message as JSON via nlohmann::json + JsonMapping<T> + UniversalDeserializer (declarative -- same pattern as arena_mappings.h)

• writer/sources: PipeFrameDataSource<Adapter> -- streaming IFrameDataSource over OS pipes, format-agnostic via a caller-supplied adapter. New public header sdk/include/vtx/writer/sources/pipe_frame_source.h -- length-prefixed framing on the wire ([uint32 LE size][payload], zero-size sentinel = clean EOF) with the payload format determined by the Adapter (sample uses JSON). Three transport modes selected by Config:

  • stdin (pipe_path empty) -- for shell pipelines producer | vtx. Sets _O_BINARY on Windows so CRLF translation doesn't corrupt the wire.
  • client (pipe_path set, as_server = false) -- VTX connects to a pipe / FIFO a producer already created (fopen("rb")).
  • server (pipe_path set, as_server = true) -- VTX creates the pipe and blocks waiting for a producer to connect. Windows: CreateNamedPipeA(PIPE_ACCESS_INBOUND, PIPE_TYPE_BYTE) then ConnectNamedPipe, with the resulting HANDLE wrapped in a FILE* via _open_osfhandle + _fdopen so ReadExact / GetNextFrame stay platform-agnostic. POSIX: mkfifo(0666) (tolerates EEXIST) then fopen -- blocks until a writer opens the other end. The FIFO is unlinked on destruction.

  Server mode is the mode for an external, independent producer -- e.g. a game injector that opens \\.\pipe\vtx as a client when its game launches. The injector needs nothing from the VTX SDK: it speaks the [uint32][payload] framing and writes a zero-size sentinel when the session ends. Self-contained contract -- any language

• writer/sources: IFramePayloadAdapter concept (sdk/include/vtx/writer/sources/frame_payload_adapter.h) -- shared compile-time contract for adapters used by streaming sources (PipeFrameDataSource, WebSocketFrameDataSource). Single requirement: bool ParseFrame(std::span<const std::byte> payload, VTX::Frame&, GameTime::GameTimeRegister&). The source owns the transport + framing; the adapter is the only place that knows the wire format (JSON / Protobuf / custom binary). Applied as the template constraint on WebSocketFrameDataSource

• samples: vtx_sample_websocket_consumer (samples/websocket_consumer.cpp) -- connects to a WebSocket server as a client and records the stream into a .vtx. Uses a JsonWebSocketAdapter built on VTX::UniversalDeserializer<>::Load<WsFrame>(JsonAdapter) -- the JSON->struct step is fully declarative (JsonMapping<WsFrame> + JsonMapping<WsEntity>), only the struct->VTX::Frame mapping is spelled out. Companion samples/websocket_server.py -- minimal Python WebSocket server using the websockets library; streams JSON frames continuously (~20 entities/frame) until the client disconnects or the user presses Ctrl+C. CLI: vtx_sample_websocket_consumer ws://127.0.0.1:8765/ out.vtx schema.json

• samples: vtx_sample_pipe_producer (samples/pipe_producer.cpp) + vtx_sample_pipe_consumer (samples/pipe_consumer.cpp) -- pair of CLI tools exercising the pipe data source. Producer emits length-prefixed JSON frames to stdout; consumer reads either stdin (-), a connected pipe (<path>), or creates one (serve:<path>). Producer supports two modes: bounded (producer N) sends exactly N frames then a sentinel; continuous (producer 0) streams indefinitely at ~60 fps until the user presses Enter in its terminal -- a detached stdin watcher thread sets an std::atomic<bool> stop flag, which makes the loop emit the sentinel cleanly and exit, so the consumer finalises a valid .vtx. Three demo .bat scripts:

  • samples/pipe_demo.bat -- anonymous-pipe demo (producer | consumer, single launcher, bounded run).
  • samples/named_pipe_vtx.bat + samples/named_pipe_producer.bat -- two-terminal demo of independent processes meeting over a Windows named pipe. VTX bat runs the consumer in server mode (serve:\\.\pipe\vtx); producer bat retry-connects (the redirect to a not-yet-existing pipe fails, so the retry loop IS the "wait for VTX" handshake -- mirrors how a real named-pipe client like a game injector waits for its server)

• dependencies: IXWebSocket v12.0.0 + mbedTLS v3.6.2 via FetchContent (cmake/VtxDependencies.cmake). Same pattern as FlatBuffers / zstd: one pinned version on every platform, no system packages, nothing shipped at runtime. IXWebSocket consumed through a new VTX::deps::ixwebsocket INTERFACE target; linked PRIVATE into vtx_writer (and so never reaches the public SDK headers thanks to the websocket-client PIMPL). mbedTLS detection inside IXWebSocket is fragile across versions, so MBEDTLS_INCLUDE_DIRS + MBEDTLS_LIBRARY + MBEDTLS_X509_LIBRARY + MBEDTLS_CRYPTO_LIBRARY are pre-seeded in the CMake cache before FetchContent_MakeAvailable(ixwebsocket); IXWebSocket's bundled FindMbedTLS.cmake's find_path / find_library short-circuit on the already-set cache entries and resolve to our FetchContent targets. IXWebSocket's bundled MbedTLS 3.x version detection misses the header reorganisation in mbedTLS 3.x, so we force IXWEBSOCKET_USE_MBED_TLS_MIN_VERSION_3 on the ixwebsocket target unconditionally (we pin mbedTLS 3.6, so we know). IXWebSocket's USE_ZLIB is disabled (no permessage-deflate extension -- avoids pulling in zlib for a feature we do not need)

• docs: docs/SDK_API.md "Writing Replays" gains a "Streaming sinks" subsection covering NetworkWriterFacadeConfig + the two CreateXNetworkWriterFacade factories. "Integration Primitives" gains subsections for PipeFrameDataSource, WebSocketFrameDataSource, and IFramePayloadAdapter. docs/SAMPLES.md updated with the three new sample targets + the demo scripts. docs/ARCHITECTURE.md module overview notes the new sinks + sources and lists IXWebSocket / mbedTLS under vtx_writer dependencies. docs/BUILD.md dependency-resolution section gains IXWebSocket and mbedTLS rows + the FetchContent narrative updated accordingly. README.md feature list gets a "Live streaming transports" bullet, "Write a replay" gains a #### Optional: stream frames live subsection with a PipeFrameDataSource snippet and a note on external-producer ergonomics; the Requirements blurb is corrected to reflect that FlatBuffers / zstd / IXWebSocket / mbedTLS all come from CMake FetchContent rather than system packages

• legal: NOTICE and THIRD_PARTY_LICENSES.md extended with the two new third-party components added under Apache-2.0 §4(d). IXWebSocket (BSD-3-Clause, Machine Zone, Inc.) and mbedTLS (Apache-2.0, The Mbed TLS Contributors) each get their own section with the upstream URL, pinned version, license name, and the relevant license / NOTICE text -- mbedTLS's Apache-2.0 text shares VTX's own LICENSE per the existing FlatBuffers pattern; IXWebSocket carries the full BSD-3-Clause text inline

[0.2.0] - 2026-05-12

Added

• writer/api: writer-side frame post-processor pipeline. A new hook fires inside ReplayWriter::RecordFrame after timer validation and before Serializer::FromNative consumes the native Frame, so whatever the processor mutates is what gets serialised to the on-disk .vtx. Three new public headers and three new facade methods materialise the feature:

  • sdk/include/vtx/writer/core/vtx_frame_post_processor.h -- IFramePostProcessor interface (Init / Process / Clear / PrintInfo), FramePostProcessorChain composable container, FramePostProcessorInitContext (frame_accessor + total_frames + schema/format version) and FramePostProcessContext (per-frame: global_frame_index, schema_version, frame_accessor) carriers. Chain execution: Init/Process/PrintInfo in registration order; Clear in reverse (destructor-like teardown); last writer wins on shared property mutations.
  • sdk/include/vtx/writer/core/vtx_frame_mutation_view.h -- write-side mirror of EntityView / FrameAccessor. EntityMutator (non-owning wrapper over PropertyContainer* with Get<T> + Set<T> + GetMutableView + GetMutableArray<T>); BucketMutator (mutable iteration + structural mutation: AddEntity / RemoveEntity / RemoveIf / Clear); FrameMutationView (entry point the processor receives -- wraps Frame& + borrows a FrameAccessor* so processors can resolve schema names without coupling to reader internals). Hot-path cost is identical to EntityView::Get -- single non-owning pointer indirection, fully inlinable.
  • IVtxWriterFacade::SetPostProcessor(std::shared_ptr<IFramePostProcessor>) / GetPostProcessor() / ClearPostProcessor() -- registration API on the writer facade, forwarded to both FlatBuffersWriterFacadeImpl and ProtobuffWriterFacadeImpl. Init() runs synchronously inside SetPostProcessor BEFORE the new processor becomes visible to any RecordFrame() -- this is the right place to resolve every PropertyKey<T> upfront since the schema is constant for the recording session. The writer is single-threaded by design (RecordFrame called sequentially from the capture loop) so no mutex is needed on post_processor_. The destructor invokes Clear() on whatever is currently registered. SetPostProcessor does NOT call Clear on the previously-registered processor; the caller keeps the shared_ptr and calls Clear explicitly if they need outgoing teardown -- use ClearPostProcessor() for the common case of explicit pre-destruction reset

• scripts/codegen: scripts/vtx_codegen.py extended to emit, per schema struct, in addition to the existing XView read-only wrapper:

  • XMutator -- write-capable wrapper around EntityMutator. All Get* methods identical to the View; adds Set*(value) for scalars and GetMutable*() returning std::span<T> (arrays) or EntityMutator (nested structs). PropertyKey<T> resolution stays cached in static locals per-method on first use, so registering a processor doesn't trigger a one-time hash sweep.
  • ForEachX(BucketMutator&, FrameAccessor&, Fn) -- template helper that filters a bucket by entity_type_id (matching EntityType::X) and invokes the lambda with an XMutator&. Read-only counterpart ForEachXView(const Bucket&, FrameAccessor&, Fn) paralleled. Result: processors operate on strongly-typed views (p.SetHealth(...)) with zero hardcoded schema strings, zero PropertyKey<T> members on the processor, and no manual entity_type_id gating -- if the schema changes, regenerating the header makes new properties available; if a property is renamed or removed, code fails to compile early instead of silently mismatching at runtime

• samples: vtx_sample_post_process_write target (samples/post_process_write.cpp) -- minimum end-to-end demo of the writer-side post-processor. Builds synthetic frames with intentionally out-of-range Health values via the codegen-generated PlayerMutator, registers a PlayerHealthProcessor (clamp [0, 100], derive IsAlive=false when Health<=0, cross-frame low-health counter, lifecycle hooks), records 30 frames, then re-opens the .vtx with OpenReplayFile and uses ForEachPlayerView (also codegen-generated) to print the persisted values -- proving the on-disk bytes contain the post-processed state, not the raw input

• samples: samples/advance_write.cpp extended to register an ArenaConsistencyProcessor on each of the three pipelines (JSON / Protobuf / FlatBuffers source). Same processor instance per pipeline using VTX::ArenaSchema::ForEachPlayer -- demonstrates that frame post-processing is orthogonal to the source format: the same logic runs on the canonical VTX::Frame regardless of whether it came from JSON, Protobuf, or FlatBuffers

• tests: tests/writer/test_frame_post_processor.cpp with 10 cases:

  • WriterPostProcessor_MutationViewUnit.SetThenGetRoundTrips and WriterPostProcessor_ChainUnit.OrderAndRemove -- standalone unit smokes for the mutation view + chain primitives.
  • WriterPostProcessorTest.NoProcessorBaselineUnchanged -- behaviour identical when no processor is registered.
  • WriterPostProcessorTest.DoubleHealthIsPersistedToDisk -- Init resolves the Health key, processor doubles values pre-serialise, readback confirms 200.0f on disk.
  • WriterPostProcessorTest.ChainLastWriterWinsOnDisk and .ChainRemoveDropsAndOtherStillFires -- chain ordering + Remove semantics from disk.
  • WriterPostProcessorTest.GhostInjectorEntityIsOnDisk -- BucketMutator::AddEntity injects a synthetic entity with entity_type_id set explicitly, readback confirms it persisted.
  • WriterPostProcessorTest.TeamTwoFilterDropsEntitiesFromDisk -- BucketMutator::RemoveIf filters entities pre-serialise.
  • WriterPostProcessorTest.GlobalFrameIndexIsMonotonic -- ctx.global_frame_index monotonically increments across RecordFrame calls.
  • WriterPostProcessorTest.ClearPostProcessorCallsClearAndUnregisters -- explicit teardown invokes Clear and subsequent RecordFrame calls bypass the processor entirely

• docs: new docs/POST_PROCESSING.md -- dedicated reference covering the feature pipeline diagram, lifecycle (Init synchronous before first Process, Clear on destructor / explicit teardown), threading model (single-threaded writer, no mutex needed), two ways to write a processor (generic with raw PropertyKey<T> vs codegen-driven strongly-typed XMutator / ForEachX), patterns for cross-frame state / chains / replay-level metadata / schema-version branching / structural mutation, error handling (exceptions swallowed at hook boundary), performance characteristics (zero overhead when unused; same hot path as EntityView when active), gotchas (the FlatBuffers serialiser drops entities with entity_type_id < 0, the writer renames bucket[0] to "data" / bucket[1] to "bone_data" / drops bucket[2+] silently, type_ranges invalidated after RemoveIf but rebuilt by the serializer), and pointers to the runnable demos

• docs: docs/SDK_API.md new "Frame Post-Processor" section between "Writing Replays" and "Diffing Frames" -- API cheat-sheet covering processor implementation, registration on the writer, chain composition, the strongly-typed codegen alternative, and the mutation view API surface. Links to POST_PROCESSING.md for the full reference

• docs: docs/SAMPLES.md updated for the two new sample targets (vtx_sample_post_process_write and the post-processor addition in vtx_sample_advance_write) plus the extended arena_generated.h codegen output (now includes *Mutator classes + ForEachX helpers). "What each sample teaches" table gains four new rows covering the post-processor and codegen-driven typed accessor patterns

• docs: README.md "Write a replay" snippet gains a sub-section showing a minimal IFramePostProcessor implementation and writer->SetPostProcessor registration. In-tree docs list updated to include POST_PROCESSING.md

• scripts: scripts/check_clang_format.py (+ .sh and .bat wrappers) -- local mirror of the CI clang-format diff-gate. Validates only the lines you've modified vs a base ref (default origin/main), matching the CI's exclusion list (thirdparty/, *generated/, arena_generated.h, portable-file-dialogs.h) and scope (.cpp / .cc / .h / .hpp). Three modes via --fix / --base <ref>: read-only check, apply fixes in place, or check against a different ref. Auto-detects clang-format-diff.py under Program Files\LLVM\share\clang\ on Windows when it's not on PATH. Cross-platform wrappers delegate to the Python implementation. Exit codes match CI semantics: 0 clean, 1 violations, 2 tooling missing

• scripts: scripts/git-hooks/pre-push -- versioned pre-push hook that runs check_clang_format.py and aborts the push on violation. Opt-in per clone via git config core.hooksPath scripts/git-hooks (built-in to git ≥ 2.9, no Husky / pre-commit dependency). Bypass for a one-off push with git push --no-verify

• docs: docs/BUILD.md "Formatting gate" subsection extended with the local helper script usage (read-only check, --fix, --base arg) and the pre-push hook activation one-liner. Same coverage in CONTRIBUTING.md under "Validate formatting before pushing" + "Pre-push hook" so contributors landing on either doc find the workflow

Changed

• sdk/include layout: vtx_frame_accessor.h moved from sdk/include/vtx/reader/core/ to sdk/include/vtx/common/. The header is fundamentally a schema utility (FrameAccessor resolves names against PropertyAddressCache, EntityView is a generic read-only wrapper over PropertyContainer); pre-move it lived under reader/ for historical reasons, which forced the new writer-side post-processor headers (vtx_frame_mutation_view.h, vtx_frame_post_processor.h) to either re-implement the schema-name resolution or include from reader/ (creating a writer→reader cross-dependency). Post-move writer/core/ and reader/core/ share common/vtx_frame_accessor.h directly and have zero include-path edges between each other. 11 include sites updated to the new path (benchmarks, tests, samples, the codegen script, vtx_reader.h, the two new writer-side headers, and writer.h); the codegen template emits the new path so regenerating arena_generated.h produces a correct include

[0.1.1] - 2026-04-28

Added

• scripts: scripts/release_sdk.sh -- Linux/macOS counterpart to scripts/release_sdk.bat. Builds the SDK libs + vtx_cli in Release mode and installs into ./dist. Removes the build/release script asymmetry between Windows and Linux
• reader/api: ReaderContext::IsReady(), IsReadyFailed(), GetReadyError(), WaitUntilReady() + WaitUntilReady(std::chrono::milliseconds) for explicit "first chunk in RAM" signalling, plus new ReplayReaderEvents::OnReady / OnReadyFailed callbacks. Previously ReaderContext::Loaded() flipped to true the instant OpenReplayFile() returned -- header and footer parsed, property-address cache built, seek table ready, but zero chunks decompressed in RAM. The first GetFrameSync() call still paid the full ZSTD + deserialise cost synchronously, and the Inspector already carried a redundant is_file_loaded_ flag alongside Loaded() to paper over the gap (tools/inspector/include/inspector_session.h:25). Now OpenReplayFile() eagerly kicks off an async load of chunk 0 as part of opening (via the existing WarmAt(0) / UpdateCacheWindow pipeline; empty 0-frame replays flip the flag vacuously through a new MarkReadyVacuous() facade hook so waiters never hang). Callers consume the signal in whichever style they prefer: poll (while (!ctx.IsReady()) ...), block (ctx.WaitUntilReady(2s)), or register a callback (OnReady / OnReadyFailed fire exactly once each, single-shot guarded under ready_mutex_ so racing async + sync load paths cannot double-fire). Failure semantics: a corrupt or unreadable chunk 0 does NOT fail OpenReplayFile() itself -- the reader is still constructed, IsReadyFailed() returns true, GetReadyError() carries the message, and downstream GetFrame*() calls behave as before (return nullptr / empty). The header-parsed-ok-but-chunk-zero-broken state stays useful to inspector-style tools that want to show partial file info. Destructor best-effort unblocks any waiter by flipping ready_failed_ + notifying the condition variable under ready_mutex_; callers remain responsible for joining their waiter threads before destroying the ReaderContext (C++ standard requires no blocked waiters at condition-variable destruction time)
• tests: six new cases in tests/reader/test_reader_context.cpp under "§READY: chunk-0 ready signalling". ReaderContextHappy.ReadyFlipsWithinTimeoutOnValidReplay asserts WaitUntilReady(5s) returns true on a well-formed replay; ReadyIsStableAcrossRepeatedQueries pins the terminal-state stability guarantee; WaitUntilReadyIsIdempotent asserts repeated calls after ready return immediately; ReaderContextReady.OnReadyFiresOnDirectFacadeWithPreWiredEvents uses CreateFlatBuffersFacade() directly, wires events before WarmAt(0), and polls an atomic counter to verify single-shot firing; ReadyIsVacuousForZeroFrameReplay exercises the MarkReadyVacuous path with a GTEST_SKIP fallback if the writer refuses a 0-frame replay; ReadyFailsOnCorruptChunkZero writes a valid file then overwrites its middle third with 0xFF bytes and verifies WaitUntilReady returns false + IsReadyFailed() + non-empty GetReadyError(). No destruction-race test: destroying std::condition_variable / std::mutex while waiters are blocked is UB per the standard, so the API contract is "join waiters before destroying" and the dtor's notify_all is best-effort only

Changed

• repo layout: all five build/clean/release wrappers moved from the repo root into scripts/ (build_sdk.bat, build_sdk.sh, clean.bat, clean.sh, release_sdk.bat). Each script now cds to the repo root internally so invocations like ./scripts/build_sdk.sh or scripts\build_sdk.bat work from any working directory. Documentation references (README, CONTRIBUTING, docs/BUILD.md) updated accordingly
• repo layout: reports/benchmarks/ renamed to docs/benchmarks/ to signal that the committed baseline outputs are reference documentation (co-located with docs/PERFORMANCE.md which narrates them) rather than stray CI artefacts. reports/ directory removed. References in docs/PERFORMANCE.md, docs/BUILD.md, and the benchmark write-ups updated
• reader/api: OpenReplayFile() now triggers an eager prefetch of chunk 0 via the existing async pipeline before returning. Open latency on the calling thread is unchanged because the load runs on the same background thread WarmAt / UpdateCacheWindow already dispatches to; the prior "first GetFrame*() is slow" cost is moved off the first access onto the open-time spawn path (same total work, just overlapped with caller init). Only chunk 0 is warmed -- the facade temporarily narrows the cache window to (0, 0) around the warm call and restores it to the default (2, 2) immediately after, so callers that set a narrow window right after OpenReplayFile() (memory-constrained tools, tests that isolate a single chunk) observe exactly the cache contents they asked for. ReaderContext::Loaded() semantics are unchanged: still means "reader object exists". New concept is IsReady() == "chunk 0 decompressed and deserialised in RAM"

[0.1.0] - 2026-04-24

Added

• reader: IVtxReaderFacade::WarmAt(int32_t frame_index) (§3.A) -- explicit prefetch hint. If the enclosing chunk is cached or in flight, this is a no-op; otherwise it kicks off an asynchronous load and returns immediately. Intended use is to call WarmAt(target_frame) at the end of a seek gesture so the ZSTD decompress overlaps with any UI teardown, eliminating the "first frame after seek is slow" stutter. Implemented by routing through UpdateCacheWindow, which means WarmAt also updates the §1.B EWMA -- from the reader's perspective it is indistinguishable from a "virtual" access

• tests: ReaderApiFlatBuffers.CancelledPrefetchReEntersWindow -- focused regression for the UpdateCacheWindow cancel + re-enter bug (see Fixed). Runs the cancel + re-enter pattern (prime chunks 0..2, jump to chunk 10 to cancel, jump to chunk 2 to revive) 50 times against a fresh reader each iteration. Pre-fix this fails ~every run under TSan and flakes at single-digit-% on stock release; post-fix it is deterministic green on both

• tests: two new regression tests in tests/reader/test_reader_api.cpp.

  • ReaderApiFlatBuffers.RandomAccessSkipsLateralPrefetches -- writes a 20-chunk replay, opens with SetCacheWindow(2, 2), performs 10 far-apart jumps (distance 10 chunks each, well above window=2). Asserts that the total distinct chunks loaded is <= 2 * jump_count; pre-§1.B this would be ~5x. The bound is conservative enough to tolerate the first two EWMA bootstrap samples still triggering laterals, tight enough to catch a regression
  • ReaderApiFlatBuffers.WarmAtTriggersAsyncLoadWithoutReading -- opens a 5-chunk replay with SetCacheWindow(0, 0), calls WarmAt(30), polls ReaderChunkState::GetSnapshot() with a 5s deadline, asserts chunk 3 is present. Pins the WarmAt contract: load happens asynchronously, no GetFrame required

• tests: 47 new tests across 8 new files, driven by a targeted SDK audit. Test suite total: 89 -> 187 passing + 1 intentionally skipped (awaiting a fixture schema with a Map field).

  • tests/common/test_flat_array_edges.cpp (12 tests) -- repeated OOB PushBack, empty-span ReplaceSubArray at non-zero indices, insert-at-end-equals-PushBack, erase-last-remaining-subarray, zero-length EraseRange, CreateEmptySubArray interactions, SSO-boundary operations on FlatArray<std::string>, FlatBoolArray = FlatArray<uint8_t> pin
  • tests/reader/test_corrupt_files.cpp (8 tests) -- empty file, file smaller than magic bytes, valid magic but truncated header, truncated before footer, corrupt footer_size, chunk offset beyond EOF, negative / out-of-range frame indices (A1 / A3 regression tests)
  • tests/writer/test_writer_edges.cpp (6 tests) -- zero-frame replay, single-frame replay, chunk_max_frames = 1, RecordFrame after Stop, double-Stop idempotency (A5 regression), frame larger than chunk_max_bytes
  • tests/differ/test_diff_edges.cpp (6 tests) -- empty buckets, differing bucket contents, byte arrays, nested any_struct_properties, map properties (skipped pending schema fixture), entity replaced under same unique_id
  • tests/common/test_schema_registry_errors.cpp (5 tests) -- empty / malformed / missing-required / duplicate-struct / unknown-typeId JSON inputs must not crash
  • tests/common/test_vtx_game_times_state.cpp (6 tests) -- rollback without prior snapshot, zero-frame resolve, setup-after-data documents contract, clear-then-reuse, snapshot+rollback roundtrip, InsertLiveChunkTimes monotonicity
  • tests/common/test_content_hash_edges.cpp (5 tests) -- NaN determinism, distinct NaN bit patterns, empty-vs-default equivalence, signed zero distinguishing, move stability
  • tests/reader/test_open_replay_edges.cpp (4 tests) -- directory path, missing file size_in_mb zeroing, relative paths resolving against cwd, non-ASCII filenames

• tests/sanitizer_suppressions/: four suppression files (asan.supp, lsan.supp, ubsan.supp, tsan.supp) with header comments documenting the respective rule formats. Committed empty; populated on demand when a specific finding is judged to be third-party noise

• benchmarks: BM_FrameAccessor_Creation (isolated reader->CreateAccessor() cost, ~6.77 µs/iter) and BM_EntityView_SingleGet (isolated EntityView view(entity); view.Get(key) cost, 1.70 ns/iter = 585 M reads/s ceiling). Fills the "what does each accessor layer cost on its own?" framing gap -- previously every accessor number was bundled with entity iteration overhead, so customers asking "how fast is your property read?" only had the mixed number. Paired with the existing BM_AccessorKeyResolution (74 ns/key), the three new benchmarks decompose the public property-access API (FrameAccessor -> PropertyKey<T> -> EntityView) into three independently measurable layers

• reports/benchmarks/: raw output of the 2026-04-23 16:20 canonical run committed (bench_20260423_162008.{txt,json}) for baseline tracking and future regression comparison. First run to separately measure the three accessor layers (FrameAccessor creation, PropertyKey resolution, EntityView::Get). Narrative interpretation lives in docs/PERFORMANCE.md rather than in a per-run markdown snapshot -- evergreen doc, one source of truth

• samples: vtx_sample_generate target -- simulates a 5v5 arena match (3600 frames @ 60 FPS) and exports three data-source files (arena_replay_data.{json,proto.bin,fbs.bin}) representing raw game telemetry

• samples: vtx_sample_advance_write target -- demonstrates the full data-source pipeline with three IFrameDataSource implementations (JSON / Protobuf / FlatBuffers) driving the writer through SDK-native mapping primitives

• samples: arena_mappings.h (JSON data model + VTX::JsonMapping<T> specialisations), arena_generated.h (autogenerated schema-field constants + typed views), schemas/arena_data.proto + schemas/arena_data.fbs (arena game-side schemas in arena_pb:: / arena_fb::)

• samples: CMake codegen rules for protoc + flatc --gen-object-api, wired into samples/CMakeLists.txt so schema edits rebuild automatically

• samples/basic_diff.cpp: --fail-on-empty flag used by the sample smoke test registered in tests/CMakeLists.txt. Guards against regressions where the differ silently returns empty patches

• docs: new docs/PERFORMANCE.md landing page -- visual health-check table, headline numbers, three-layer accessor breakdown, cache-window finding with its 59 %-slower trap, format-choice caveat, honest "what these numbers do not prove" section. Linked from README.md and docs/BUILD.md

• docs: README.md now has a "Performance at a glance" section up top with a headline table and the traffic-light health check. Non-engineering readers landing on the GitHub page get a truthful one-screen summary before the build instructions

• docs/BUILD.md: new "Running benchmarks locally" section mirroring the sanitizer section -- covers configure, full-suite run, filtered run, fixture requirements

• docs/BUILD.md: "Running sanitizers locally" section covering the ASan+UBsan and TSan invocations, the vm.mmap_rnd_bits workaround, and the fix-vs-suppress decision for findings

• docs: new docs/SAMPLES.md -- per-sample walkthrough, folder layout, mapping-strategy comparison, codegen explanation

• docs/SDK_API.md: new "Integration Primitives" section documenting IFrameDataSource, JsonMapping<T>, ProtoBinding<T>, FlatBufferBinding<T>

• docs: README gets a CI status badge; docs/BUILD.md gets a "Continuous Integration" section describing the matrix and failure-artefact workflow

• build: VTX_SANITIZE CMake option enables gcc/clang runtime sanitizers (OFF, address, undefined, address,undefined, thread). Gated to NOT MSVC. When enabled, appends -fsanitize=<mode> + -fno-omit-frame-pointer + -g to compile and link options. Documented in docs/BUILD.md under "Running sanitizers locally"

• build: cross-platform support for Linux and macOS. Core SDK (vtx_common, vtx_writer, vtx_reader, vtx_differ), CLI tool (vtx_cli), all five sample programs, and the full test suite now build and run on Linux. macOS builds the SDK + CLI; GUI tools (inspector, schema_creator) remain Windows-only until their INI-based settings persistence is ported to XDG

• build: cmake/VtxDependencies.cmake -- central dependency resolution module exposing VTX::deps::protobuf, VTX::deps::flatbuffers, VTX::deps::zstd imported targets plus VTX_PROTOC_EXE / VTX_FLATC_EXE cache variables and the vtx_copy_runtime_deps() helper. On Windows, VTX_DEPENDENCY_SOURCE (AUTO / PACKAGE_MANAGER / BUNDLED) picks between the vcpkg manifest and the legacy thirdparty/protobuf/ bundle; on Linux/macOS, Protobuf comes from the system package manager (with a CONFIG-then-MODULE fallback -- Ubuntu 22.04's libprotobuf-dev doesn't ship ProtobufConfig.cmake). FlatBuffers + zstd are unconditionally fetched from pinned source (FetchContent: v24.12.23 and v1.5.6) so the wire format version and compression library are identical on every platform

• build: VTX_BUILD_SHARED option (default OFF) -- when enabled the four SDK libraries build as shared libraries (.dll / .so / .dylib) instead of static. The generated protobuf/flatbuffers sources live in an OBJECT library (vtx_generated_code) that each Windows DLL embeds, because CMAKE_WINDOWS_EXPORT_ALL_SYMBOLS cannot re-export protobuf's _default_instance_ data globals -- letting each SDK DLL compile its own copy makes each a self-contained linking unit. Includes RUNTIME DESTINATION bin in the install export and a static-vs-shared comparison in docs/BUILD.md

• build: vcpkg.json manifest for Windows package-manager builds. Currently lists only protobuf; FlatBuffers and zstd never need vcpkg because FetchContent covers them

• build: build_sdk.sh -- Linux/macOS counterpart to build_sdk.bat. Honours BUILD_TYPE, CLEAN, SKIP_TESTS, JOBS, INSTALL_PREFIX env overrides. Runs the full pipeline: configure -> build -> ctest -> install

• build: clean.sh -- Linux/macOS counterpart to clean.bat

• docs/BUILD.md: Linux / macOS dependency lists per distro (Ubuntu/Debian/Fedora/macOS), environment-variable-driven script usage, VTX_DEPENDENCY_SOURCE documentation, and platform-specific troubleshooting

• cmake: root CMakeLists.txt wires the new tests/ directory through an add_subdirectory(tests) block guarded by VTX_BUILD_TESTS (default ON). sdk/src/vtx_common/CMakeLists.txt exposes its generated-code directory as the cache variable VTX_COMMON_GENERATED_DIR so the test target can include the same protobuf / flatbuffers headers vtx_common compiles

• ci: dedicated clang-format job in .github/workflows/build.yml that runs clang-format --dry-run --Werror against every C++ file added or modified by a PR / push (vs base branch on PR, vs HEAD~1 on push). Pre-existing files are not checked -- about 90% of the codebase predates .clang-format, so a full-repo strict check would be permanently red. This "diff gate" catches new formatting regressions without requiring a blame-destroying style sweep. Uses clang-format-15 pinned for reproducibility. docs/BUILD.md documents the one-liner for running a full-tree sweep locally when the team is ready

• ci: .github/workflows/build.yml expanded from a single Windows build (no tests) into a six-job matrix: Windows and Linux, each in Release-static / Release-shared / Debug-static. Every push and every pull request runs the full ctest suite plus the sample smoke test on all six configurations. Concurrency group cancels superseded runs; failed jobs upload build/Testing/ and sample test_output/ artefacts for inspection. GUI tools (vtx_inspector, vtx_schema_creator) are disabled in CI for speed -- the SDK, vtx_cli, and samples are fully covered

• ci: two new Linux matrix jobs in .github/workflows/build.yml -- Linux / ASan+UBsan / Debug and Linux / TSan / Debug. Both run the full ctest suite with sanitizer-specific runtime options (ASAN_OPTIONS, UBSAN_OPTIONS, TSAN_OPTIONS, LSAN_OPTIONS). Test timeout 120 -> 180 seconds; matrix timeout-minutes 30 -> 45. Sanitizer jobs run sudo sysctl -w vm.mmap_rnd_bits=28 before the test step to work around TSan's shadow-memory-layout limit on recent Linux kernels (upstream: google/sanitizers#1716)

• ci: sanitizer jobs run sudo sysctl -w vm.mmap_rnd_bits=28 before the test step to work around TSan's shadow-memory-layout limit on recent Linux kernels (vm.mmap_rnd_bits=32 by default on Ubuntu 24.04+ / WSL2 on Win11 -- TSan aborts at startup with FATAL: ThreadSanitizer: unexpected memory mapping otherwise). Upstream tracker: google/sanitizers#1716

• legal: NOTICE file (Apache 2.0 §4(d) compliance) and THIRD_PARTY_LICENSES.md enumerating every third-party component with its version, upstream URL, license name, and the full text of each license (MIT, BSD-2-Clause, BSD-3-Clause, Apache-2.0, zlib/libpng). Each bundled dependency under thirdparty/ now ships a local LICENSE file as well (thirdparty/jsonlohmann/LICENSE.MIT, thirdparty/xxhash/LICENSE, thirdparty/protobuf/LICENSE). Fixes the missing attribution for nlohmann/json, xxHash, Protocol Buffers, FlatBuffers, zstd, GoogleTest, Dear ImGui, and GLFW

• docs: README.md License section expanded with pointers to the new NOTICE and THIRD_PARTY_LICENSES.md

Changed

• reader/perf: ReplayReader::UpdateCacheWindow now cancels stale prefetches (§1.A) and skips lateral prefetches under random-access patterns (§1.B). Each in-flight chunk load gets its own std::stop_source in the new PendingLoad struct, replacing the single global stop_source_; when the window shifts, prefetches whose target chunk fell outside the new range have request_stop() called on them so AsyncLoadTask / ProcessChunkData short-circuit at their next stop-token checkpoint instead of running to completion and evicting immediately afterwards. A second layer (§1.B) tracks an EWMA (α=0.3) of chunk-index distance between consecutive accesses; when the moving average exceeds cache_backward_ + cache_forward_ the trigger loop only loads current_idx and skips the [start..end] lateral sweep, because random seeks repopulate the full window on every call and the laterals are pure waste. Measured on the cache-sweep benchmark (10k-frame FlatBuffers replay, 50 random accesses per iter, single-iter per config, seed=42):

  cache_window	CPU baseline	CPU after	Δ CPU	items/s baseline	items/s after
  0	5.2M us	5.4M us	+4%	9.5	9.3
  2	24.6M us	14.5M us	-41%	2.0	3.4
  5	22.8M us	16.3M us	-29%	2.2	3.1
  10	5.5M us	5.2M us	-5%	9.3	9.6
  20	5.5M us	5.5M us	0%	9.2	9.1

  The pathological regime (small symmetric windows under random access, where every jump triggered a prefetch storm that evicted itself before use) sees the largest gain: CPU is cut by 29-41% and random-access throughput improves 40-70%. Wall time moves less (-7% to -20%) because ZSTD decompress dominates the sync path and the cancellation only short-circuits post-decompress work; §1.B is what moves the CPU needle. Cache windows of 0 and >=10 are essentially unchanged, as expected -- no storm to cancel in those regimes

• build: find_package(VTX) is now self-contained. VTXConfig.cmake calls find_dependency(Protobuf) so consumers automatically pull a matching protobuf runtime, and the zstd static library is installed alongside VTX (exported as VTX::libzstd_static) and hooked into vtx_common via $<INSTALL_INTERFACE:...>. A downstream project can link VTX::vtx_reader with nothing more than find_package(VTX REQUIRED) + a reachable Protobuf (vcpkg / apt / brew)

• build: thirdparty/zstd/ and thirdparty/flatbuffers/ deleted from the repo -- both libraries now come exclusively from FetchContent (pinned v1.5.6 and v24.12.23 respectively). thirdparty/protobuf/ stays as the Windows CI / local-dev fast path; Protobuf from vcpkg / system package managers continues to work on any platform via the existing VTX_DEPENDENCY_SOURCE=PACKAGE_MANAGER / AUTO code path

• build: root CMakeLists.txt is now platform-aware. MSVC-specific settings (CMAKE_MSVC_RUNTIME_LIBRARY, PROTOBUF_STATIC_LIBRARY) are gated by if(MSVC) + VTX_NEEDS_PROTOBUF_STATIC_DEFINE. Non-MSVC builds enable CMAKE_POSITION_INDEPENDENT_CODE and set $ORIGIN/../lib (Linux) / @loader_path/../lib (macOS) RPATH so installed binaries find bundled shared libraries

• build: per-target hardcoded thirdparty/protobuf/include / thirdparty/flatbuffers/include entries removed -- headers now reach every consumer transitively through vtx_common's PUBLIC interface and VTX::deps::* imported targets

• build: generated protobuf / FlatBuffers C++ code now lives under ${CMAKE_CURRENT_BINARY_DIR}/generated (build tree) instead of sdk/src/vtx_common/src/generated (source tree). The old location caused cross-platform crashes when the source tree got shared between machines with different flatc versions -- sharing between Windows (flatc 25.x) and Linux (flatc 2.x from apt) produced headers that hardcoded FLATBUFFERS_VERSION_MAJOR == 25 and static-assert-failed against libflatbuffers 2.x. Consumers (vtx_reader, vtx_writer, vtx_differ, samples, tests, tools/cli, tools/shared) reference the VTX_COMMON_GENERATED_DIR INTERNAL cache variable instead of hardcoding the old path

• build: FlatBuffers is now consumed header-only via FetchContent (pinned v24.12.23) on every platform. Previously Windows linked a static flatbuffers.lib from thirdparty/flatbuffers/, while Linux tried find_package(flatbuffers) against libflatbuffers-dev (v2.x on Ubuntu 22.04) -- a version mismatch that produced "Non-compatible flatbuffers version included" static-assert failures. flatc is built from source as part of the CMake graph ($<TARGET_FILE:flatc>). Removed thirdparty/flatbuffers/bin/ and thirdparty/flatbuffers/lib/ (dead code). Ubuntu no longer needs flatbuffers-compiler or libflatbuffers-dev apt packages

• build: zstd is consumed via FetchContent (pinned v1.5.6, facebook/zstd), built as a static library and linked into the SDK modules. No runtime DLL / .so dependency on any platform; no libzstd-dev / Homebrew zstd requirement. thirdparty/zstd/ stays in the repo for now as an unused fallback that a later cleanup commit will delete

• build: target_link_libraries(vtx_common ... $<BUILD_INTERFACE:VTX::deps::...>) keeps bundled thirdparty imports out of the install export, so find_package(VTX) consumers are expected to bring their own protobuf (documented in docs/BUILD.md)

• build: per-target zstd DLL copy commands replaced with the central vtx_copy_runtime_deps(target) helper. With zstd now statically linked from FetchContent, the helper is effectively a no-op stub -- kept at all call sites so reintroducing a runtime dep is a one-line change in VtxDependencies.cmake

• tools: BUILD_VTX_INSPECTOR and BUILD_VTX_SCHEMA_CREATOR default to OFF on Linux/macOS (Windows-only glue around INI settings persistence and dialog integration). BUILD_VTX_CLI stays ON everywhere

• tools: tools/CMakeLists.txt only fetches ImGui + GLFW (and only adds tools/shared) when at least one GUI tool is enabled. Headless Linux builds no longer pull in X11 as a build requirement -- docs/BUILD.md lists the X11 packages needed if you opt the GUI tools back in

• tools/shared: platform link libraries -- opengl32 on Windows, GL/dl/pthread on Linux, -framework OpenGL on macOS. NOMINMAX / WIN32_LEAN_AND_MEAN now gated to Windows

• tools/shared/src/gui/gui_app.cpp: previously-unguarded #define GLFW_EXPOSE_NATIVE_WIN32 + #include <GLFW/glfw3native.h> wrapped in #if defined(_WIN32) so the file compiles on Linux

• ci: .github/workflows/build.yml Linux jobs (Release static, Release shared, Debug static) are now active -- they were scaffolded but commented out in the previous push because the VTX sources didn't yet build on Linux. Workflow apt install now covers just cmake g++ ninja-build protobuf-compiler libprotobuf-dev (FlatBuffers + zstd via FetchContent)

• README.md: requirements row lists Windows + Linux + macOS; adds a "Using vcpkg on Windows" quick-start block; thirdparty/ described as "Header-only deps + legacy Windows binary fallback"

• samples/basic_read.cpp, samples/basic_diff.cpp: default replay path updated to content/reader/arena/arena_from_fbs_ds.vtx

• samples/basic_diff.cpp: rewritten to exercise VtxDiff::IVtxDifferFacade::DiffRawFrames instead of manually hash-comparing entities (the target named "diff" previously never touched the differ module it links against)

• samples/generate_replay.cpp: removed the +30s UTC workaround now that the underlying SDK bug is fixed; replaced with a fixed historical timestamp (2025-04-19) for reproducible output

• content/ layout: arena schema + three data sources live under content/writer/arena/; generated .vtx replays land under content/reader/arena/

• sdk/include/vtx/differ/core/vtx_default_tree_diff.h: standardised xxhash include to <xxh3.h> -- now consistent with vtx_types_helpers.h

• docs/ARCHITECTURE.md, docs/BUILD.md: IFrameDataSource documented in the writer section; build outputs list all five sample executables; cross-reference to SAMPLES.md

Fixed

• reader: ReplayReader::UpdateCacheWindow no longer leaves a chunk permanently stuck when it is cancelled by a window shift and then immediately re-requested before the worker thread has started running. The §1.A cancellation path (PR #4) flags out-of-window prefetches via request_stop() on their per-PendingLoad stop_source but leaves the map entry in place. If the chunk re-entered the window before its worker was scheduled, the trigger() lambda saw the stale entry, assumed a load was already in flight, and skipped spawning a new task. The original worker then ran, observed stop_requested() == true at its entry check in PerformHeavyLoading, returned an empty CachedChunk, and AsyncLoadTask skipped the cache write (correctly, because the stop was still requested). The future resolved cleanly, but the cache stayed empty. A synchronous caller waiting on that future in GetFramePtrSync then read the empty cache and returned nullptr -- manifesting as a spurious load failure under random-seek patterns. Fix: trigger() now detects a pending entry whose stop is already requested and replaces it with a fresh PendingLoad; the orphaned worker exits on its own and its stop_requested()-gated cache write still cannot pollute anything. Exposed by the TSan CI job on ReaderApiFlatBuffers.RandomAccessSkipsLateralPrefetches; the scheduling overhead of the ThreadSanitizer runtime makes the "cancelled before scheduled" race far more likely to manifest. Stock release builds had been papering over it by the workers happening to get past the entry check before cancellation landed

5 new correctness bugs surfaced by a targeted SDK audit:

• A1 -- vtx_reader.h ReadFooter(): footer_size was read from the stream but used without checking stream.gcount() or stream.fail(). On a truncated file the uninitialised value drove a seek to garbage and crashed the reader (SEH access violation in tests). Now validates stream reads, checks file size, and rejects implausibly large footer sizes

• A1 extended -- vtx_reader.h ReadHeader(): same hardening applied (bounds check on declared header size against remaining file)

• A2 -- vtx_reader.h PerformHeavyLoading(): stream.read() can produce partial reads; the previous code only checked raw_buffer.size() <= 4 which doesn't reflect actual bytes read. Now validates via stream.gcount() and logs the specific shortfall

• A3 -- vtx_reader.h PerformHeavyLoading(): corrupt seek tables with file_offset + chunk_size_bytes > file_size previously seeked past EOF and crashed the deserialiser. Now validates the chunk extent against the actual file size before seeking

• A4 -- vtx_reader.h SetEvents() + callback call-sites: events_ was read (via events_.OnChunkLoadFinished etc.) without synchronisation while SetEvents() could overwrite it from another thread. Reading a std::function while another thread writes is UB. Introduced events_mutex_ + a GetEventsSnapshot() helper used at every callback site; the actual callback invocations happen on the local snapshot outside the lock

• A5 -- vtx_writer_facade.cpp WriterFacadeImpl: RecordFrame, Flush, and subsequent Stop calls after the initial Stop() could overwrite the already-finalised file and truncate previously-recorded frames. WriterFacadeImpl now tracks a stopped_ flag and silently no-ops all three methods after Stop(). Stop() itself is idempotent

• vtx_common (vtx_types.h): VTXGameTimes::AddTimeRegistry no longer rejects valid historical UTC timestamps -- the regression check now correctly requires a prior frame to exist before flagging a repeat

• vtx_common (vtx_types.h): VTXGameTimes() constructor no longer seeds start_utc_ with GetUtcNowTicks(); field starts at 0 and is populated when real data arrives. Clear() updated for consistency

• vtx_common (vtx_types.h): OnlyIncreasing / OnlyDecreasing game-time filters no longer reject the very first frame of a replay whose game_time is 0

• vtx_common (vtx_types.h): three warning messages in AddTimeRegistry used printf specifiers (%lld, %f) inside std::format calls -- the values were never printed. Converted to {} placeholders

• vtx_reader (vtx_reader_facade.h): swapped member declaration order in ReaderContext so reader is destroyed before chunk_state. The reader's async chunk-load callbacks capture a raw pointer into chunk_state; the previous order created a potential use-after-free during context teardown

• vtx_reader (vtx_reader.h): PerformHeavyLoading no longer silently swallows deserialization exceptions -- logs the exception message before returning an empty chunk

• sdk/src/schemas/vtx_schema.proto: removed six stray // Ale author comments from the public Protobuf schema

Notes

• An earlier revision of the Windows CI dependency work also moved Windows CI onto vcpkg manifest mode and deleted the bundled thirdparty/protobuf/. That change was reverted: vcpkg on GitHub's Windows runners builds Protobuf + abseil from source on every cache miss (~15-30 min per job), which made CI timeouts the common case. vcpkg.json is still committed for contributors who prefer the package-manager flow locally, but CI sticks with the bundled-binary fast path

[0.0.1] - 2026-04-16

Added

• vtx_common: Core type system with SoA-based PropertyContainer, Frame, Bucket, and Transform types
• vtx_common: Dual serialization backend support (Protocol Buffers and FlatBuffers)
• vtx_common: Schema registry with JSON-based schema definitions and PropertyAddressCache for O(1) property lookup
• vtx_common: zstd compression for chunks and headers
• vtx_common: xxHash-based content hashing for fast frame comparison
• vtx_common: Thread-safe logger with configurable sinks (VTX_INFO, VTX_WARN, VTX_ERROR, VTX_DEBUG)
• vtx_writer: IVtxWriterFacade for recording frame data into chunked .vtx files
• vtx_writer: ChunkedFileSink with configurable chunk size, compression, and seek table generation
• vtx_writer: Protobuf and FlatBuffers serialization policies
• vtx_reader: IVtxReaderFacade with random-access and streaming frame access
• vtx_reader: Async chunk-based caching with configurable cache window
• vtx_reader: FrameAccessor for type-safe, O(1) property access via PropertyKey<T>
• vtx_reader: Seek table for O(1) chunk lookup by frame index
• vtx_differ: DefaultTreeDiff<TNodeView> for structural comparison of replay trees
• vtx_differ: PatchIndex and DiffIndexOp for tracking add/remove/modify operations
• vtx_differ: Configurable float epsilon for approximate comparisons
• vtx_differ: Protobuf and FlatBuffers view adapters
• Tools: VTX Inspector -- ImGui-based GUI for browsing replay files
• Tools: VTX CLI -- Headless JSON inspector for scripting and AI agents
• Tools: Schema Creator -- Interactive schema definition tool
• Build: CMake build system with modular options and CMake Presets support
• Build: build_sdk.bat one-click build script for Windows