feat: add db.setCompression() for live per-CF compression changes

[kriszyp]

Summary

RocksDB's DB::Open() opens every existing column family in one call, so a caller can never re-specify compression for a column family that's already been transitively opened without closing every other handle sharing that process-global DBDescriptor. This blocks Harper's boot sequence, which necessarily opens its default CF (transitively opening every other CF) before it can read its own catalog CF to learn each table's desired codec — see the linked issue for the full repro.

Adds db.setCompression(algorithm, level?), backed by rocksdb::DB::SetOptions(), which RocksDB's own headers document as dynamically mutable for exactly the fields needed (ColumnFamilyOptions::compression, blob_compression_type). Verified against the linked RocksDB build with a standalone C++ probe (not just assumed) — see the PR's self-review for the full writeup, including why this was chosen over a per-CF options map on the root open call (the alternative doesn't fit Harper's actual use case: it doesn't know a table's codec until after reading its own catalog CF, which is itself opened by that same call).

An independent cross-model review (codex + gemini + grok + a Harper-domain adjudicator) on the initial implementation surfaced real native-layer gaps — no in-flight operation registration (a shutdown-race UAF hazard matching AGENTS.md's documented teardown discipline), no read-only guard, no handling for a split live/durable state on an OPTIONS-persist failure, and an unbounded per-call cost given SetOptions() is RocksDB's own documented "slow, infrequent-use" call. All are fixed (second commit) — see the self-review for details.

Test plan

• pnpm check (type-check, lint, format) clean
• pnpm test — 720 passed, 2 pre-existing skips (unrelated)
• Native binding (pnpm build:binding) and TS bundle (pnpm build:bundle) both rebuild clean
• New tests: live getter effect, level round-trip/reset, on-disk-size proof of live algorithm change (SST + blob threshold), cold-reopen durability, already-open-CF conflict-check interplay, read-only rejection, validation
• README.md Compression section updated with the new API's semantics and verified caveats
• Independent cross-model review: round 1 (pre-hardening) completed with real findings, all addressed in the second commit; round 2 (post-hardening) could not complete after 5 attempts due to the review CLI's background process being killed externally each time (machine under heavy concurrent multi-agent load) — see the PR's self-review for the full account. The hardening commit mirrors exact patterns (ACQUIRE_OPERATIONS_LOCK, THROW_IF_READONLY) already used by sibling native ops in this file, which is a partial mitigation but not a substitute for independent review of that specific diff.

🤖 Generated with Claude Code

https://claude.ai/code/session_01WV3oB7zEeJvKFugn8DQ93e

_{Review-Coverage: authored=unknown; ran=none; rounds=1 @ 4d4194e}

_{Human-Review-Need: 4 @ 4d4194e}

[gemini-code-assist]

Code Review

This pull request introduces the setCompression method to dynamically change the compression algorithm and optional level for an already-open database's column family without requiring a close and reopen. This feature is implemented across the TypeScript API, the native C++ binding using RocksDB's DB::SetOptions(), and is thoroughly documented in the README and verified with a comprehensive suite of unit tests. I have no feedback to provide as the implementation is robust, well-documented, and includes excellent test coverage.

[github-actions]

📊 Benchmark Results

get-sync.bench.ts

getSync() > random keys - small key size (100 records)

Implementation	Rank	Operations/sec	Mean (ms)	Min (ms)	Max (ms)	RME (%)	Samples
🥇 lmdb	1	23.32K ops/sec	42.89	41.50	496.985	0.112	116,584
🥈 rocksdb	2	10.86K ops/sec	92.06	87.18	31,419.513	1.24	54,312

getSync() > sequential keys - small key size (100 records)

Implementation	Rank	Operations/sec	Mean (ms)	Min (ms)	Max (ms)	RME (%)	Samples
🥇 lmdb	1	28.54K ops/sec	35.04	33.82	570.501	0.107	142,689
🥈 rocksdb	2	11.44K ops/sec	87.40	85.40	581.469	0.052	57,206

ranges.bench.ts

getRange() > small range (100 records, 50 range)

Implementation	Rank	Operations/sec	Mean (ms)	Min (ms)	Max (ms)	RME (%)	Samples
🥇 lmdb	1	24.81K ops/sec	40.31	36.50	1,977.29	0.304	124,031
🥈 rocksdb	2	16.54K ops/sec	60.46	52.37	1,105.232	0.122	82,706

realistic-load.bench.ts

Realistic write load with workers > write variable records with transaction log

Implementation	Rank	Operations/sec	Mean (ms)	Min (ms)	Max (ms)	RME (%)	Samples
🥇 rocksdb	1	374.82 ops/sec	2,667.975	117.459	28,069.363	9.22	750
🥈 lmdb	2	26.40 ops/sec	37,876.857	422.169	1,191,651.657	136.548	64.00

transaction-log.bench.ts

Transaction log > read 100 iterators while write log with 100 byte records

Implementation	Rank	Operations/sec	Mean (ms)	Min (ms)	Max (ms)	RME (%)	Samples
🥇 rocksdb	1	38.65K ops/sec	25.87	11.72	451.168	0.242	193,258
🥈 lmdb	2	443.41 ops/sec	2,255.269	103.262	8,116.097	1.14	2,218

Transaction log > read one entry from random position from log with 1000 100 byte records

Implementation	Rank	Operations/sec	Mean (ms)	Min (ms)	Max (ms)	RME (%)	Samples
🥇 rocksdb	1	728.60K ops/sec	1.37	1.20	5,231.168	0.219	3,642,978
🥈 lmdb	2	447.78K ops/sec	2.23	1.10	5,732.14	0.815	2,238,909

worker-put-sync.bench.ts

putSync() > random keys - small key size (100 records, 10 workers)

Implementation	Rank	Operations/sec	Mean (ms)	Min (ms)	Max (ms)	RME (%)	Samples
🥇 rocksdb	1	821.19 ops/sec	1,217.75	1,045.153	3,124.784	0.384	1,643
🥈 lmdb	2	1.16 ops/sec	860,945.334	801,249.067	954,056.92	4.21	10.00

worker-transaction-log.bench.ts

Transaction log with workers > write log with 100 byte records

Implementation	Rank	Operations/sec	Mean (ms)	Min (ms)	Max (ms)	RME (%)	Samples
🥇 rocksdb	1	22.73K ops/sec	44.00	30.52	525.706	0.575	45,460
🥈 lmdb	2	809.73 ops/sec	1,234.976	197.946	13,277.574	5.34	1,620

---

Results from commit 27e2b6e

[kriszyp]

Parking this for future consideration rather than merging it now — moving to draft.

Opened #743 as the narrower alternative: compressionForAllColumnFamilies on the open call, which applies the requested codec to every column family instead of only the target. That covers the case actually blocking Harper today (a database upgraded from 5.1, where every family is persisted at none and the per-table reconcile open cannot move them), without live mutation of an already-open family.

The distinction is which model we want:

• One codec per deployment — the codec is known from config before any open, so it can just be supplied at open time. Add compressionForAllColumnFamilies so a database can adopt one codec #743 is enough, and there is no SetOptions() cost or new mutable native state.
• A codec per table — you cannot know a table's codec until you have read the catalog, which is itself opened by that same DB::Open, so the only way to act on it afterwards is to change an open family's codec. That is this PR, and nothing else substitutes for it.

Harper's per-table dimension today is only enabled-vs-disabled, derived from one global storage.compression value at table-creation time, so the deployment-global model fits what exists. If we later want genuine per-table codecs (a @table(compression:) directive, say), this PR is the thing that unblocks it.

Nothing here is wrong — the native hardening in the second commit (in-flight operation registration, read-only guard, split live/durable state on an OPTIONS-persist failure) is good work and worth keeping on the branch. Parking rather than closing so it's ready if the per-table case comes up.

Generated by Claude Opus 5.