chore(release): eql-3.0.1#409
Merged
Merged
Conversation
… for consistency
Make the encrypted-JSON domains follow the same naming convention as every
scalar family: the bare family name is the storage-only domain, a suffix marks
the searchable variant.
- New: public.eql_v3_json — a ciphertext-only, storage-only encrypted-JSON
domain ({v,i,c}, no index terms; the JSON analogue of public.eql_v3_boolean),
generated by the scalar materializer like any storage-only scalar.
- Renamed (3.0.0-alpha surface): the searchable SteVec document
public.eql_v3_json -> public.eql_v3_json_search, public.eql_v3_jsonb_entry ->
public.eql_v3_json_entry, eql_v3.query_jsonb -> eql_v3.query_json.
The `json` family is now a mixed family: three hand-written SteVec domains plus
one generated scalar storage domain. eql-codegen's SQL/bindings drivers iterate
a new families_with_scalar_domains() seam and render each family's scalar
domains only; the family stays non-scalar so scalar_families() (list-types, the
matrix) still excludes it. The generated Json struct is re-exported from the
hand-written json.rs so inventory/payload resolve super::json::Json.
Regenerated SQL/bindings/TS/JSON committed; changeset + U-007 upgrade note added.
The storage-domain tests used inline {v,i,c} payloads, which technically violate the project rule that tests must use real ciphertexts, never synthetic blobs. Port PR #397's real-ciphertext approach (adapted to this branch's generated architecture and `json` module naming): - New `v3_json_storage` fixture: encrypts JSON through cipherstash-client with `.storage_only()` (no index) into a real {v,i,c} envelope, column-typed `public.eql_v3_json` so the domain CHECK runs at load. - New `v3_json_storage_tests` suite: a real client payload parses into the generated `Json` binding; `@>` on a real fixture row raises the firewall; the storage/search CHECKs are mutually exclusive on real data. - Fix `v3_convert::targets_for`: an unindexed jsonb fixture now converts to the storage-only `eql_v3_json` target (was unconditionally `eql_v3_json_search`), with a unit test pinning the bare {v,i,c} envelope. - Wire-up: allowlist `public.eql_v3_json`, register the fixture module + generation, gitignore the generated .sql. The inline jsonb_check tests are kept — they cover what fixtures can't (malformed-envelope rejection, the full blocked-operator matrix, and the path-operator fall-through characterization).
…parity Bring the storage-only public.eql_v3_json suite to parity with boolean's caps=[storage] matrix arm, over the real v3_json_storage fixture, and remove the synthetic duplication left behind by the first pass (89a1da5). v3_json_storage_tests (3 -> 9 tests), mirroring tests/sqlx/src/matrix.rs: - storage_fixture_shape: {v,i,c} with no sv/k and no term keys (hm/ob/bf/op); real wire shape parses into the generated Json binding. - storage_comparison_blockers: = <> < <= > >= @> <@ raise, both operand orders. - storage_path_op_blockers: explicitly-typed -> / ->> raise. - storage_native_jsonb_blockers: ? ?| ?& @? @@ #> #>> - #- || raise. - storage_native_absent_ops: ~~ / ~~* -> "operator does not exist". - storage_aggregate_typecheck: eql_v3.min/max must not resolve; asserts on SQLSTATE 42883 (undefined_function) or 42725 (ambiguous_function), exactly as matrix_boolean_storage_aggregate_typecheck does. - storage_path_operators_unknown_literal_fall_through: with an UNKNOWN-type literal RHS, -> / ->> / ? resolve to native jsonb (plaintext-safe) instead of the domain blocker. Documents that resolution turns on literal typing alone; the blocked, explicitly-typed forms are storage_path_op_blockers. - storage_row_count, storage_and_search_domain_checks_are_mutually_exclusive. Two behaviours pinned by probing the real surface, not assumed: - Path-op resolution on public.eql_v3_json is literal-typing dependent — an exact-match domain blocker beats the base-type jsonb operator only when the RHS type is known; an unknown literal falls through. Both are pinned. - eql_v3.min/max(eql_v3_json) errors 42725 (ambiguous_function) because the jsonb-backed value coerces equally to the many ordered-domain min/max overloads, none specific to storage. Asserting on SQLSTATE (not the message) makes this a stable proof the storage domain owns no aggregate — the same 42883-or-42725 contract the boolean matrix driver uses. jsonb_check.rs: delete the two synthetic storage tests now covered by real fixtures (blocks_native_comparison_operators, path_operators_fall_through); rename the malformed-corpus test to json_storage_check_rejects_malformed and drop its positive {v,i,c} accept rows (acceptance is proven by the fixture loading through the CHECK at INSERT and by storage_fixture_shape). Test-only; no SQL/catalog/bindings change, no changeset.
…stall smoke
Address review feedback on the clean-install v3 smoke test:
- The storage-only negative test cast a `{"v":"3","i":{},"sv":[...]}` payload,
which differs from a valid `{v,i,c}` envelope in three ways (string v, empty
i, sv-not-c). Rewrite it to integer v=3 + a populated i{t,c} keeping sv and no
root c, so the payload differs from a valid storage envelope by exactly the
c->sv swap and the storage CHECK's has-c rule is the isolated failure.
- The searchable smoke insert's identity object was inverted
(`{"c":"v3_json_smoke","t":"encrypted"}`); correct it to `{"t":"v3_json_smoke",
"c":"e"}` (t=table, c=column), matching the storage smoke test's format.
Validated end to end via tasks/test/clean_install_v3.sh (clean v3 install OK).
…mily bindings seam Address review of the JSON storage/search rename: - CLAUDE.md + render_scalar_only_bindings doc comment referenced the pre-rename jsonb.rs/jsonb_storage.rs/Jsonb/&JSONB names; corrected to json.rs/json_storage.rs/Json/&JSON. - render_bindings hardcoded eql_domains::JSON for the mixed-family storage module; replaced with the generic families_with_scalar_domains().filter(|f| !f.is_scalar()) seam so a second mixed family is picked up automatically, mirroring the SQL side. Byte-identical output today (drift check clean).
…-search-rename feat(eql_v3): add storage-only public.eql_v3_json; rename JSON family for consistency
Contributor
|
Important Review skippedToo many files! This PR contains 235 files, which is 135 over the limit of 100. To get a review, narrow the scope: Upgrade to a paid plan to raise the limit. ⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Run ID: ⛔ Files ignored due to path filters (26)
📒 Files selected for processing (235)
You can disable this status message by setting the Use the checkbox below for a quick retry:
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
The bloom fuzzy match on the text match/search domains (`public.eql_v3_text_match` / `text_search` / `text_search_ore`) was spelled with the containment operators `@>` / `<@`, backed by `eql_v3.contains` / `eql_v3.contained_by`. That promised containment semantics it does not provide: it is array containment over two bloom filters (downcased 3-gram token sets) — a probabilistic, order-insensitive, one-sided match. It is now a single directional operator `@@` backed by `eql_v3.matches`; the reverse `contained_by` direction is dropped, and `@>` / `<@` now RAISE on these domains. Genuine JSON containment (`public.eql_v3_json_search` `@>`/`<@`, `ste_vec_contains`) is untouched. Generator (the surface is generated from the catalog, not hand-written): - `Term::Bloom` advertises `@@` (was `@>`/`<@`), routing to `match_term`. - `@@` gains a per-signature `blocker_only` flag: the three symmetric domain/jsonb overloads are the supported match; the `(domain, jsonpath)` overload stays a blocker on every domain (the "no silent native jsonb ops" guarantee). - `Operator::wrapper_function_name()` maps supported `@@` -> `eql_v3.matches` while the blocker keeps `eql_v3_internal."@@"`; `Operator::body_operator()` keeps the wrapper body as `match_term(a) @> match_term(b)`, so `col @@ $1` inlines to the same GIN-indexable array-containment (no new SEM operator). Regenerated in place: `src/v3/scalars/*`, Rust/TS/JSON bindings, and the bundled SQL installers (crate + npm) so packaged installs match the advertised API. `@@` symmetric overloads render as blockers on non-Bloom domains, consistent with how `@>`/`<@` already do. Tests/docs: rewrote the text match/smoke suites, property + matrix match arms, operator-equivalents list, splinter allowlist; regenerated the public-surface, matrix-name, and cargo-expand snapshots; added the U-008 upgrade note and a `major` changeset; corrected rollback guidance, "bloom fuzzy match" wording, `text_search_ore` coverage in the reference matrices, and the BloomFilter / unsupported-blocker doc surface. Client code emitting `eql_v3.contains(` for text match must switch to `eql_v3.matches(` in lockstep. BREAKING CHANGE: `@>` / `<@` and `eql_v3.contains` / `eql_v3.contained_by` on the text match/search domains now raise. Migrate to `@@` / `eql_v3.matches`.
Replace the stringly-typed `Operator.symbol: &'static str` field with a closed `OpSymbol` enum (20 variants, one per catalog operator) whose `as_str` is the single source of truth for the SQL operator text. `wrapper_function_name()` and `body_operator()` are now exhaustive `match self.symbol` with no wildcard `_ =>` arm — a future operator that needs a divergent wrapper name or body operator must be classified here (compile error until it is), instead of silently inheriting the default. Pure behaviour-preserving refactor: generated SQL/bindings/TS/JSON are byte-identical (codegen:parity → PARITY OK, types:check synced, all eql-codegen tests green). No changeset — internal codegen refactor with no observable change to any caller.
…rator feat(eql_v3)!: rename text bloom fuzzy match to `@@` / `eql_v3.matches`
…l-enum refactor(eql-codegen): model operator symbol as OpSymbol enum
The prerelease pipeline was hardwired to the retired eql_v3 branch, so no prerelease could be cut from a release batching branch (e.g. eql-3.0.1) without merging to main. Generalize classify to grant prerelease mode to any non-main branch carrying a chore(release): marker commit, dispatched via workflow_dispatch. Safety is unchanged — it comes from the marker commit, the prerelease-shaped version guard, and the tag-idempotency short-circuit, not the branch name. Drop the dead eql_v3 entry from on: push.
The blanket ord_term→ord_term_ore rename and the _ord→Term::Ope flip swept up the domains that moved *to* OPE, inverting their descriptions. A maintainer reading these would "fix" a test in the wrong direction. Corrected against the catalog ground truth across the shipped Doxygen block, test comments, the v3.0 upgrade guide, the privilege test, and the matrix-snapshot tooling. - ope_cllw/types.sql: `op` appears on `_ord` and `_ord_ope`; contrast the removed ore_cllw (`oc`) with the live ore_block_256 (`ob`). Regenerated the bundled SQL (crates/eql-bindings/sql, packages/eql/sql) via the release prepare script + GNU tsort at version 3.0.0, refreshing the release-manifest SHA256s; the installer SQL is byte-identical apart from this comment. - fixture_oracle / scalar_domains / matrix: text_search is [Hm,Ope,Bloom]; SteVec entries order by `op` not `oc`; ord identity is per-twin (ord_term/`op` for OPE, ord_term_ore/`ob` for ORE); text fixture spec is [Unique,Ore,Match,Ope]. - v3.0.md: U-note cross-links _ord→U-005, text_search→U-006. - v3_privilege_tests: ORD_QUERY now describes the OPE path (ord_term → ope_cllw) and drops the dead `GRANT USAGE ON SCHEMA extensions` — the default `_ord` path is native bytea compare and never calls pgcrypto. - mise.toml / snapshots: drop the inert `routes_through_ordering_term` grep alternative (every such test name already matches `_ord`). - known_failure: public.double_eq → public.eql_v3_double_eq (CIP-3472 prefix).
Five test fixes/additions closing coverage gaps left by the block-ORE -> CLLW-OPE migration, where a naive rename left several controls vacuous or inverted: - R2a (v3_ore_fallback_tests): re-point the superuser positive control at `_ord_ore` (ORE, poisonable) instead of `_ord` (OPE, UNPOISONABLE by construction). The old control cast the OPE domain, which can never be poisoned, so it proved nothing; the ORE domain is the meaningful "superuser install does not poison" check. - PR#392 (edge_cases): split the stale `check_rejects_payload_missing_ob` into two tests. Post-flip `_ord` is OPE and its CHECK requires `op` (not `ob`), so the old test passed for the wrong reason. Now `_ord` rejects a payload missing `op` and `_ord_ore` rejects a payload missing `ob`, each proving the domain needs its own ordering term. - T18 (float_special): parameterise the signed-zero (-0.0 vs +0.0) pins over both `real` (F4) and `double` (F8) via a local `SignedZeroPair` trait and generic per-behaviour helpers. `real` shares the crypto path but was asserted nowhere; the `known_failure` inversion is preserved per type. - R3a (fixture_oracle): add `search_ore_fn_oracle` for `Variant::SearchOre` (`text_search_ore`, [Hm, Ore, Bloom]), the ORE twin of the existing `search_fn_oracle` (`text_search`, OPE). Mirrors it exactly but exercises the block-ORE extractor (`ord_term_ore`/`ob`). - T10 (v3_privilege_tests): add `runtime_role_ope_ok_ore_needs_extensions`, pinning the pgcrypto boundary. Under identical grants (eql_v3 + eql_v3_internal + fixture, but NOT `extensions`), the CLLW-OPE `_ord` path succeeds while the block-ORE `_ord_ore` path fails at pgcrypto `encrypt()`. NOT done: R2b (adding "1.0" to NUMERIC_FIXTURES). Empirically breaks CI: "1" and "1.0" parse to the same rust_decimal value, tripping the deliberate `numeric_value_guards::fixtures_are_distinct_by_value` guard (which names this exact pair) and producing duplicate-plaintext fixture rows that break `fetch_fixture_payload`'s `fetch_one` and the numeric oracle. `cargo test -p eql-domains` passes (a false green: the catalog keys numeric fixtures by literal string), but `cargo test --lib -p eql_tests` fails. Surfaced for a decision rather than shipping a CI-breaking change.
…heck (CIP-3489)
The CLLW-OPE migration removed the `ore_cllw` surface, but its splinter allowlist
rows survived and nothing could catch them: the awk only tested `if (key in allow)`
per finding and never iterated `allow[]`, so no allowlist row could ever be
reported as unused — a future `ore_cllw` would arrive pre-waived.
- Remove the 10 inert rows: eql_v3_internal.ore_cllw_{eq,neq,lt,lte,gt,gte} and
eql_v3.{ore_cllw,has_ore_cllw} + eql_v3_internal.{ore_cllw,has_ore_cllw}. No
such functions exist in src/v3 any more, so the rows waived nothing.
- Fix regressed prose on the two live rows that still routed "via ore_cllw":
eql_v3.lt (jsonb_entry lt now routes ord_term → ope_cllw) and eql_v3.-> (the
chained ste_vec recipe extracts via eq_term / ord_term).
- Add a `for (k in allow)` unused-row check: allowlist rows that match no
finding now fail the gate (with a CI step-summary section), enforcing the same
"registered but referenced by nothing" invariant the known-failures gate uses.
Comment/blank heredoc lines are skipped so they are never counted as unused.
Verified end-to-end against a live install (splinter@55db5b1): the gate stays
clean (raw=1180, allowlisted=1180, unmatched=0, unused=0, exit 0) after the row
removals, and an injected bogus row is correctly reported unused and fails the
gate (exit 1).
…ation-left-stale-and-inverted-comments-docs-and docs(eql_v3): fix stale/inverted CLLW-OPE migration comments (CIP-3490)
…3488) The known-failure gate could report success while verifying nothing, or fail with the wrong reason. Four independent defects, all raised in review on #389 and still live on main: - T14 (fail-open): the registry parser was fully anchored and hardcoded `u64`; any deviation (u32, indentation, a trailing comment, `1_000`) parsed zero rows, printed "nothing to verify", and exit 0. Now the strict parse is tolerant of indentation / trailing comments / `_` digit separators, and a declared-vs-parsed mismatch (counted independently with a loose matcher) is a hard error (exit 2). - T3 (transient reads as missing): switched from `gh issue view … || echo MISSING` to `gh api repos/<repo>/issues/<n>`, classifying a genuine `(HTTP 404)` as NOT_FOUND and every other gh/network error as a distinct transient ERROR, so a network blip no longer masquerades as "issue missing". - T15 (PR numbers pass as issues): the REST issues object carries a `pull_request` key iff the number is a PR; PR numbers are now rejected instead of validating as open issues. - T16 (bash 3.2): replaced `mapfile` (bash 4+) with a 3.2-safe while-read loop — mise invokes this as `bash …` and stock macOS /bin/bash is 3.2. Verified end-to-end against the live registry and crafted fixtures (u32 drift → exit 2; PR #399 → rejected; missing #99999999 → NOT_FOUND; `3_87` + indentation + comment → parsed 387, OPEN), and under /bin/bash 3.2.57.
…-stale-ore_cllw-allowlist-rows-and-no-unused ci(splinter): drop 10 stale ore_cllw allowlist rows, add unused-row check (CIP-3489)
…own-failuressh-ci-gate-multiple-fail-open-false ci(known-failures): close four fail-open / false-positive paths (CIP-3488)
Contributor
|
Closing in order to pick up some additional changes in the release. |
github-merge-queue
Bot
removed this pull request from the merge queue due to failed status checks
Jul 17, 2026
github-merge-queue
Bot
removed this pull request from the merge queue due to failed status checks
Jul 17, 2026
github-merge-queue
Bot
removed this pull request from the merge queue due to failed status checks
Jul 17, 2026
The merge queue ejects PRs when the sqlx shards fail with `53200 out of
shared memory`. Despite the wording this is not RAM exhaustion — it is
the shared lock table running out of slots, exactly as the error's own
hint ("You might need to increase max_locks_per_transaction") says.
max_locks_per_transaction is not a per-transaction cap; it sizes the
CLUSTER-WIDE lock table as max_locks_per_transaction * max_connections.
The shipped uninstaller drops both eql_v3 schemas with DROP ... CASCADE,
which locks every one of the ~5,682 objects it removes and holds them all
until commit (sqlx::raw_sql sends the script as one implicit
transaction). Measured: 5,689 locks for a single uninstall against a
stock table of 64 * 100 = 6,400 slots — 89% of the entire cluster's
budget in one transaction. Overlapping uninstalls exhaust it, and the
backends that then fail are mostly innocent tests that were merely
installing EQL, which is why this surfaces as `failed to apply
migrations` in unrelated tests.
Sharding is what tipped it over: the queue runs 2 shards where PRs run 4,
doubling test density per shard so the uninstall-heavy tests overlap.
Partitioning is a deterministic hash, so they land in the same shard on
every run — hence shard 2/2 failing on all four PG versions while the
same commit passes on the PR. This is not specific to one PR: #382, #401
and #402 ejected with the identical signature.
Raise max_locks_per_transaction to 1024 (102,400 slots), sized for the
worst case of nextest's 16 concurrent tests each uninstalling (~91k
locks). Verified on postgres:17: 6 concurrent uninstalls fail 5/6 at the
default and pass 6/6 at 1024. Costs ~42MB of shared memory (145MB ->
187MB), well inside the container's 2GB limit.
Add lock_capacity_tests.rs as a preflight gate so an undersized lock
table fails once, immediately, with a message naming the cause and the
fix — instead of six unrelated tests dying 20 minutes into CI with what
reads like an OOM. NB: a Postgres container created before this change
keeps the old setting, so local devs must recreate it
(mise run postgres:down && mise run postgres:up postgres-17).
Deliberately not "fixed" by giving the queue 4 shards: that would only
hide an undersized lock table until the catalog grew again.
freshtonic
enabled auto-merge
July 17, 2026 06:55
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Release batching branch for eql-3.0.1. Aggregates all eql-3.0.1 changes; merges to
mainat release time.Merged
#398 — JSON family rename + storage-only domain (minor)
public.eql_v3_json: ciphertext-only storage-only domain ({v,i,c}, no index terms).eql_v3_json→eql_v3_json_search(document),eql_v3_jsonb_entry→eql_v3_json_entry(entry),query_jsonb→query_json(needle).jsonis now a mixed family (hand-written SteVec + one generated scalar storage domain); codegen renders scalar domains via newfamilies_with_scalar_domains();scalar_families()/matrix still exclude it.#399 — fix stale/inverted CLLW-OPE migration comments (CIP-3490) (docs, no changeset)
ord_term→ord_term_ore/_ord→Term::Opesweep inverted the descriptions of the domains that moved to OPE; every fix here is checked againsteql-domains::CATALOGground truth (_ord/_ord_ope=Ope/op/ord_term;_ord_ore=Ore/ob/ord_term_ore).GRANT USAGE ON SCHEMA extensionsinv3_privilege_tests, and fixes apublic.double_eq→public.eql_v3_double_eqref.#401 — harden known-failures CI gate (CIP-3488) (CI, no changeset)
tasks/test/known-failures.sh(the feat(v3)!: back the_orddomains with CLLW-OPE instead of block-ORE #389 gate): tolerant registry parse with a declared-vs-parsed mismatch now a hardexit 2; transientgherrors distinguished from real 404s; PR numbers rejected as issues;mapfile→bash 3.2-safe loop for stock macOS.#402 — drop stale ore_cllw splinter allowlist rows + add unused-row check (CIP-3489) (CI, no changeset)
ore_cllwallowlist rows left by the CLLW-OPE migration (0ore_cllwrefs remain insrc/v3) and fixes regressed prose on the 2 liveope_cllwrows.for (k in allow)unused-row check so allowlist rows matching no finding now fail the gate — the same "registered but referenced by nothing" invariant the known-failures gate enforces.#406 — text bloom fuzzy-match operator rename (major)
eql_v3_text_match,eql_v3_text_search,eql_v3_text_search_ore):@>/<@(contains/contained_by) → single directional@@backed byeql_v3.matches.@>/<@now raise on these domains. JSON containment (eql_v3.ste_vec_contains) unaffected.contained_bydirection dropped; GIN recipe unchanged (GIN (eql_v3.match_term(col))).contains()→matches()rename; changeset + U-008 upgrade note.col @> $1/eql_v3.contains(col,…)→col @@ $1/eql_v3.matches(col,…).#408 —
OpSymbolenum for the codegen operator symbol (internal, no changeset)eql-codegen'sOperator.symbolis now a closedOpSymbolenum (20 variants, one per catalog operator) with aconst fn as_str()as the single source of truth for the SQL text, replacing the stringly-typed&'static str.wrapper_function_name()andbody_operator()are now exhaustivematchwith no wildcard arm — a future operator needing a divergent wrapper name or body operator is a compile error until classified, instead of silently inheriting the default.codegen:parity→PARITY OK,types:checksynced). No changeset — internal codegen refactor with no observable change to any caller.