Implement the PartiQL “AT” construct for unnesting with ordinality#4112
Merged
Conversation
RobertBrunel
force-pushed
the
partiql-at
branch
5 times, most recently
from
03a0ac3 to
d1f39b3
Compare
hatyo
requested changes
May 5, 2026
RobertBrunel
force-pushed
the
partiql-at
branch
2 times, most recently
from
425709c to
a0ab96e
Compare
normen662
previously requested changes
May 7, 2026
This change adds support for the PartiQL `AT` clause in array unnesting queries. For example, `SELECT … FROM T1, T1.array1 AS element AT pos` produces a `pos` field carrying the 1-based ordinal position of each array element. The ordinal has type `INTEGER` (not `BIGINT`), aligned with the array subscript operator.
Without `A`T, the `EXPLODE` plan operation produces just the array elements as before. When `AT` is present, it instead produces a struct {element, ordinal} (with anonymous field names) as a `DynamicMessage`. Downstream `FLATMAP` and `FILTER` nodes bind these fields through normal `FieldValue` accessors (no special-case unwrapping is needed).
Summary of changes:
1. Parser: Add the optional `AT <atAlias>` syntax to the `atomTableItem` rule. Only `tableSourceItem` can have an `AT`; other `FROM` item types reject it with SQLSTATE 42809.
2. AST transformation: In `QueryVisitor#visitAtomTableItem()`, extract the `AT` alias and pass it to `LogicalOperator#generateAccess()`.
3. Logical plan: In `LogicalOperator#generateCorrelatedFieldAccess()`, construct an `ExplodeExpression` with the `WITH ORDINALITY` flag and build appropriate output attributes for the element/ordinal pair.
4. Relational expressions: Extend `ExplodeExpression` as appropriate to incorporate the new `withOrdinality` field and the element/ordinal result type.
5. Physical plan nodes: Extend `RecordQueryExplodePlan` similarly with a `withOrdinality` boolean, including the proto representation. `executePlan()` builds `DynamicMessage` structs with 1-based ordinals when ordinality is requested.
6. Planner rules: `ImplementExplodeRule` passes `withOrdinality` through to the plan.
7. EXPLAIN output: In `ExplainPlanVisitor`, add the `WITH ORDINALITY` keywords to the EXPLODE node to indicate that it produces the element/ordinal pair.
Testing:
* Integration tests in `array-join-at.yamsql`.
* Unit tests for `EXPLODE` changes in `ExplodePlanTest`.
Resolves FoundationDB#4117.
📊 Metrics Diff Analysis ReportSummary
ℹ️ About this analysisThis automated analysis compares query planner metrics between the base branch and this PR. It categorizes changes into:
The last category in particular may indicate planner regressions that should be investigated. New QueriesCount of new queries by file:
|
hatyo
approved these changes
May 15, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This change adds support for the PartiQL
ATclause in array unnesting queries. For example,SELECT … FROM T1, T1.array1 AS element AT posproduces aposfield carrying the 1-based ordinal position of each array element. The ordinal has typeINTEGER(notBIGINT), aligned with the array subscript operator.Without
AT, theEXPLODEplan operation produces just the array elements as before. WhenATis present, it instead produces a struct {element, ordinal} (with anonymous field names) as aDynamicMessage. DownstreamFLATMAPandFILTERnodes bind these fields through normalFieldValueaccessors (no special-case unwrapping is needed).Summary of changes:
AT <atAlias>syntax to theatomTableItemrule. OnlytableSourceItemcan have anAT; otherFROMitem types reject it with SQLSTATE 42809.QueryVisitor#visitAtomTableItem(), extract theATalias and pass it toLogicalOperator#generateAccess().LogicalOperator#generateCorrelatedFieldAccess(), construct anExplodeExpressionwith theWITH ORDINALITYflag and build appropriate output attributes for the element/ordinal pair.ExplodeExpressionas appropriate to incorporate the newwithOrdinalityfield and the element/ordinal result type.RecordQueryExplodePlansimilarly with awithOrdinalityboolean, including the proto representation.executePlan()buildsDynamicMessagestructs with 1-based ordinals when ordinality is requested.ImplementExplodeRulepasseswithOrdinalitythrough to the plan.ExplainPlanVisitor, add theWITH ORDINALITYkeywords to the EXPLODE node to indicate that it produces the element/ordinal pair.Testing:
array-join-at.yamsql.EXPLODEchanges inExplodePlanTest.Resolves #4117.