Skip to content

Improve handling of nested arrays and arrays of nullable values#4171

Open
hazefully wants to merge 6 commits into
FoundationDB:mainfrom
hazefully:array-nullability-improvments
Open

Improve handling of nested arrays and arrays of nullable values#4171
hazefully wants to merge 6 commits into
FoundationDB:mainfrom
hazefully:array-nullability-improvments

Conversation

@hazefully
Copy link
Copy Markdown
Contributor

This PR improves handling of the user including nested arrays as a selected element, which is not possible due to the limitation of record-layer discussed in #4167, by rejecting the query at parsing time due to the lack of support for nested arrays. If the nested arrays are instead used to evaluate some other values (for example in a comparison), it is allowed as in https://github.com/hazefully/fdb-record-layer/blob/e19cb847765c7801d0d2e0d47ed0e0e374874d57/yaml-tests/src/test/resources/arrays-operators.yamsql#L259-L262.

The PR also improves handling of constructing arrays using underlying nullable values, by adding a promotion to the non-nullable variant of the same type, as arrays with non-nullable elements are also not supported. This used to cause issues in continuations, when attempting to parse the ResultSetMetadata that is serialised in the continuation. In addition, the array construction is migrated to make use of the __internal_array built-in function since it now supports promoting record types properly while computing the max type of its elements.

Finally, the PR contains an improvement to fdb-relational-grpc to maintain the struct type names in the ResultSetMetadata to support asserting on these names in yaml-tests. The struct names were already maintained with embedded connections, so this makes the gRPC connection maintain the struct type names as well.

@hazefully hazefully force-pushed the array-nullability-improvments branch from e19cb84 to 2e56331 Compare May 15, 2026 15:13
@hazefully hazefully force-pushed the array-nullability-improvments branch from 2e56331 to 4010ce2 Compare May 15, 2026 15:32
@hazefully hazefully added the bug fix Change that fixes a bug label May 15, 2026
@github-actions
Copy link
Copy Markdown

📊 Metrics Diff Analysis Report

Summary

  • New queries: 1
  • Dropped queries: 0
  • Plan changed + metrics changed: 0
  • Plan unchanged + metrics changed: 0
ℹ️ About this analysis

This automated analysis compares query planner metrics between the base branch and this PR. It categorizes changes into:

  • New queries: Queries added in this PR
  • Dropped queries: Queries removed in this PR. These should be reviewed to ensure we are not losing coverage.
  • Plan changed + metrics changed: The query plan has changed along with planner metrics.
  • Metrics only changed: Same plan but different metrics

The last category in particular may indicate planner regressions that should be investigated.

New Queries

Count of new queries by file:

  • yaml-tests/src/test/resources/arrays.metrics.yaml: 1

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug fix Change that fixes a bug

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant