[FLINK-37924][table]Introduce Built-in Function to Access field or element in the Variant by JinkunLiu · Pull Request #27330 · apache/flink

JinkunLiu · 2025-12-09T16:17:34Z

What is the purpose of the change

Introduce a built-in function to access a field or element in a Variant. This is part of FLIP-521.

SQL Function	Table Function	Description
variant ‘[’ INT ‘]’	VARIANT.at(INT)	If the VARIANT is an ARRAY value, returns a VARIANT whose value is the element at the specified index. Otherwise, this operation returns NULL
variant ‘[’ STRING ‘]’	VARIANT.at(STRING)	If the VARIANT is a MAP value that has an element with this key, a VARIANT holding the associated value is returned. Otherwise, NULL is returned.

Brief change log

Support accessing VARIANT elements using [INT] or VARIANT.at(INT) when the VARIANT contains an ARRAY value in SQL or table expressions.
Support accessing VARIANT elements using ['STRING'] or VARIANT.at('STRING') when the VARIANT contains a MAP value in SQL or table expressions.

Verifying this change

This change added tests and can be verified as follows:

Added six new test at org.apache.flink.table.planner.plan.nodes.exec.stream.VariantSemanticTest

Does this pull request potentially affect one of the following parts:

Dependencies (does it add or upgrade a dependency): (no)
The public API, i.e., is any changed class annotated with @Public(Evolving): (no)
The serializers: (no)
The runtime per-record code paths (performance sensitive): (no)
Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: (no)
The S3 file system connector: (no)

…ement in the Variant

flinkbot · 2025-12-09T16:30:44Z

CI report:

26d8f28 UNKNOWN
80d4638 Azure: FAILURE

Bot commands

The @flinkbot bot supports the following commands:

@flinkbot run azure re-run the last Azure build

.../java/org/apache/flink/table/types/inference/strategies/ItemAtIndexArgumentTypeStrategy.java

...-table-common/src/main/java/org/apache/flink/table/functions/BuiltInFunctionDefinitions.java

...src/test/java/org/apache/flink/table/planner/plan/nodes/exec/stream/VariantSemanticTest.java

…ues with incorrect types

…r Variant access

JinkunLiu · 2025-12-23T04:33:50Z

Hello @Sxnan Could you take a look at this PR when you have time? Thanks

yunfengzhou-hub

Thanks for the PR @JinkunLiu. Left some comments as below.

...mmon/src/main/java/org/apache/flink/table/types/inference/strategies/ItemAtTypeStrategy.java

...planner/src/main/scala/org/apache/flink/table/planner/codegen/calls/ScalarOperatorGens.scala

...-table-planner/src/main/scala/org/apache/flink/table/planner/codegen/ExprCodeGenerator.scala

flink-table/flink-sql-parser/src/main/java/org/apache/calcite/sql/fun/SqlItemOperator.java

JinkunLiu · 2026-01-20T12:38:04Z

Hi @yunfengzhou-hub, thanks for the review! I have updated the code according to your suggestions. Could you please take another look at this PR when you have time? Thank you very much!

yunfengzhou-hub

Thanks for the update. Generally LGTM. Some minor comments are as below.

docs/data/sql_functions.yml

.../java/org/apache/flink/table/types/inference/strategies/ItemAtIndexArgumentTypeStrategy.java

...flink-table-planner/src/main/scala/org/apache/flink/table/planner/codegen/CodeGenUtils.scala

JinkunLiu · 2026-01-23T14:20:00Z

Hi @davidradl , thanks for the review! Could you take an another look. Thanks

xuyangzhong

Thanks for driving this. This pr LGTM overall. I just left some minor comments.

.../java/org/apache/flink/table/types/inference/strategies/ItemAtIndexArgumentTypeStrategy.java

...planner/src/main/scala/org/apache/flink/table/planner/codegen/calls/ScalarOperatorGens.scala

docs/data/sql_functions.yml

flink-table/flink-sql-parser/src/main/java/org/apache/calcite/sql/fun/SqlItemOperator.java

docs/data/sql_functions.yml

xuyangzhong

Thanks for updating, just one more minor comment.

...planner/src/main/scala/org/apache/flink/table/planner/codegen/calls/ScalarOperatorGens.scala

… index types

xuyangzhong

+1 👍

yunfengzhou-hub

+1

[FLINK-37924][table]Introduce Built-in Function to Access field or el…

53f4993

…ement in the Variant

JinkunLiu force-pushed the feature/FLINK-37924 branch from efb13bf to 53f4993 Compare December 9, 2025 16:27

JinkunLiu changed the title ~~[FLINK-37924][sql]Introduce Built-in Function to Access field or element in the Variant~~ [FLINK-37924][table]Introduce Built-in Function to Access field or element in the Variant Dec 9, 2025

JinkunLiu marked this pull request as ready for review December 9, 2025 16:28

davidradl reviewed Dec 9, 2025

View reviewed changes

.../java/org/apache/flink/table/types/inference/strategies/ItemAtIndexArgumentTypeStrategy.java Outdated Show resolved Hide resolved

davidradl reviewed Dec 9, 2025

View reviewed changes

.../java/org/apache/flink/table/types/inference/strategies/ItemAtIndexArgumentTypeStrategy.java Outdated Show resolved Hide resolved

davidradl reviewed Dec 9, 2025

View reviewed changes

...-table-common/src/main/java/org/apache/flink/table/functions/BuiltInFunctionDefinitions.java Show resolved Hide resolved

davidradl reviewed Dec 9, 2025

View reviewed changes

...src/test/java/org/apache/flink/table/planner/plan/nodes/exec/stream/VariantSemanticTest.java Show resolved Hide resolved

github-actions bot added the community-reviewed PR has been reviewed by the community. label Dec 10, 2025

JinkunLiu added 2 commits December 10, 2025 09:26

[FLINK-37924][table] Improve error messages for accessing Variant val…

5ef9f93

…ues with incorrect types

[FLINK-37924][table] Update expected argument types and signatures fo…

e036ebc

…r Variant access

JinkunLiu force-pushed the feature/FLINK-37924 branch from c27b41a to e036ebc Compare December 10, 2025 13:11

[FLINK-37924][table] Add error tests

2d5a8ef

JinkunLiu marked this pull request as draft December 11, 2025 16:22

JinkunLiu added 2 commits December 18, 2025 00:03

[FLINK-37924][table] Support CALCITE-7325 change

06d5c7c

[FLINK-37924][table] Add Doc for Variant Built-in Function

43de8ce

JinkunLiu marked this pull request as ready for review December 19, 2025 17:43

JinkunLiu requested a review from davidradl December 20, 2025 04:02

yunfengzhou-hub reviewed Jan 8, 2026

View reviewed changes

JinkunLiu added 2 commits January 18, 2026 18:41

[FLINK-37924][table] Opt generateVariantGet code generate logic

1ff8ce8

[FLINK-37924][table] Fix ItemAtTypeStrategy Function comment

e7fe43b

JinkunLiu requested a review from yunfengzhou-hub January 20, 2026 02:53

yunfengzhou-hub reviewed Jan 23, 2026

View reviewed changes

xuyangzhong reviewed Jan 26, 2026

View reviewed changes

[FLINK-37924][table] Improve comments and formatting code

26d8f28

JinkunLiu force-pushed the feature/FLINK-37924 branch from d08b908 to 26d8f28 Compare January 26, 2026 15:47

JinkunLiu added 2 commits January 26, 2026 23:48

[FLINK-37924][table] Update Document

2985b11

[FLINK-37924][table] Update key type checks for array type variant

fc3a69f

JinkunLiu requested review from xuyangzhong and yunfengzhou-hub January 26, 2026 16:02

davidradl reviewed Jan 27, 2026

View reviewed changes

docs/data/sql_functions.yml Show resolved Hide resolved

davidradl reviewed Jan 27, 2026

View reviewed changes

docs/data/sql_functions.yml Show resolved Hide resolved

JinkunLiu requested a review from davidradl January 27, 2026 17:30

xuyangzhong reviewed Jan 30, 2026

View reviewed changes

...planner/src/main/scala/org/apache/flink/table/planner/codegen/calls/ScalarOperatorGens.scala Show resolved Hide resolved

[FLINK-37924][table] Add test for variant array access with different…

80d4638

… index types

xuyangzhong approved these changes Jan 30, 2026

View reviewed changes

yunfengzhou-hub approved these changes Jan 30, 2026

View reviewed changes

yunfengzhou-hub merged commit 80e26ca into apache:master Feb 2, 2026

Conversation

JinkunLiu commented Dec 9, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What is the purpose of the change

Brief change log

Verifying this change

Does this pull request potentially affect one of the following parts:

Uh oh!

flinkbot commented Dec 9, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

CI report:

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

JinkunLiu commented Dec 23, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

yunfengzhou-hub left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

JinkunLiu commented Jan 20, 2026

Uh oh!

yunfengzhou-hub left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

JinkunLiu commented Jan 23, 2026

Uh oh!

xuyangzhong left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

xuyangzhong left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

xuyangzhong left a comment

Choose a reason for hiding this comment

Uh oh!

yunfengzhou-hub left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

JinkunLiu commented Dec 9, 2025 •

edited

Loading

flinkbot commented Dec 9, 2025 •

edited

Loading

JinkunLiu commented Dec 23, 2025 •

edited

Loading