Skip to content

[CALCITE-7595] Support FILTER clause with window functions#5019

Merged
xuzifu666 merged 1 commit into
apache:mainfrom
xuzifu666:7595
Jun 18, 2026
Merged

[CALCITE-7595] Support FILTER clause with window functions#5019
xuzifu666 merged 1 commit into
apache:mainfrom
xuzifu666:7595

Conversation

@xuzifu666

Copy link
Copy Markdown
Member

@mihaibudiu mihaibudiu left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Where is the documentation about the new grammar and its semantics?

Comment thread core/src/test/resources/sql/winagg.iq
Comment thread site/_docs/reference.md

!ok

# [CALCITE-6442] Support FILTER clause with window functions

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please add here the following:

  • next 4 tests are related to this issue
  • results were validated on Postgres

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK, done.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The execution results differ from PostgreSQL.
@xuzifu666
https://onecompiler.com/postgresql/44smkpfxb

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This data source should be different. I used the standard Scott data source for the test set, and it passed the syntax verification. I'll investigate the Calcite data source later. @iwanttobepowerful

NotJsonImplementor.of(
new MethodImplementor(BuiltInMethod.IS_JSON_SCALAR.method,
NullPolicy.NONE, false)));
define(FILTER, new FilterImplementor());

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't understand where this is used. Can you please explain?

@xuzifu666 xuzifu666 Jun 17, 2026

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The FILTER operator implements conditional aggregation in SQL, such as:
SUM(salary) FILTER (WHERE department = 'Sales') COUNT(*) FILTER (WHERE age > 30)
Processing Pipeline:

  1. SQL Parsing → FILTER (WHERE condition) parsed as SqlKind.FILTER node
  2. Validation → Checks aggregate-FILTER compatibility (we reject COUNT(DISTINCT) + FILTER in window functions)
  3. SQL-to-Relational Conversion → FILTER condition extracted and stored
  4. Code Generation → Registers FilterImplementor to translate FILTER into executable code
    I had added a comment about it.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How come this does not affect other aggregates that have filters and only applies to window aggregates?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In my view restriction is in SqlOverOperator.validateCall(), which is only called for window functions (the OVER operator).

if (hasFilter && aggCall.getKind() == SqlKind.COUNT
      && aggCall.getFunctionQuantifier() != null) {
    throw validator.newValidationError(aggCall, RESOURCE.overNonAggregate());
  }
  1. Normal aggregates bypass SqlOverOperator entirely—they're validated by their own operator (e.g., SqlCountAggFunction)
  2. Window aggregates MUST go through SqlOverOperator because they have the OVER clause
  3. Our restriction is injected into the SqlOverOperator validation, so it naturally only affects window functions

@mihaibudiu mihaibudiu left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have approved, but I still have a question

@mihaibudiu

Copy link
Copy Markdown
Contributor

Interesting, can you add a comment in the RexImplTable about this?

@xuzifu666

Copy link
Copy Markdown
Member Author

Interesting, can you add a comment in the RexImplTable about this?

OK, I had added comments about it.

@xuzifu666

Copy link
Copy Markdown
Member Author

Do you happy with these changes? If OK, I would squash the commits~ @mihaibudiu

@mihaibudiu

Copy link
Copy Markdown
Contributor

I have approved

@sonarqubecloud

Copy link
Copy Markdown

@xuzifu666 xuzifu666 added the LGTM-will-merge-soon Overall PR looks OK. Only minor things left. label Jun 17, 2026
@xuzifu666 xuzifu666 merged commit 6e08166 into apache:main Jun 18, 2026
52 of 53 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

LGTM-will-merge-soon Overall PR looks OK. Only minor things left.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants