Skip to content

Make SelectMergeRule greedily merge all child selects#3850

Open
g31pranjal wants to merge 14 commits intoFoundationDB:mainfrom
g31pranjal:select_merge
Open

Make SelectMergeRule greedily merge all child selects#3850
g31pranjal wants to merge 14 commits intoFoundationDB:mainfrom
g31pranjal:select_merge

Conversation

@g31pranjal
Copy link
Copy Markdown
Member

@g31pranjal g31pranjal commented Jan 15, 2026

This PR makes the SelectMergeRule to merge all its children select expressions. This improves on the earlier approach that used to merge only certain children that were deemed mergable. A child was considered mergable if there is no correlation pointing to it from other children. Hence, the merge was partial in some sense. The current approach relaxes that constraint. In a nutshell it does the following

  1. iterates over the children in the topological order, given by the dependency among them. That is the child appearing before could have children appearing later dependant on it.
  2. For each child, "rebase" its children with a "running" translationMap and add to the new top select. Apply the same translation on predicates and put them in top select.
  3. Translate child resultValue and put it in the "running" translationMap - to be used by future children.

Resolves: #4012

@g31pranjal g31pranjal added the enhancement New feature or request label Jan 16, 2026
@g31pranjal g31pranjal changed the title Select merge make SelectMergeRule greedily merge all child selects Jan 16, 2026
@g31pranjal g31pranjal force-pushed the select_merge branch 5 times, most recently from b262c38 to 006418a Compare February 4, 2026 12:18
@g31pranjal g31pranjal added the Run mixed-mode Label to add to Pull Requests to have it run mixed mode tests label Mar 6, 2026
@g31pranjal g31pranjal marked this pull request as ready for review March 6, 2026 16:29
Copy link
Copy Markdown
Collaborator

@alecgrieser alecgrieser left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The PR description could also probably use a bit more motivation explaining what this is for. This may be a bit harder to add, but I believe the thought was that this should fix some queries that were previously unable to plan due to index matching problems, and it would be nice if those could be added to a yaml test case

Arguments.of(ImmutableMap.of(250L, 50L, 40L, 10L), LEVEL, List.of(250L, 40L, 50L, 10L, 10L, 1L, 1L)),
Arguments.of(ImmutableMap.of(250L, 50L, 40L, 10L), PREORDER, List.of(250L, 50L, 10L, 1L, 40L, 10L, 1L)),
Arguments.of(ImmutableMap.of(250L, 50L, 40L, 10L), ANY, List.of(250L, 50L, 10L, 1L, 40L, 10L, 1L)),
Arguments.of(ImmutableMap.of(250L, 50L, 40L, 10L), ANY, List.of(250L, 40L, 50L, 10L, 10L, 1L, 1L)),
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you know what happened here? Does it have something the do with the fact that the order of quantifiers (and their children) can be modified by the SelectMergeRule if they don't start in a topologically sorted order?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't know what changed here - but it is definitely not because of the order of the quantifiers as I did try with presereved ordering and that didn't change the result. I think it is the side-effect of complete merging. However, I didn't look deeper as the results are correct for ANY ordering

@g31pranjal g31pranjal requested a review from alecgrieser March 19, 2026 18:12
@g31pranjal g31pranjal changed the title make SelectMergeRule greedily merge all child selects Make SelectMergeRule greedily merge all child selects Mar 19, 2026
@g31pranjal g31pranjal changed the title Make SelectMergeRule greedily merge all child selects Make SelectMergeRule greedily merge all child selects Mar 19, 2026
@github-actions
Copy link
Copy Markdown

📊 Metrics Diff Analysis Report

Summary

  • New queries: 15
  • Dropped queries: 0
  • Plan changed + metrics changed: 23
  • Plan unchanged + metrics changed: 32
ℹ️ About this analysis

This automated analysis compares query planner metrics between the base branch and this PR. It categorizes changes into:

  • New queries: Queries added in this PR
  • Dropped queries: Queries removed in this PR. These should be reviewed to ensure we are not losing coverage.
  • Plan changed + metrics changed: The query plan has changed along with planner metrics.
  • Metrics only changed: Same plan but different metrics

The last category in particular may indicate planner regressions that should be investigated.

New Queries

Count of new queries by file:

  • yaml-tests/src/test/resources/aggregate-index-tests.metrics.yaml: 15

Plan and Metrics Changed

These queries experienced both plan and metrics changes. This generally indicates that there was some planner change
that means the planning for this query may be substantially different. Some amount of query plan metrics change is expected,
but the reviewer should still validate that these changes are not excessive.

Total: 23 queries

Statistical Summary (Plan and Metrics Changed)

task_count:

  • Average change: +54.0
  • Average regression: +54.0
  • Median change: +54
  • Median regression: +54
  • Standard deviation: 0.0
  • Standard deviation of regressions: 0.0
  • Range: +54 to +54
  • Range of regressions: +54 to +54
  • Queries changed: 1
  • Queries regressed: 1

transform_count:

  • Average change: +10.0
  • Average regression: +10.0
  • Median change: +10
  • Median regression: +10
  • Standard deviation: 0.0
  • Standard deviation of regressions: 0.0
  • Range: +10 to +10
  • Range of regressions: +10 to +10
  • Queries changed: 1
  • Queries regressed: 1

transform_yield_count:

  • Average change: +4.0
  • Average regression: +4.0
  • Median change: +4
  • Median regression: +4
  • Standard deviation: 0.0
  • Standard deviation of regressions: 0.0
  • Range: +4 to +4
  • Range of regressions: +4 to +4
  • Queries changed: 1
  • Queries regressed: 1

insert_new_count:

  • Average change: -1.5
  • Median change: -1
  • Standard deviation: 1.2
  • Range: -6 to -1
  • Queries changed: 23
  • No regressions! 🎉

insert_reused_count:

  • Average change: -1.0
  • Median change: -1
  • Standard deviation: 0.0
  • Range: -1 to -1
  • Queries changed: 1
  • No regressions! 🎉

There were no queries with significant regressions detected.

Minor Changes (Plan and Metrics Changed)

In addition, there were 23 queries with minor changes.

Only Metrics Changed

These queries experienced only metrics changes without any plan changes. If these metrics have substantially changed,
then a planner change has been made which affects planner performance but does not correlate with any new outcomes,
which could indicate a regression.

Total: 32 queries

Statistical Summary (Only Metrics Changed)

task_count:

  • Average change: -107.0
  • Median change: -22
  • Standard deviation: 85.0
  • Range: -192 to -22
  • Queries changed: 4
  • No regressions! 🎉

transform_yield_count:

  • Average change: -9.0
  • Median change: -2
  • Standard deviation: 7.0
  • Range: -16 to -2
  • Queries changed: 4
  • No regressions! 🎉

insert_new_count:

  • Average change: -12.8
  • Median change: -2
  • Standard deviation: 38.3
  • Range: -160 to -1
  • Queries changed: 32
  • No regressions! 🎉

insert_reused_count:

  • Average change: +9.0
  • Average regression: +9.0
  • Median change: +16
  • Median regression: +16
  • Standard deviation: 7.0
  • Standard deviation of regressions: 7.0
  • Range: +2 to +16
  • Range of regressions: +2 to +16
  • Queries changed: 4
  • Queries regressed: 4

There were no queries with significant regressions detected.

Minor Changes (Only Metrics Changed)

In addition, there were 32 queries with minor changes.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request Run mixed-mode Label to add to Pull Requests to have it run mixed mode tests

Projects

None yet

Development

Successfully merging this pull request may close these issues.

SelectMergeRule do not merge all the child select expression

2 participants