Skip to content

[test](regression) Enable segment limit pushdown fuzzy#64479

Merged
BiteTheDDDDt merged 1 commit into
apache:masterfrom
BiteTheDDDDt:codex/enable-segment-limit-fuzzy
Jun 15, 2026
Merged

[test](regression) Enable segment limit pushdown fuzzy#64479
BiteTheDDDDt merged 1 commit into
apache:masterfrom
BiteTheDDDDt:codex/enable-segment-limit-fuzzy

Conversation

@BiteTheDDDDt

Copy link
Copy Markdown
Contributor

What problem does this PR solve?

Issue Number: None

Related PR: #64285

Problem Summary:

enable_segment_limit_pushdown is the replacement switch for the old common expr pushdown session variables that were removed as no-op FE variables. The variable was already marked fuzzy, but fuzzy initialization still skipped randomizing it because many regression cases explicitly pinned the old variables.

This PR migrates regression cases that set enable_common_expr_pushdown or enable_common_expr_pushdown_for_inverted_index to use enable_segment_limit_pushdown instead, including both true and false settings. Cases that had conflicting old settings in one SET_VAR are normalized to a single enable_segment_limit_pushdown setting. After the case cleanup, fuzzy mode now randomizes enableSegmentLimitPushdown.

Release note

None

Check List (For Author)

  • Test: Manual test
    • git diff --check HEAD^ HEAD
    • git diff --cached --check
    • rg verification that regression suites no longer reference the old common expr pushdown variables outside data files
    • rg verification that no regression hint has conflicting or duplicate enable_segment_limit_pushdown settings
  • Behavior changed: No
  • Does this need documentation: No

### What problem does this PR solve?

Issue Number: None

Related PR: None

Problem Summary: enable_segment_limit_pushdown was marked as fuzzy but omitted from fuzzy initialization because regression cases still pinned removed common expr pushdown variables. Those removed variables no longer control BE pushdown, so fuzzy mode could disable segment limit pushdown for cases that need it. Replace true common expr pushdown settings in regression cases with enable_segment_limit_pushdown=true and enable the fuzzy randomization.

### Release note

None

### Check List (For Author)

- Test: Manual test
    - git diff --check
    - git diff --cached --check
    - rg verification for removed common expr pushdown true settings outside data files
    - rg verification for duplicate enable_segment_limit_pushdown=true hints
- Behavior changed: No
- Does this need documentation: No
@hello-stephen

Copy link
Copy Markdown
Contributor

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@BiteTheDDDDt BiteTheDDDDt marked this pull request as ready for review June 13, 2026 06:43
@BiteTheDDDDt

Copy link
Copy Markdown
Contributor Author

run buildall

@BiteTheDDDDt

Copy link
Copy Markdown
Contributor Author

/review

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review opinion: no blocking issues found.

Checkpoint conclusions:

  • Goal and proof: The PR enables fuzzy randomization for enableSegmentLimitPushdown and migrates regression pins from removed/no-op enable_common_expr_pushdown* variables to enable_segment_limit_pushdown. The changed suites now set the live session variable, and the patch removes old variable references from regression suites outside expected compatibility/deprecated definitions.
  • Scope and focus: The functional change is small: one fuzzy-mode assignment plus mechanical regression hint updates. I noticed whitespace-only cleanup in some touched regression files, but it does not affect behavior.
  • Concurrency: No new concurrent execution paths, shared mutable state, or locking changes are introduced.
  • Lifecycle/static initialization: No new lifecycle ownership, static initialization dependency, or release-order concern is introduced.
  • Configuration/session variables: No new variable is added. The existing enable_segment_limit_pushdown variable is already declared fuzzy=true and needForward=true; the change now actually randomizes it in fuzzy mode and still forwards it through TQueryOptions to BE.
  • Compatibility: Deprecated thrift fields for the removed common-expression variables and the FE removed-variable no-op handling remain in place, so old client SET statements continue to be ignored instead of rejected.
  • Parallel code paths: The relevant FE-to-BE forwarding path and BE runtime-state consumption path were checked; the variable gates SegmentIterator LIMIT/TopN pushdown, while common-expression pushdown itself is not gated by the removed variables.
  • Conditional checks: No new special runtime condition or defensive branch is added.
  • Test coverage: This PR changes regression tests rather than adding a new engine feature. The migration covers the existing tests that explicitly pinned the removed variables. I did not run the full regression suite locally in this review.
  • Test result files: No .out files are changed.
  • Observability: No new runtime path requiring additional logs or metrics is introduced.
  • Transactions/persistence/data writes: Not applicable; this is session-variable initialization and regression test hint cleanup only.
  • FE/BE variable passing: SessionVariable.toThrift() already forwards enableSegmentLimitPushdown; BE reads it from RuntimeState::enable_segment_limit_pushdown() with default-true compatibility.
  • Performance: Randomizing the variable intentionally exercises both LIMIT-pushdown states in fuzzy mode. No hot-path implementation change is introduced.
  • User focus: No additional user-provided review focus was supplied.

Reviewed existing inline review context first; no existing inline comments or replies were present.

@hello-stephen

Copy link
Copy Markdown
Contributor
TPC-H: Total hot run time: 29262 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 92679383e47f23b498fa239533bc0c59b8196a27, data reload: false

------ Round 1 ----------------------------------
orders	Doris	NULL	NULL	0	0	0	NULL	0	NULL	NULL	2023-12-26 18:27:23	2023-12-26 18:42:55	NULL	utf-8	NULL	NULL	
============================================
q1	17653	4208	4281	4208
q2	q3	10777	1370	796	796
q4	4684	471	342	342
q5	7552	879	577	577
q6	184	187	142	142
q7	771	811	639	639
q8	9501	1569	1615	1569
q9	5802	4460	4480	4460
q10	6702	1794	1544	1544
q11	425	273	249	249
q12	622	426	282	282
q13	18417	3300	2715	2715
q14	270	261	249	249
q15	q16	814	770	709	709
q17	996	908	952	908
q18	7131	5643	5581	5581
q19	1313	1298	1024	1024
q20	543	412	262	262
q21	6133	2876	2694	2694
q22	468	369	312	312
Total cold run time: 100758 ms
Total hot run time: 29262 ms

----- Round 2, with runtime_filter_mode=off -----
orders	Doris	NULL	NULL	150000000	42	6422171781	NULL	22778155	NULL	NULL	2023-12-26 18:27:23	2023-12-26 18:42:55	NULL	utf-8	NULL	NULL	
============================================
q1	5223	4875	5014	4875
q2	q3	4834	5232	4674	4674
q4	2104	2210	1406	1406
q5	4763	4862	4644	4644
q6	235	181	132	132
q7	1963	1769	1536	1536
q8	2377	2084	2077	2077
q9	7925	7518	7447	7447
q10	4753	4675	4226	4226
q11	539	383	347	347
q12	726	739	539	539
q13	2952	3526	2836	2836
q14	269	288	242	242
q15	q16	681	691	608	608
q17	1296	1248	1274	1248
q18	7294	6937	6699	6699
q19	1094	1081	1088	1081
q20	2224	2225	1941	1941
q21	5263	4564	4414	4414
q22	520	462	416	416
Total cold run time: 57035 ms
Total hot run time: 51388 ms

@hello-stephen

Copy link
Copy Markdown
Contributor
TPC-DS: Total hot run time: 168743 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 92679383e47f23b498fa239533bc0c59b8196a27, data reload: false

query5	4317	622	472	472
query6	475	191	172	172
query7	4811	556	298	298
query8	359	214	203	203
query9	8773	4090	4054	4054
query10	440	301	253	253
query11	5919	2406	2168	2168
query12	151	99	95	95
query13	1343	595	425	425
query14	6364	5390	5044	5044
query14_1	4339	4368	4370	4368
query15	203	196	171	171
query16	1014	424	418	418
query17	1130	682	543	543
query18	2460	464	332	332
query19	191	188	138	138
query20	112	108	107	107
query21	206	134	115	115
query22	13676	13591	13378	13378
query23	17335	16398	16081	16081
query23_1	16191	16208	16235	16208
query24	7555	1821	1300	1300
query24_1	1317	1313	1309	1309
query25	551	469	407	407
query26	1314	324	172	172
query27	2713	580	338	338
query28	4435	2032	2012	2012
query29	1091	633	493	493
query30	310	232	204	204
query31	1110	1082	964	964
query32	115	64	59	59
query33	548	315	246	246
query34	1198	1133	661	661
query35	746	771	668	668
query36	1341	1399	1257	1257
query37	151	109	89	89
query38	3235	3144	3044	3044
query39	930	904	883	883
query39_1	893	881	876	876
query40	224	122	99	99
query41	62	62	60	60
query42	96	93	91	91
query43	326	329	286	286
query44	
query45	200	182	180	180
query46	1110	1198	732	732
query47	2363	2363	2234	2234
query48	401	384	283	283
query49	621	460	351	351
query50	989	347	264	264
query51	4323	4261	4262	4261
query52	87	88	78	78
query53	244	274	192	192
query54	271	218	197	197
query55	79	75	72	72
query56	231	225	226	225
query57	1444	1386	1291	1291
query58	244	215	204	204
query59	1570	1618	1418	1418
query60	276	242	228	228
query61	153	153	161	153
query62	685	644	582	582
query63	228	186	190	186
query64	2536	794	618	618
query65	
query66	1785	451	341	341
query67	29662	29653	29518	29518
query68	
query69	464	301	271	271
query70	931	923	963	923
query71	309	231	212	212
query72	2913	2592	2328	2328
query73	925	754	434	434
query74	5172	4941	4733	4733
query75	2674	2576	2263	2263
query76	2346	1179	790	790
query77	350	373	308	308
query78	12267	12213	11763	11763
query79	1296	1037	753	753
query80	511	454	377	377
query81	448	271	243	243
query82	234	158	122	122
query83	264	282	251	251
query84	
query85	834	492	405	405
query86	324	284	274	274
query87	3338	3349	3253	3253
query88	3711	2745	2701	2701
query89	398	383	341	341
query90	2184	185	183	183
query91	167	155	130	130
query92	63	59	56	56
query93	1451	1464	863	863
query94	539	359	270	270
query95	685	469	335	335
query96	1028	806	363	363
query97	2690	2715	2582	2582
query98	205	208	199	199
query99	1144	1165	1038	1038
Total cold run time: 249854 ms
Total hot run time: 168743 ms

@hello-stephen

Copy link
Copy Markdown
Contributor

FE UT Coverage Report

Increment line coverage 0.00% (0/1) 🎉
Increment coverage report
Complete coverage report

@hello-stephen

Copy link
Copy Markdown
Contributor

FE Regression Coverage Report

Increment line coverage 100.00% (1/1) 🎉
Increment coverage report
Complete coverage report

@hello-stephen

Copy link
Copy Markdown
Contributor

FE Regression Coverage Report

Increment line coverage 1.61% (1/62) 🎉
Increment coverage report
Complete coverage report

@github-actions github-actions Bot added the approved Indicates a PR has been approved by one committer. label Jun 15, 2026
@github-actions

Copy link
Copy Markdown
Contributor

PR approved by at least one committer and no changes requested.

@github-actions

Copy link
Copy Markdown
Contributor

PR approved by anyone and no changes requested.

@BiteTheDDDDt BiteTheDDDDt merged commit ec3346d into apache:master Jun 15, 2026
34 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by one committer. reviewed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants