Skip to content

[fix](nereids) partition topn optimization requires all window expressions are in the same order#56622

Merged
englefly merged 2 commits intoapache:masterfrom
englefly:partition_topn_orderkey
Oct 15, 2025
Merged

[fix](nereids) partition topn optimization requires all window expressions are in the same order#56622
englefly merged 2 commits intoapache:masterfrom
englefly:partition_topn_orderkey

Conversation

@englefly
Copy link
Contributor

@englefly englefly commented Sep 29, 2025

What problem does this PR solve?

Issue Number: close #xxx

Related PR: #xxx

Problem Summary:

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason

  • Behavior changed:

    • No.
    • Yes.

  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@Thearas
Copy link
Contributor

Thearas commented Sep 29, 2025

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@englefly
Copy link
Contributor Author

englefly commented Sep 29, 2025

run buildall

morrySnow
morrySnow reviewed Sep 29, 2025
View reviewed changes
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/logical/LogicalWindow.java Outdated
Copy link
Contributor

@morrySnow morrySnow Sep 29, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

add feut

@doris-robot
Copy link

doris-robot commented Sep 29, 2025

ClickBench: Total hot run time: 30.55 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 6f2b4a036c2a40e8a6da6a05532d415cded85e82, data reload: false

query1	0.06	0.05	0.05
query2	0.09	0.05	0.06
query3	0.25	0.08	0.08
query4	1.61	0.12	0.12
query5	0.28	0.26	0.25
query6	1.19	0.66	0.64
query7	0.03	0.03	0.03
query8	0.06	0.05	0.05
query9	0.64	0.52	0.53
query10	0.57	0.58	0.58
query11	0.17	0.12	0.11
query12	0.15	0.12	0.12
query13	0.63	0.62	0.62
query14	1.04	1.05	1.05
query15	0.87	0.88	0.86
query16	0.41	0.42	0.40
query17	1.04	1.07	1.07
query18	0.21	0.20	0.20
query19	2.00	1.85	1.86
query20	0.01	0.01	0.01
query21	15.41	0.94	0.57
query22	0.76	1.20	0.98
query23	14.77	1.40	0.68
query24	6.95	1.46	0.49
query25	0.46	0.13	0.05
query26	0.66	0.16	0.14
query27	0.07	0.06	0.05
query28	9.44	1.35	0.93
query29	12.57	3.92	3.24
query30	0.28	0.13	0.11
query31	2.82	0.60	0.40
query32	3.24	0.57	0.47
query33	3.12	3.09	3.07
query34	16.19	5.57	4.85
query35	4.91	4.95	4.95
query36	0.68	0.52	0.51
query37	0.10	0.08	0.08
query38	0.07	0.05	0.05
query39	0.04	0.03	0.03
query40	0.17	0.15	0.14
query41	0.08	0.03	0.03
query42	0.04	0.04	0.03
query43	0.05	0.04	0.04
Total cold run time: 104.19 s
Total hot run time: 30.55 s

@hello-stephen
Copy link
Contributor

hello-stephen commented Sep 29, 2025

FE UT Coverage Report

Increment line coverage 44.44% (4/9) 🎉
Increment coverage report
Complete coverage report

@englefly englefly force-pushed the partition_topn_orderkey branch from 6f2b4a0 to 6ca52cc Compare September 29, 2025 08:09
@englefly
Copy link
Contributor Author

englefly commented Sep 29, 2025

run buildall

@doris-robot
Copy link

doris-robot commented Sep 29, 2025

TPC-DS: Total hot run time: 190309 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 6ca52cc92c1d462f3dbd2cd3bb21fd1af9b252ef, data reload: false

query1	1080	441	407	407
query2	6558	1755	1692	1692
query3	6750	221	223	221
query4	26605	23660	23222	23222
query5	5223	691	494	494
query6	347	254	230	230
query7	4674	519	297	297
query8	334	275	264	264
query9	8749	2574	2579	2574
query10	557	345	302	302
query11	16070	15370	14773	14773
query12	193	126	114	114
query13	1689	573	460	460
query14	12665	9132	9237	9132
query15	219	194	172	172
query16	7659	680	466	466
query17	1585	784	644	644
query18	2157	447	376	376
query19	240	211	191	191
query20	136	130	133	130
query21	228	154	121	121
query22	4637	4752	4627	4627
query23	34801	34758	33897	33897
query24	8336	2538	2538	2538
query25	592	545	480	480
query26	1308	295	169	169
query27	3188	520	394	394
query28	4357	2213	2194	2194
query29	817	673	525	525
query30	315	247	221	221
query31	926	884	799	799
query32	84	77	108	77
query33	591	407	344	344
query34	863	879	535	535
query35	841	860	769	769
query36	1035	1075	943	943
query37	133	127	88	88
query38	3596	3546	3530	3530
query39	1483	1401	1405	1401
query40	232	137	122	122
query41	79	76	58	58
query42	122	121	110	110
query43	486	503	461	461
query44	1383	865	829	829
query45	192	180	170	170
query46	887	1066	654	654
query47	1755	1871	1702	1702
query48	391	436	325	325
query49	773	494	401	401
query50	684	722	409	409
query51	4023	4084	3764	3764
query52	109	111	102	102
query53	247	273	210	210
query54	603	588	521	521
query55	89	95	86	86
query56	320	313	318	313
query57	1183	1211	1115	1115
query58	288	283	310	283
query59	2583	2687	2480	2480
query60	356	341	349	341
query61	162	159	187	159
query62	809	730	673	673
query63	246	201	207	201
query64	4540	1256	949	949
query65	4047	3972	3963	3963
query66	1060	424	337	337
query67	15707	15316	15218	15218
query68	8193	971	591	591
query69	500	323	300	300
query70	1406	1264	1349	1264
query71	509	341	316	316
query72	6032	4873	4791	4791
query73	685	596	360	360
query74	9192	9072	8869	8869
query75	4074	3357	2894	2894
query76	3791	1177	744	744
query77	822	423	313	313
query78	9712	9809	8870	8870
query79	2618	878	615	615
query80	694	580	512	512
query81	524	336	227	227
query82	479	160	143	143
query83	269	267	262	262
query84	261	110	93	93
query85	908	469	425	425
query86	393	323	328	323
query87	3793	3752	3670	3670
query88	3707	2254	2216	2216
query89	400	317	295	295
query90	2019	223	228	223
query91	246	163	142	142
query92	95	70	65	65
query93	2141	992	647	647
query94	705	444	335	335
query95	401	322	313	313
query96	495	577	286	286
query97	2928	2967	2861	2861
query98	245	219	218	218
query99	1351	1417	1357	1357
Total cold run time: 284565 ms
Total hot run time: 190309 ms

@doris-robot
Copy link

doris-robot commented Sep 29, 2025

ClickBench: Total hot run time: 30.35 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 6ca52cc92c1d462f3dbd2cd3bb21fd1af9b252ef, data reload: false

query1	0.06	0.05	0.04
query2	0.09	0.05	0.05
query3	0.25	0.08	0.08
query4	1.60	0.11	0.12
query5	0.27	0.26	0.24
query6	1.15	0.65	0.65
query7	0.03	0.03	0.03
query8	0.06	0.05	0.05
query9	0.61	0.52	0.51
query10	0.58	0.59	0.58
query11	0.18	0.11	0.14
query12	0.16	0.12	0.11
query13	0.64	0.62	0.61
query14	1.03	1.03	1.03
query15	0.88	0.84	0.87
query16	0.40	0.40	0.39
query17	1.06	1.05	1.06
query18	0.22	0.20	0.20
query19	1.92	1.93	1.82
query20	0.02	0.01	0.01
query21	15.42	0.92	0.56
query22	0.77	1.16	0.65
query23	15.00	1.40	0.61
query24	7.09	1.04	0.85
query25	0.50	0.09	0.06
query26	0.65	0.16	0.14
query27	0.06	0.06	0.05
query28	9.53	1.34	0.93
query29	12.62	3.99	3.26
query30	0.28	0.14	0.11
query31	2.82	0.60	0.39
query32	3.26	0.56	0.48
query33	3.09	3.07	3.13
query34	16.18	5.48	4.94
query35	4.91	4.88	4.94
query36	0.70	0.52	0.50
query37	0.10	0.07	0.08
query38	0.07	0.05	0.04
query39	0.04	0.04	0.03
query40	0.19	0.17	0.13
query41	0.09	0.04	0.03
query42	0.04	0.03	0.03
query43	0.05	0.03	0.03
Total cold run time: 104.67 s
Total hot run time: 30.35 s

@hello-stephen
Copy link
Contributor

hello-stephen commented Sep 29, 2025

FE UT Coverage Report

Increment line coverage 66.67% (6/9) 🎉
Increment coverage report
Complete coverage report

@hello-stephen
Copy link
Contributor

hello-stephen commented Sep 29, 2025

FE Regression Coverage Report

Increment line coverage 66.67% (6/9) 🎉
Increment coverage report
Complete coverage report

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Oct 14, 2025
@github-actions
Copy link
Contributor

github-actions bot commented Oct 14, 2025

PR approved by at least one committer and no changes requested.

@github-actions
Copy link
Contributor

github-actions bot commented Oct 14, 2025

PR approved by anyone and no changes requested.

@englefly englefly merged commit fffbeab into apache:master Oct 15, 2025
27 of 28 checks passed
@englefly englefly deleted the partition_topn_orderkey branch October 15, 2025 02:17
github-actions bot pushed a commit that referenced this pull request Oct 15, 2025
@englefly
[fix](nereids) partition topn optimization requires all window expres…
c87b803 
…sions are in the same order (#56622)

### What problem does this PR solve?
fix bug: partition topn optimization requires all window expressions are in the same order
@github-actions github-actions bot mentioned this pull request Oct 15, 2025
Merged
yiguolei pushed a commit that referenced this pull request Oct 16, 2025
@github-actions @englefly
branch-4.0: [fix](nereids) partition topn optimization requires all w…
a41f409 
…indow expressions are in the same order #56622 (#56976)

Cherry-picked from #56622

Co-authored-by: minghong <zhouminghong@selectdb.com>
@wm1581066 wm1581066 added the p0_w label Oct 16, 2025
@yiguolei yiguolei mentioned this pull request Nov 5, 2025
Closed
github-actions bot pushed a commit that referenced this pull request Nov 17, 2025
@englefly
[fix](nereids) partition topn optimization requires all window expres…
43398cf 
…sions are in the same order (#56622)

### What problem does this PR solve?
fix bug: partition topn optimization requires all window expressions are in the same order
@github-actions github-actions bot mentioned this pull request Nov 17, 2025
Merged
morrySnow pushed a commit that referenced this pull request Nov 25, 2025
@github-actions @englefly
branch-3.1: [fix](nereids) partition topn optimization requires all w…
bfca441 
…indow expressions are in the same order #56622 (#58080)

Cherry-picked from #56622

Co-authored-by: minghong <zhouminghong@selectdb.com>
yiguolei pushed a commit to yiguolei/incubator-doris that referenced this pull request Dec 30, 2025
@selectdb-robot @englefly
branch-selectdb-doris-3.0: [fix](nereids) partition topn optimization…
135dd1e 
… requires all window expressions are in the same order apache#56622 (apache#5752)

cherry-picks from apache#56622
Related to apache#5098

Co-authored-by: minghong <zhouminghong@selectdb.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@morrySnow morrySnow morrySnow left review comments

@starocean999 starocean999 starocean999 approved these changes

@zhangstar333 zhangstar333 zhangstar333 approved these changes

Assignees

No one assigned

Labels

approved Indicates a PR has been approved by one committer. dev/3.1.4-merged dev/4.0.1-merged p0_w reviewed

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

8 participants

@englefly @Thearas @doris-robot @hello-stephen @starocean999 @zhangstar333 @morrySnow @wm1581066

Comments