Skip to content

[opt](nereids) use one fragment to execute point query#53541

Merged
morrySnow merged 1 commit intoapache:masterfrom
924060929:opt-point-query
Jul 18, 2025
Merged

[opt](nereids) use one fragment to execute point query#53541
morrySnow merged 1 commit intoapache:masterfrom
924060929:opt-point-query

Conversation

@924060929
Copy link
Contributor

@924060929 924060929 commented Jul 18, 2025 •
edited
Loading

What problem does this PR solve?

the nereids planner use two fragments to execute point query and cause some performance degradation, this pr optimize it by generate one fragment

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason

  • Behavior changed:

    • No.
    • Yes.

  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@924060929
Copy link
Contributor Author

924060929 commented Jul 18, 2025

run buildall

@doris-robot
Copy link

doris-robot commented Jul 18, 2025

TPC-H: Total hot run time: 34287 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 80930302268e3abda2c22425c8d2d3d0d01ba8aa, data reload: false

------ Round 1 ----------------------------------
q1	17586	5301	5165	5165
q2	1950	293	194	194
q3	10288	1278	695	695
q4	10238	1008	507	507
q5	7501	2426	2339	2339
q6	190	173	137	137
q7	943	770	642	642
q8	9345	1312	1052	1052
q9	6925	5192	5151	5151
q10	6907	2433	1989	1989
q11	493	305	290	290
q12	363	366	233	233
q13	17782	3720	3097	3097
q14	243	252	259	252
q15	586	531	548	531
q16	448	439	396	396
q17	607	881	369	369
q18	7688	7277	7065	7065
q19	1243	975	610	610
q20	352	362	242	242
q21	4075	2524	2344	2344
q22	1081	1046	987	987
Total cold run time: 106834 ms
Total hot run time: 34287 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5151	5396	5155	5155
q2	267	351	248	248
q3	2182	2691	2319	2319
q4	1359	1781	1295	1295
q5	4321	4545	4553	4545
q6	223	179	141	141
q7	2007	1907	1808	1808
q8	2604	2675	2610	2610
q9	7271	7316	7194	7194
q10	3319	3318	2890	2890
q11	583	503	494	494
q12	696	817	626	626
q13	3591	3974	3318	3318
q14	304	312	293	293
q15	563	510	533	510
q16	473	516	465	465
q17	1170	1588	1452	1452
q18	8036	7655	7417	7417
q19	800	881	870	870
q20	1983	2113	1937	1937
q21	4953	4590	4567	4567
q22	1163	1159	1078	1078
Total cold run time: 53019 ms
Total hot run time: 51232 ms

@doris-robot
Copy link

doris-robot commented Jul 18, 2025

TPC-DS: Total hot run time: 191815 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 80930302268e3abda2c22425c8d2d3d0d01ba8aa, data reload: false

query1	1023	395	420	395
query2	6544	1764	1703	1703
query3	6765	229	219	219
query4	26437	23848	23414	23414
query5	4437	719	568	568
query6	332	238	231	231
query7	4654	514	314	314
query8	385	341	336	336
query9	8660	2685	2666	2666
query10	517	390	323	323
query11	15974	15137	14709	14709
query12	190	130	122	122
query13	1693	593	480	480
query14	10799	6053	6062	6053
query15	223	206	187	187
query16	7659	685	492	492
query17	1610	742	597	597
query18	2027	436	324	324
query19	205	217	173	173
query20	145	130	127	127
query21	234	141	127	127
query22	4470	4729	4440	4440
query23	34937	34146	33693	33693
query24	8104	2401	2457	2401
query25	550	469	420	420
query26	1231	274	158	158
query27	2709	520	404	404
query28	4332	2206	2166	2166
query29	736	564	460	460
query30	295	233	203	203
query31	1012	923	794	794
query32	83	82	72	72
query33	579	399	332	332
query34	802	858	540	540
query35	811	836	784	784
query36	998	1019	920	920
query37	114	106	84	84
query38	4181	4193	4094	4094
query39	1543	1483	1513	1483
query40	230	141	137	137
query41	106	101	143	101
query42	137	123	116	116
query43	500	522	501	501
query44	1379	860	848	848
query45	192	186	181	181
query46	853	1062	664	664
query47	1795	1826	1811	1811
query48	401	439	394	394
query49	820	570	478	478
query50	666	711	442	442
query51	5449	5540	5706	5540
query52	120	119	106	106
query53	248	283	201	201
query54	607	608	534	534
query55	93	92	90	90
query56	339	341	313	313
query57	1217	1272	1154	1154
query58	316	309	316	309
query59	2599	2775	2591	2591
query60	363	352	341	341
query61	153	151	149	149
query62	808	732	696	696
query63	249	209	214	209
query64	4330	1134	730	730
query65	4369	4231	4247	4231
query66	1231	594	494	494
query67	15788	15669	15588	15588
query68	8265	940	553	553
query69	507	345	306	306
query70	1266	1154	1189	1154
query71	506	351	331	331
query72	5555	4881	5031	4881
query73	747	693	373	373
query74	9000	9203	9108	9108
query75	3915	3288	2724	2724
query76	3747	1167	785	785
query77	852	461	373	373
query78	10074	10141	9426	9426
query79	2714	873	603	603
query80	703	584	531	531
query81	483	270	238	238
query82	497	140	112	112
query83	326	287	271	271
query84	304	115	109	109
query85	839	437	359	359
query86	402	317	322	317
query87	4557	4479	4457	4457
query88	3491	2405	2387	2387
query89	422	344	313	313
query90	1944	239	238	238
query91	162	154	132	132
query92	91	77	71	71
query93	1865	1004	609	609
query94	680	406	333	333
query95	395	321	317	317
query96	503	587	292	292
query97	2779	2767	2649	2649
query98	248	220	218	218
query99	1479	1444	1288	1288
Total cold run time: 282676 ms
Total hot run time: 191815 ms

@doris-robot
Copy link

doris-robot commented Jul 18, 2025

ClickBench: Total hot run time: 32.61 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 80930302268e3abda2c22425c8d2d3d0d01ba8aa, data reload: false

query1	0.05	0.04	0.04
query2	0.08	0.04	0.05
query3	0.25	0.07	0.07
query4	1.61	0.10	0.11
query5	0.44	0.44	0.40
query6	1.16	0.66	0.66
query7	0.03	0.02	0.02
query8	0.05	0.04	0.04
query9	0.60	0.55	0.51
query10	0.61	0.59	0.57
query11	0.16	0.12	0.13
query12	0.17	0.13	0.12
query13	0.64	0.61	0.62
query14	0.81	0.81	0.83
query15	0.90	0.88	0.87
query16	0.40	0.41	0.40
query17	1.08	1.06	1.09
query18	0.23	0.23	0.22
query19	1.98	1.85	1.93
query20	0.02	0.02	0.02
query21	15.37	0.88	0.55
query22	0.78	1.24	0.71
query23	14.82	1.38	0.59
query24	6.86	2.16	0.48
query25	0.41	0.18	0.22
query26	0.66	0.15	0.14
query27	0.08	0.06	0.06
query28	9.53	0.86	0.45
query29	12.61	4.02	3.35
query30	3.17	3.08	3.05
query31	2.83	0.62	0.39
query32	3.23	0.57	0.48
query33	3.11	3.08	3.05
query34	15.90	5.45	4.83
query35	4.86	4.89	4.86
query36	0.69	0.51	0.50
query37	0.11	0.07	0.07
query38	0.07	0.05	0.05
query39	0.05	0.04	0.04
query40	0.19	0.16	0.15
query41	0.09	0.03	0.03
query42	0.04	0.04	0.03
query43	0.05	0.04	0.04
Total cold run time: 106.78 s
Total hot run time: 32.61 s

@github-actions
Copy link
Contributor

github-actions bot commented Jul 18, 2025

PR approved by at least one committer and no changes requested.

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Jul 18, 2025
@github-actions
Copy link
Contributor

github-actions bot commented Jul 18, 2025

PR approved by anyone and no changes requested.

@morrySnow morrySnow merged commit bc54bdd into apache:master Jul 18, 2025
30 of 32 checks passed
github-actions bot pushed a commit that referenced this pull request Jul 18, 2025
@924060929
[opt](nereids) use one fragment to execute point query (#53541)
a0264ea 
the nereids planner use two fragments to execute point query and cause
some performance degradation, this pr optimize it by generate one
fragment
@github-actions github-actions bot mentioned this pull request Jul 18, 2025
Merged
github-actions bot pushed a commit that referenced this pull request Jul 18, 2025
@924060929
[opt](nereids) use one fragment to execute point query (#53541)
2f8873f 
the nereids planner use two fragments to execute point query and cause
some performance degradation, this pr optimize it by generate one
fragment
@github-actions github-actions bot mentioned this pull request Jul 18, 2025
Merged
github-actions bot pushed a commit that referenced this pull request Jul 18, 2025
@924060929
[opt](nereids) use one fragment to execute point query (#53541)
5b10435 
the nereids planner use two fragments to execute point query and cause
some performance degradation, this pr optimize it by generate one
fragment
@github-actions github-actions bot mentioned this pull request Jul 18, 2025
Merged
morrySnow pushed a commit that referenced this pull request Jul 19, 2025
@github-actions @924060929
branch-3.1: [opt](nereids) use one fragment to execute point query #5…
5ca653a 
…3541 (#53575)

Cherry-picked from #53541

Co-authored-by: 924060929 <lanhuajian@selectdb.com>
yiguolei pushed a commit that referenced this pull request Jul 25, 2025
@github-actions @924060929
branch-2.1: [opt](nereids) use one fragment to execute point query #5…
77ff75b 
…3541 (#53574)

Cherry-picked from #53541

Co-authored-by: 924060929 <lanhuajian@selectdb.com>
dataroaring pushed a commit that referenced this pull request Aug 14, 2025
@github-actions @924060929
branch-3.0: [opt](nereids) use one fragment to execute point query #5…
c1ab85e 
…3541 (#53573)

Cherry-picked from #53541

Co-authored-by: 924060929 <lanhuajian@selectdb.com>
@924060929 924060929 deleted the opt-point-query branch August 29, 2025 07:02
@gavinchou gavinchou mentioned this pull request Sep 1, 2025
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@starocean999 starocean999 starocean999 approved these changes

@morrySnow morrySnow morrySnow approved these changes

Assignees

No one assigned

Labels

approved Indicates a PR has been approved by one committer. dev/2.1.11-merged dev/3.0.8-merged dev/3.1.0-merged reviewed

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

6 participants

@924060929 @doris-robot @starocean999 @morrySnow @yiguolei @dataroaring

Comments