Skip to content

[improve](planner) select * unique_table limit n should use one instance#53948

Merged
zhangstar333 merged 1 commit intoapache:masterfrom
zhangstar333:unique_limit
Jul 29, 2025
Merged

[improve](planner) select * unique_table limit n should use one instance#53948
zhangstar333 merged 1 commit intoapache:masterfrom
zhangstar333:unique_limit

Conversation

@zhangstar333
Copy link
Contributor

@zhangstar333 zhangstar333 commented Jul 28, 2025 •
edited
Loading

What problem does this PR solve?

Problem Summary:
the unique_table has a condition of delete_sign = 0 awalys,
so it's not have plan for one instance to scan table,
now add some check for unique_table let running only one instance for select limit n.

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason

  • Behavior changed:

    • No.
    • Yes.

  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@hello-stephen
Copy link
Contributor

hello-stephen commented Jul 28, 2025

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@zhangstar333
Copy link
Contributor Author

zhangstar333 commented Jul 28, 2025

run buildall

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Jul 28, 2025
@github-actions
Copy link
Contributor

github-actions bot commented Jul 28, 2025

PR approved by at least one committer and no changes requested.

@github-actions
Copy link
Contributor

github-actions bot commented Jul 28, 2025

PR approved by anyone and no changes requested.

HappenLee
HappenLee approved these changes Jul 28, 2025
View reviewed changes
Copy link
Contributor

@HappenLee HappenLee left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@doris-robot
Copy link

doris-robot commented Jul 28, 2025

TPC-H: Total hot run time: 34140 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit f70b2bb135faed184043deb5fbfefee33ce73fb4, data reload: false

------ Round 1 ----------------------------------
q1	17598	5190	5201	5190
q2	1927	301	183	183
q3	10377	1310	714	714
q4	10237	988	540	540
q5	7648	2326	2292	2292
q6	189	170	133	133
q7	892	779	617	617
q8	9310	1337	1097	1097
q9	6802	5183	5092	5092
q10	6908	2365	1970	1970
q11	466	277	266	266
q12	362	367	225	225
q13	17794	3516	3080	3080
q14	233	233	222	222
q15	539	466	470	466
q16	429	432	369	369
q17	604	862	377	377
q18	7532	7123	7139	7123
q19	2844	1002	560	560
q20	339	317	233	233
q21	3445	3151	2393	2393
q22	1033	1105	998	998
Total cold run time: 107508 ms
Total hot run time: 34140 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5365	5262	5276	5262
q2	230	325	212	212
q3	2094	2553	2174	2174
q4	1309	1717	1306	1306
q5	4215	4595	4516	4516
q6	219	176	133	133
q7	2021	1927	1815	1815
q8	2557	2525	2553	2525
q9	7223	7376	7312	7312
q10	3410	3363	2943	2943
q11	539	529	485	485
q12	748	802	591	591
q13	3537	3800	3326	3326
q14	271	298	313	298
q15	502	472	462	462
q16	487	519	451	451
q17	1215	1468	1386	1386
q18	7958	7778	7645	7645
q19	12071	868	919	868
q20	2915	1985	1987	1985
q21	14413	4345	4271	4271
q22	1079	1044	961	961
Total cold run time: 74378 ms
Total hot run time: 50927 ms

@doris-robot
Copy link

doris-robot commented Jul 28, 2025

TPC-DS: Total hot run time: 173216 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit f70b2bb135faed184043deb5fbfefee33ce73fb4, data reload: false

============================================
query1	1005	398	452	398
query2	6510	1847	1706	1706
query3	6743	227	223	223
query4	26818	23869	23425	23425
query5	4384	639	551	551
query6	322	243	243	243
query7	4658	538	302	302
query8	289	255	225	225
query9	8589	2980	2988	2980
query10	482	353	315	315
query11	15825	15047	14761	14761
query12	184	136	137	136
query13	1664	576	432	432
query14	8754	5980	6000	5980
query15	213	192	166	166
query16	7186	671	469	469
query17	1227	779	649	649
query18	2205	464	329	329
query19	232	218	198	198
query20	155	141	139	139
query21	220	178	109	109
query22	4133	3939	3955	3939
query23	34059	34274	34317	34274
query24	8099	2434	2453	2434
query25	556	532	437	437
query26	729	291	162	162
query27	2571	515	353	353
query28	4284	2359	2348	2348
query29	757	610	505	505
query30	295	259	210	210
query31	895	815	723	723
query32	95	83	84	83
query33	569	438	366	366
query34	834	867	525	525
query35	827	812	768	768
query36	1024	1083	936	936
query37	140	112	103	103
query38	4007	4095	4137	4095
query39	1454	1384	1360	1360
query40	242	147	133	133
query41	63	83	57	57
query42	142	123	132	123
query43	518	537	460	460
query44	1427	882	892	882
query45	203	186	178	178
query46	952	1075	679	679
query47	1816	1845	1743	1743
query48	413	455	332	332
query49	711	535	428	428
query50	675	698	426	426
query51	5538	5518	5414	5414
query52	138	162	123	123
query53	265	298	213	213
query54	657	659	559	559
query55	92	89	89	89
query56	356	356	359	356
query57	1192	1225	1147	1147
query58	341	337	344	337
query59	2511	2707	2492	2492
query60	434	386	412	386
query61	129	122	129	122
query62	811	728	666	666
query63	258	216	231	216
query64	3199	1126	826	826
query65	4152	4129	4046	4046
query66	952	454	354	354
query67	query68	17923	1046	1112	1046
query69	1040	293	296	293
query70	1489	1127	1104	1104
query71	690	318	323	318
query72	9141	2317	2385	2317
query73	3828	645	366	366
query74	9021	8913	8837	8837
query75	7369	3140	2629	2629
query76	8595	1260	803	803
query77	1176	401	401	401
query78	9604	11182	9504	9504
query79	14606	662	589	589
query80	878	549	497	497
query81	550	267	243	243
query82	438	156	115	115
query83	304	282	262	262
query84	305	102	92	92
query85	1055	364	338	338
query86	352	316	298	298
query87	4342	4315	4246	4246
query88	5576	2312	2302	2302
query89	482	369	326	326
query90	2572	246	243	243
query91	149	140	115	115
query92	95	76	72	72
query93	6094	1003	670	670
query94	1106	403	284	284
query95	419	332	321	321
query96	523	597	294	294
query97	2776	2775	2646	2646
query98	248	238	219	219
query99	1461	1411	1282	1282
Total cold run time: 305387 ms
Total hot run time: 173216 ms

@doris-robot
Copy link

doris-robot commented Jul 28, 2025

ClickBench: Total hot run time: 33.48 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit f70b2bb135faed184043deb5fbfefee33ce73fb4, data reload: false

query1	0.04	0.04	0.04
query2	0.08	0.05	0.04
query3	0.25	0.08	0.08
query4	1.61	0.11	0.10
query5	0.43	0.44	0.45
query6	1.14	0.71	0.70
query7	0.02	0.02	0.02
query8	0.05	0.04	0.05
query9	0.55	0.47	0.49
query10	0.53	0.52	0.51
query11	0.16	0.10	0.11
query12	0.14	0.11	0.11
query13	0.65	0.68	0.66
query14	0.92	1.11	1.14
query15	0.90	0.91	0.90
query16	0.41	0.39	0.38
query17	1.08	1.07	1.05
query18	0.23	0.20	0.21
query19	2.07	1.96	1.98
query20	0.02	0.01	0.02
query21	15.35	0.82	0.53
query22	0.75	1.13	0.67
query23	14.97	1.16	0.64
query24	6.61	0.85	0.98
query25	0.47	0.24	0.10
query26	0.70	0.16	0.15
query27	0.07	0.06	0.05
query28	9.76	0.87	0.45
query29	12.59	3.89	3.41
query30	3.03	2.99	2.91
query31	2.82	0.57	0.40
query32	3.25	0.56	0.51
query33	3.03	3.18	3.23
query34	16.29	5.35	4.93
query35	4.85	4.95	4.93
query36	0.70	0.51	0.49
query37	0.10	0.07	0.07
query38	0.06	0.05	0.04
query39	0.03	0.03	0.03
query40	0.17	0.14	0.13
query41	0.08	0.02	0.02
query42	0.03	0.03	0.03
query43	0.05	0.04	0.04
Total cold run time: 107.04 s
Total hot run time: 33.48 s

@hello-stephen
Copy link
Contributor

hello-stephen commented Jul 28, 2025

BE UT Coverage Report

Increment line coverage 100.00% (3/3) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 57.59% (15960/27713)
Line Coverage 46.34% (143527/309732)
Region Coverage 35.74% (108222/302770)
Branch Coverage 38.29% (47759/124725)

@hello-stephen
Copy link
Contributor

hello-stephen commented Jul 28, 2025

BE Regression && UT Coverage Report

Increment line coverage 100.00% (3/3) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 81.23% (22096/27203)
Line Coverage 73.83% (228369/309318)
Region Coverage 61.48% (190918/310518)
Branch Coverage 65.23% (82277/126133)

@hello-stephen
Copy link
Contributor

hello-stephen commented Jul 28, 2025

FE UT Coverage Report

Increment line coverage 13.33% (2/15) 🎉
Increment coverage report
Complete coverage report

@hello-stephen
Copy link
Contributor

hello-stephen commented Jul 28, 2025

BE Regression && UT Coverage Report

Increment line coverage 100.00% (3/3) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 81.23% (22096/27203)
Line Coverage 73.83% (228369/309318)
Region Coverage 61.48% (190918/310518)
Branch Coverage 65.23% (82277/126133)

@hello-stephen
Copy link
Contributor

hello-stephen commented Jul 28, 2025

BE Regression && UT Coverage Report

Increment line coverage 100.00% (3/3) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 81.22% (22095/27203)
Line Coverage 73.82% (228353/309318)
Region Coverage 61.47% (190884/310518)
Branch Coverage 65.22% (82264/126133)

@zhangstar333 zhangstar333 merged commit 90ce174 into apache:master Jul 29, 2025
27 of 30 checks passed
@yiguolei yiguolei added usercase Important user case type label dev/3.0.x dev/3.1.x labels Aug 1, 2025
github-actions bot pushed a commit that referenced this pull request Aug 1, 2025
@zhangstar333
[improve](planner) select * unique_table limit n should use one insta…
646c20a 
…nce (#53948)

### What problem does this PR solve?
Problem Summary:
the unique_table has a condition of delete_sign = 0 awalys,
so it's not have plan for one instance to scan table,
now add some check for unique_table let running only one instance for
select limit n.
@github-actions github-actions bot mentioned this pull request Aug 1, 2025
Merged
github-actions bot pushed a commit that referenced this pull request Aug 1, 2025
@zhangstar333
[improve](planner) select * unique_table limit n should use one insta…
65c3634 
…nce (#53948)

### What problem does this PR solve?
Problem Summary:
the unique_table has a condition of delete_sign = 0 awalys,
so it's not have plan for one instance to scan table,
now add some check for unique_table let running only one instance for
select limit n.
@github-actions github-actions bot mentioned this pull request Aug 1, 2025
Merged
morrySnow pushed a commit that referenced this pull request Aug 4, 2025
@github-actions @zhangstar333
branch-3.1: [improve](planner) select * unique_table limit n should u…
434171d 
…se one instance #53948 (#54186)

Cherry-picked from #53948

Co-authored-by: zhangstar333 <zhangsida@selectdb.com>
dataroaring pushed a commit that referenced this pull request Aug 13, 2025
@github-actions @zhangstar333
branch-3.0: [improve](planner) select * unique_table limit n should u…
48c8fc7 
…se one instance #53948 (#54182)

Cherry-picked from #53948

Co-authored-by: zhangstar333 <zhangsida@selectdb.com>
@gavinchou gavinchou mentioned this pull request Sep 1, 2025
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@924060929 924060929 924060929 approved these changes

@HappenLee HappenLee HappenLee approved these changes

Assignees

No one assigned

Labels

approved Indicates a PR has been approved by one committer. dev/3.0.8-merged dev/3.1.0-merged reviewed usercase Important user case type label

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

8 participants

@zhangstar333 @hello-stephen @doris-robot @924060929 @HappenLee @yiguolei @dataroaring @morrySnow

Comments