[fix](coordinator) fix query cache throw CacheSourceOperator only support one scan range#51202

924060929 · 2025-05-23T10:54:39Z

What problem does this PR solve?

fix old coordinator use query cache maybe throw exception

CacheSourceOperator only support one scan range, plan error

this exception will be thrown when the aggregate key is the distribute key:

SELECT  `pk` AS field3 -- the tbl distributed by hash(pk)
FROM tbl
GROUP BY field3

Test
- Regression test
- Unit Test
- Manual test (add detailed scripts or steps below)
- No need to test or manual test. Explain why:
  - This is a refactor/code format and no logic has been changed.
  - Previous test can cover this change.
  - No code files have been changed.
  - Other reason
Behavior changed:
- No.
- Yes.
Does this need documentation?
- No.
- Yes.

Check List (For Reviewer who merge this PR)

Confirm the release note
Confirm test cases
Confirm document
Add branch pick label

Thearas · 2025-05-23T10:55:02Z

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

What problem was fixed (it's best to include specific error reporting information). How it was fixed.
Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
What features were added. Why was this function added?
Which code was refactored and why was this part of the code refactored?
Which functions were optimized and what is the difference before and after the optimization?

924060929 · 2025-05-23T10:57:04Z

run buildall

github-actions · 2025-05-23T11:39:45Z

PR approved by at least one committer and no changes requested.

github-actions · 2025-05-23T11:39:48Z

PR approved by anyone and no changes requested.

doris-robot · 2025-05-23T12:30:16Z

TPC-H: Total hot run time: 33866 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit acf60621ead0c4ab6cf99fd967315404cce0194d, data reload: false

------ Round 1 ----------------------------------
q1	26567	5074	5094	5074
q2	2097	299	201	201
q3	10477	1226	718	718
q4	10237	995	510	510
q5	7687	2411	2277	2277
q6	180	165	135	135
q7	924	760	614	614
q8	9319	1288	1020	1020
q9	6779	5033	5145	5033
q10	6868	2310	1901	1901
q11	482	303	283	283
q12	360	360	221	221
q13	17770	3662	3119	3119
q14	232	243	216	216
q15	549	483	490	483
q16	415	438	376	376
q17	642	851	390	390
q18	7824	7211	7194	7194
q19	1871	950	558	558
q20	335	348	219	219
q21	3812	3173	2379	2379
q22	1046	1021	945	945
Total cold run time: 116473 ms
Total hot run time: 33866 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5220	5122	5090	5090
q2	239	327	239	239
q3	2201	2617	2333	2333
q4	1355	1782	1436	1436
q5	4416	4397	4394	4394
q6	216	166	127	127
q7	1997	1911	1753	1753
q8	2574	2609	2529	2529
q9	7177	7217	7026	7026
q10	3028	3171	2734	2734
q11	586	519	521	519
q12	706	802	610	610
q13	3499	3926	3319	3319
q14	282	309	299	299
q15	539	496	468	468
q16	443	494	442	442
q17	1180	1756	1363	1363
q18	7873	7677	7477	7477
q19	817	836	1055	836
q20	2068	1973	1842	1842
q21	4896	4484	4423	4423
q22	1109	1034	1014	1014
Total cold run time: 52421 ms
Total hot run time: 50273 ms

doris-robot · 2025-05-23T12:42:35Z

TPC-DS: Total hot run time: 193249 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit acf60621ead0c4ab6cf99fd967315404cce0194d, data reload: false

query1	1470	1100	1082	1082
query2	6285	1873	1819	1819
query3	11037	4583	4448	4448
query4	53166	25553	23378	23378
query5	5180	587	487	487
query6	363	216	208	208
query7	4880	527	298	298
query8	317	270	246	246
query9	5624	2669	2664	2664
query10	428	332	266	266
query11	15190	15216	14811	14811
query12	157	114	101	101
query13	1040	520	413	413
query14	10169	6249	6383	6249
query15	221	205	180	180
query16	7156	653	510	510
query17	1108	762	629	629
query18	1596	415	329	329
query19	219	199	182	182
query20	139	134	130	130
query21	209	131	114	114
query22	4361	4458	4264	4264
query23	34679	33705	33690	33690
query24	6975	2407	2482	2407
query25	490	484	415	415
query26	705	273	156	156
query27	2391	505	342	342
query28	3307	2141	2151	2141
query29	604	582	527	527
query30	279	224	197	197
query31	891	864	771	771
query32	83	75	66	66
query33	465	383	325	325
query34	783	882	542	542
query35	830	857	732	732
query36	974	1013	922	922
query37	118	105	84	84
query38	4249	4222	4209	4209
query39	1521	1442	1461	1442
query40	229	126	115	115
query41	60	55	52	52
query42	135	116	115	115
query43	511	540	492	492
query44	1391	858	857	857
query45	188	175	168	168
query46	873	1041	659	659
query47	1861	1881	1816	1816
query48	400	440	328	328
query49	666	540	448	448
query50	669	683	418	418
query51	4188	4270	4175	4175
query52	117	116	110	110
query53	231	266	188	188
query54	607	583	532	532
query55	85	85	92	85
query56	332	343	302	302
query57	1222	1203	1157	1157
query58	274	272	262	262
query59	2685	2861	2652	2652
query60	352	327	326	326
query61	151	125	130	125
query62	737	751	703	703
query63	233	193	194	193
query64	1795	1034	724	724
query65	4321	4234	4225	4225
query66	733	400	309	309
query67	16094	15793	15515	15515
query68	6815	901	541	541
query69	542	318	295	295
query70	1186	1108	1166	1108
query71	511	337	297	297
query72	6005	4913	5019	4913
query73	1514	706	372	372
query74	9313	9205	8664	8664
query75	4023	3184	2687	2687
query76	4136	1197	758	758
query77	661	434	290	290
query78	10115	10069	9321	9321
query79	3758	796	563	563
query80	646	510	447	447
query81	496	250	221	221
query82	479	132	98	98
query83	335	250	240	240
query84	295	106	82	82
query85	885	357	329	329
query86	371	300	292	292
query87	4356	4496	4273	4273
query88	3918	2321	2316	2316
query89	442	323	287	287
query90	1805	211	212	211
query91	154	150	111	111
query92	76	65	55	55
query93	3033	929	588	588
query94	664	417	302	302
query95	373	299	293	293
query96	494	581	287	287
query97	2736	2749	2712	2712
query98	242	220	203	203
query99	1445	1390	1266	1266
Total cold run time: 302623 ms
Total hot run time: 193249 ms

doris-robot · 2025-05-23T12:47:45Z

ClickBench: Total hot run time: 29.08 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit acf60621ead0c4ab6cf99fd967315404cce0194d, data reload: false

query1	0.04	0.04	0.03
query2	0.13	0.11	0.12
query3	0.25	0.19	0.20
query4	1.60	0.20	0.19
query5	0.45	0.44	0.44
query6	1.50	0.68	0.67
query7	0.03	0.02	0.02
query8	0.04	0.04	0.04
query9	0.56	0.53	0.52
query10	0.57	0.57	0.58
query11	0.16	0.11	0.11
query12	0.15	0.12	0.12
query13	0.62	0.60	0.61
query14	0.80	0.82	0.83
query15	0.89	0.85	0.87
query16	0.39	0.39	0.40
query17	1.04	1.07	1.07
query18	0.22	0.21	0.21
query19	1.91	1.86	1.81
query20	0.01	0.02	0.01
query21	15.40	0.89	0.53
query22	0.74	1.10	0.67
query23	14.99	1.38	0.65
query24	6.95	1.39	0.57
query25	0.50	0.24	0.06
query26	0.53	0.16	0.16
query27	0.06	0.05	0.06
query28	9.74	0.95	0.45
query29	12.55	4.14	3.43
query30	0.25	0.09	0.06
query31	2.82	0.59	0.40
query32	3.23	0.54	0.46
query33	3.01	3.06	3.07
query34	15.74	5.05	4.48
query35	4.55	4.52	4.49
query36	0.66	0.49	0.48
query37	0.09	0.07	0.06
query38	0.04	0.04	0.04
query39	0.03	0.03	0.02
query40	0.17	0.12	0.13
query41	0.08	0.03	0.03
query42	0.04	0.02	0.02
query43	0.04	0.02	0.03
Total cold run time: 103.57 s
Total hot run time: 29.08 s

924060929 · 2025-05-25T08:11:05Z

run buildall

doris-robot · 2025-05-25T08:59:46Z

TPC-H: Total hot run time: 33682 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 50a2eaff6bf3d296a210525a9f0d85677891fadb, data reload: false

------ Round 1 ----------------------------------
q1	25796	5025	4974	4974
q2	2062	282	182	182
q3	10405	1237	707	707
q4	10226	980	501	501
q5	7521	2369	2343	2343
q6	177	161	134	134
q7	871	760	607	607
q8	9321	1252	1058	1058
q9	6835	5069	5082	5069
q10	7252	2363	1945	1945
q11	485	290	269	269
q12	349	357	222	222
q13	17773	3633	3061	3061
q14	239	230	220	220
q15	530	483	502	483
q16	412	429	374	374
q17	608	845	372	372
q18	7623	7137	7087	7087
q19	1583	954	541	541
q20	334	333	224	224
q21	3721	3134	2370	2370
q22	1016	1022	939	939
Total cold run time: 115139 ms
Total hot run time: 33682 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5096	5048	5044	5044
q2	241	324	232	232
q3	2190	2674	2318	2318
q4	1286	1749	1396	1396
q5	4402	4388	4356	4356
q6	211	166	133	133
q7	2020	1906	1819	1819
q8	2586	2514	2511	2511
q9	7092	7175	6913	6913
q10	3044	3191	2757	2757
q11	576	520	484	484
q12	667	789	638	638
q13	3496	3842	3270	3270
q14	277	310	302	302
q15	535	478	495	478
q16	431	486	441	441
q17	1126	1591	1363	1363
q18	7707	7478	7455	7455
q19	780	744	817	744
q20	1990	2003	1804	1804
q21	4621	4454	4362	4362
q22	1079	1025	1024	1024
Total cold run time: 51453 ms
Total hot run time: 49844 ms

doris-robot · 2025-05-25T09:12:00Z

TPC-DS: Total hot run time: 190313 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 50a2eaff6bf3d296a210525a9f0d85677891fadb, data reload: false

query1	1411	1075	1051	1051
query2	6192	1775	1804	1775
query3	11044	4527	4450	4450
query4	54335	25055	23114	23114
query5	5160	575	459	459
query6	342	205	194	194
query7	4898	523	301	301
query8	316	244	222	222
query9	5739	2621	2611	2611
query10	406	322	260	260
query11	14999	15078	14864	14864
query12	157	111	107	107
query13	1073	550	421	421
query14	10190	6230	6090	6090
query15	188	199	183	183
query16	7113	658	509	509
query17	1085	729	571	571
query18	1593	406	318	318
query19	202	202	183	183
query20	126	123	121	121
query21	203	131	143	131
query22	4363	4452	4216	4216
query23	34213	33576	33668	33576
query24	6611	2386	2419	2386
query25	481	479	414	414
query26	660	273	157	157
query27	2135	522	345	345
query28	2938	2150	2163	2150
query29	572	572	449	449
query30	276	228	193	193
query31	862	916	792	792
query32	76	75	77	75
query33	473	386	331	331
query34	792	870	552	552
query35	825	857	765	765
query36	950	997	892	892
query37	114	111	76	76
query38	4233	4274	4185	4185
query39	1552	1476	1487	1476
query40	215	136	113	113
query41	70	67	64	64
query42	136	121	116	116
query43	518	490	479	479
query44	1372	872	842	842
query45	190	185	205	185
query46	856	1042	650	650
query47	1864	1913	1824	1824
query48	391	423	336	336
query49	686	543	458	458
query50	663	707	404	404
query51	4300	4246	4167	4167
query52	113	112	106	106
query53	224	262	186	186
query54	599	587	564	564
query55	92	84	86	84
query56	331	325	286	286
query57	1159	1176	1133	1133
query58	272	261	249	249
query59	2671	2782	2664	2664
query60	346	329	318	318
query61	136	127	138	127
query62	749	769	667	667
query63	229	203	198	198
query64	1466	1079	702	702
query65	4315	4235	4235	4235
query66	742	413	306	306
query67	15928	15931	15602	15602
query68	6817	890	539	539
query69	537	314	265	265
query70	1253	1141	1119	1119
query71	503	338	302	302
query72	5532	2364	5153	2364
query73	1450	774	360	360
query74	9155	9255	8943	8943
query75	4040	3206	2695	2695
query76	4313	1206	758	758
query77	781	461	284	284
query78	9982	10067	9376	9376
query79	5186	836	576	576
query80	646	552	454	454
query81	482	255	221	221
query82	633	138	99	99
query83	294	259	234	234
query84	294	106	94	94
query85	799	389	380	380
query86	332	314	290	290
query87	4344	4361	4338	4338
query88	3066	2329	2299	2299
query89	451	319	296	296
query90	1962	220	219	219
query91	145	142	116	116
query92	74	64	59	59
query93	2461	934	592	592
query94	683	412	286	286
query95	373	300	295	295
query96	504	574	293	293
query97	2704	2787	2641	2641
query98	231	217	200	200
query99	1435	1443	1296	1296
Total cold run time: 300993 ms
Total hot run time: 190313 ms

doris-robot · 2025-05-25T09:17:12Z

ClickBench: Total hot run time: 29.1 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 50a2eaff6bf3d296a210525a9f0d85677891fadb, data reload: false

query1	0.03	0.03	0.04
query2	0.12	0.10	0.10
query3	0.26	0.19	0.19
query4	1.60	0.20	0.11
query5	0.43	0.40	0.43
query6	1.16	0.66	0.66
query7	0.02	0.02	0.01
query8	0.05	0.04	0.03
query9	0.57	0.52	0.51
query10	0.57	0.60	0.55
query11	0.15	0.11	0.11
query12	0.14	0.11	0.12
query13	0.61	0.61	0.60
query14	0.77	0.83	0.80
query15	0.88	0.84	0.86
query16	0.39	0.38	0.38
query17	1.07	1.03	1.05
query18	0.22	0.21	0.20
query19	1.89	1.86	1.83
query20	0.01	0.02	0.01
query21	15.39	0.90	0.55
query22	0.75	1.12	0.62
query23	15.07	1.37	0.65
query24	7.14	1.73	0.99
query25	0.49	0.19	0.08
query26	0.57	0.18	0.15
query27	0.05	0.05	0.05
query28	9.35	0.81	0.42
query29	12.56	4.00	3.29
query30	0.25	0.09	0.07
query31	2.81	0.59	0.39
query32	3.23	0.55	0.47
query33	2.99	3.06	3.06
query34	15.76	5.05	4.52
query35	4.47	4.50	4.50
query36	0.68	0.50	0.49
query37	0.08	0.06	0.06
query38	0.05	0.04	0.03
query39	0.03	0.03	0.02
query40	0.18	0.14	0.13
query41	0.08	0.02	0.02
query42	0.04	0.02	0.03
query43	0.04	0.03	0.02
Total cold run time: 103 s
Total hot run time: 29.1 s

github-actions · 2025-05-26T13:30:28Z

PR approved by at least one committer and no changes requested.

…51204) cherry pick from #47883, #47961, #51202 --------- Co-authored-by: HappenLee <happenlee@selectdb.com>

…port one scan range (apache#51202) fix old coordinator use query cache maybe throw exception ``` CacheSourceOperator only support one scan range, plan error ``` this exception will be thrown when the aggregate key is the distribute key: ```sql SELECT `pk` AS field3 -- the tbl distributed by hash(pk) FROM tbl GROUP BY field3 ```

fix

acf6062

924060929 added p0_w dev/3.0.x labels May 23, 2025

morrySnow previously approved these changes May 23, 2025

View reviewed changes

github-actions bot added the approved Indicates a PR has been approved by one committer. label May 23, 2025

github-actions bot added the reviewed label May 23, 2025

924060929 mentioned this pull request May 23, 2025

[Pick](query-cache) some query-cache bugfix (#47883)(#47961)(#51202) #51204

Merged

fix

50a2eaf

924060929 dismissed morrySnow’s stale review via 50a2eaf May 25, 2025 08:10

github-actions bot removed the approved Indicates a PR has been approved by one committer. label May 25, 2025

starocean999 approved these changes May 26, 2025

View reviewed changes

github-actions bot added the approved Indicates a PR has been approved by one committer. label May 26, 2025

morrySnow approved these changes May 27, 2025

View reviewed changes

924060929 merged commit ebc26bc into apache:master May 27, 2025
26 of 27 checks passed

924060929 deleted the fix_query_cache branch May 27, 2025 02:47

github-actions bot added the dev/3.0.x-conflict label May 27, 2025

924060929 removed the dev/3.0.x-conflict label May 27, 2025

dataroaring pushed a commit that referenced this pull request May 27, 2025

[Pick](query-cache) some query-cache bugfix (#47883)(#47961)(#51202) (#…

6e38b0a

…51204) cherry pick from #47883, #47961, #51202 --------- Co-authored-by: HappenLee <happenlee@selectdb.com>

dataroaring added dev/3.0.6-merged and removed dev/3.0.x labels May 27, 2025

Conversation

924060929 commented May 23, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What problem does this PR solve?

Check List (For Reviewer who merge this PR)

Uh oh!

Thearas commented May 23, 2025

Uh oh!

924060929 commented May 23, 2025

Uh oh!

github-actions bot commented May 23, 2025

Uh oh!

github-actions bot commented May 23, 2025

Uh oh!

doris-robot commented May 23, 2025

Uh oh!

doris-robot commented May 23, 2025

Uh oh!

doris-robot commented May 23, 2025

Uh oh!

924060929 commented May 25, 2025

Uh oh!

doris-robot commented May 25, 2025

Uh oh!

doris-robot commented May 25, 2025

Uh oh!

doris-robot commented May 25, 2025

Uh oh!

github-actions bot commented May 26, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

Comments

924060929 commented May 23, 2025 •

edited

Loading