Skip to content

[fix](load) update scanned rows and loaded bytes progressively#54606

Merged
liaoxin01 merged 5 commits intoapache:masterfrom
kaijchen:DORIS-20541
Aug 14, 2025
Merged

[fix](load) update scanned rows and loaded bytes progressively#54606
liaoxin01 merged 5 commits intoapache:masterfrom
kaijchen:DORIS-20541

Conversation

@kaijchen
Copy link
Member

What problem does this PR solve?

Issue Number: DORIS-20541

Related PR: #29802

Problem Summary:

Fix an issue where the scanned rows and loaded bytes metrics were not updated progressively in the FE.
Although the BE periodically reports execution status, the FE was ignoring these reports due to a change introduced in PR #29802, which skips processing reports without the isDone flag.

This fix ensures that intermediate execution reports are processed, allowing progressive updates of scanned rows and loaded bytes during query execution.

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason
  • Behavior changed:

    • No.
    • Yes.
  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@Thearas
Copy link
Contributor

Thearas commented Aug 12, 2025

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@kaijchen
Copy link
Member Author

run buildall

@kaijchen kaijchen changed the title [fix](load) update scanned rows and loaded bytes progressively in FE [fix](load) update scanned rows and loaded bytes progressively Aug 12, 2025
@kaijchen
Copy link
Member Author

run buildall

@hello-stephen
Copy link
Contributor

FE UT Coverage Report

Increment line coverage 2.33% (1/43) 🎉
Increment coverage report
Complete coverage report

@doris-robot
Copy link

TPC-H: Total hot run time: 33856 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit b08e2e8d94396188cafa0ebbe67231f613848edd, data reload: false

------ Round 1 ----------------------------------
q1	17592	5257	5115	5115
q2	1914	282	178	178
q3	10346	1334	699	699
q4	10212	1007	535	535
q5	7541	2382	2337	2337
q6	175	162	131	131
q7	882	747	611	611
q8	9315	1296	1105	1105
q9	6908	5108	5051	5051
q10	6869	2415	1953	1953
q11	480	283	273	273
q12	346	353	221	221
q13	17753	3660	3057	3057
q14	230	259	211	211
q15	555	492	490	490
q16	432	420	365	365
q17	585	845	355	355
q18	7391	7089	7074	7074
q19	1106	948	580	580
q20	357	347	220	220
q21	3668	2552	2332	2332
q22	1072	1026	963	963
Total cold run time: 105729 ms
Total hot run time: 33856 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5198	5067	5135	5067
q2	243	316	227	227
q3	2157	2677	2299	2299
q4	1341	1780	1330	1330
q5	4206	4369	4538	4369
q6	216	173	134	134
q7	2003	1958	1830	1830
q8	2656	2597	2613	2597
q9	7332	7244	7324	7244
q10	3185	3269	2861	2861
q11	594	508	492	492
q12	704	807	630	630
q13	3561	4096	3222	3222
q14	290	314	292	292
q15	525	471	492	471
q16	455	516	460	460
q17	1193	1586	1376	1376
q18	7972	7619	7552	7552
q19	819	926	1129	926
q20	2036	2023	1942	1942
q21	5034	4457	4326	4326
q22	1063	1026	1017	1017
Total cold run time: 52783 ms
Total hot run time: 50664 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 185508 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit b08e2e8d94396188cafa0ebbe67231f613848edd, data reload: false

query1	996	395	400	395
query2	6513	1745	1788	1745
query3	6744	225	214	214
query4	26125	23422	23206	23206
query5	4378	613	503	503
query6	310	213	207	207
query7	4622	501	286	286
query8	277	229	223	223
query9	8594	2928	2884	2884
query10	460	331	291	291
query11	15872	15515	14724	14724
query12	174	116	112	112
query13	1657	537	424	424
query14	9968	5921	5805	5805
query15	216	184	164	164
query16	7661	638	494	494
query17	1285	735	602	602
query18	2041	424	325	325
query19	197	191	165	165
query20	130	115	114	114
query21	213	122	108	108
query22	4260	4304	4154	4154
query23	34400	33342	33234	33234
query24	8008	2420	2352	2352
query25	544	480	389	389
query26	1235	275	154	154
query27	2729	500	342	342
query28	4346	2246	2216	2216
query29	735	551	445	445
query30	294	222	218	218
query31	891	813	749	749
query32	81	74	82	74
query33	571	394	349	349
query34	803	844	514	514
query35	788	828	741	741
query36	967	1001	912	912
query37	122	102	88	88
query38	4052	4051	3990	3990
query39	1444	1405	1393	1393
query40	217	122	118	118
query41	61	60	82	60
query42	119	114	117	114
query43	493	524	480	480
query44	1395	868	863	863
query45	176	177	166	166
query46	873	1016	643	643
query47	1746	1783	1716	1716
query48	385	439	313	313
query49	723	520	395	395
query50	664	691	408	408
query51	4153	4059	4043	4043
query52	117	115	104	104
query53	232	273	204	204
query54	611	585	532	532
query55	94	85	87	85
query56	321	341	315	315
query57	1170	1201	1135	1135
query58	278	273	261	261
query59	2660	2672	2570	2570
query60	336	340	327	327
query61	133	125	127	125
query62	838	715	659	659
query63	226	198	187	187
query64	4271	1060	734	734
query65	4320	4150	4214	4150
query66	1131	412	317	317
query67	15551	15314	15251	15251
query68	8720	914	573	573
query69	464	340	288	288
query70	1258	1151	1072	1072
query71	485	333	307	307
query72	5302	4756	4776	4756
query73	736	630	356	356
query74	9308	9008	8890	8890
query75	4008	3086	2658	2658
query76	3701	1153	762	762
query77	808	426	319	319
query78	9565	9772	8803	8803
query79	2046	841	587	587
query80	622	552	490	490
query81	466	253	297	253
query82	437	144	107	107
query83	250	252	237	237
query84	251	99	90	90
query85	792	374	336	336
query86	343	315	298	298
query87	4286	4297	4266	4266
query88	2843	2192	2197	2192
query89	389	312	287	287
query90	1936	217	219	217
query91	143	141	109	109
query92	90	71	69	69
query93	1104	990	642	642
query94	686	423	307	307
query95	391	315	311	311
query96	481	587	278	278
query97	2691	2665	2569	2569
query98	233	217	221	217
query99	1444	1391	1282	1282
Total cold run time: 274276 ms
Total hot run time: 185508 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 32.62 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit b08e2e8d94396188cafa0ebbe67231f613848edd, data reload: false

query1	0.04	0.04	0.03
query2	0.08	0.04	0.04
query3	0.24	0.07	0.07
query4	1.64	0.11	0.11
query5	0.42	0.41	0.39
query6	1.18	0.64	0.64
query7	0.03	0.02	0.02
query8	0.05	0.03	0.03
query9	0.62	0.53	0.52
query10	0.57	0.56	0.56
query11	0.16	0.11	0.11
query12	0.15	0.11	0.11
query13	0.62	0.61	0.61
query14	0.79	0.82	0.84
query15	0.91	0.84	0.85
query16	0.40	0.40	0.39
query17	1.06	1.07	1.01
query18	0.21	0.19	0.20
query19	1.92	1.78	1.82
query20	0.02	0.01	0.01
query21	15.45	0.87	0.57
query22	0.79	1.16	0.74
query23	14.91	1.36	0.64
query24	6.33	2.54	0.81
query25	0.46	0.17	0.23
query26	0.60	0.17	0.13
query27	0.06	0.05	0.05
query28	9.77	0.92	0.43
query29	12.54	3.94	3.25
query30	3.10	3.00	2.96
query31	2.84	0.57	0.37
query32	3.25	0.54	0.47
query33	3.02	3.16	3.15
query34	16.16	5.44	4.86
query35	4.91	4.90	4.99
query36	0.69	0.51	0.49
query37	0.10	0.08	0.07
query38	0.06	0.04	0.05
query39	0.03	0.02	0.02
query40	0.17	0.13	0.13
query41	0.08	0.03	0.03
query42	0.04	0.03	0.03
query43	0.03	0.03	0.04
Total cold run time: 106.5 s
Total hot run time: 32.62 s

@kaijchen
Copy link
Member Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 33745 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 864fa0c9518614eb9aa3fd4bd06b70a6d7c8b726, data reload: false

------ Round 1 ----------------------------------
q1	17578	5225	5087	5087
q2	1908	292	181	181
q3	10311	1245	695	695
q4	10211	1014	520	520
q5	7450	2351	2260	2260
q6	173	161	133	133
q7	896	743	610	610
q8	9311	1275	1106	1106
q9	7269	5082	5086	5082
q10	6889	2372	1977	1977
q11	460	295	265	265
q12	341	347	233	233
q13	17751	3647	3034	3034
q14	230	247	207	207
q15	541	482	478	478
q16	423	418	371	371
q17	566	847	353	353
q18	7417	7108	7119	7108
q19	1092	935	555	555
q20	325	332	222	222
q21	3923	3179	2284	2284
q22	1070	1031	984	984
Total cold run time: 106135 ms
Total hot run time: 33745 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5123	5078	5069	5069
q2	245	323	218	218
q3	2171	2634	2312	2312
q4	1305	1771	1291	1291
q5	4191	4408	4486	4408
q6	215	185	136	136
q7	2039	1882	1800	1800
q8	2647	2544	2581	2544
q9	7395	7238	7336	7238
q10	3099	3316	2830	2830
q11	625	515	530	515
q12	682	777	632	632
q13	3487	4038	3318	3318
q14	292	323	319	319
q15	637	480	469	469
q16	434	487	422	422
q17	1152	1565	1350	1350
q18	7784	7522	7526	7522
q19	801	953	879	879
q20	1971	1995	1930	1930
q21	5044	4277	4271	4271
q22	1062	1028	980	980
Total cold run time: 52401 ms
Total hot run time: 50453 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 185134 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 864fa0c9518614eb9aa3fd4bd06b70a6d7c8b726, data reload: false

query1	977	372	401	372
query2	6519	1772	1662	1662
query3	6735	214	221	214
query4	26271	23202	23366	23202
query5	4391	606	472	472
query6	295	204	194	194
query7	4615	487	277	277
query8	261	229	224	224
query9	8615	2859	2850	2850
query10	475	327	301	301
query11	15813	15043	14865	14865
query12	168	119	116	116
query13	1669	538	412	412
query14	8675	5723	5651	5651
query15	202	191	170	170
query16	7323	624	441	441
query17	1227	725	608	608
query18	1989	420	329	329
query19	188	193	167	167
query20	125	130	125	125
query21	215	125	108	108
query22	4125	4123	4197	4123
query23	34252	33369	33224	33224
query24	8194	2300	2302	2300
query25	547	470	394	394
query26	1235	265	153	153
query27	2757	490	348	348
query28	4327	2224	2201	2201
query29	771	605	450	450
query30	290	220	190	190
query31	859	808	725	725
query32	88	76	74	74
query33	538	371	336	336
query34	776	821	503	503
query35	815	801	735	735
query36	986	1010	932	932
query37	122	103	83	83
query38	4010	4060	4034	4034
query39	1470	1410	1399	1399
query40	217	126	115	115
query41	67	55	53	53
query42	117	112	107	107
query43	485	493	450	450
query44	1331	839	837	837
query45	173	168	165	165
query46	871	985	625	625
query47	1792	1809	1740	1740
query48	388	412	313	313
query49	721	494	419	419
query50	618	695	385	385
query51	4063	4131	4040	4040
query52	110	121	100	100
query53	224	250	198	198
query54	579	576	542	542
query55	84	86	80	80
query56	314	295	300	295
query57	1200	1196	1132	1132
query58	270	266	265	265
query59	2661	2679	2689	2679
query60	343	330	311	311
query61	131	144	134	134
query62	835	738	662	662
query63	224	181	179	179
query64	4357	1053	714	714
query65	4282	4212	4184	4184
query66	1167	403	352	352
query67	15452	15382	15206	15206
query68	8419	910	563	563
query69	472	317	283	283
query70	1251	1149	1165	1149
query71	467	318	297	297
query72	5695	4672	4718	4672
query73	732	608	348	348
query74	9024	9037	8950	8950
query75	3789	3079	2696	2696
query76	3688	1110	729	729
query77	800	389	313	313
query78	9754	9637	8879	8879
query79	2424	841	597	597
query80	615	537	489	489
query81	490	258	217	217
query82	457	142	103	103
query83	254	244	229	229
query84	259	100	95	95
query85	796	382	339	339
query86	386	314	319	314
query87	4405	4201	4239	4201
query88	3394	2215	2160	2160
query89	382	310	277	277
query90	1836	222	218	218
query91	141	139	109	109
query92	90	69	65	65
query93	1702	997	638	638
query94	719	413	312	312
query95	388	319	315	315
query96	479	576	274	274
query97	2622	2686	2551	2551
query98	246	221	205	205
query99	1668	1397	1301	1301
Total cold run time: 273830 ms
Total hot run time: 185134 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 32.46 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 864fa0c9518614eb9aa3fd4bd06b70a6d7c8b726, data reload: false

query1	0.04	0.04	0.04
query2	0.07	0.04	0.04
query3	0.25	0.07	0.07
query4	1.61	0.11	0.12
query5	0.43	0.41	0.40
query6	1.15	0.64	0.66
query7	0.03	0.02	0.01
query8	0.04	0.03	0.03
query9	0.61	0.53	0.52
query10	0.57	0.57	0.56
query11	0.16	0.12	0.11
query12	0.14	0.11	0.12
query13	0.62	0.61	0.60
query14	0.80	0.84	0.84
query15	0.88	0.84	0.87
query16	0.38	0.39	0.39
query17	1.05	1.05	1.03
query18	0.21	0.19	0.20
query19	1.94	1.80	1.85
query20	0.02	0.01	0.01
query21	15.39	0.90	0.55
query22	0.77	1.24	0.75
query23	14.83	1.41	0.62
query24	6.38	1.81	0.78
query25	0.49	0.25	0.06
query26	0.45	0.17	0.12
query27	0.06	0.06	0.05
query28	10.53	0.95	0.44
query29	12.53	3.97	3.30
query30	3.11	3.05	3.00
query31	2.82	0.58	0.38
query32	3.25	0.54	0.46
query33	3.14	3.05	3.11
query34	15.75	5.45	4.83
query35	4.94	4.91	4.98
query36	0.69	0.52	0.49
query37	0.10	0.07	0.06
query38	0.04	0.04	0.03
query39	0.03	0.02	0.03
query40	0.17	0.14	0.14
query41	0.07	0.03	0.03
query42	0.04	0.02	0.03
query43	0.03	0.03	0.03
Total cold run time: 106.61 s
Total hot run time: 32.46 s

@hello-stephen
Copy link
Contributor

FE UT Coverage Report

Increment line coverage 2.33% (1/43) 🎉
Increment coverage report
Complete coverage report

@doris-robot
Copy link

BE UT Coverage Report

Increment line coverage 0.00% (0/10) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 59.03% (16616/28149)
Line Coverage 47.92% (150714/314499)
Region Coverage 36.76% (112939/307260)
Branch Coverage 39.68% (50114/126298)

@dataroaring
Copy link
Contributor

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 33899 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 360675dc56fa4c9049b64ada693f04c83165ee2d, data reload: false

------ Round 1 ----------------------------------
q1	17589	5271	5061	5061
q2	1928	288	182	182
q3	10303	1273	749	749
q4	10231	1015	512	512
q5	7534	2423	2315	2315
q6	173	163	133	133
q7	904	769	622	622
q8	9319	1323	1119	1119
q9	6892	5179	5178	5178
q10	6877	2365	1975	1975
q11	485	289	261	261
q12	349	351	223	223
q13	17756	3629	3055	3055
q14	226	228	216	216
q15	539	480	480	480
q16	431	411	377	377
q17	595	870	365	365
q18	7390	7186	6940	6940
q19	1088	938	563	563
q20	357	343	229	229
q21	3816	3244	2350	2350
q22	1063	1007	994	994
Total cold run time: 105845 ms
Total hot run time: 33899 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5207	5134	5081	5081
q2	229	324	214	214
q3	2157	2711	2310	2310
q4	1367	1798	1338	1338
q5	4187	4460	4636	4460
q6	217	168	128	128
q7	2031	1926	1806	1806
q8	2687	2680	2578	2578
q9	7261	7324	7312	7312
q10	3085	3437	2867	2867
q11	573	526	489	489
q12	738	759	644	644
q13	3428	3883	3316	3316
q14	316	315	331	315
q15	543	471	477	471
q16	450	475	444	444
q17	1194	1578	1429	1429
q18	8016	7860	7560	7560
q19	877	867	1000	867
q20	2007	2097	1899	1899
q21	4855	4390	4320	4320
q22	1111	1075	1035	1035
Total cold run time: 52536 ms
Total hot run time: 50883 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 184716 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 360675dc56fa4c9049b64ada693f04c83165ee2d, data reload: false

query1	983	428	420	420
query2	6511	1743	1698	1698
query3	6745	218	219	218
query4	26414	23590	23201	23201
query5	4354	592	476	476
query6	298	233	200	200
query7	4639	508	292	292
query8	272	224	207	207
query9	8598	2870	2825	2825
query10	463	335	290	290
query11	15951	14898	14789	14789
query12	168	110	116	110
query13	1655	541	425	425
query14	9339	5782	5739	5739
query15	200	188	174	174
query16	7640	637	471	471
query17	1194	686	602	602
query18	2031	405	303	303
query19	191	186	156	156
query20	139	120	113	113
query21	207	116	108	108
query22	4094	4368	4130	4130
query23	34440	33149	33216	33149
query24	8190	2396	2323	2323
query25	529	468	424	424
query26	1237	268	162	162
query27	2732	498	349	349
query28	4350	2232	2221	2221
query29	730	564	449	449
query30	281	218	191	191
query31	923	828	726	726
query32	82	72	75	72
query33	538	369	361	361
query34	814	832	509	509
query35	789	828	734	734
query36	966	999	916	916
query37	117	102	86	86
query38	4093	3995	3968	3968
query39	1498	1419	1440	1419
query40	212	118	111	111
query41	57	60	56	56
query42	119	108	118	108
query43	475	508	476	476
query44	1417	841	851	841
query45	173	163	163	163
query46	847	1001	642	642
query47	1797	1794	1739	1739
query48	376	420	318	318
query49	735	488	389	389
query50	633	687	406	406
query51	4074	4269	4028	4028
query52	115	108	96	96
query53	234	261	209	209
query54	604	577	512	512
query55	86	92	89	89
query56	311	320	290	290
query57	1180	1191	1144	1144
query58	277	264	268	264
query59	2694	2676	2661	2661
query60	347	336	319	319
query61	125	118	122	118
query62	806	709	642	642
query63	226	187	198	187
query64	4432	1017	665	665
query65	4275	4212	4179	4179
query66	1086	418	323	323
query67	15526	15306	15053	15053
query68	8060	909	569	569
query69	465	332	279	279
query70	1185	1156	1066	1066
query71	420	318	308	308
query72	5571	4761	4734	4734
query73	664	616	350	350
query74	9332	8788	8908	8788
query75	3141	3046	2637	2637
query76	3230	1122	717	717
query77	464	397	322	322
query78	9555	9699	8961	8961
query79	2422	804	592	592
query80	656	533	460	460
query81	509	252	213	213
query82	200	137	114	114
query83	246	249	239	239
query84	255	108	82	82
query85	774	420	325	325
query86	382	305	297	297
query87	4206	4239	4076	4076
query88	2950	2219	2227	2219
query89	390	303	292	292
query90	1945	225	220	220
query91	145	146	111	111
query92	83	73	65	65
query93	2251	983	626	626
query94	683	403	310	310
query95	391	314	310	310
query96	478	587	277	277
query97	2608	2664	2626	2626
query98	254	224	202	202
query99	1338	1401	1295	1295
Total cold run time: 272887 ms
Total hot run time: 184716 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 32.66 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 360675dc56fa4c9049b64ada693f04c83165ee2d, data reload: false

query1	0.05	0.04	0.04
query2	0.08	0.04	0.05
query3	0.23	0.08	0.07
query4	1.63	0.11	0.11
query5	0.43	0.42	0.39
query6	1.18	0.63	0.64
query7	0.03	0.02	0.02
query8	0.04	0.03	0.04
query9	0.62	0.52	0.51
query10	0.57	0.57	0.57
query11	0.16	0.11	0.11
query12	0.15	0.11	0.12
query13	0.61	0.61	0.60
query14	0.80	0.84	0.82
query15	0.86	0.86	0.85
query16	0.37	0.39	0.38
query17	1.05	1.03	1.04
query18	0.21	0.20	0.19
query19	1.94	1.84	1.83
query20	0.02	0.02	0.01
query21	15.41	0.90	0.57
query22	0.79	1.08	0.71
query23	14.96	1.35	0.59
query24	6.45	1.89	1.01
query25	0.50	0.10	0.08
query26	0.57	0.17	0.14
query27	0.05	0.05	0.05
query28	10.42	0.90	0.42
query29	12.54	3.89	3.24
query30	3.13	3.06	3.00
query31	2.82	0.57	0.38
query32	3.26	0.55	0.47
query33	3.00	3.04	3.08
query34	16.19	5.46	4.86
query35	4.94	4.91	4.93
query36	0.71	0.51	0.50
query37	0.10	0.07	0.07
query38	0.06	0.04	0.04
query39	0.03	0.03	0.02
query40	0.17	0.15	0.14
query41	0.07	0.03	0.03
query42	0.03	0.02	0.02
query43	0.04	0.03	0.03
Total cold run time: 107.27 s
Total hot run time: 32.66 s

@hello-stephen
Copy link
Contributor

FE UT Coverage Report

Increment line coverage 2.33% (1/43) 🎉
Increment coverage report
Complete coverage report

@hello-stephen
Copy link
Contributor

BE UT Coverage Report

Increment line coverage 0.00% (0/10) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 59.43% (16748/28180)
Line Coverage 48.31% (151943/314537)
Region Coverage 37.29% (115504/309765)
Branch Coverage 40.11% (50824/126715)

@hello-stephen
Copy link
Contributor

BE Regression && UT Coverage Report

Increment line coverage 100.00% (10/10) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 81.88% (22608/27612)
Line Coverage 74.54% (234290/314313)
Region Coverage 61.75% (194532/315008)
Branch Coverage 65.88% (84093/127648)

@kaijchen
Copy link
Member Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 33894 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 749df6d61a23fe017c279ad2866885a2019ecf6a, data reload: false

------ Round 1 ----------------------------------
q1	17568	5283	5117	5117
q2	1927	294	183	183
q3	10317	1277	757	757
q4	10227	1007	535	535
q5	7547	2386	2324	2324
q6	182	160	129	129
q7	896	744	618	618
q8	9334	1305	1106	1106
q9	7007	5044	5138	5044
q10	6894	2385	1989	1989
q11	480	299	276	276
q12	340	348	225	225
q13	17758	3653	3058	3058
q14	235	225	211	211
q15	548	470	493	470
q16	423	427	369	369
q17	579	848	356	356
q18	7216	7030	7015	7015
q19	1221	948	572	572
q20	350	350	225	225
q21	3762	2614	2327	2327
q22	1051	1022	988	988
Total cold run time: 105862 ms
Total hot run time: 33894 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5225	5122	5204	5122
q2	238	321	213	213
q3	2226	2707	2279	2279
q4	1375	1849	1363	1363
q5	4216	4371	4527	4371
q6	212	168	131	131
q7	1978	2010	1837	1837
q8	2714	2624	2621	2621
q9	7205	7340	7359	7340
q10	3090	3324	2839	2839
q11	585	523	490	490
q12	693	788	637	637
q13	3553	3992	3306	3306
q14	284	312	270	270
q15	532	482	480	480
q16	451	494	449	449
q17	1397	1548	1378	1378
q18	7846	7682	7499	7499
q19	870	853	900	853
q20	1932	1988	1862	1862
q21	4734	4366	4259	4259
q22	1089	1041	997	997
Total cold run time: 52445 ms
Total hot run time: 50596 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 185861 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 749df6d61a23fe017c279ad2866885a2019ecf6a, data reload: false

query1	998	373	404	373
query2	6549	1741	1702	1702
query3	6739	217	224	217
query4	26162	23576	23517	23517
query5	4347	609	478	478
query6	301	207	199	199
query7	4631	499	285	285
query8	262	230	232	230
query9	8605	2866	2858	2858
query10	455	335	276	276
query11	15751	15022	15010	15010
query12	166	116	117	116
query13	1652	582	430	430
query14	8682	5945	5918	5918
query15	215	197	167	167
query16	7603	618	537	537
query17	1183	724	590	590
query18	2004	418	308	308
query19	189	185	156	156
query20	139	118	117	117
query21	210	122	104	104
query22	3996	4156	4042	4042
query23	34250	33251	33155	33155
query24	8163	2388	2368	2368
query25	553	478	403	403
query26	1235	272	157	157
query27	2760	505	347	347
query28	4424	2212	2202	2202
query29	768	550	457	457
query30	289	221	184	184
query31	887	780	724	724
query32	85	74	76	74
query33	576	372	338	338
query34	797	822	523	523
query35	824	814	754	754
query36	990	1029	907	907
query37	135	108	87	87
query38	4005	4090	3960	3960
query39	1496	1401	1608	1401
query40	214	123	122	122
query41	60	55	88	55
query42	115	113	111	111
query43	509	496	470	470
query44	1335	848	845	845
query45	176	166	167	166
query46	859	1008	639	639
query47	1716	1806	1702	1702
query48	377	416	313	313
query49	707	495	411	411
query50	638	680	398	398
query51	4059	4030	4059	4030
query52	114	114	103	103
query53	236	259	206	206
query54	591	583	510	510
query55	91	85	94	85
query56	323	310	301	301
query57	1198	1179	1143	1143
query58	271	265	263	263
query59	2699	2692	2509	2509
query60	347	328	332	328
query61	125	119	126	119
query62	820	709	664	664
query63	227	194	192	192
query64	4328	1017	776	776
query65	4278	4206	4187	4187
query66	1169	436	337	337
query67	15359	15251	15246	15246
query68	7897	935	577	577
query69	482	326	289	289
query70	1190	1121	1161	1121
query71	432	352	345	345
query72	5568	4800	4919	4800
query73	677	645	357	357
query74	9155	9086	8938	8938
query75	3714	3094	2627	2627
query76	3427	1130	718	718
query77	789	393	329	329
query78	9552	9696	8811	8811
query79	2429	898	615	615
query80	620	549	471	471
query81	489	258	226	226
query82	498	141	106	106
query83	253	253	236	236
query84	252	102	80	80
query85	840	373	335	335
query86	384	317	303	303
query87	4322	4300	4201	4201
query88	3691	2236	2209	2209
query89	398	310	283	283
query90	1941	225	223	223
query91	147	141	111	111
query92	91	70	70	70
query93	1928	980	620	620
query94	686	387	311	311
query95	392	318	350	318
query96	491	566	280	280
query97	2607	2672	2562	2562
query98	247	213	207	207
query99	1342	1424	1280	1280
Total cold run time: 272948 ms
Total hot run time: 185861 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 32.5 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 749df6d61a23fe017c279ad2866885a2019ecf6a, data reload: false

query1	0.04	0.03	0.03
query2	0.07	0.05	0.04
query3	0.24	0.08	0.07
query4	1.62	0.10	0.10
query5	0.43	0.41	0.41
query6	1.15	0.64	0.65
query7	0.02	0.02	0.02
query8	0.05	0.04	0.04
query9	0.60	0.52	0.52
query10	0.57	0.56	0.56
query11	0.16	0.11	0.11
query12	0.15	0.12	0.12
query13	0.65	0.62	0.61
query14	0.82	0.82	0.84
query15	0.87	0.88	0.88
query16	0.40	0.39	0.40
query17	1.03	1.07	1.08
query18	0.21	0.20	0.21
query19	1.99	1.85	1.81
query20	0.02	0.01	0.01
query21	15.39	0.91	0.55
query22	0.76	1.23	0.90
query23	14.74	1.39	0.63
query24	7.13	1.43	0.52
query25	0.49	0.18	0.10
query26	0.70	0.16	0.13
query27	0.10	0.05	0.05
query28	10.02	0.96	0.45
query29	12.64	3.93	3.24
query30	3.09	3.05	3.00
query31	2.82	0.60	0.38
query32	3.25	0.55	0.47
query33	3.10	3.04	3.20
query34	15.92	5.50	4.90
query35	4.92	4.93	4.86
query36	0.71	0.51	0.50
query37	0.10	0.07	0.06
query38	0.05	0.05	0.04
query39	0.03	0.02	0.02
query40	0.16	0.14	0.15
query41	0.08	0.03	0.02
query42	0.03	0.03	0.03
query43	0.04	0.03	0.02
Total cold run time: 107.36 s
Total hot run time: 32.5 s

@hello-stephen
Copy link
Contributor

BE UT Coverage Report

Increment line coverage 0.00% (0/10) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 59.45% (16756/28184)
Line Coverage 48.33% (152035/314574)
Region Coverage 37.32% (115598/309788)
Branch Coverage 40.13% (50865/126745)

@hello-stephen
Copy link
Contributor

BE Regression && UT Coverage Report

Increment line coverage 50.00% (5/10) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 81.95% (22631/27616)
Line Coverage 74.63% (234594/314350)
Region Coverage 61.95% (195151/315031)
Branch Coverage 66.02% (84290/127678)

Copy link
Contributor

@dataroaring dataroaring left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@dataroaring dataroaring added the usercase Important user case type label label Aug 14, 2025
@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Aug 14, 2025
@github-actions
Copy link
Contributor

PR approved by at least one committer and no changes requested.

@github-actions
Copy link
Contributor

PR approved by anyone and no changes requested.

Copy link
Contributor

@liaoxin01 liaoxin01 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@liaoxin01 liaoxin01 merged commit 8dcb614 into apache:master Aug 14, 2025
30 of 32 checks passed
kaijchen added a commit to kaijchen/doris that referenced this pull request Aug 14, 2025
…e#54606)

Issue Number: DORIS-20541

Related PR: apache#29802

Problem Summary:

Fix an issue where the scanned rows and loaded bytes metrics were not
updated progressively in the FE.
Although the BE periodically reports execution status, the FE was
ignoring these reports due to a change introduced in PR apache#29802, which
skips processing reports without the isDone flag.

This fix ensures that intermediate execution reports are processed,
allowing progressive updates of scanned rows and loaded bytes during
query execution.
kaijchen added a commit to kaijchen/doris that referenced this pull request Aug 14, 2025
…e#54606)

Issue Number: DORIS-20541

Related PR: apache#29802

Problem Summary:

Fix an issue where the scanned rows and loaded bytes metrics were not
updated progressively in the FE.
Although the BE periodically reports execution status, the FE was
ignoring these reports due to a change introduced in PR apache#29802, which
skips processing reports without the isDone flag.

This fix ensures that intermediate execution reports are processed,
allowing progressive updates of scanned rows and loaded bytes during
query execution.
dataroaring pushed a commit that referenced this pull request Aug 15, 2025
…ively #54606 (#54790)

Backport #54606

### What problem does this PR solve?

Issue Number: DORIS-20541

Related PR: #29802

Problem Summary:

Fix an issue where the scanned rows and loaded bytes metrics were not
updated progressively in the FE.
Although the BE periodically reports execution status, the FE was
ignoring these reports due to a change introduced in PR #29802, which
skips processing reports without the isDone flag.

This fix ensures that intermediate execution reports are processed,
allowing progressive updates of scanned rows and loaded bytes during
query execution.
morrySnow pushed a commit that referenced this pull request Aug 15, 2025
…ively #54606 (#54787)

Backport #54606

### What problem does this PR solve?

Issue Number: DORIS-20541

Related PR: #29802

Problem Summary:

Fix an issue where the scanned rows and loaded bytes metrics were not
updated progressively in the FE.
Although the BE periodically reports execution status, the FE was
ignoring these reports due to a change introduced in PR #29802, which
skips processing reports without the isDone flag.

This fix ensures that intermediate execution reports are processed,
allowing progressive updates of scanned rows and loaded bytes during
query execution.
@gavinchou gavinchou mentioned this pull request Sep 1, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by one committer. dev/2.1.x dev/2.1.x-conflict dev/3.0.8-merged dev/3.1.0-merged reviewed usercase Important user case type label

Projects

None yet

Development

Successfully merging this pull request may close these issues.

7 participants