[opt](filecache) Let base compaction output rowsets write filecache when cache size is enough#53801

swjtu-zhanglei · 2025-07-23T14:47:21Z

What problem does this PR solve?

Issue Number: close #xxx

Related PR: #xxx

Problem Summary:

Release note

None

Check List (For Author)

Test
- Regression test
- Unit Test
- Manual test (add detailed scripts or steps below)
- No need to test or manual test. Explain why:
  - This is a refactor/code format and no logic has been changed.
  - Previous test can cover this change.
  - No code files have been changed.
  - Other reason
Behavior changed:
- No.
- Yes.
Does this need documentation?
- No.
- Yes.

Check List (For Reviewer who merge this PR)

Confirm the release note
Confirm test cases
Confirm document
Add branch pick label

hello-stephen · 2025-07-23T14:47:27Z

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

What problem was fixed (it's best to include specific error reporting information). How it was fixed.
Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
What features were added. Why was this function added?
Which code was refactored and why was this part of the code refactored?
Which functions were optimized and what is the difference before and after the optimization?

swjtu-zhanglei · 2025-07-23T14:47:32Z

run buildall

dataroaring

Add a config for this feature, we can remove it in the future.

doris-robot · 2025-07-23T16:12:25Z

TPC-H: Total hot run time: 34369 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit f78aa824e59306da7aeaac9a7a8a65efd2b2db37, data reload: false

------ Round 1 ----------------------------------
q1	17632	5274	5081	5081
q2	1941	290	184	184
q3	10307	1336	744	744
q4	10214	1025	534	534
q5	7473	2529	2388	2388
q6	189	164	130	130
q7	919	763	615	615
q8	9314	1363	1125	1125
q9	7536	5148	5213	5148
q10	6894	2389	1960	1960
q11	485	294	278	278
q12	366	376	226	226
q13	17753	3769	3124	3124
q14	238	230	226	226
q15	548	470	493	470
q16	435	438	387	387
q17	595	895	377	377
q18	7690	7157	7229	7157
q19	1226	968	596	596
q20	355	353	220	220
q21	3969	3394	2437	2437
q22	1058	1026	962	962
Total cold run time: 107137 ms
Total hot run time: 34369 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5194	5177	5125	5125
q2	234	320	215	215
q3	2231	2686	2304	2304
q4	1351	1788	1377	1377
q5	4521	4638	4603	4603
q6	225	174	130	130
q7	2063	1974	1807	1807
q8	2741	2702	2579	2579
q9	7291	7274	7262	7262
q10	3086	3320	2937	2937
q11	581	509	513	509
q12	743	782	621	621
q13	3704	4017	3491	3491
q14	287	307	286	286
q15	533	486	472	472
q16	451	509	468	468
q17	1159	1571	1435	1435
q18	8021	7694	7660	7660
q19	906	860	1021	860
q20	2020	2045	1823	1823
q21	5003	4454	4290	4290
q22	1068	1013	997	997
Total cold run time: 53413 ms
Total hot run time: 51251 ms

doris-robot · 2025-07-23T16:24:05Z

TPC-DS: Total hot run time: 188081 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit f78aa824e59306da7aeaac9a7a8a65efd2b2db37, data reload: false

query1	986	402	413	402
query2	6520	1717	1734	1717
query3	6749	232	218	218
query4	26281	23260	23427	23260
query5	4359	624	506	506
query6	316	227	211	211
query7	4629	521	313	313
query8	277	231	226	226
query9	8599	2918	2919	2918
query10	471	322	278	278
query11	15745	15018	15254	15018
query12	161	110	116	110
query13	1646	524	399	399
query14	8580	5912	5925	5912
query15	220	191	166	166
query16	7147	656	480	480
query17	1218	734	602	602
query18	2008	420	327	327
query19	194	196	175	175
query20	125	119	125	119
query21	216	123	114	114
query22	4040	4203	4027	4027
query23	34025	33112	33011	33011
query24	8226	2448	2396	2396
query25	558	469	418	418
query26	1239	271	164	164
query27	2761	512	353	353
query28	4332	2260	2246	2246
query29	780	579	448	448
query30	282	224	191	191
query31	918	858	768	768
query32	84	78	75	75
query33	562	403	327	327
query34	819	846	538	538
query35	802	824	720	720
query36	971	1016	924	924
query37	123	103	85	85
query38	4242	4165	4018	4018
query39	1506	1482	1411	1411
query40	214	126	115	115
query41	58	55	66	55
query42	121	110	116	110
query43	488	502	464	464
query44	1406	890	868	868
query45	176	169	169	169
query46	848	1010	654	654
query47	1792	1839	1753	1753
query48	398	427	313	313
query49	736	470	386	386
query50	651	692	429	429
query51	5507	5701	5364	5364
query52	121	114	107	107
query53	243	287	189	189
query54	593	591	533	533
query55	91	92	86	86
query56	337	319	298	298
query57	1175	1178	1142	1142
query58	290	274	276	274
query59	2530	2754	2506	2506
query60	340	337	334	334
query61	130	118	120	118
query62	788	727	648	648
query63	234	189	199	189
query64	4379	1057	702	702
query65	4277	4182	4151	4151
query66	1167	436	341	341
query67	15828	15542	15331	15331
query68	7553	924	630	630
query69	508	328	295	295
query70	1205	1117	1131	1117
query71	420	330	330	330
query72	5651	4846	4987	4846
query73	702	689	363	363
query74	9200	9179	8924	8924
query75	3132	3084	2659	2659
query76	3182	1242	771	771
query77	482	389	336	336
query78	10001	10038	9233	9233
query79	2434	892	604	604
query80	605	530	479	479
query81	494	261	225	225
query82	201	135	106	106
query83	251	250	236	236
query84	240	109	88	88
query85	757	363	316	316
query86	382	304	310	304
query87	4452	4517	4320	4320
query88	3768	2326	2297	2297
query89	386	318	282	282
query90	1975	232	222	222
query91	140	135	113	113
query92	85	76	66	66
query93	1993	1005	643	643
query94	682	407	325	325
query95	429	319	312	312
query96	489	575	282	282
query97	2697	2735	2657	2657
query98	251	215	210	210
query99	1327	1399	1270	1270
Total cold run time: 273709 ms
Total hot run time: 188081 ms

doris-robot · 2025-07-23T16:29:31Z

ClickBench: Total hot run time: 32.01 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit f78aa824e59306da7aeaac9a7a8a65efd2b2db37, data reload: false

query1	0.05	0.04	0.03
query2	0.07	0.04	0.04
query3	0.24	0.07	0.07
query4	1.62	0.11	0.11
query5	0.43	0.43	0.41
query6	1.15	0.65	0.66
query7	0.03	0.02	0.02
query8	0.05	0.03	0.04
query9	0.59	0.53	0.52
query10	0.58	0.59	0.57
query11	0.16	0.11	0.10
query12	0.15	0.12	0.12
query13	0.64	0.61	0.61
query14	0.81	0.82	0.82
query15	0.89	0.87	0.86
query16	0.38	0.38	0.39
query17	1.07	1.03	1.05
query18	0.22	0.21	0.20
query19	1.95	1.80	1.78
query20	0.02	0.01	0.02
query21	15.50	0.86	0.53
query22	0.79	1.05	0.69
query23	15.00	1.39	0.65
query24	6.78	1.34	0.36
query25	0.32	0.25	0.07
query26	0.59	0.19	0.14
query27	0.06	0.05	0.05
query28	8.90	0.89	0.42
query29	12.60	3.83	3.29
query30	3.11	3.10	3.04
query31	2.82	0.58	0.40
query32	3.24	0.55	0.49
query33	3.04	3.10	3.04
query34	15.97	5.41	4.83
query35	4.87	4.81	4.85
query36	0.68	0.50	0.50
query37	0.10	0.06	0.06
query38	0.05	0.04	0.04
query39	0.03	0.03	0.03
query40	0.18	0.14	0.14
query41	0.08	0.02	0.03
query42	0.04	0.02	0.03
query43	0.05	0.03	0.04
Total cold run time: 105.9 s
Total hot run time: 32.01 s

hello-stephen · 2025-07-23T17:51:20Z

BE UT Coverage Report

Increment line coverage 19.05% (4/21) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	57.48% (15924/27703)
Line Coverage	46.28% (143044/309095)
Region Coverage	35.76% (107813/301469)
Branch Coverage	38.24% (47542/124313)

hello-stephen · 2025-07-23T18:34:02Z

BE Regression && UT Coverage Report

Increment line coverage 100.00% (21/21) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	81.12% (22057/27189)
Line Coverage	73.80% (227771/308624)
Region Coverage	61.45% (189999/309191)
Branch Coverage	65.25% (82025/125701)

swjtu-zhanglei · 2025-07-24T09:30:26Z

Add a config for this feature, we can remove it in the future.

we already have a config for force write filecache, how do you think ? @gavinchou

gavinchou · 2025-07-24T12:42:55Z

be/src/io/fs/file_writer.h

-    bool sync_file_data = true;         // Whether flush data into storage system
-    uint64_t file_cache_expiration = 0; // Absolute time
+    bool sync_file_data = true;           // Whether flush data into storage system
+    uint64_t file_cache_expiration = 0;   // Absolute time


naming file_cache_expired_at_s

naming file_cache_expired_at_s

没改过这个字段，只是增加了注释缩进，因为clang-format报错的原因

swjtu-zhanglei · 2025-07-24T14:21:17Z

run buildall

doris-robot · 2025-07-24T15:43:25Z

TPC-H: Total hot run time: 34009 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 8737d54652c56b15ee6fa0b12466a858af57c933, data reload: false

------ Round 1 ----------------------------------
q1	17560	5227	5058	5058
q2	1923	296	177	177
q3	10326	1277	732	732
q4	10219	999	544	544
q5	7508	2416	2311	2311
q6	175	158	129	129
q7	887	752	598	598
q8	9302	1311	1087	1087
q9	7103	5151	5179	5151
q10	6877	2398	1949	1949
q11	474	278	272	272
q12	346	355	218	218
q13	17768	3668	3091	3091
q14	237	249	209	209
q15	533	484	476	476
q16	428	439	382	382
q17	594	891	370	370
q18	7527	7181	7247	7181
q19	1164	953	551	551
q20	351	352	230	230
q21	3990	2659	2365	2365
q22	1072	1053	928	928
Total cold run time: 106364 ms
Total hot run time: 34009 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5136	5040	5124	5040
q2	245	332	221	221
q3	2188	2634	2311	2311
q4	1419	1792	1327	1327
q5	4205	4481	4635	4481
q6	214	170	132	132
q7	2028	1923	1786	1786
q8	2631	2665	2613	2613
q9	7351	7427	7298	7298
q10	3088	3280	2857	2857
q11	588	530	497	497
q12	680	773	633	633
q13	3543	4166	3298	3298
q14	265	301	263	263
q15	544	486	478	478
q16	509	495	450	450
q17	1184	1536	1418	1418
q18	8096	7769	7612	7612
q19	843	840	903	840
q20	2022	2044	1938	1938
q21	5061	4306	4294	4294
q22	1099	1016	976	976
Total cold run time: 52939 ms
Total hot run time: 50763 ms

doris-robot · 2025-07-24T15:55:08Z

TPC-DS: Total hot run time: 187211 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 8737d54652c56b15ee6fa0b12466a858af57c933, data reload: false

query1	998	387	422	387
query2	6509	1700	1663	1663
query3	6741	228	224	224
query4	26351	23767	22872	22872
query5	4349	633	500	500
query6	315	227	186	186
query7	4630	492	289	289
query8	258	233	202	202
query9	8553	2931	2891	2891
query10	469	328	279	279
query11	15951	14914	14716	14716
query12	167	113	117	113
query13	1688	556	443	443
query14	8591	5808	5826	5808
query15	206	194	170	170
query16	7122	643	445	445
query17	961	706	582	582
query18	1972	413	312	312
query19	188	183	160	160
query20	124	118	117	117
query21	203	122	104	104
query22	4076	4297	4125	4125
query23	33823	33030	32855	32855
query24	8204	2397	2368	2368
query25	526	461	448	448
query26	1231	273	189	189
query27	2728	501	349	349
query28	4389	2238	2241	2238
query29	808	563	447	447
query30	285	216	191	191
query31	894	807	768	768
query32	79	80	72	72
query33	555	363	332	332
query34	788	854	512	512
query35	788	826	749	749
query36	1004	989	921	921
query37	126	112	87	87
query38	4164	4123	4044	4044
query39	1462	1433	1411	1411
query40	227	130	122	122
query41	69	63	61	61
query42	124	114	111	111
query43	491	494	469	469
query44	1369	882	867	867
query45	178	173	164	164
query46	851	1006	652	652
query47	1826	1811	1802	1802
query48	395	434	325	325
query49	730	488	401	401
query50	679	707	420	420
query51	5517	5520	5393	5393
query52	116	111	101	101
query53	231	266	187	187
query54	616	617	550	550
query55	92	89	88	88
query56	336	324	325	324
query57	1196	1212	1142	1142
query58	287	282	277	277
query59	2466	2604	2613	2604
query60	361	334	323	323
query61	127	122	124	122
query62	784	764	661	661
query63	221	183	193	183
query64	4379	1038	766	766
query65	4275	4205	4158	4158
query66	1167	430	340	340
query67	16112	15684	15458	15458
query68	8563	913	576	576
query69	474	307	292	292
query70	1250	1188	1040	1040
query71	447	379	318	318
query72	5778	4878	5125	4878
query73	785	715	366	366
query74	9205	9005	8948	8948
query75	3895	3078	2606	2606
query76	3626	1126	722	722
query77	795	376	314	314
query78	9979	10066	9266	9266
query79	5156	815	610	610
query80	686	531	467	467
query81	461	256	218	218
query82	720	134	111	111
query83	286	251	236	236
query84	291	106	84	84
query85	780	365	338	338
query86	346	311	285	285
query87	4306	4454	4372	4372
query88	2858	2260	2249	2249
query89	452	318	292	292
query90	1906	213	215	213
query91	149	141	106	106
query92	83	72	66	66
query93	2720	964	638	638
query94	680	399	289	289
query95	394	314	307	307
query96	483	592	280	280
query97	2716	2710	2596	2596
query98	233	216	214	214
query99	1450	1379	1274	1274
Total cold run time: 279451 ms
Total hot run time: 187211 ms

doris-robot · 2025-07-24T16:00:37Z

ClickBench: Total hot run time: 32.09 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 8737d54652c56b15ee6fa0b12466a858af57c933, data reload: false

query1	0.04	0.04	0.04
query2	0.08	0.04	0.04
query3	0.24	0.07	0.07
query4	1.64	0.11	0.11
query5	0.41	0.43	0.41
query6	1.16	0.65	0.68
query7	0.03	0.02	0.02
query8	0.05	0.03	0.03
query9	0.59	0.51	0.53
query10	0.58	0.57	0.57
query11	0.16	0.11	0.11
query12	0.14	0.11	0.12
query13	0.61	0.62	0.61
query14	0.79	0.83	0.81
query15	0.88	0.88	0.87
query16	0.39	0.38	0.40
query17	1.06	1.04	1.07
query18	0.21	0.20	0.20
query19	2.00	1.79	1.84
query20	0.01	0.01	0.01
query21	15.42	0.88	0.53
query22	0.78	1.18	0.75
query23	14.77	1.37	0.63
query24	6.79	1.54	0.41
query25	0.55	0.24	0.16
query26	0.68	0.16	0.15
query27	0.07	0.05	0.05
query28	9.61	0.91	0.43
query29	12.66	3.94	3.23
query30	3.06	3.05	2.97
query31	2.82	0.60	0.37
query32	3.23	0.54	0.47
query33	3.01	3.15	3.04
query34	15.93	5.49	4.83
query35	4.89	4.90	4.88
query36	0.69	0.50	0.48
query37	0.10	0.07	0.06
query38	0.05	0.04	0.04
query39	0.03	0.02	0.03
query40	0.18	0.15	0.14
query41	0.07	0.02	0.02
query42	0.04	0.02	0.02
query43	0.03	0.04	0.03
Total cold run time: 106.53 s
Total hot run time: 32.09 s

doris-robot · 2025-07-24T17:35:38Z

BE UT Coverage Report

Increment line coverage 21.05% (4/19) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	57.55% (15936/27691)
Line Coverage	46.34% (143269/309138)
Region Coverage	35.79% (107991/301729)
Branch Coverage	38.30% (47642/124392)

hello-stephen · 2025-07-24T18:22:35Z

BE Regression && UT Coverage Report

Increment line coverage 100.00% (19/19) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	81.08% (22041/27185)
Line Coverage	73.71% (227553/308714)
Region Coverage	61.37% (189937/309481)
Branch Coverage	65.13% (81932/125800)

swjtu-zhanglei · 2025-07-25T02:40:21Z

run buildall

swjtu-zhanglei · 2025-07-25T02:41:03Z

add a config enable_file_cache_adaptive_write for be

doris-robot · 2025-07-25T04:08:06Z

TPC-H: Total hot run time: 34501 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 2d40e65fe7569f889944786394843e44b0b39a2e, data reload: false

------ Round 1 ----------------------------------
q1	17593	5271	5148	5148
q2	1940	285	174	174
q3	10336	1293	712	712
q4	10217	1043	530	530
q5	7545	2443	2353	2353
q6	184	157	131	131
q7	898	756	602	602
q8	9331	1416	1175	1175
q9	7604	5159	5164	5159
q10	6888	2371	1984	1984
q11	485	293	284	284
q12	348	368	222	222
q13	17784	3709	3183	3183
q14	250	244	212	212
q15	541	485	480	480
q16	437	450	381	381
q17	625	895	372	372
q18	7555	7291	7227	7227
q19	1211	961	563	563
q20	337	350	231	231
q21	4063	3182	2410	2410
q22	1079	1028	968	968
Total cold run time: 107251 ms
Total hot run time: 34501 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5239	5475	5164	5164
q2	245	328	212	212
q3	2168	2693	2306	2306
q4	1391	1804	1359	1359
q5	4502	4625	4524	4524
q6	218	172	128	128
q7	1998	1980	1822	1822
q8	2705	2610	2525	2525
q9	7305	7239	7355	7239
q10	3086	3300	2858	2858
q11	575	494	496	494
q12	693	766	644	644
q13	3716	3949	3463	3463
q14	297	319	307	307
q15	559	497	488	488
q16	451	534	436	436
q17	1264	1614	1392	1392
q18	8027	7686	7608	7608
q19	880	805	895	805
q20	1918	1956	1794	1794
q21	4796	4434	4300	4300
q22	1116	1035	1002	1002
Total cold run time: 53149 ms
Total hot run time: 50870 ms

doris-robot · 2025-07-25T04:19:50Z

TPC-DS: Total hot run time: 188064 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 2d40e65fe7569f889944786394843e44b0b39a2e, data reload: false

query1	996	393	397	393
query2	6531	1650	1596	1596
query3	6740	222	218	218
query4	26476	23736	23049	23049
query5	4339	665	533	533
query6	333	210	200	200
query7	4631	519	289	289
query8	291	226	241	226
query9	8592	2967	2934	2934
query10	482	355	299	299
query11	15972	15000	14855	14855
query12	166	119	116	116
query13	1666	569	430	430
query14	9149	6034	5876	5876
query15	223	193	175	175
query16	7225	638	418	418
query17	1182	719	612	612
query18	1975	412	304	304
query19	191	189	175	175
query20	130	115	113	113
query21	214	123	107	107
query22	4205	4178	4146	4146
query23	33966	32864	32948	32864
query24	8129	2451	2431	2431
query25	564	469	426	426
query26	1235	275	156	156
query27	2750	511	353	353
query28	4361	2250	2222	2222
query29	771	586	464	464
query30	292	227	198	198
query31	928	836	771	771
query32	80	81	76	76
query33	544	397	342	342
query34	811	846	523	523
query35	808	822	743	743
query36	978	1019	882	882
query37	120	107	87	87
query38	4133	4169	4098	4098
query39	1482	1411	1410	1410
query40	261	127	120	120
query41	62	58	55	55
query42	122	116	116	116
query43	534	500	473	473
query44	1414	867	872	867
query45	179	174	174	174
query46	894	1022	662	662
query47	1778	1846	1742	1742
query48	397	464	323	323
query49	719	491	386	386
query50	678	706	418	418
query51	5510	5599	5437	5437
query52	113	110	104	104
query53	238	273	192	192
query54	617	593	540	540
query55	91	92	89	89
query56	316	345	325	325
query57	1180	1192	1133	1133
query58	281	288	279	279
query59	2505	2679	2569	2569
query60	360	346	333	333
query61	130	123	130	123
query62	823	725	645	645
query63	226	195	198	195
query64	4319	1041	789	789
query65	4285	4170	4170	4170
query66	1157	424	339	339
query67	15842	15378	15549	15378
query68	6299	937	570	570
query69	482	338	299	299
query70	1227	1210	1160	1160
query71	403	328	307	307
query72	5820	4678	4677	4677
query73	639	599	361	361
query74	9267	9062	9082	9062
query75	3167	3111	2716	2716
query76	3213	1151	782	782
query77	476	400	331	331
query78	9884	10061	9291	9291
query79	2153	876	610	610
query80	675	562	486	486
query81	498	250	227	227
query82	194	142	111	111
query83	257	259	240	240
query84	249	110	92	92
query85	772	382	339	339
query86	380	310	284	284
query87	4387	4381	4347	4347
query88	2945	2365	2284	2284
query89	390	327	289	289
query90	1961	226	226	226
query91	144	142	112	112
query92	98	74	69	69
query93	1769	1005	647	647
query94	689	405	303	303
query95	409	323	320	320
query96	496	612	283	283
query97	2671	2730	2669	2669
query98	237	210	212	210
query99	1327	1446	1274	1274
Total cold run time: 272270 ms
Total hot run time: 188064 ms

doris-robot · 2025-07-25T04:25:18Z

ClickBench: Total hot run time: 32.79 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 2d40e65fe7569f889944786394843e44b0b39a2e, data reload: false

query1	0.04	0.03	0.04
query2	0.08	0.04	0.04
query3	0.25	0.07	0.07
query4	1.63	0.11	0.11
query5	0.41	0.40	0.40
query6	1.16	0.64	0.67
query7	0.03	0.02	0.02
query8	0.05	0.04	0.03
query9	0.60	0.52	0.52
query10	0.59	0.56	0.57
query11	0.16	0.12	0.11
query12	0.15	0.12	0.12
query13	0.63	0.61	0.61
query14	0.80	0.82	0.84
query15	0.89	0.87	0.86
query16	0.39	0.38	0.39
query17	1.07	1.05	1.07
query18	0.21	0.20	0.20
query19	2.00	1.88	1.81
query20	0.01	0.01	0.02
query21	15.38	0.90	0.54
query22	0.78	1.24	0.69
query23	14.85	1.39	0.64
query24	6.52	1.98	1.04
query25	0.44	0.19	0.19
query26	0.58	0.16	0.13
query27	0.05	0.05	0.05
query28	9.20	0.95	0.44
query29	12.54	3.98	3.31
query30	3.12	2.97	2.99
query31	2.82	0.58	0.38
query32	3.23	0.56	0.48
query33	3.12	3.03	3.09
query34	16.04	5.47	4.84
query35	4.85	4.86	4.90
query36	0.69	0.51	0.49
query37	0.10	0.07	0.06
query38	0.05	0.04	0.03
query39	0.03	0.03	0.02
query40	0.17	0.14	0.14
query41	0.08	0.02	0.03
query42	0.04	0.02	0.03
query43	0.04	0.04	0.03
Total cold run time: 105.87 s
Total hot run time: 32.79 s

doris-robot · 2025-08-11T12:13:50Z

TPC-H: Total hot run time: 34131 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit c720f15118715f4def89af9739b9ab453d8f056d, data reload: false

------ Round 1 ----------------------------------
q1	17566	5305	5183	5183
q2	1946	287	172	172
q3	10304	1298	718	718
q4	10209	1017	530	530
q5	7483	2435	2397	2397
q6	187	168	130	130
q7	908	761	625	625
q8	9311	1291	1069	1069
q9	7110	5139	5167	5139
q10	6900	2399	1986	1986
q11	490	286	263	263
q12	342	350	220	220
q13	17786	3718	3025	3025
q14	226	240	212	212
q15	556	483	488	483
q16	443	423	368	368
q17	594	871	360	360
q18	7738	7101	7189	7101
q19	1092	975	578	578
q20	344	347	234	234
q21	3760	3208	2366	2366
q22	1064	1039	972	972
Total cold run time: 106359 ms
Total hot run time: 34131 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5162	5146	5117	5117
q2	249	323	222	222
q3	2217	2660	2418	2418
q4	1371	1774	1337	1337
q5	4255	4616	4537	4537
q6	222	171	130	130
q7	2025	1947	1798	1798
q8	2695	2714	2631	2631
q9	7333	7334	7319	7319
q10	3152	3390	2858	2858
q11	575	504	492	492
q12	731	781	623	623
q13	3557	3875	3216	3216
q14	305	292	452	292
q15	603	480	473	473
q16	459	504	440	440
q17	1192	1548	1408	1408
q18	7875	7722	7496	7496
q19	863	846	850	846
q20	2018	2053	1998	1998
q21	5175	4618	4303	4303
q22	1070	1031	997	997
Total cold run time: 53104 ms
Total hot run time: 50951 ms

doris-robot · 2025-08-11T12:25:26Z

TPC-DS: Total hot run time: 185281 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit c720f15118715f4def89af9739b9ab453d8f056d, data reload: false

query1	984	393	432	393
query2	6533	1743	1715	1715
query3	6747	223	214	214
query4	26851	23947	23074	23074
query5	5033	617	501	501
query6	315	223	220	220
query7	4644	494	293	293
query8	289	244	240	240
query9	8646	2907	2889	2889
query10	521	336	297	297
query11	15871	14990	14779	14779
query12	174	118	117	117
query13	1697	542	420	420
query14	9650	5914	5893	5893
query15	192	186	162	162
query16	7641	646	498	498
query17	1157	690	596	596
query18	2040	409	308	308
query19	182	179	160	160
query20	128	124	115	115
query21	218	119	105	105
query22	4136	4222	3980	3980
query23	34141	33342	33127	33127
query24	8114	2360	2384	2360
query25	516	481	423	423
query26	1227	269	154	154
query27	2753	501	332	332
query28	4356	2243	2222	2222
query29	743	599	444	444
query30	291	220	187	187
query31	875	826	738	738
query32	84	78	80	78
query33	553	387	329	329
query34	803	844	522	522
query35	817	816	766	766
query36	991	988	937	937
query37	117	105	85	85
query38	4082	3990	3963	3963
query39	1442	1429	1431	1429
query40	221	130	119	119
query41	66	56	57	56
query42	120	111	106	106
query43	502	518	488	488
query44	1349	867	859	859
query45	177	170	165	165
query46	849	1017	644	644
query47	1742	1798	1712	1712
query48	401	404	317	317
query49	729	516	398	398
query50	634	702	397	397
query51	4052	4081	4066	4066
query52	110	115	126	115
query53	235	267	196	196
query54	587	598	532	532
query55	88	86	89	86
query56	316	319	300	300
query57	1148	1199	1120	1120
query58	281	269	265	265
query59	2591	2703	2661	2661
query60	329	345	320	320
query61	136	126	125	125
query62	811	729	666	666
query63	236	189	191	189
query64	4283	1031	678	678
query65	4291	4255	4235	4235
query66	1085	412	326	326
query67	15523	15390	15301	15301
query68	8046	918	578	578
query69	496	316	283	283
query70	1171	1117	1148	1117
query71	464	334	322	322
query72	5516	4900	4866	4866
query73	697	658	353	353
query74	8970	9120	8780	8780
query75	3366	3198	2601	2601
query76	3408	1136	737	737
query77	776	408	323	323
query78	9512	9632	8768	8768
query79	2508	826	577	577
query80	607	552	472	472
query81	546	253	222	222
query82	186	144	110	110
query83	251	245	237	237
query84	248	100	90	90
query85	758	372	332	332
query86	378	307	318	307
query87	4314	4399	4140	4140
query88	3845	2222	2220	2220
query89	397	303	293	293
query90	1981	232	229	229
query91	140	140	120	120
query92	92	73	71	71
query93	1995	968	632	632
query94	663	402	307	307
query95	378	317	310	310
query96	484	576	288	288
query97	2602	2700	2574	2574
query98	241	219	216	216
query99	1309	1426	1299	1299
Total cold run time: 274791 ms
Total hot run time: 185281 ms

doris-robot · 2025-08-11T12:30:54Z

ClickBench: Total hot run time: 32.33 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit c720f15118715f4def89af9739b9ab453d8f056d, data reload: false

query1	0.04	0.03	0.04
query2	0.07	0.05	0.04
query3	0.24	0.07	0.07
query4	1.63	0.11	0.11
query5	0.43	0.41	0.40
query6	1.17	0.64	0.65
query7	0.02	0.02	0.02
query8	0.04	0.03	0.04
query9	0.60	0.52	0.51
query10	0.57	0.57	0.59
query11	0.16	0.12	0.11
query12	0.15	0.12	0.11
query13	0.62	0.60	0.61
query14	0.81	0.81	0.85
query15	0.89	0.84	0.85
query16	0.38	0.39	0.40
query17	1.06	1.05	1.02
query18	0.21	0.19	0.20
query19	1.96	1.85	1.83
query20	0.01	0.00	0.01
query21	15.40	0.94	0.58
query22	0.79	1.14	0.80
query23	14.77	1.38	0.65
query24	6.47	1.36	0.56
query25	0.54	0.29	0.08
query26	0.62	0.16	0.12
query27	0.05	0.06	0.05
query28	9.48	0.94	0.43
query29	12.71	3.92	3.23
query30	3.10	3.06	3.00
query31	2.82	0.57	0.38
query32	3.23	0.56	0.48
query33	3.14	3.07	3.27
query34	15.89	5.42	4.85
query35	4.91	4.91	4.94
query36	0.72	0.50	0.50
query37	0.10	0.07	0.06
query38	0.06	0.04	0.04
query39	0.03	0.02	0.02
query40	0.16	0.15	0.13
query41	0.08	0.03	0.02
query42	0.03	0.03	0.02
query43	0.03	0.03	0.03
Total cold run time: 106.19 s
Total hot run time: 32.33 s

freemandealer

LGTM

doris-robot · 2025-08-11T13:27:48Z

BE UT Coverage Report

Increment line coverage 43.33% (13/30) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	59.02% (16607/28139)
Line Coverage	47.88% (150520/314358)
Region Coverage	36.70% (112713/307085)
Branch Coverage	39.64% (50018/126192)

hello-stephen · 2025-08-11T15:10:58Z

BE Regression && UT Coverage Report

Increment line coverage 100.00% (30/30) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	81.55% (22519/27614)
Line Coverage	74.20% (233241/314357)
Region Coverage	61.48% (193884/315347)
Branch Coverage	65.52% (83692/127740)

swjtu-zhanglei · 2025-08-12T01:55:54Z

run cloud_p0

hello-stephen · 2025-08-12T04:01:48Z

BE Regression && UT Coverage Report

Increment line coverage 100.00% (30/30) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	81.55% (22519/27614)
Line Coverage	74.20% (233241/314357)
Region Coverage	61.48% (193884/315347)
Branch Coverage	65.52% (83692/127740)

dataroaring

LGTM

github-actions · 2025-08-13T04:19:20Z

PR approved by at least one committer and no changes requested.

github-actions · 2025-08-13T04:19:22Z

PR approved by anyone and no changes requested.

…hen cache size is enough (apache#53801)

… filecache when cache size is enough (#53801) (#54693)

… filecache when cache size is enough #53801 (#54692) picked from #53801

dataroaring reviewed Jul 23, 2025

View reviewed changes

swjtu-zhanglei force-pushed the master-fix12 branch from f78aa82 to a91aacd Compare July 24, 2025 09:25

swjtu-zhanglei marked this pull request as ready for review July 24, 2025 09:25

swjtu-zhanglei requested a review from gavinchou as a code owner July 24, 2025 09:25

swjtu-zhanglei force-pushed the master-fix12 branch from a91aacd to 8737d54 Compare July 24, 2025 09:28

dataroaring added dev/3.0.x dev/3.1.x usercase Important user case type label labels Jul 24, 2025

gavinchou reviewed Jul 24, 2025

View reviewed changes

swjtu-zhanglei changed the title ~~[opt](filecache) Let base compaction output rowsets write filecache w…~~ [opt](filecache) Let base compaction output rowsets write filecache when cache size is enough Jul 25, 2025

swjtu-zhanglei force-pushed the master-fix12 branch from 8737d54 to 2d40e65 Compare July 25, 2025 02:39

freemandealer approved these changes Aug 11, 2025

View reviewed changes

dataroaring approved these changes Aug 13, 2025

View reviewed changes

gavinchou approved these changes Aug 13, 2025

View reviewed changes

gavinchou merged commit fe7f709 into apache:master Aug 13, 2025
28 of 32 checks passed

github-actions bot added the approved Indicates a PR has been approved by one committer. label Aug 13, 2025

github-actions bot added the reviewed label Aug 13, 2025

github-actions bot added dev/3.0.x-conflict dev/3.1.x-conflict labels Aug 13, 2025

swjtu-zhanglei added a commit to swjtu-zhanglei/incubator-doris that referenced this pull request Aug 13, 2025

[opt](filecache) Let base compaction output rowsets write filecache w…

82de25c

…hen cache size is enough (apache#53801)

swjtu-zhanglei added a commit to swjtu-zhanglei/incubator-doris that referenced this pull request Aug 13, 2025

[opt](filecache) Let base compaction output rowsets write filecache w…

0ac03f9

…hen cache size is enough (apache#53801)

dataroaring pushed a commit that referenced this pull request Aug 14, 2025

branch-3.0: [opt](filecache) Let base compaction output rowsets write…

2a23e4f

… filecache when cache size is enough (#53801) (#54693)

dataroaring mentioned this pull request Aug 14, 2025

branch-3.0: [opt](filecache) Let base compaction output rowsets write filecache when cache size is enough (#53801) #54693

Merged

16 tasks

dataroaring added dev/3.0.8-merged and removed dev/3.0.x dev/3.0.x-conflict labels Aug 14, 2025

morrySnow mentioned this pull request Aug 14, 2025

branch-3.1: [opt](filecache) Let base compaction output rowsets write filecache when cache size is enough #53801 #54692

Merged

morrySnow pushed a commit that referenced this pull request Aug 14, 2025

branch-3.1: [opt](filecache) Let base compaction output rowsets write…

002fd51

… filecache when cache size is enough #53801 (#54692) picked from #53801

morrySnow added dev/3.1.0-merged and removed dev/3.1.x dev/3.1.x-conflict labels Aug 14, 2025

gavinchou mentioned this pull request Sep 1, 2025

Release Note 3.0.8 #55553

Open

Conversation

swjtu-zhanglei commented Jul 23, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What problem does this PR solve?

Release note

Check List (For Author)

Check List (For Reviewer who merge this PR)

Uh oh!

hello-stephen commented Jul 23, 2025

Uh oh!

swjtu-zhanglei commented Jul 23, 2025

Uh oh!

dataroaring left a comment

Choose a reason for hiding this comment

Uh oh!

doris-robot commented Jul 23, 2025

Uh oh!

doris-robot commented Jul 23, 2025

Uh oh!

doris-robot commented Jul 23, 2025

Uh oh!

hello-stephen commented Jul 23, 2025

BE UT Coverage Report

Uh oh!

hello-stephen commented Jul 23, 2025

BE Regression && UT Coverage Report

Uh oh!

swjtu-zhanglei commented Jul 24, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

gavinchou Jul 24, 2025

Choose a reason for hiding this comment

Uh oh!

swjtu-zhanglei Jul 25, 2025

Choose a reason for hiding this comment

Uh oh!

swjtu-zhanglei commented Jul 24, 2025

Uh oh!

doris-robot commented Jul 24, 2025

Uh oh!

doris-robot commented Jul 24, 2025

Uh oh!

doris-robot commented Jul 24, 2025

Uh oh!

doris-robot commented Jul 24, 2025

BE UT Coverage Report

Uh oh!

hello-stephen commented Jul 24, 2025

BE Regression && UT Coverage Report

Uh oh!

swjtu-zhanglei commented Jul 25, 2025

Uh oh!

swjtu-zhanglei commented Jul 25, 2025

Uh oh!

doris-robot commented Jul 25, 2025

Uh oh!

doris-robot commented Jul 25, 2025

Uh oh!

doris-robot commented Jul 25, 2025

Uh oh!

doris-robot commented Aug 11, 2025

Uh oh!

doris-robot commented Aug 11, 2025

Uh oh!

doris-robot commented Aug 11, 2025

Uh oh!

freemandealer left a comment

Choose a reason for hiding this comment

Uh oh!

doris-robot commented Aug 11, 2025

BE UT Coverage Report

Uh oh!

hello-stephen commented Aug 11, 2025

BE Regression && UT Coverage Report

Uh oh!

swjtu-zhanglei commented Aug 12, 2025

Uh oh!

hello-stephen commented Aug 12, 2025

BE Regression && UT Coverage Report

Uh oh!

swjtu-zhanglei commented Jul 23, 2025 •

edited

Loading

swjtu-zhanglei commented Jul 24, 2025 •

edited

Loading