Skip to content

branch-3.0: [opt](filecache) Let base compaction output rowsets write filecache when cache size is enough (#53801)#54693

Merged
dataroaring merged 1 commit intoapache:branch-3.0from
swjtu-zhanglei:branch-3.0-pick
Aug 14, 2025
Merged

branch-3.0: [opt](filecache) Let base compaction output rowsets write filecache when cache size is enough (#53801)#54693
dataroaring merged 1 commit intoapache:branch-3.0from
swjtu-zhanglei:branch-3.0-pick

Conversation

@swjtu-zhanglei
Copy link
Contributor

@swjtu-zhanglei swjtu-zhanglei commented Aug 13, 2025

What problem does this PR solve?

pick #53801
Issue Number: close #xxx

Related PR: #xxx

Problem Summary:

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason
  • Behavior changed:

    • No.
    • Yes.
  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@swjtu-zhanglei
Copy link
Contributor Author

run buildall

@Thearas
Copy link
Contributor

Thearas commented Aug 13, 2025

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@doris-robot
Copy link

BE UT Coverage Report

Increment line coverage 43.33% (13/30) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 42.07% (11188/26592)
Line Coverage 32.56% (95926/294609)
Region Coverage 31.72% (49592/156362)
Branch Coverage 28.18% (25452/90332)

@doris-robot
Copy link

TPC-H: Total hot run time: 40064 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 0ac03f9756f9f6dae72015fef597a6c7ee034cf9, data reload: false

------ Round 1 ----------------------------------
q1	17770	6869	6601	6601
q2	2070	171	161	161
q3	10666	1143	1211	1143
q4	10486	775	763	763
q5	7734	2898	2820	2820
q6	217	135	134	134
q7	978	617	609	609
q8	9350	1969	2049	1969
q9	6684	6459	6452	6452
q10	6971	2289	2299	2289
q11	453	269	265	265
q12	393	213	222	213
q13	17789	2990	2955	2955
q14	232	213	204	204
q15	525	473	443	443
q16	475	392	377	377
q17	983	627	569	569
q18	7339	6658	6704	6658
q19	1451	1085	1088	1085
q20	482	196	211	196
q21	3918	3202	3290	3202
q22	1112	956	1010	956
Total cold run time: 108078 ms
Total hot run time: 40064 ms

----- Round 2, with runtime_filter_mode=off -----
q1	6661	6575	6628	6575
q2	342	243	225	225
q3	2973	2917	2925	2917
q4	2057	1812	1792	1792
q5	5689	5733	5765	5733
q6	214	130	128	128
q7	2240	1775	1825	1775
q8	3379	3550	3552	3550
q9	8760	8987	8932	8932
q10	3546	3539	3516	3516
q11	595	496	492	492
q12	803	588	609	588
q13	8586	3219	3134	3134
q14	303	268	275	268
q15	505	462	469	462
q16	498	433	453	433
q17	1858	1637	1627	1627
q18	8289	7756	7740	7740
q19	1723	1446	1601	1446
q20	2049	1800	1886	1800
q21	5311	4969	5018	4969
q22	1126	1030	1018	1018
Total cold run time: 67507 ms
Total hot run time: 59120 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 196902 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 0ac03f9756f9f6dae72015fef597a6c7ee034cf9, data reload: false

query1	1288	961	907	907
query2	6241	1954	1944	1944
query3	10941	4532	4698	4532
query4	33329	23458	23853	23458
query5	4146	443	434	434
query6	273	198	186	186
query7	3977	321	313	313
query8	310	218	225	218
query9	9338	2605	2599	2599
query10	485	260	258	258
query11	17881	15236	15191	15191
query12	166	105	102	102
query13	1570	425	432	425
query14	9468	6635	7072	6635
query15	247	184	190	184
query16	8055	495	505	495
query17	1586	613	608	608
query18	2123	317	319	317
query19	247	171	168	168
query20	131	130	117	117
query21	216	106	109	106
query22	4841	4508	4457	4457
query23	35355	34337	34498	34337
query24	11260	2942	2888	2888
query25	640	424	434	424
query26	1314	167	166	166
query27	2524	359	361	359
query28	7338	2160	2182	2160
query29	867	471	463	463
query30	253	164	163	163
query31	1031	840	877	840
query32	91	54	55	54
query33	770	290	313	290
query34	951	507	515	507
query35	912	727	718	718
query36	1107	952	977	952
query37	130	68	67	67
query38	4070	4024	3946	3946
query39	1545	1620	1489	1489
query40	193	97	101	97
query41	51	50	50	50
query42	118	106	104	104
query43	531	495	486	486
query44	1274	817	814	814
query45	193	176	169	169
query46	1137	734	731	731
query47	2040	1938	1904	1904
query48	484	396	411	396
query49	929	388	414	388
query50	820	416	430	416
query51	7468	7179	7339	7179
query52	101	90	88	88
query53	252	197	189	189
query54	1147	464	464	464
query55	83	76	80	76
query56	268	242	259	242
query57	1285	1204	1184	1184
query58	225	217	216	216
query59	3347	3062	3170	3062
query60	278	264	257	257
query61	106	109	112	109
query62	860	694	696	694
query63	224	188	200	188
query64	4240	649	658	649
query65	3380	3259	3229	3229
query66	762	322	288	288
query67	16323	15703	15564	15564
query68	4635	573	569	569
query69	440	264	263	263
query70	1193	1142	1142	1142
query71	318	257	260	257
query72	6279	3994	4068	3994
query73	755	341	353	341
query74	10510	8996	8879	8879
query75	3372	2597	2669	2597
query76	2816	1111	1179	1111
query77	393	267	268	267
query78	10541	9515	9496	9496
query79	2335	583	598	583
query80	1129	432	418	418
query81	556	222	220	220
query82	929	90	93	90
query83	232	149	145	145
query84	239	72	80	72
query85	1298	303	286	286
query86	438	285	277	277
query87	4363	4281	4234	4234
query88	4138	2357	2338	2338
query89	412	287	294	287
query90	1942	185	185	185
query91	181	144	149	144
query92	65	52	52	52
query93	2309	560	549	549
query94	787	308	254	254
query95	361	248	258	248
query96	617	281	280	280
query97	3309	3137	3200	3137
query98	216	195	207	195
query99	1552	1323	1282	1282
Total cold run time: 303796 ms
Total hot run time: 196902 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 29.86 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 0ac03f9756f9f6dae72015fef597a6c7ee034cf9, data reload: false

query1	0.03	0.03	0.03
query2	0.08	0.02	0.04
query3	0.23	0.06	0.06
query4	1.63	0.11	0.10
query5	0.52	0.49	0.51
query6	1.14	0.73	0.73
query7	0.02	0.01	0.02
query8	0.04	0.03	0.03
query9	0.58	0.50	0.50
query10	0.58	0.55	0.56
query11	0.14	0.10	0.10
query12	0.13	0.10	0.11
query13	0.61	0.60	0.59
query14	0.79	0.79	0.79
query15	0.84	0.81	0.83
query16	0.38	0.39	0.41
query17	1.05	1.00	1.08
query18	0.24	0.22	0.21
query19	1.92	1.90	1.92
query20	0.01	0.01	0.01
query21	15.39	0.57	0.57
query22	2.28	1.69	1.54
query23	17.09	0.87	0.84
query24	3.53	0.91	1.51
query25	0.24	0.08	0.07
query26	0.43	0.15	0.14
query27	0.03	0.04	0.05
query28	9.92	0.50	0.45
query29	12.59	3.17	3.16
query30	0.24	0.06	0.06
query31	2.86	0.37	0.39
query32	3.25	0.46	0.47
query33	3.00	3.03	3.07
query34	17.35	4.54	4.53
query35	4.55	4.52	4.57
query36	0.65	0.47	0.46
query37	0.08	0.06	0.06
query38	0.04	0.04	0.03
query39	0.03	0.02	0.03
query40	0.16	0.13	0.12
query41	0.08	0.03	0.02
query42	0.03	0.02	0.02
query43	0.03	0.02	0.02
Total cold run time: 104.81 s
Total hot run time: 29.86 s

Copy link
Contributor

@dataroaring dataroaring left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Aug 13, 2025
@github-actions
Copy link
Contributor

PR approved by at least one committer and no changes requested.

@github-actions
Copy link
Contributor

PR approved by anyone and no changes requested.

@dataroaring dataroaring merged commit 2a23e4f into apache:branch-3.0 Aug 14, 2025
21 of 23 checks passed
@gavinchou gavinchou mentioned this pull request Sep 1, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by one committer. reviewed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants

Comments