[opt](nereids)move runtime filter info from instance level to BE level#56978

englefly · 2025-10-15T02:45:45Z

What problem does this PR solve?

Runtime filter info is shared among all instances within a BE.
This change saves thrift transfering effort.

Issue Number: close #xxx

Related PR: #xxx

Problem Summary:

Release note

None

Check List (For Author)

Test
- Regression test
- Unit Test
- Manual test (add detailed scripts or steps below)
- No need to test or manual test. Explain why:
  - This is a refactor/code format and no logic has been changed.
  - Previous test can cover this change.
  - No code files have been changed.
  - Other reason
Behavior changed:
- No.
- Yes.
Does this need documentation?
- No.
- Yes.

Check List (For Reviewer who merge this PR)

Confirm the release note
Confirm test cases
Confirm document
Add branch pick label

hello-stephen · 2025-10-15T02:45:50Z

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

What problem was fixed (it's best to include specific error reporting information). How it was fixed.
Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
What features were added. Why was this function added?
Which code was refactored and why was this part of the code refactored?
Which functions were optimized and what is the difference before and after the optimization?

englefly · 2025-10-15T02:45:56Z

run buildall

hello-stephen · 2025-10-15T03:10:19Z

Cloud UT Coverage Report

Increment line coverage 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	83.27% (1638/1967)
Line Coverage	67.86% (28934/42638)
Region Coverage	68.09% (14257/20937)
Branch Coverage	58.41% (7591/12996)

doris-robot · 2025-10-15T03:44:12Z

TPC-DS: Total hot run time: 188193 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit a18aabdeebb19f8557ee5ee5acce618f581a7b1d, data reload: false

query1	1058	446	405	405
query2	6560	1694	1695	1694
query3	6761	222	221	221
query4	26619	23828	23650	23650
query5	4613	659	502	502
query6	346	260	229	229
query7	4671	505	301	301
query8	308	296	264	264
query9	8738	2564	2555	2555
query10	515	358	301	301
query11	15376	15058	15666	15058
query12	215	138	132	132
query13	1847	684	477	477
query14	11594	9713	9603	9603
query15	346	200	190	190
query16	8169	699	531	531
query17	2361	780	651	651
query18	2235	430	342	342
query19	310	278	201	201
query20	317	184	145	145
query21	212	135	114	114
query22	4586	4920	4641	4641
query23	35145	34175	33423	33423
query24	8088	2339	2390	2339
query25	566	510	437	437
query26	1224	268	156	156
query27	2680	496	353	353
query28	4118	2138	2152	2138
query29	776	611	479	479
query30	295	220	192	192
query31	916	826	751	751
query32	77	66	65	65
query33	555	369	320	320
query34	800	860	519	519
query35	796	826	744	744
query36	932	1004	910	910
query37	124	113	81	81
query38	3576	3601	3461	3461
query39	1533	1411	1424	1411
query40	217	125	120	120
query41	63	59	60	59
query42	118	107	113	107
query43	482	487	473	473
query44	1312	822	814	814
query45	185	183	168	168
query46	844	988	643	643
query47	1765	1809	1727	1727
query48	397	422	325	325
query49	755	525	424	424
query50	648	688	411	411
query51	3939	3933	3920	3920
query52	113	108	97	97
query53	228	270	200	200
query54	608	586	522	522
query55	92	82	84	82
query56	342	326	294	294
query57	1206	1186	1139	1139
query58	292	267	301	267
query59	2491	2703	2521	2521
query60	343	332	318	318
query61	160	154	151	151
query62	814	738	659	659
query63	231	198	193	193
query64	4218	1154	823	823
query65	4104	3969	3965	3965
query66	1069	459	338	338
query67	15572	15581	15270	15270
query68	8775	941	643	643
query69	469	315	284	284
query70	1377	1338	1306	1306
query71	477	344	308	308
query72	5968	2612	5310	2612
query73	652	775	368	368
query74	9097	9237	8659	8659
query75	4256	3352	2841	2841
query76	3380	1155	738	738
query77	932	405	316	316
query78	9691	9928	8858	8858
query79	2030	844	591	591
query80	641	566	487	487
query81	515	268	234	234
query82	388	174	136	136
query83	273	279	253	253
query84	258	113	99	99
query85	993	462	418	418
query86	358	307	311	307
query87	3750	3789	3695	3695
query88	3830	2245	2232	2232
query89	406	334	298	298
query90	1919	218	220	218
query91	170	166	131	131
query92	93	70	64	64
query93	1822	984	645	645
query94	686	429	339	339
query95	401	322	315	315
query96	493	587	289	289
query97	2971	2964	2890	2890
query98	242	234	219	219
query99	1468	1458	1384	1384
Total cold run time: 281347 ms
Total hot run time: 188193 ms

doris-robot · 2025-10-15T03:49:48Z

ClickBench: Total hot run time: 31.17 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit a18aabdeebb19f8557ee5ee5acce618f581a7b1d, data reload: false

query1	0.05	0.05	0.04
query2	0.12	0.06	0.06
query3	0.30	0.07	0.07
query4	1.61	0.09	0.09
query5	0.27	0.25	0.25
query6	1.17	0.64	0.65
query7	0.03	0.02	0.02
query8	0.07	0.05	0.05
query9	0.68	0.54	0.53
query10	0.58	0.59	0.59
query11	0.25	0.14	0.13
query12	0.26	0.14	0.14
query13	0.65	0.64	0.64
query14	1.04	1.04	1.07
query15	0.94	0.88	0.88
query16	0.40	0.39	0.39
query17	1.07	1.09	1.06
query18	0.24	0.23	0.23
query19	2.02	1.86	1.87
query20	0.01	0.01	0.02
query21	15.40	1.02	0.70
query22	0.92	1.08	0.87
query23	14.69	1.53	0.76
query24	5.44	0.59	0.31
query25	0.18	0.10	0.09
query26	0.55	0.22	0.18
query27	0.09	0.09	0.09
query28	11.03	1.67	1.06
query29	12.53	4.09	3.35
query30	0.33	0.11	0.10
query31	2.83	0.62	0.43
query32	3.25	0.59	0.52
query33	3.12	3.06	3.13
query34	16.91	5.56	4.81
query35	4.84	4.85	4.93
query36	0.65	0.54	0.52
query37	0.22	0.19	0.20
query38	0.19	0.17	0.17
query39	0.06	0.05	0.05
query40	0.22	0.20	0.18
query41	0.12	0.06	0.06
query42	0.07	0.06	0.05
query43	0.06	0.06	0.05
Total cold run time: 105.46 s
Total hot run time: 31.17 s

hello-stephen · 2025-10-15T04:49:57Z

BE UT Coverage Report

Increment line coverage 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	52.53% (17751/33790)
Line Coverage	37.75% (161459/427754)
Region Coverage	32.18% (123122/382598)
Branch Coverage	33.62% (54104/160941)

hello-stephen · 2025-10-15T05:18:27Z

BE Regression && UT Coverage Report

Increment line coverage 100% (0/0) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	71.24% (23598/33125)
Line Coverage	57.70% (246679/427500)
Region Coverage	52.90% (205032/387548)
Branch Coverage	54.71% (88539/161830)

hello-stephen · 2025-10-15T05:29:34Z

FE Regression Coverage Report

Increment line coverage 100.00% (10/10) 🎉
Increment coverage report
Complete coverage report

github-actions · 2025-10-16T06:59:20Z

PR approved by at least one committer and no changes requested.

github-actions · 2025-10-16T06:59:23Z

PR approved by anyone and no changes requested.

#56978) ### What problem does this PR solve? Runtime filter info is shared among all instances within a BE. This change saves thrift transfering effort.

…l to BE level #56978 (#57047) Cherry-picked from #56978 Co-authored-by: minghong <zhouminghong@selectdb.com>

…57108) set RuntimeFilterInfo only on BE which is merge node. note: RuntimeFilterInfo only contains runtime-filter merge information, not runtime filter descriptor In the previous pull request #56978, we moved the runtime info (which records how runtime filters are merged) from the instance level to the Backend (BE) level. In a multi-BE environment, when a runtime filter needs to be merged, the BE does not perform the merge using the mergeInstance specified by the Runtime Filter descriptor. Instead, it selects a BE that has the RuntimeInfo set to perform the merge. This causes BEs that should not be responsible for merging to wait for other nodes to send local runtime filters, resulting in a time

…pache#57108) set RuntimeFilterInfo only on BE which is merge node. note: RuntimeFilterInfo only contains runtime-filter merge information, not runtime filter descriptor In the previous pull request apache#56978, we moved the runtime info (which records how runtime filters are merged) from the instance level to the Backend (BE) level. In a multi-BE environment, when a runtime filter needs to be merged, the BE does not perform the merge using the mergeInstance specified by the Runtime Filter descriptor. Instead, it selects a BE that has the RuntimeInfo set to perform the merge. This causes BEs that should not be responsible for merging to wait for other nodes to send local runtime filters, resulting in a time

move rf info from instance level to be level

a18aabd

morrySnow approved these changes Oct 16, 2025

View reviewed changes

englefly added the dev/4.0.x label Oct 16, 2025

BiteTheDDDDt approved these changes Oct 16, 2025

View reviewed changes

github-actions bot added the approved Indicates a PR has been approved by one committer. label Oct 16, 2025

englefly merged commit 456af82 into apache:master Oct 16, 2025
43 of 47 checks passed

github-actions bot added the reviewed label Oct 16, 2025

englefly deleted the rf-thrift-fe-1015 branch October 16, 2025 06:59

github-actions bot mentioned this pull request Oct 16, 2025

branch-4.0: [opt](nereids)move runtime filter info from instance level to BE level #56978 #57047

Merged

yiguolei pushed a commit that referenced this pull request Oct 16, 2025

branch-4.0: [opt](nereids)move runtime filter info from instance leve…

85824c5

…l to BE level #56978 (#57047) Cherry-picked from #56978 Co-authored-by: minghong <zhouminghong@selectdb.com>

yiguolei added dev/4.0.1-merged and removed dev/4.0.x labels Oct 16, 2025

englefly mentioned this pull request Oct 20, 2025

[fix](nereids)set RuntimeFilterInfo only on BE which is merge node #57108

Merged

16 tasks

yiguolei mentioned this pull request Nov 5, 2025

4.0.1 Release Notes #57717

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[opt](nereids)move runtime filter info from instance level to BE level#56978