[fix](nereids)PushDownJoinOnAssertNumRows create invalid LogicalProject node#57804

starocean999 · 2025-11-07T06:09:20Z

when push alias into child node, we should always create a new project node for it.

input:

Alias->cast(col_int_undef_signed#30 as DOUBLE) AS `cast(col_int_undef_signed as DOUBLE)`#70
childProject->LogicalProject(projects=[col_int_undef_signed2#50 AS `col_int_undef_signed`#30])

error case:
LogicalProject(projects=[col_int_undef_signed2#50 AS `col_int_undef_signed`#30, cast(col_int_undef_signed#30 as DOUBLE) AS `cast(col_int_undef_signed as DOUBLE)`#70])

correct:

LogicalProject(projects=[col_int_undef_signed#30, cast(col_int_undef_signed#30 as DOUBLE) AS `cast(col_int_undef_signed as DOUBLE)`#70])
LogicalProject(projects=[col_int_undef_signed2#50 AS `col_int_undef_signed`#30])

Issue Number: close #xxx

Related PR: (#57414)

Problem Summary:

Release note

None

Check List (For Author)

Test
- Regression test
- Unit Test
- Manual test (add detailed scripts or steps below)
- No need to test or manual test. Explain why:
  - This is a refactor/code format and no logic has been changed.
  - Previous test can cover this change.
  - No code files have been changed.
  - Other reason
Behavior changed:
- No.
- Yes.
Does this need documentation?
- No.
- Yes.

Check List (For Reviewer who merge this PR)

Confirm the release note
Confirm test cases
Confirm document
Add branch pick label

hello-stephen · 2025-11-07T06:09:25Z

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

What problem was fixed (it's best to include specific error reporting information). How it was fixed.
Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
What features were added. Why was this function added?
Which code was refactored and why was this part of the code refactored?
Which functions were optimized and what is the difference before and after the optimization?

starocean999 · 2025-11-07T06:09:33Z

run buildall

doris-robot · 2025-11-07T06:38:23Z

TPC-DS: Total hot run time: 188851 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 2ba9bb3c4716d4870bd441740b3c07c9d3c40959, data reload: false

query1	1043	416	384	384
query2	6559	1686	1725	1686
query3	6752	222	218	218
query4	26340	23843	23234	23234
query5	4542	620	459	459
query6	341	252	252	252
query7	4656	497	311	311
query8	312	265	258	258
query9	8685	2586	2563	2563
query10	513	339	284	284
query11	15563	15097	14774	14774
query12	186	128	128	128
query13	1696	573	460	460
query14	11261	9154	9122	9122
query15	212	183	171	171
query16	7657	673	489	489
query17	1381	747	619	619
query18	2033	404	355	355
query19	233	211	182	182
query20	148	149	145	145
query21	218	139	125	125
query22	4627	4582	4459	4459
query23	34585	33794	33641	33641
query24	9225	2483	2565	2483
query25	618	537	468	468
query26	1279	275	167	167
query27	2849	519	381	381
query28	4957	2259	2249	2249
query29	826	665	537	537
query30	301	243	202	202
query31	967	827	737	737
query32	81	80	71	71
query33	637	374	340	340
query34	819	897	528	528
query35	857	876	794	794
query36	962	976	895	895
query37	121	110	86	86
query38	3525	3482	3473	3473
query39	1474	1416	1396	1396
query40	212	128	114	114
query41	58	55	56	55
query42	118	108	115	108
query43	480	484	461	461
query44	1251	733	736	733
query45	176	175	169	169
query46	881	982	644	644
query47	1736	1804	1729	1729
query48	385	430	309	309
query49	767	492	413	413
query50	650	681	394	394
query51	3952	3892	3846	3846
query52	107	108	101	101
query53	236	269	193	193
query54	309	288	276	276
query55	87	88	82	82
query56	325	320	309	309
query57	1171	1174	1111	1111
query58	290	269	267	267
query59	2548	2643	2528	2528
query60	373	331	314	314
query61	175	158	154	154
query62	788	722	684	684
query63	233	195	190	190
query64	4467	1159	843	843
query65	3998	3917	3935	3917
query66	1081	441	327	327
query67	15390	15083	15016	15016
query68	5871	937	611	611
query69	499	333	282	282
query70	1297	1376	1210	1210
query71	448	337	319	319
query72	6063	4843	4989	4843
query73	608	568	358	358
query74	9144	8860	8972	8860
query75	3307	3345	2861	2861
query76	3327	1147	706	706
query77	504	425	319	319
query78	9369	9646	8919	8919
query79	1925	839	599	599
query80	709	600	493	493
query81	525	262	233	233
query82	241	159	139	139
query83	268	256	244	244
query84	259	114	94	94
query85	879	489	437	437
query86	382	316	314	314
query87	3715	3689	3663	3663
query88	3000	2290	2266	2266
query89	387	338	294	294
query90	1926	229	222	222
query91	156	166	136	136
query92	70	71	61	61
query93	1165	993	656	656
query94	677	452	338	338
query95	406	331	319	319
query96	507	573	287	287
query97	2946	2948	2849	2849
query98	246	220	208	208
query99	1306	1421	1298	1298
Total cold run time: 273417 ms
Total hot run time: 188851 ms

doris-robot · 2025-11-07T06:43:30Z

ClickBench: Total hot run time: 27.49 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 2ba9bb3c4716d4870bd441740b3c07c9d3c40959, data reload: false

query1	0.06	0.05	0.04
query2	0.09	0.05	0.05
query3	0.26	0.08	0.08
query4	1.61	0.12	0.12
query5	0.27	0.28	0.25
query6	1.17	0.64	0.64
query7	0.03	0.02	0.02
query8	0.06	0.05	0.04
query9	0.61	0.53	0.50
query10	0.56	0.57	0.57
query11	0.17	0.12	0.12
query12	0.15	0.11	0.12
query13	0.62	0.60	0.60
query14	1.03	1.00	1.00
query15	0.86	0.83	0.86
query16	0.40	0.40	0.39
query17	1.03	1.00	1.02
query18	0.22	0.20	0.21
query19	1.88	1.77	1.78
query20	0.02	0.01	0.02
query21	15.44	0.20	0.14
query22	5.04	0.08	0.05
query23	15.66	0.26	0.10
query24	3.35	0.67	0.54
query25	0.08	0.05	0.05
query26	0.14	0.15	0.14
query27	0.08	0.05	0.05
query28	4.63	1.18	0.94
query29	12.60	3.95	3.21
query30	0.28	0.14	0.12
query31	2.82	0.60	0.38
query32	3.23	0.55	0.47
query33	3.03	3.04	3.12
query34	15.76	5.22	4.51
query35	4.57	4.56	4.56
query36	0.67	0.50	0.50
query37	0.10	0.06	0.08
query38	0.07	0.04	0.04
query39	0.04	0.03	0.03
query40	0.18	0.14	0.13
query41	0.08	0.03	0.04
query42	0.04	0.03	0.03
query43	0.04	0.04	0.03
Total cold run time: 99.03 s
Total hot run time: 27.49 s

github-actions · 2025-11-11T06:53:32Z

PR approved by at least one committer and no changes requested.

github-actions · 2025-11-11T06:53:35Z

PR approved by anyone and no changes requested.

morrySnow · 2025-11-11T06:55:40Z

...e-core/src/main/java/org/apache/doris/nereids/rules/rewrite/PushDownJoinOnAssertNumRows.java

add ut for this function

…ct node

starocean999 · 2025-11-11T07:33:39Z

run buildall

doris-robot · 2025-11-11T08:12:26Z

TPC-H: Total hot run time: 34387 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 9ca02eaa1cd17f3ac5d120a4393dd21f22baddc2, data reload: false

------ Round 1 ----------------------------------
q1	17604	5242	5117	5117
q2	2042	309	212	212
q3	10265	1284	704	704
q4	10234	921	386	386
q5	7491	2419	2346	2346
q6	181	166	137	137
q7	913	759	624	624
q8	9335	1333	1105	1105
q9	6970	5150	5159	5150
q10	6874	2211	1822	1822
q11	486	308	282	282
q12	344	388	256	256
q13	18053	3659	3064	3064
q14	240	253	225	225
q15	598	512	519	512
q16	1056	1022	965	965
q17	598	879	373	373
q18	7777	7275	6993	6993
q19	1219	939	565	565
q20	357	328	224	224
q21	3742	3224	2336	2336
q22	1068	1040	989	989
Total cold run time: 107447 ms
Total hot run time: 34387 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5215	5131	5142	5131
q2	255	335	243	243
q3	2159	2671	2336	2336
q4	1381	1765	1311	1311
q5	4198	4689	4364	4364
q6	217	169	133	133
q7	2046	1958	1864	1864
q8	2664	2677	2582	2582
q9	7418	7364	7418	7364
q10	3051	3317	2819	2819
q11	597	533	499	499
q12	700	798	618	618
q13	3666	3871	3416	3416
q14	296	326	302	302
q15	560	508	510	508
q16	1126	1099	1081	1081
q17	1206	1541	1411	1411
q18	7833	7734	7600	7600
q19	811	879	902	879
q20	1989	2008	1829	1829
q21	4713	4402	4303	4303
q22	1091	1060	1003	1003
Total cold run time: 53192 ms
Total hot run time: 51596 ms

doris-robot · 2025-11-11T08:24:08Z

TPC-DS: Total hot run time: 188885 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 9ca02eaa1cd17f3ac5d120a4393dd21f22baddc2, data reload: false

query1	1039	407	393	393
query2	6631	1700	1693	1693
query3	6758	237	221	221
query4	26980	23312	23636	23312
query5	4441	670	496	496
query6	329	244	231	231
query7	4648	516	295	295
query8	308	259	255	255
query9	8690	2639	2615	2615
query10	534	353	281	281
query11	15475	15190	14994	14994
query12	179	153	116	116
query13	1681	563	444	444
query14	10422	9238	9350	9238
query15	193	185	183	183
query16	7552	681	526	526
query17	1234	750	633	633
query18	2008	413	315	315
query19	220	205	175	175
query20	137	139	127	127
query21	218	137	119	119
query22	4057	4121	3920	3920
query23	33836	32847	32920	32847
query24	8555	2421	2471	2421
query25	603	526	452	452
query26	1231	292	165	165
query27	2734	519	347	347
query28	4380	2203	2183	2183
query29	807	633	533	533
query30	300	223	211	211
query31	896	780	727	727
query32	85	75	74	74
query33	597	384	330	330
query34	806	859	542	542
query35	821	837	763	763
query36	945	999	895	895
query37	123	112	96	96
query38	3480	3578	3598	3578
query39	1455	1424	1406	1406
query40	220	130	122	122
query41	63	60	61	60
query42	131	117	111	111
query43	511	486	477	477
query44	1286	758	747	747
query45	191	186	173	173
query46	898	998	659	659
query47	1743	1784	1715	1715
query48	391	433	323	323
query49	803	542	441	441
query50	669	692	408	408
query51	3957	3926	3899	3899
query52	110	120	112	112
query53	250	297	210	210
query54	314	305	278	278
query55	93	89	84	84
query56	331	335	312	312
query57	1174	1216	1109	1109
query58	305	299	302	299
query59	2539	2679	2492	2492
query60	370	382	349	349
query61	194	194	189	189
query62	781	761	717	717
query63	245	200	208	200
query64	4626	1310	1011	1011
query65	4046	3985	3992	3985
query66	1160	480	361	361
query67	15195	15289	15179	15179
query68	7460	898	604	604
query69	521	351	310	310
query70	1327	1282	1304	1282
query71	447	354	331	331
query72	6255	5017	4787	4787
query73	635	596	366	366
query74	8815	9051	9038	9038
query75	3334	3332	2831	2831
query76	3309	1166	775	775
query77	528	428	341	341
query78	9573	9893	8893	8893
query79	2178	812	598	598
query80	1509	569	484	484
query81	570	259	225	225
query82	412	157	132	132
query83	274	270	251	251
query84	260	116	101	101
query85	899	477	462	462
query86	444	320	282	282
query87	3745	3701	3645	3645
query88	2956	2293	2291	2291
query89	399	337	292	292
query90	1958	224	220	220
query91	175	165	139	139
query92	85	72	67	67
query93	2652	977	643	643
query94	784	455	347	347
query95	396	312	321	312
query96	489	566	278	278
query97	2914	2956	2871	2871
query98	242	230	211	211
query99	1310	1438	1302	1302
Total cold run time: 274839 ms
Total hot run time: 188885 ms

doris-robot · 2025-11-11T08:29:15Z

ClickBench: Total hot run time: 27.43 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 9ca02eaa1cd17f3ac5d120a4393dd21f22baddc2, data reload: false

query1	0.05	0.05	0.05
query2	0.09	0.05	0.05
query3	0.26	0.08	0.08
query4	1.61	0.12	0.11
query5	0.28	0.25	0.26
query6	1.19	0.63	0.65
query7	0.03	0.02	0.02
query8	0.06	0.04	0.04
query9	0.59	0.53	0.52
query10	0.58	0.57	0.57
query11	0.16	0.11	0.11
query12	0.15	0.12	0.12
query13	0.62	0.60	0.60
query14	1.01	1.00	0.99
query15	0.86	0.83	0.83
query16	0.39	0.39	0.40
query17	1.04	1.08	1.01
query18	0.22	0.20	0.19
query19	1.94	1.77	1.78
query20	0.02	0.01	0.01
query21	15.44	0.21	0.14
query22	4.93	0.07	0.05
query23	15.68	0.25	0.10
query24	2.97	0.82	0.38
query25	0.09	0.06	0.06
query26	0.13	0.14	0.13
query27	0.07	0.06	0.05
query28	4.73	1.15	0.93
query29	12.66	3.89	3.34
query30	0.28	0.15	0.11
query31	2.82	0.59	0.39
query32	3.23	0.55	0.47
query33	3.02	3.02	3.16
query34	15.66	5.20	4.54
query35	4.63	4.59	4.52
query36	0.68	0.49	0.49
query37	0.09	0.07	0.07
query38	0.07	0.04	0.04
query39	0.04	0.03	0.03
query40	0.17	0.14	0.14
query41	0.09	0.03	0.03
query42	0.04	0.04	0.03
query43	0.04	0.03	0.04
Total cold run time: 98.71 s
Total hot run time: 27.43 s

hello-stephen · 2025-11-11T10:18:54Z

FE Regression Coverage Report

Increment line coverage 100.00% (3/3) 🎉
Increment coverage report
Complete coverage report

hello-stephen · 2025-11-12T04:37:20Z

FE Regression Coverage Report

Increment line coverage 100.00% (3/3) 🎉
Increment coverage report
Complete coverage report

github-actions · 2025-11-13T02:03:44Z

PR approved by at least one committer and no changes requested.

hello-stephen · 2025-11-13T03:51:36Z

FE Regression Coverage Report

Increment line coverage 100.00% (3/3) 🎉
Increment coverage report
Complete coverage report

…ct node (apache#57804) when push alias into child node, we should always create a new project node for it. input: ``` Alias->cast(col_int_undef_signed#30 as DOUBLE) AS `cast(col_int_undef_signed as DOUBLE)`apache#70 childProject->LogicalProject(projects=[col_int_undef_signed2#50 AS `col_int_undef_signed`apache#30]) ``` error case: ```LogicalProject(projects=[col_int_undef_signed2#50 AS `col_int_undef_signed`apache#30, cast(col_int_undef_signed#30 as DOUBLE) AS `cast(col_int_undef_signed as DOUBLE)`apache#70])``` correct: ``` LogicalProject(projects=[col_int_undef_signed#30, cast(col_int_undef_signed#30 as DOUBLE) AS `cast(col_int_undef_signed as DOUBLE)`apache#70]) LogicalProject(projects=[col_int_undef_signed2#50 AS `col_int_undef_signed`apache#30]) ```

…bug (#58174) ### What problem does this PR solve? pr #57414 will generate error project: LogicalProject(a as b, b as c), in which 'b as c' is not correct, because 'b' is not in output of project.child() pr #57804 fixed above issue, but made another issue, that is when the bottom project is not resued in result plan, some aliases definitions are lost.

…bug (apache#58174) ### What problem does this PR solve? pr apache#57414 will generate error project: LogicalProject(a as b, b as c), in which 'b as c' is not correct, because 'b' is not in output of project.child() pr apache#57804 fixed above issue, but made another issue, that is when the bottom project is not resued in result plan, some aliases definitions are lost.

starocean999 added the dev/4.0.x label Nov 7, 2025

starocean999 marked this pull request as ready for review November 10, 2025 02:04

englefly previously approved these changes Nov 11, 2025

View reviewed changes

github-actions bot added approved Indicates a PR has been approved by one committer. reviewed labels Nov 11, 2025

morrySnow previously approved these changes Nov 11, 2025

View reviewed changes

morrySnow reviewed Nov 11, 2025

View reviewed changes

starocean999 added 2 commits November 11, 2025 15:26

[fix](nereids)PushDownJoinOnAssertNumRows create invalid LogicalProje…

cf660a9

…ct node

add fe ut

9ca02ea

starocean999 dismissed stale reviews from morrySnow and englefly via 9ca02ea November 11, 2025 07:33

starocean999 force-pushed the nereids_1107 branch from 2ba9bb3 to 9ca02ea Compare November 11, 2025 07:33

github-actions bot removed the approved Indicates a PR has been approved by one committer. label Nov 11, 2025

morrySnow approved these changes Nov 13, 2025

View reviewed changes

github-actions bot added the approved Indicates a PR has been approved by one committer. label Nov 13, 2025

englefly approved these changes Nov 13, 2025

View reviewed changes

starocean999 merged commit acc1bcf into apache:master Nov 13, 2025
29 of 30 checks passed

github-actions bot added the dev/4.0.x-conflict label Nov 13, 2025

englefly mentioned this pull request Nov 20, 2025

[fix](nereids)PushDownJoinOnAssertNumRows rule process project alias bug #58174

Merged

16 tasks

morrySnow removed dev/4.0.x dev/4.0.x-conflict labels Nov 27, 2025

Conversation

starocean999 commented Nov 7, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Release note

Check List (For Author)

Check List (For Reviewer who merge this PR)

Uh oh!

hello-stephen commented Nov 7, 2025

Uh oh!

starocean999 commented Nov 7, 2025

Uh oh!

doris-robot commented Nov 7, 2025

Uh oh!

doris-robot commented Nov 7, 2025

Uh oh!

github-actions bot commented Nov 11, 2025

Uh oh!

github-actions bot commented Nov 11, 2025

Uh oh!

morrySnow Nov 11, 2025

Choose a reason for hiding this comment

Uh oh!

starocean999 commented Nov 11, 2025

Uh oh!

doris-robot commented Nov 11, 2025

Uh oh!

doris-robot commented Nov 11, 2025

Uh oh!

doris-robot commented Nov 11, 2025

Uh oh!

hello-stephen commented Nov 11, 2025

FE Regression Coverage Report

Uh oh!

hello-stephen commented Nov 12, 2025

FE Regression Coverage Report

Uh oh!

github-actions bot commented Nov 13, 2025

Uh oh!

hello-stephen commented Nov 13, 2025

FE Regression Coverage Report

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Comments

starocean999 commented Nov 7, 2025 •

edited

Loading