Skip to content

[Fix](agg) fix push agg op in nullable column before projection#58234

Merged
BiteTheDDDDt merged 1 commit intoapache:masterfrom
HappenLee:20875
Nov 22, 2025
Merged

[Fix](agg) fix push agg op in nullable column before projection#58234
BiteTheDDDDt merged 1 commit intoapache:masterfrom
HappenLee:20875

Conversation

@HappenLee
Copy link
Contributor

@HappenLee HappenLee commented Nov 21, 2025

What problem does this PR solve?

Fix Fix push aggregate operation in nullable column before projection

Changes

Core Logic Adjustment (AggregateStrategies.java):
Removed redundant Count function-specific nullable slot checking logic, unifying aggregate function argument validation into a single stream-based check
Enhanced argument type verification: For SlotReference and numeric-type Cast (with SlotReference as child) arguments, uniformly collect slots to checkNullSlots for nullability validation
Fixed inconsistent aggregate pushdown judgment for nested projection scenarios, ensuring consistent validation logic for aggregate function arguments across different code branches

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason
  • Behavior changed:

    • No.
    • Yes.
  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@Thearas
Copy link
Contributor

Thearas commented Nov 21, 2025

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@HappenLee
Copy link
Contributor Author

run buildall

924060929
924060929 previously approved these changes Nov 21, 2025
@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Nov 21, 2025
@github-actions
Copy link
Contributor

PR approved by at least one committer and no changes requested.

@github-actions
Copy link
Contributor

PR approved by anyone and no changes requested.

@doris-robot
Copy link

TPC-H: Total hot run time: 34251 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit b354ccbf3b04a2ae90b99079caab34b1482fdf7f, data reload: false

------ Round 1 ----------------------------------
q1	17626	5059	4878	4878
q2	2027	310	200	200
q3	10277	1278	705	705
q4	10230	922	367	367
q5	7507	2372	2287	2287
q6	182	167	135	135
q7	906	756	622	622
q8	9343	1326	1071	1071
q9	7028	5308	5312	5308
q10	6846	2241	1788	1788
q11	485	292	278	278
q12	327	352	215	215
q13	17832	3637	3031	3031
q14	239	249	231	231
q15	586	541	522	522
q16	1025	1018	953	953
q17	594	867	360	360
q18	7468	7243	7226	7226
q19	1097	965	559	559
q20	365	342	232	232
q21	4159	2557	2302	2302
q22	1048	1011	981	981
Total cold run time: 107197 ms
Total hot run time: 34251 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4946	4919	4910	4910
q2	329	396	314	314
q3	2219	2619	2294	2294
q4	1325	1826	1382	1382
q5	4209	4237	4498	4237
q6	218	175	131	131
q7	2042	1984	1801	1801
q8	2675	2604	2549	2549
q9	7513	7494	7573	7494
q10	3091	3280	2775	2775
q11	602	518	518	518
q12	655	771	606	606
q13	3481	3901	3276	3276
q14	302	321	271	271
q15	535	504	508	504
q16	1048	1132	1069	1069
q17	1164	1565	1415	1415
q18	7965	7486	7513	7486
q19	792	833	747	747
q20	1868	1964	1816	1816
q21	4543	4302	4321	4302
q22	1055	1017	999	999
Total cold run time: 52577 ms
Total hot run time: 50896 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 187244 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit b354ccbf3b04a2ae90b99079caab34b1482fdf7f, data reload: false

query1	1055	414	407	407
query2	6575	1691	1707	1691
query3	6754	225	221	221
query4	26955	23182	23056	23056
query5	4377	625	497	497
query6	350	242	227	227
query7	4656	492	299	299
query8	325	258	252	252
query9	8734	2584	2581	2581
query10	498	338	291	291
query11	15634	15088	14878	14878
query12	180	117	112	112
query13	1704	580	457	457
query14	10930	9185	9208	9185
query15	197	187	172	172
query16	7283	692	521	521
query17	1264	771	627	627
query18	2014	438	337	337
query19	215	233	178	178
query20	128	131	121	121
query21	213	140	123	123
query22	3974	4108	4006	4006
query23	34030	33043	32859	32859
query24	8473	2419	2451	2419
query25	611	559	477	477
query26	1242	324	166	166
query27	2714	488	357	357
query28	4363	2171	2168	2168
query29	825	601	480	480
query30	313	226	190	190
query31	903	796	697	697
query32	81	74	73	73
query33	584	365	324	324
query34	797	865	527	527
query35	829	844	755	755
query36	954	969	867	867
query37	120	114	91	91
query38	3559	3525	3434	3434
query39	1487	1413	1410	1410
query40	230	133	119	119
query41	70	65	63	63
query42	124	116	112	112
query43	482	512	486	486
query44	1231	767	766	766
query45	184	173	177	173
query46	873	977	643	643
query47	1749	1772	1762	1762
query48	392	429	317	317
query49	770	484	402	402
query50	645	676	416	416
query51	3930	3892	3918	3892
query52	118	114	103	103
query53	240	273	196	196
query54	358	290	278	278
query55	92	94	84	84
query56	308	331	311	311
query57	1143	1168	1128	1128
query58	278	275	267	267
query59	2449	2589	2455	2455
query60	347	351	320	320
query61	161	159	157	157
query62	759	709	654	654
query63	225	194	200	194
query64	4494	1164	883	883
query65	4058	3957	3913	3913
query66	1165	435	329	329
query67	15117	15032	15130	15032
query68	8207	949	638	638
query69	483	325	284	284
query70	1350	1305	1249	1249
query71	488	347	317	317
query72	5979	4902	4745	4745
query73	654	573	365	365
query74	8794	8899	8966	8899
query75	3951	3354	2818	2818
query76	3761	1146	729	729
query77	839	381	322	322
query78	9510	9613	8845	8845
query79	2022	797	604	604
query80	630	564	510	510
query81	487	259	228	228
query82	446	159	137	137
query83	277	266	252	252
query84	256	103	94	94
query85	936	472	455	455
query86	343	298	296	296
query87	3657	3767	3601	3601
query88	3304	2233	2244	2233
query89	389	319	287	287
query90	1979	218	218	218
query91	161	160	130	130
query92	88	67	63	63
query93	1229	1026	688	688
query94	715	415	341	341
query95	411	318	308	308
query96	491	565	275	275
query97	2936	3002	2814	2814
query98	239	219	212	212
query99	1378	1372	1299	1299
Total cold run time: 274282 ms
Total hot run time: 187244 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 27.23 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit b354ccbf3b04a2ae90b99079caab34b1482fdf7f, data reload: false

query1	0.05	0.05	0.05
query2	0.10	0.05	0.04
query3	0.26	0.08	0.08
query4	1.61	0.11	0.11
query5	0.28	0.25	0.25
query6	1.20	0.65	0.64
query7	0.03	0.02	0.03
query8	0.06	0.04	0.04
query9	0.58	0.53	0.51
query10	0.57	0.58	0.57
query11	0.16	0.11	0.12
query12	0.15	0.12	0.12
query13	0.62	0.61	0.60
query14	1.01	1.01	0.99
query15	0.86	0.84	0.83
query16	0.40	0.40	0.39
query17	1.02	1.06	1.05
query18	0.22	0.20	0.20
query19	1.95	1.87	1.87
query20	0.02	0.02	0.01
query21	15.46	0.17	0.15
query22	5.10	0.07	0.05
query23	15.68	0.26	0.10
query24	2.67	0.46	0.27
query25	0.08	0.07	0.06
query26	0.13	0.12	0.13
query27	0.06	0.06	0.06
query28	3.90	1.13	0.92
query29	12.57	3.84	3.17
query30	0.28	0.13	0.11
query31	2.81	0.61	0.39
query32	3.23	0.54	0.47
query33	3.06	3.06	3.02
query34	15.78	5.17	4.49
query35	4.54	4.52	4.55
query36	0.66	0.50	0.49
query37	0.10	0.07	0.07
query38	0.06	0.04	0.04
query39	0.04	0.03	0.03
query40	0.16	0.13	0.14
query41	0.09	0.03	0.03
query42	0.04	0.03	0.03
query43	0.04	0.03	0.03
Total cold run time: 97.69 s
Total hot run time: 27.23 s

@HappenLee
Copy link
Contributor Author

run buildall

@github-actions github-actions bot removed the approved Indicates a PR has been approved by one committer. label Nov 21, 2025
@HappenLee HappenLee force-pushed the 20875 branch 2 times, most recently from 28ee2a5 to 08d0c92 Compare November 21, 2025 10:53
@HappenLee
Copy link
Contributor Author

run buildall

@HappenLee
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 34093 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit a8688620c6c87791125f313134f5511f236092d2, data reload: false

------ Round 1 ----------------------------------
q1	17636	5049	4902	4902
q2	2030	313	210	210
q3	10243	1296	704	704
q4	10221	873	365	365
q5	7524	2330	2373	2330
q6	178	162	134	134
q7	909	766	617	617
q8	9330	1349	1085	1085
q9	7110	5300	5369	5300
q10	6924	2244	1780	1780
q11	497	297	283	283
q12	367	356	221	221
q13	17807	3660	3043	3043
q14	230	234	213	213
q15	582	513	493	493
q16	1003	994	935	935
q17	591	855	364	364
q18	7613	7155	7110	7110
q19	1367	932	547	547
q20	345	336	223	223
q21	3782	3180	2289	2289
q22	1075	1034	945	945
Total cold run time: 107364 ms
Total hot run time: 34093 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5011	4945	4909	4909
q2	320	392	318	318
q3	2168	2688	2307	2307
q4	1326	1787	1343	1343
q5	4184	4395	4432	4395
q6	211	180	130	130
q7	2062	1932	1859	1859
q8	2638	2550	2464	2464
q9	7540	7635	7640	7635
q10	3054	3238	2809	2809
q11	587	558	527	527
q12	675	823	617	617
q13	3593	3887	3326	3326
q14	280	295	273	273
q15	531	491	510	491
q16	1054	1121	1038	1038
q17	1211	1625	1386	1386
q18	8126	7634	7579	7579
q19	754	757	804	757
q20	1961	1942	1841	1841
q21	4740	4372	4305	4305
q22	1065	1041	1000	1000
Total cold run time: 53091 ms
Total hot run time: 51309 ms

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Nov 21, 2025
@github-actions
Copy link
Contributor

PR approved by at least one committer and no changes requested.

@doris-robot
Copy link

TPC-DS: Total hot run time: 186869 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit a8688620c6c87791125f313134f5511f236092d2, data reload: false

query1	1019	407	427	407
query2	6568	1655	1702	1655
query3	6755	223	215	215
query4	26116	23064	22727	22727
query5	4412	617	460	460
query6	330	233	221	221
query7	4645	487	297	297
query8	289	241	236	236
query9	8697	2572	2549	2549
query10	522	348	271	271
query11	15314	15163	14758	14758
query12	169	113	112	112
query13	1679	554	439	439
query14	10254	9068	9194	9068
query15	202	181	171	171
query16	7150	677	504	504
query17	1043	752	645	645
query18	2000	429	329	329
query19	212	203	176	176
query20	129	127	123	123
query21	217	136	117	117
query22	3987	4034	3994	3994
query23	33828	32980	32951	32951
query24	8473	2430	2364	2364
query25	603	509	425	425
query26	1239	268	159	159
query27	2782	503	351	351
query28	4382	2187	2165	2165
query29	814	600	479	479
query30	297	222	200	200
query31	910	797	741	741
query32	81	74	72	72
query33	598	362	320	320
query34	786	864	515	515
query35	798	823	727	727
query36	951	1009	891	891
query37	117	113	85	85
query38	3537	3498	3467	3467
query39	1478	1443	1458	1443
query40	225	128	117	117
query41	67	61	62	61
query42	123	120	111	111
query43	488	495	450	450
query44	1229	773	761	761
query45	184	177	170	170
query46	868	968	631	631
query47	1758	1762	1729	1729
query48	389	413	322	322
query49	795	492	406	406
query50	646	684	415	415
query51	3889	3967	4043	3967
query52	109	112	104	104
query53	238	268	204	204
query54	300	278	279	278
query55	90	85	85	85
query56	316	310	292	292
query57	1202	1159	1116	1116
query58	289	274	273	273
query59	2528	2659	2443	2443
query60	333	322	323	322
query61	160	157	155	155
query62	801	715	661	661
query63	229	191	201	191
query64	4540	1174	976	976
query65	3994	3930	3930	3930
query66	1182	461	345	345
query67	15273	15047	14744	14744
query68	4671	963	691	691
query69	498	322	289	289
query70	1284	1213	1306	1213
query71	446	343	344	343
query72	6573	5057	5126	5057
query73	707	672	381	381
query74	8958	9124	9042	9042
query75	3384	3298	2763	2763
query76	3254	1143	735	735
query77	521	396	303	303
query78	9407	9818	8855	8855
query79	1012	829	582	582
query80	1249	568	492	492
query81	553	257	232	232
query82	402	166	139	139
query83	353	258	239	239
query84	293	119	106	106
query85	927	481	443	443
query86	387	323	307	307
query87	3735	3744	3615	3615
query88	2874	2274	2237	2237
query89	393	330	288	288
query90	1747	219	224	219
query91	168	163	145	145
query92	75	70	69	69
query93	1124	988	691	691
query94	696	470	350	350
query95	416	342	332	332
query96	482	599	291	291
query97	2912	2975	2858	2858
query98	248	226	211	211
query99	1334	1416	1263	1263
Total cold run time: 266711 ms
Total hot run time: 186869 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 27.31 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit a8688620c6c87791125f313134f5511f236092d2, data reload: false

query1	0.06	0.05	0.04
query2	0.09	0.05	0.04
query3	0.25	0.08	0.08
query4	1.61	0.11	0.10
query5	0.27	0.25	0.26
query6	1.16	0.64	0.64
query7	0.03	0.03	0.02
query8	0.05	0.04	0.04
query9	0.58	0.52	0.52
query10	0.58	0.57	0.55
query11	0.16	0.11	0.11
query12	0.15	0.12	0.11
query13	0.62	0.60	0.62
query14	1.00	0.99	1.00
query15	0.84	0.82	0.85
query16	0.40	0.41	0.40
query17	1.06	1.02	1.02
query18	0.21	0.19	0.20
query19	1.95	1.81	1.83
query20	0.02	0.02	0.02
query21	15.44	0.17	0.13
query22	5.19	0.07	0.05
query23	15.68	0.24	0.10
query24	2.90	0.90	0.28
query25	0.08	0.07	0.06
query26	0.13	0.13	0.13
query27	0.06	0.06	0.05
query28	3.78	1.11	0.93
query29	12.57	3.89	3.20
query30	0.28	0.14	0.11
query31	2.81	0.60	0.40
query32	3.23	0.55	0.46
query33	2.94	3.11	3.13
query34	15.80	5.22	4.54
query35	4.61	4.55	4.59
query36	0.68	0.50	0.50
query37	0.11	0.07	0.07
query38	0.06	0.04	0.04
query39	0.04	0.03	0.02
query40	0.18	0.15	0.14
query41	0.09	0.04	0.03
query42	0.05	0.03	0.03
query43	0.05	0.04	0.03
Total cold run time: 97.85 s
Total hot run time: 27.31 s

@hello-stephen
Copy link
Contributor

FE UT Coverage Report

Increment line coverage 48.00% (12/25) 🎉
Increment coverage report
Complete coverage report

@hello-stephen
Copy link
Contributor

FE Regression Coverage Report

Increment line coverage 72.00% (18/25) 🎉
Increment coverage report
Complete coverage report

@BiteTheDDDDt BiteTheDDDDt merged commit 703badd into apache:master Nov 22, 2025
27 of 28 checks passed
github-actions bot pushed a commit that referenced this pull request Nov 24, 2025
[Fix](chrome-extension://dbjibobgilijgolhjdcbdebjhejelffo/agg) Fix push
aggregate operation in nullable column before projection
# Changes
Core Logic Adjustment (AggregateStrategies.java):
Removed redundant Count function-specific nullable slot checking logic,
unifying aggregate function argument validation into a single
stream-based check
Enhanced argument type verification: For SlotReference and numeric-type
Cast (with SlotReference as child) arguments, uniformly collect slots to
checkNullSlots for nullability validation
Fixed inconsistent aggregate pushdown judgment for nested projection
scenarios, ensuring consistent validation logic for aggregate function
arguments across different code branches
yiguolei pushed a commit that referenced this pull request Nov 25, 2025
…ection #58234 (#58281)

Cherry-picked from #58234

Co-authored-by: HappenLee <happenlee@selectdb.com>
@yiguolei yiguolei mentioned this pull request Dec 2, 2025
nagisa-kunhah pushed a commit to nagisa-kunhah/doris that referenced this pull request Dec 14, 2025
…he#58234)

[Fix](chrome-extension://dbjibobgilijgolhjdcbdebjhejelffo/agg) Fix push
aggregate operation in nullable column before projection
# Changes
Core Logic Adjustment (AggregateStrategies.java):
Removed redundant Count function-specific nullable slot checking logic,
unifying aggregate function argument validation into a single
stream-based check
Enhanced argument type verification: For SlotReference and numeric-type
Cast (with SlotReference as child) arguments, uniformly collect slots to
checkNullSlots for nullability validation
Fixed inconsistent aggregate pushdown judgment for nested projection
scenarios, ensuring consistent validation logic for aggregate function
arguments across different code branches
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by one committer. dev/4.0.2-merged reviewed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

7 participants