Skip to content

[fix](hudi) Fix Hudi query error "do not support DLA type HUDI"#58413

Merged
morningman merged 2 commits intoapache:masterfrom
suxiaogang223:fix_hudi
Nov 27, 2025
Merged

[fix](hudi) Fix Hudi query error "do not support DLA type HUDI"#58413
morningman merged 2 commits intoapache:masterfrom
suxiaogang223:fix_hudi

Conversation

@suxiaogang223
Copy link
Contributor

@suxiaogang223 suxiaogang223 commented Nov 26, 2025 •
edited by morningman
Loading

What problem does this PR solve?

Related PR: #56802

Problem Summary:

LogicalHudiScan did not override withTableAlias() and withCachedOutput() methods, causing HUDI table type information to be lost during optimization, and the table was incorrectly processed as PhysicalFileScan.

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason

  • Behavior changed:

    • No.
    • Yes.

  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@Thearas
Copy link
Contributor

Thearas commented Nov 26, 2025

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@suxiaogang223 suxiaogang223 changed the title Fix hudi [fix](hudi) Fix Hudi query error "do not support DLA type HUDI" Nov 26, 2025
@suxiaogang223
Copy link
Contributor Author

suxiaogang223 commented Nov 26, 2025

run buildall

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Nov 26, 2025
@github-actions
Copy link
Contributor

github-actions bot commented Nov 26, 2025

PR approved by at least one committer and no changes requested.

@github-actions
Copy link
Contributor

github-actions bot commented Nov 26, 2025

PR approved by anyone and no changes requested.

@doris-robot
Copy link

doris-robot commented Nov 26, 2025

TPC-H: Total hot run time: 35116 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 4bc4f7e4b4f9022014d61883c3e4f7bcabbd2cd9, data reload: false

------ Round 1 ----------------------------------
q1	17612	5072	4949	4949
q2	2031	323	206	206
q3	10234	1312	736	736
q4	10227	910	354	354
q5	7542	2417	2349	2349
q6	183	166	136	136
q7	925	767	622	622
q8	9344	1352	1098	1098
q9	7064	5295	5509	5295
q10	6832	2252	1841	1841
q11	486	298	277	277
q12	323	363	229	229
q13	17755	3679	3077	3077
q14	245	235	215	215
q15	573	525	513	513
q16	1050	1006	942	942
q17	594	873	357	357
q18	7545	7313	7131	7131
q19	1127	947	546	546
q20	369	350	229	229
q21	4095	3252	3016	3016
q22	1104	1016	998	998
Total cold run time: 107260 ms
Total hot run time: 35116 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5014	4952	4976	4952
q2	345	404	323	323
q3	2208	2760	2261	2261
q4	1353	1775	1358	1358
q5	4230	4584	4669	4584
q6	217	170	129	129
q7	2098	2004	1804	1804
q8	2608	2634	2573	2573
q9	7607	7461	7462	7461
q10	3162	3330	2872	2872
q11	593	511	491	491
q12	706	785	636	636
q13	3595	4017	3291	3291
q14	279	322	291	291
q15	564	519	541	519
q16	1130	1120	1080	1080
q17	1158	1583	1352	1352
q18	8007	7769	7768	7768
q19	849	824	891	824
q20	2080	2142	1901	1901
q21	5050	4490	4401	4401
q22	1117	1076	1035	1035
Total cold run time: 53970 ms
Total hot run time: 51906 ms

@doris-robot
Copy link

doris-robot commented Nov 26, 2025

TPC-DS: Total hot run time: 184341 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 4bc4f7e4b4f9022014d61883c3e4f7bcabbd2cd9, data reload: false

query1	1073	417	394	394
query2	6568	1598	1568	1568
query3	6760	228	221	221
query4	25593	22577	22604	22577
query5	4424	626	487	487
query6	328	237	234	234
query7	4651	498	313	313
query8	301	269	268	268
query9	8699	2545	2596	2545
query10	525	373	310	310
query11	15291	14854	14583	14583
query12	180	121	115	115
query13	1691	580	458	458
query14	10805	8887	8889	8887
query15	221	206	197	197
query16	7750	704	548	548
query17	1260	763	633	633
query18	2056	450	343	343
query19	232	235	189	189
query20	135	128	127	127
query21	221	136	116	116
query22	3832	4120	3842	3842
query23	33187	32107	32048	32048
query24	8502	2446	2445	2445
query25	640	551	481	481
query26	1247	281	166	166
query27	2723	495	361	361
query28	4339	2151	2161	2151
query29	800	635	524	524
query30	314	243	217	217
query31	854	735	632	632
query32	90	77	75	75
query33	608	404	370	370
query34	782	858	529	529
query35	807	823	755	755
query36	916	937	873	873
query37	121	116	91	91
query38	3283	3395	3262	3262
query39	1468	1420	1442	1420
query40	224	130	119	119
query41	65	61	62	61
query42	132	117	112	112
query43	453	454	442	442
query44	1229	761	769	761
query45	200	187	183	183
query46	876	996	640	640
query47	1695	1737	1631	1631
query48	397	427	328	328
query49	769	502	424	424
query50	660	682	407	407
query51	3883	4053	3976	3976
query52	113	111	107	107
query53	246	269	203	203
query54	300	308	269	269
query55	94	95	92	92
query56	325	320	320	320
query57	1172	1205	1097	1097
query58	287	280	283	280
query59	2440	2571	2380	2380
query60	363	339	336	336
query61	179	156	162	156
query62	788	734	672	672
query63	239	191	194	191
query64	4459	1200	889	889
query65	4107	3974	3986	3974
query66	1143	441	331	331
query67	15258	14933	15104	14933
query68	8341	960	635	635
query69	507	336	308	308
query70	1339	1227	1252	1227
query71	490	337	317	317
query72	5654	4954	4897	4897
query73	681	572	348	348
query74	8892	8777	8405	8405
query75	4038	3328	2831	2831
query76	3792	1123	714	714
query77	807	412	316	316
query78	9530	9679	8819	8819
query79	1665	826	576	576
query80	652	585	483	483
query81	501	270	246	246
query82	422	156	133	133
query83	308	269	251	251
query84	309	119	100	100
query85	922	491	446	446
query86	334	315	284	284
query87	3512	3538	3366	3366
query88	2826	2259	2254	2254
query89	381	333	301	301
query90	1930	218	215	215
query91	178	173	145	145
query92	82	74	63	63
query93	1090	991	662	662
query94	710	450	349	349
query95	492	404	395	395
query96	492	564	276	276
query97	2945	2983	2865	2865
query98	234	217	207	207
query99	1431	1370	1238	1238
Total cold run time: 270860 ms
Total hot run time: 184341 ms

@doris-robot
Copy link

doris-robot commented Nov 26, 2025

ClickBench: Total hot run time: 27.58 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 4bc4f7e4b4f9022014d61883c3e4f7bcabbd2cd9, data reload: false

query1	0.05	0.05	0.05
query2	0.10	0.05	0.04
query3	0.25	0.09	0.08
query4	1.60	0.11	0.11
query5	0.27	0.26	0.24
query6	1.17	0.65	0.64
query7	0.02	0.02	0.03
query8	0.05	0.04	0.04
query9	0.60	0.53	0.52
query10	0.57	0.58	0.58
query11	0.16	0.10	0.11
query12	0.15	0.12	0.11
query13	0.63	0.62	0.61
query14	1.01	0.99	1.00
query15	0.85	0.83	0.84
query16	0.39	0.39	0.39
query17	1.03	1.02	1.03
query18	0.21	0.20	0.19
query19	1.91	1.80	1.79
query20	0.02	0.01	0.02
query21	15.46	0.20	0.13
query22	5.10	0.08	0.05
query23	15.68	0.25	0.10
query24	2.56	0.59	1.19
query25	0.09	0.07	0.07
query26	0.14	0.13	0.13
query27	0.06	0.05	0.06
query28	5.33	1.14	0.93
query29	12.54	3.89	3.21
query30	0.27	0.14	0.11
query31	2.81	0.60	0.38
query32	3.23	0.56	0.47
query33	3.08	3.05	3.02
query34	15.77	5.17	4.54
query35	4.58	4.63	4.60
query36	0.68	0.50	0.49
query37	0.10	0.07	0.06
query38	0.07	0.04	0.04
query39	0.04	0.03	0.03
query40	0.19	0.14	0.15
query41	0.08	0.02	0.02
query42	0.04	0.03	0.03
query43	0.04	0.04	0.03
Total cold run time: 98.98 s
Total hot run time: 27.58 s

@hello-stephen
Copy link
Contributor

hello-stephen commented Nov 26, 2025

FE UT Coverage Report

Increment line coverage 0.00% (0/14) 🎉
Increment coverage report
Complete coverage report

@hello-stephen
Copy link
Contributor

hello-stephen commented Nov 26, 2025

FE Regression Coverage Report

Increment line coverage 0.00% (0/14) 🎉
Increment coverage report
Complete coverage report

@englefly
Copy link
Contributor

englefly commented Nov 27, 2025

run cloud_p0

@morningman morningman merged commit 333d274 into apache:master Nov 27, 2025
34 of 35 checks passed
@seawinde seawinde mentioned this pull request Dec 2, 2025
Merged
16 tasks
morrySnow pushed a commit that referenced this pull request Dec 4, 2025
@seawinde
[fix](mtmv) Fix hudi materialized view union all rewritten plan execu…
a128c14 
…te fail because of invalid slot (#58643)

### What problem does this PR solve?

Related PR: #57558 #58413

Problem Summary:

This fix addresses the following three issues:
1. When invoking the method
org.apache.doris.nereids.trees.plans.logical.LogicalHudiScan#withRelationId,
the output needs to be recalculated to meet expectations.
2. After compensating with a union all due to partial partition
invalidation of a materialized view, during the next round of
transparent rewriting, the rewriting for the child of the union
allshould use the query partitioncorresponding to the specific relation
id to prevent infinite loops.
3. Currently, in the `test_hudi_rewrite_mtmv` test, if the plan
rewritten by the materialized view transparent rewriting is not selected
by the CBO, it is difficult to troubleshoot because explain memo planis
not used. Therefore, the corresponding test method is modified.
seawinde added a commit to seawinde/doris that referenced this pull request Dec 4, 2025
@seawinde
[fix](mtmv) Fix hudi materialized view union all rewritten plan execu…
eb2494e 
…te fail because of invalid slot (apache#58643)

Related PR: apache#57558 apache#58413

Problem Summary:

This fix addresses the following three issues:
1. When invoking the method
org.apache.doris.nereids.trees.plans.logical.LogicalHudiScan#withRelationId,
the output needs to be recalculated to meet expectations.
2. After compensating with a union all due to partial partition
invalidation of a materialized view, during the next round of
transparent rewriting, the rewriting for the child of the union
allshould use the query partitioncorresponding to the specific relation
id to prevent infinite loops.
3. Currently, in the `test_hudi_rewrite_mtmv` test, if the plan
rewritten by the materialized view transparent rewriting is not selected
by the CBO, it is difficult to troubleshoot because explain memo planis
not used. Therefore, the corresponding test method is modified.
seawinde added a commit to seawinde/doris that referenced this pull request Dec 4, 2025
@seawinde
[fix](mtmv) Fix hudi materialized view union all rewritten plan execu…
c5428ea 
…te fail because of invalid slot (apache#58643)

Related PR: apache#57558 apache#58413

Problem Summary:

This fix addresses the following three issues:
1. When invoking the method
org.apache.doris.nereids.trees.plans.logical.LogicalHudiScan#withRelationId,
the output needs to be recalculated to meet expectations.
2. After compensating with a union all due to partial partition
invalidation of a materialized view, during the next round of
transparent rewriting, the rewriting for the child of the union
allshould use the query partitioncorresponding to the specific relation
id to prevent infinite loops.
3. Currently, in the `test_hudi_rewrite_mtmv` test, if the plan
rewritten by the materialized view transparent rewriting is not selected
by the CBO, it is difficult to troubleshoot because explain memo planis
not used. Therefore, the corresponding test method is modified.
@suxiaogang223 suxiaogang223 deleted the fix_hudi branch December 5, 2025 06:22
nagisa-kunhah pushed a commit to nagisa-kunhah/doris that referenced this pull request Dec 14, 2025
@suxiaogang223 @nagisa-kunhah
[fix](hudi) Fix Hudi query error "do not support DLA type HUDI" (apac…
918b737 
…he#58413)

### What problem does this PR solve?

Related PR: apache#56802

Problem Summary:

`LogicalHudiScan` did not override `withTableAlias()` and
`withCachedOutput()` methods, causing HUDI table type information to be
lost during optimization, and the table was incorrectly processed as
`PhysicalFileScan`.
nagisa-kunhah pushed a commit to nagisa-kunhah/doris that referenced this pull request Dec 14, 2025
@seawinde @nagisa-kunhah
[fix](mtmv) Fix hudi materialized view union all rewritten plan execu…
b743cfb 
…te fail because of invalid slot (apache#58643)

### What problem does this PR solve?

Related PR: apache#57558 apache#58413

Problem Summary:

This fix addresses the following three issues:
1. When invoking the method
org.apache.doris.nereids.trees.plans.logical.LogicalHudiScan#withRelationId,
the output needs to be recalculated to meet expectations.
2. After compensating with a union all due to partial partition
invalidation of a materialized view, during the next round of
transparent rewriting, the rewriting for the child of the union
allshould use the query partitioncorresponding to the specific relation
id to prevent infinite loops.
3. Currently, in the `test_hudi_rewrite_mtmv` test, if the plan
rewritten by the materialized view transparent rewriting is not selected
by the CBO, it is difficult to troubleshoot because explain memo planis
not used. Therefore, the corresponding test method is modified.
@suxiaogang223 suxiaogang223 mentioned this pull request Jan 9, 2026
Merged
morningman pushed a commit that referenced this pull request Jan 10, 2026
@suxiaogang223
branch-4.0: Fix: Ensure HUDI tables are processed correctly by throwi…
7c8df01 
…ng an error for incorrect PhysicalFileScan usage (#59709)

bp: #58413
LogicalHudiScan did not override withCachedOutput() method, causing HUDI
table type information to be lost during optimization, and the table was
incorrectly processed as PhysicalFileScan.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@englefly englefly englefly approved these changes

@morningman morningman morningman approved these changes

Assignees

No one assigned

Labels

approved Indicates a PR has been approved by one committer. dev/4.0.3-merged dev/4.1.x reviewed

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

6 participants

@suxiaogang223 @Thearas @doris-robot @hello-stephen @englefly @morningman

Comments