[fix](routine load) Include missing execution properties in SHOW CREATE ROUTINE LOAD - #69932

Aranya7 · 2026-11-01T22:47:09Z

What problem does this PR solve?

Issue Number: N/A (related to #69928)

Related PR: #69929

Problem Summary:

SHOW CREATE ROUTINE LOAD currently omits two supported job properties from its generated CREATE statement:

send_batch_parallelism
load_to_single_tablet

Although both properties are accepted by CREATE ROUTINE LOAD and represented in the job's runtime configuration, they are not included by RoutineLoadJob.getShowCreateInfo().

Consequently, the generated DDL does not faithfully represent these execution settings. Recreating a job using the generated statement can cause the omitted properties to revert to their defaults.

Changes

Include send_batch_parallelism and load_to_single_tablet in the generated PROPERTIES clause.
Use the existing property constants, runtime getters, and SQL formatting helper.
Update the existing exact-output unit test to verify both default values.
Add a unit test verifying nondefault values and correct property formatting.

Behavior and Compatibility

Both properties are emitted with their effective runtime values, including when their defaults are used. This is consistent with the existing behavior of SHOW CREATE ROUTINE LOAD, which renders effective values for other job properties.

This change affects only the generated DDL. It does not modify Routine Load execution, metadata serialization, deserialization, property validation, or Kafka progress.

The fix is independently mergeable from #69929, which restores configured runtime properties during FE recovery. Both changes are complementary but have separate responsibilities.

Scope and Follow-up

This PR intentionally addresses only send_batch_parallelism and load_to_single_tablet.

Additional supported properties, including Unique Key update settings, workload-group configuration, and CSV formatting options, are also omitted from SHOW CREATE ROUTINE LOAD.

These omissions will be investigated and addressed separately, with additional validation for their respective configuration semantics and SQL round-trip behavior.

This PR does not claim to provide complete SHOW CREATE ROUTINE LOAD configuration fidelity.

Release note

SHOW CREATE ROUTINE LOAD now includes send_batch_parallelism and load_to_single_tablet in the generated CREATE statement.

Check List (For Author)

Test
- Regression test
- Unit Test
- Manual test (add detailed scripts or steps below)
- No need to test or manual test. Explain why:
  - This is a refactor/code format and no logic has been changed.
  - Previous test can cover this change.
  - No code files have been changed.
  - Other reason

Unit Test

Extended RoutineLoadJobTest to verify:

Both properties are included with their default values.
Nondefault values are correctly represented.
Property ordering and SQL formatting remain correct.

Also executed ShowCreateRoutineLoadCommandTest to validate the surrounding SHOW CREATE command.

Manual Test

Use an existing paused Routine Load job configured with:
- send_batch_parallelism = 2
- load_to_single_tablet = true
Capture the job's configuration and generated CREATE statement before deploying the fix.
Deploy a development FE containing the SHOW CREATE change.

Execute:

SHOW ALL ROUTINE LOAD FOR rl_recovery_new;
SHOW ALL CREATE ROUTINE LOAD FOR rl_recovery_new;

Verify the generated CREATE statement now includes:

"send_batch_parallelism" = "2",
"load_to_single_tablet" = "true",

Confirm the job ID, state, effective configuration, and stored Kafka progress remain unchanged.
Restore the original development FE binary.

Test Results

Unit Tests — PASSED
- RoutineLoadJobTest — including the updated exact-output test and the new test for nondefault property values.
- ShowCreateRoutineLoadCommandTest — existing command-level tests passed.
FE Build — PASSED
- ./build.sh --fe completed successfully.
Manual Verification — PASSED
- Verified using the existing Routine Load job rl_recovery_new in rl_recovery_verify.
- Confirmed that SHOW CREATE ROUTINE LOAD previously omitted send_batch_parallelism and load_to_single_tablet.
- Verified that the updated output includes the configured values:
```
"send_batch_parallelism" = "2",
"load_to_single_tablet" = "true",
```
- Confirmed that the job ID, state, configuration, and stored Kafka progress remained unchanged.
Behavior changed:
- No.
- Yes.
SHOW CREATE ROUTINE LOAD now includes the two previously omitted properties.
Does this need documentation?
- No.
- Yes.
No new SQL syntax or configuration options are introduced.

Check List (For Reviewer who merge this PR)

Confirm the release note
Confirm test cases
Confirm document
Add branch pick label

hello-stephen · 2026-11-01T22:47:16Z

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

What problem was fixed (it's best to include specific error reporting information). How it was fixed.
Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
What features were added. Why was this function added?
Which code was refactored and why was this part of the code refactored?
Which functions were optimized and what is the difference before and after the optimization?

Aranya7 · 2026-11-01T22:48:35Z

run buildall

hello-stephen · 2026-11-01T23:18:44Z

TPC-H: Total hot run time: 28325 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 2a4100ccc91f3333d36862f6adaf00fe3b7394a4, data reload: false

------ Round 1 ----------------------------------
============================================
q1	17594	5136	5061	5061
q2	2128	293	255	255
q3	10261	1360	815	815
q4	4688	460	333	333
q5	7440	796	532	532
q6	186	187	155	155
q7	750	789	601	601
q8	9264	1326	1046	1046
q9	5518	4166	4186	4166
q10	6821	1290	1009	1009
q11	429	245	224	224
q12	638	394	275	275
q13	18095	3571	3019	3019
q14	267	265	243	243
q15	q16	728	712	660	660
q17	1181	748	585	585
q18	6996	6138	6065	6065
q19	1096	954	575	575
q20	381	328	220	220
q21	5300	2235	2255	2235
q22	340	291	251	251
Total cold run time: 100101 ms
Total hot run time: 28325 ms

----- Round 2, with runtime_filter_mode=off -----
============================================
q1	5586	5597	5594	5594
q2	700	588	531	531
q3	5200	5394	4897	4897
q4	1978	2142	1547	1547
q5	5258	5187	5546	5187
q6	258	199	166	166
q7	2292	2035	1847	1847
q8	3008	2680	2708	2680
q9	7985	7971	7945	7945
q10	3735	3883	3258	3258
q11	607	441	429	429
q12	734	703	510	510
q13	3509	3751	3333	3333
q14	329	323	295	295
q15	q16	713	727	631	631
q17	8306	7132	6753	6753
q18	12954	12146	12863	12146
q19	898	780	826	780
q20	2165	2165	1925	1925
q21	5756	4882	4792	4792
q22	512	487	416	416
Total cold run time: 72483 ms
Total hot run time: 65662 ms

hello-stephen · 2026-11-01T23:28:30Z

TPC-DS: Total hot run time: 150684 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 2a4100ccc91f3333d36862f6adaf00fe3b7394a4, data reload: false

query5	4319	585	436	436
query6	418	204	193	193
query7	4817	434	228	228
query8	316	164	147	147
query9	8769	3149	3082	3082
query10	469	307	265	265
query11	6053	3845	3548	3548
query12	145	88	87	87
query13	1264	451	322	322
query14	6112	3894	3629	3629
query14_1	3381	3383	3392	3383
query15	207	200	190	190
query16	927	421	412	412
query17	841	653	536	536
query18	2434	422	325	325
query19	198	168	132	132
query20	84	88	86	86
query21	207	126	107	107
query22	12900	12897	12790	12790
query23	13264	12740	12435	12435
query23_1	12482	12566	12543	12543
query24	7377	824	485	485
query24_1	480	474	500	474
query25	529	414	352	352
query26	1270	259	141	141
query27	2768	440	264	264
query28	4641	2028	2043	2028
query29	1580	588	463	463
query30	301	217	187	187
query31	1017	935	816	816
query32	155	93	96	93
query33	521	300	245	245
query34	906	859	517	517
query35	734	744	659	659
query36	813	844	711	711
query37	129	104	83	83
query38	1756	1727	1647	1647
query39	791	792	742	742
query39_1	730	746	742	742
query40	237	117	97	97
query41	73	69	68	68
query42	87	88	84	84
query43	375	366	308	308
query44	1310	693	726	693
query45	176	191	169	169
query46	845	976	584	584
query47	2842	2917	2826	2826
query48	334	295	217	217
query49	558	394	287	287
query50	677	288	212	212
query51	10610	10405	10560	10405
query52	75	77	69	69
query53	179	206	147	147
query54	232	207	184	184
query55	67	65	68	65
query56	231	209	204	204
query57	1627	1634	1560	1560
query58	281	257	254	254
query59	2153	2215	1981	1981
query60	278	236	222	222
query61	144	145	135	135
query62	391	335	285	285
query63	186	158	153	153
query64	2739	963	759	759
query65	3325	3255	3267	3255
query66	1773	409	293	293
query67	19954	19924	19951	19924
query68	2943	1014	609	609
query69	380	292	251	251
query70	843	747	763	747
query71	301	229	223	223
query72	2649	2577	2318	2318
query73	523	523	294	294
query74	4573	4499	4302	4302
query75	2340	2286	1880	1880
query76	2328	1003	581	581
query77	338	373	289	289
query78	10004	9685	9116	9116
query79	991	853	524	524
query80	805	416	342	342
query81	570	323	265	265
query82	578	131	106	106
query83	307	202	197	197
query84	305	129	107	107
query85	873	427	357	357
query86	349	235	232	232
query87	2000	1934	1789	1789
query88	3760	2761	2744	2744
query89	340	278	243	243
query90	1787	166	155	155
query91	165	151	123	123
query92	98	86	89	86
query93	1006	997	569	569
query94	532	332	266	266
query95	573	336	370	336
query96	688	479	237	237
query97	2448	2439	2311	2311
query98	159	154	147	147
query99	758	761	653	653
Total cold run time: 233960 ms
Total hot run time: 150684 ms

hello-stephen · 2026-11-01T23:33:22Z

ClickBench: Total hot run time: 24.29 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 2a4100ccc91f3333d36862f6adaf00fe3b7394a4, data reload: false

query1	0.00	0.00	0.00
query2	0.10	0.05	0.05
query3	0.26	0.08	0.09
query4	1.61	0.10	0.10
query5	0.24	0.24	0.24
query6	1.18	0.70	0.68
query7	0.04	0.01	0.00
query8	0.06	0.04	0.04
query9	0.42	0.35	0.35
query10	0.58	0.60	0.58
query11	0.20	0.14	0.14
query12	0.18	0.16	0.15
query13	0.49	0.48	0.50
query14	1.04	1.01	1.01
query15	0.62	0.59	0.60
query16	0.35	0.35	0.36
query17	1.10	1.14	1.17
query18	0.22	0.22	0.20
query19	2.15	1.97	1.95
query20	0.02	0.01	0.01
query21	15.49	0.20	0.14
query22	4.80	0.05	0.06
query23	16.25	0.32	0.12
query24	2.73	1.00	0.38
query25	0.13	0.06	0.05
query26	1.63	0.22	0.16
query27	0.04	0.03	0.04
query28	3.02	0.79	0.36
query29	12.48	4.15	3.26
query30	0.28	0.13	0.10
query31	2.77	0.55	0.33
query32	3.24	0.59	0.51
query33	3.23	3.18	3.13
query34	15.43	4.03	3.40
query35	3.33	3.34	3.33
query36	0.59	0.45	0.47
query37	0.10	0.07	0.06
query38	0.05	0.04	0.04
query39	0.03	0.04	0.04
query40	0.17	0.16	0.14
query41	0.09	0.03	0.02
query42	0.04	0.02	0.02
query43	0.04	0.03	0.04
Total cold run time: 96.82 s
Total hot run time: 24.29 s

Aranya7 · 2026-11-02T05:21:48Z

run check_coverage

Fix missing Routine Load properties in SHOW CREATE

2a4100c

Aranya7 requested review from gavinchou and liaoxin01 as code owners November 1, 2026 22:47

Navigation Menu

Repository navigation

[fix](routine load) Include missing execution properties in SHOW CREATE ROUTINE LOAD - #69932