Skip to content

[Fix](Streamingjob) fix postgres incr consumer too slow#59919

Merged
JNSimba merged 11 commits intoapache:masterfrom
JNSimba:fix_incr_consumer_slow
Jan 16, 2026
Merged

[Fix](Streamingjob) fix postgres incr consumer too slow#59919
JNSimba merged 11 commits intoapache:masterfrom
JNSimba:fix_incr_consumer_slow

Conversation

@JNSimba
Copy link
Member

@JNSimba JNSimba commented Jan 15, 2026 •
edited
Loading

What problem does this PR solve?

Issue Number: close #xxx

Related PR: #59798

If you close the reader before committing the LSN, the commit will not be completed.

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason

  • Behavior changed:

    • No.
    • Yes.

  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@JNSimba
Copy link
Member Author

JNSimba commented Jan 15, 2026

run buildall

@JNSimba
Copy link
Member Author

JNSimba commented Jan 15, 2026

run buildall

@doris-robot
Copy link

doris-robot commented Jan 15, 2026

TPC-H: Total hot run time: 31466 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 41548b98b998c7cd0cd8e7303a7308edc5cff597, data reload: false

------ Round 1 ----------------------------------
q1	17623	4299	4038	4038
q2	2063	348	247	247
q3	10159	1244	710	710
q4	10194	815	304	304
q5	7551	2105	1843	1843
q6	189	170	137	137
q7	932	790	645	645
q8	9260	1434	1095	1095
q9	4911	4610	4614	4610
q10	6787	1794	1394	1394
q11	534	304	280	280
q12	691	755	584	584
q13	17795	3788	3102	3102
q14	288	294	274	274
q15	597	528	511	511
q16	682	661	657	657
q17	659	778	525	525
q18	6499	6402	6442	6402
q19	1242	979	627	627
q20	408	349	245	245
q21	3058	2414	2249	2249
q22	1030	1055	987	987
Total cold run time: 103152 ms
Total hot run time: 31466 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4103	4076	4030	4030
q2	328	414	316	316
q3	2078	2556	2210	2210
q4	1308	1759	1316	1316
q5	4162	4042	4190	4042
q6	200	165	131	131
q7	1870	1797	1687	1687
q8	2833	2512	2530	2512
q9	7290	7047	7082	7047
q10	2551	2802	2293	2293
q11	565	498	456	456
q12	747	747	631	631
q13	3663	4141	3459	3459
q14	287	311	285	285
q15	562	521	498	498
q16	645	692	636	636
q17	1162	1296	1353	1296
q18	8242	7787	7875	7787
q19	890	875	953	875
q20	2040	2051	1889	1889
q21	4436	4280	4111	4111
q22	1042	1060	949	949
Total cold run time: 51004 ms
Total hot run time: 48456 ms

@doris-robot
Copy link

doris-robot commented Jan 15, 2026

TPC-DS: Total hot run time: 174775 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 41548b98b998c7cd0cd8e7303a7308edc5cff597, data reload: false

query5	4386	629	520	520
query6	380	242	216	216
query7	4228	455	263	263
query8	359	254	241	241
query9	8701	2869	2907	2869
query10	546	381	331	331
query11	15298	15157	14921	14921
query12	179	117	125	117
query13	1256	504	422	422
query14	6113	3114	2844	2844
query14_1	2753	2769	2712	2712
query15	205	198	175	175
query16	979	490	472	472
query17	1114	677	585	585
query18	2442	443	347	347
query19	232	220	200	200
query20	120	112	112	112
query21	220	134	116	116
query22	4040	4045	4129	4045
query23	15938	15610	15383	15383
query23_1	15709	15474	15575	15474
query24	7156	1566	1160	1160
query24_1	1157	1178	1208	1178
query25	566	467	447	447
query26	1243	263	155	155
query27	2764	441	282	282
query28	4557	2129	2126	2126
query29	813	550	475	475
query30	305	245	201	201
query31	821	625	561	561
query32	84	81	71	71
query33	516	333	309	309
query34	889	879	516	516
query35	709	756	695	695
query36	887	870	845	845
query37	129	92	86	86
query38	2714	2698	2751	2698
query39	765	742	724	724
query39_1	704	714	708	708
query40	223	130	117	117
query41	65	61	63	61
query42	102	102	102	102
query43	489	449	469	449
query44	1339	732	728	728
query45	192	185	180	180
query46	823	931	569	569
query47	1457	1470	1348	1348
query48	311	316	234	234
query49	599	423	373	373
query50	616	277	199	199
query51	3823	3904	3850	3850
query52	107	103	97	97
query53	286	325	273	273
query54	297	266	258	258
query55	80	78	76	76
query56	301	304	309	304
query57	1025	970	936	936
query58	265	264	251	251
query59	2146	2125	2121	2121
query60	333	335	317	317
query61	153	151	164	151
query62	406	349	332	332
query63	296	273	260	260
query64	4990	1268	993	993
query65	3873	3774	3741	3741
query66	1444	426	326	326
query67	15499	15625	15414	15414
query68	2435	1093	746	746
query69	435	366	323	323
query70	988	909	946	909
query71	324	309	286	286
query72	5401	3295	3342	3295
query73	588	717	301	301
query74	8733	8751	8537	8537
query75	2770	2790	2452	2452
query76	2284	1035	674	674
query77	358	373	313	313
query78	9726	9880	9152	9152
query79	1059	890	624	624
query80	1275	594	500	500
query81	539	264	240	240
query82	1256	153	109	109
query83	331	263	235	235
query84	256	122	93	93
query85	905	504	459	459
query86	442	318	278	278
query87	2851	2920	2761	2761
query88	3498	2572	2542	2542
query89	380	359	322	322
query90	1982	176	161	161
query91	174	159	136	136
query92	72	75	69	69
query93	923	914	529	529
query94	662	328	289	289
query95	580	393	321	321
query96	637	523	233	233
query97	2360	2404	2299	2299
query98	215	203	205	203
query99	594	564	529	529
Total cold run time: 247803 ms
Total hot run time: 174775 ms

@doris-robot
Copy link

doris-robot commented Jan 15, 2026

ClickBench: Total hot run time: 26.63 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 41548b98b998c7cd0cd8e7303a7308edc5cff597, data reload: false

query1	0.04	0.05	0.05
query2	0.10	0.04	0.05
query3	0.25	0.08	0.08
query4	1.61	0.11	0.11
query5	0.29	0.25	0.25
query6	1.14	0.68	0.66
query7	0.03	0.03	0.03
query8	0.05	0.04	0.03
query9	0.58	0.51	0.50
query10	0.56	0.55	0.53
query11	0.14	0.10	0.10
query12	0.14	0.11	0.11
query13	0.59	0.58	0.58
query14	0.95	0.93	0.93
query15	0.79	0.78	0.79
query16	0.39	0.39	0.41
query17	1.09	1.00	0.98
query18	0.23	0.21	0.21
query19	1.98	1.87	1.86
query20	0.02	0.02	0.01
query21	15.46	0.25	0.14
query22	5.31	0.05	0.05
query23	16.09	0.27	0.10
query24	1.06	0.27	0.54
query25	0.13	0.07	0.05
query26	0.14	0.13	0.12
query27	0.11	0.05	0.08
query28	3.93	1.08	0.89
query29	12.55	3.89	3.13
query30	0.27	0.14	0.11
query31	2.82	0.64	0.39
query32	3.24	0.55	0.46
query33	2.96	3.01	3.01
query34	15.95	5.09	4.39
query35	4.46	4.45	4.47
query36	0.66	0.50	0.49
query37	0.11	0.06	0.07
query38	0.07	0.04	0.03
query39	0.04	0.02	0.03
query40	0.17	0.15	0.12
query41	0.09	0.03	0.02
query42	0.04	0.02	0.03
query43	0.05	0.03	0.04
Total cold run time: 96.68 s
Total hot run time: 26.63 s

@hello-stephen
Copy link
Contributor

hello-stephen commented Jan 15, 2026

FE UT Coverage Report

Increment line coverage 0.00% (0/1) 🎉
Increment coverage report
Complete coverage report

@hello-stephen
Copy link
Contributor

hello-stephen commented Jan 15, 2026

FE Regression Coverage Report

Increment line coverage 0.00% (0/1) 🎉
Increment coverage report
Complete coverage report

@JNSimba
Copy link
Member Author

JNSimba commented Jan 15, 2026

run buildall

@JNSimba
Copy link
Member Author

JNSimba commented Jan 15, 2026

run buildall

@JNSimba JNSimba force-pushed the fix_incr_consumer_slow branch from 2be9b48 to 6aef060 Compare January 15, 2026 12:58
@JNSimba
Copy link
Member Author

JNSimba commented Jan 15, 2026

rum buildall

@doris-robot
Copy link

doris-robot commented Jan 15, 2026

TPC-H: Total hot run time: 31949 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 2be9b4871efe9d131803a235d959c2a90cfc0f99, data reload: false

------ Round 1 ----------------------------------
q1	17619	4241	4042	4042
q2	2068	354	258	258
q3	10103	1271	712	712
q4	10224	853	307	307
q5	7512	2052	1908	1908
q6	194	167	138	138
q7	934	773	656	656
q8	9275	1374	1186	1186
q9	4825	4588	4503	4503
q10	6722	1775	1459	1459
q11	536	300	280	280
q12	697	751	569	569
q13	17780	3822	3124	3124
q14	282	291	272	272
q15	585	522	507	507
q16	696	678	645	645
q17	672	858	445	445
q18	6662	6332	6731	6332
q19	1254	1025	764	764
q20	439	397	260	260
q21	3233	2741	2590	2590
q22	1185	1083	992	992
Total cold run time: 103497 ms
Total hot run time: 31949 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4280	4372	4315	4315
q2	340	427	329	329
q3	2258	2875	2516	2516
q4	1505	1891	1482	1482
q5	4757	4345	4288	4288
q6	206	170	130	130
q7	2048	1954	1723	1723
q8	2535	2418	2438	2418
q9	7314	7321	7130	7130
q10	2497	2787	2404	2404
q11	538	480	493	480
q12	783	795	672	672
q13	3597	4033	3266	3266
q14	268	292	261	261
q15	571	491	478	478
q16	632	653	603	603
q17	1101	1291	1320	1291
q18	7467	7278	7348	7278
q19	793	783	792	783
q20	1867	1936	1779	1779
q21	4514	4231	4166	4166
q22	1073	1018	985	985
Total cold run time: 50944 ms
Total hot run time: 48777 ms

@doris-robot
Copy link

doris-robot commented Jan 15, 2026

TPC-DS: Total hot run time: 173647 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 2be9b4871efe9d131803a235d959c2a90cfc0f99, data reload: false

query5	4573	628	490	490
query6	333	232	210	210
query7	4229	450	252	252
query8	349	261	248	248
query9	8749	2946	2897	2897
query10	507	384	339	339
query11	15227	15070	14821	14821
query12	185	115	111	111
query13	1240	481	373	373
query14	6495	3039	2873	2873
query14_1	2736	2716	2707	2707
query15	195	192	168	168
query16	997	513	473	473
query17	1116	686	573	573
query18	2706	446	340	340
query19	222	225	200	200
query20	153	113	114	113
query21	219	141	123	123
query22	3945	4009	3822	3822
query23	15991	15567	15266	15266
query23_1	15602	15319	15522	15319
query24	7157	1524	1192	1192
query24_1	1179	1195	1203	1195
query25	566	472	442	442
query26	1256	269	156	156
query27	2764	442	278	278
query28	4565	2128	2135	2128
query29	781	550	454	454
query30	315	242	207	207
query31	829	636	579	579
query32	86	79	76	76
query33	549	372	320	320
query34	939	876	549	549
query35	749	791	690	690
query36	828	942	798	798
query37	138	99	89	89
query38	2735	2722	2669	2669
query39	781	774	734	734
query39_1	709	702	716	702
query40	222	140	124	124
query41	73	75	68	68
query42	106	105	108	105
query43	422	456	428	428
query44	1340	752	744	744
query45	189	192	182	182
query46	896	963	594	594
query47	1333	1461	1413	1413
query48	320	334	251	251
query49	625	461	358	358
query50	611	264	206	206
query51	3906	3802	3759	3759
query52	109	109	99	99
query53	300	335	280	280
query54	308	295	313	295
query55	82	81	78	78
query56	317	324	309	309
query57	1035	1037	961	961
query58	278	248	247	247
query59	2126	2183	2114	2114
query60	338	335	320	320
query61	162	154	158	154
query62	401	361	317	317
query63	296	288	279	279
query64	4878	1340	981	981
query65	3802	3693	3766	3693
query66	1386	419	326	326
query67	15610	15632	15517	15517
query68	2500	1088	742	742
query69	450	417	326	326
query70	988	967	970	967
query71	317	313	294	294
query72	5415	3259	2451	2451
query73	596	717	308	308
query74	8724	8780	8503	8503
query75	2716	2816	2505	2505
query76	2306	1051	661	661
query77	378	410	314	314
query78	9887	9891	9136	9136
query79	2331	885	566	566
query80	1772	579	502	502
query81	557	268	234	234
query82	1263	143	110	110
query83	334	254	242	242
query84	257	121	92	92
query85	915	515	454	454
query86	409	303	293	293
query87	2896	2878	2769	2769
query88	3510	2611	2585	2585
query89	388	351	335	335
query90	2002	178	172	172
query91	183	170	138	138
query92	80	74	68	68
query93	1096	911	525	525
query94	654	314	302	302
query95	587	405	331	331
query96	654	495	229	229
query97	2335	2364	2334	2334
query98	211	198	206	198
query99	620	574	500	500
Total cold run time: 250779 ms
Total hot run time: 173647 ms

@JNSimba
Copy link
Member Author

JNSimba commented Jan 15, 2026

run buildall

@doris-robot
Copy link

doris-robot commented Jan 15, 2026

TPC-H: Total hot run time: 31706 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 6aef0601a0c2d4ee553a1e92770019d6bc285cf3, data reload: false

------ Round 1 ----------------------------------
q1	17583	4204	4046	4046
q2	2054	356	238	238
q3	10135	1248	737	737
q4	10209	830	298	298
q5	7516	2070	1816	1816
q6	182	168	135	135
q7	915	786	660	660
q8	9283	1406	1125	1125
q9	4909	4618	4476	4476
q10	6777	1774	1444	1444
q11	498	316	275	275
q12	680	736	604	604
q13	17776	3801	3053	3053
q14	291	310	275	275
q15	597	513	504	504
q16	656	686	632	632
q17	664	809	465	465
q18	6562	6445	6571	6445
q19	1322	1035	705	705
q20	408	362	266	266
q21	3191	2690	2443	2443
q22	1124	1177	1064	1064
Total cold run time: 103332 ms
Total hot run time: 31706 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4416	4202	4219	4202
q2	318	414	345	345
q3	2323	2777	2466	2466
q4	1497	1848	1405	1405
q5	4534	4342	4587	4342
q6	210	166	126	126
q7	1921	1911	1778	1778
q8	2567	2491	2412	2412
q9	7179	7030	7143	7030
q10	2460	2688	2306	2306
q11	558	473	447	447
q12	723	794	603	603
q13	3307	3765	3063	3063
q14	273	297	261	261
q15	521	495	496	495
q16	601	639	608	608
q17	1082	1245	1257	1245
q18	7422	7330	7129	7129
q19	812	789	763	763
q20	1864	1971	1818	1818
q21	4453	4313	4057	4057
q22	1076	1036	974	974
Total cold run time: 50117 ms
Total hot run time: 47875 ms

@doris-robot
Copy link

doris-robot commented Jan 15, 2026

TPC-DS: Total hot run time: 173659 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 6aef0601a0c2d4ee553a1e92770019d6bc285cf3, data reload: false

query5	4381	668	480	480
query6	334	233	225	225
query7	4237	493	258	258
query8	379	256	244	244
query9	8714	2843	2841	2841
query10	463	373	321	321
query11	15284	15169	14801	14801
query12	180	120	117	117
query13	1248	506	394	394
query14	6371	3016	2779	2779
query14_1	2680	2657	2630	2630
query15	189	188	172	172
query16	996	481	457	457
query17	1082	670	531	531
query18	2432	430	324	324
query19	223	221	193	193
query20	118	109	114	109
query21	212	142	114	114
query22	3910	3981	3928	3928
query23	15823	15396	15507	15396
query23_1	15299	15438	15378	15378
query24	7295	1614	1158	1158
query24_1	1151	1150	1170	1150
query25	525	440	390	390
query26	1230	278	160	160
query27	2759	465	275	275
query28	4545	2109	2093	2093
query29	785	585	412	412
query30	308	241	203	203
query31	779	627	531	531
query32	80	76	72	72
query33	532	355	296	296
query34	901	942	525	525
query35	728	759	685	685
query36	887	915	805	805
query37	123	90	82	82
query38	2671	2695	2679	2679
query39	769	748	732	732
query39_1	706	704	720	704
query40	215	135	117	117
query41	65	65	62	62
query42	108	99	97	97
query43	420	467	443	443
query44	1360	725	722	722
query45	182	185	175	175
query46	883	987	570	570
query47	1304	1404	1368	1368
query48	305	322	237	237
query49	602	434	338	338
query50	638	276	198	198
query51	3774	3801	3768	3768
query52	108	108	96	96
query53	299	333	269	269
query54	280	267	256	256
query55	80	78	74	74
query56	297	336	297	297
query57	994	1016	930	930
query58	269	254	260	254
query59	2120	2032	2012	2012
query60	333	332	308	308
query61	159	155	158	155
query62	380	344	308	308
query63	296	273	266	266
query64	4999	1275	987	987
query65	3811	3753	3691	3691
query66	1439	427	317	317
query67	15553	15617	15444	15444
query68	2431	1091	741	741
query69	439	349	320	320
query70	1004	853	939	853
query71	328	306	288	288
query72	5484	3603	3657	3603
query73	591	771	312	312
query74	8740	8780	8511	8511
query75	2866	2853	2473	2473
query76	2291	1133	657	657
query77	364	397	313	313
query78	9861	10038	9213	9213
query79	1136	861	572	572
query80	1128	595	494	494
query81	527	263	230	230
query82	1339	146	108	108
query83	370	286	242	242
query84	246	116	96	96
query85	921	533	447	447
query86	373	316	280	280
query87	2844	2855	2762	2762
query88	3467	2571	2555	2555
query89	386	352	326	326
query90	1854	166	166	166
query91	166	166	136	136
query92	75	77	72	72
query93	940	941	525	525
query94	562	339	306	306
query95	582	339	313	313
query96	641	528	225	225
query97	2360	2336	2292	2292
query98	217	206	193	193
query99	589	559	503	503
Total cold run time: 246445 ms
Total hot run time: 173659 ms

@doris-robot
Copy link

doris-robot commented Jan 15, 2026

ClickBench: Total hot run time: 26.77 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 6aef0601a0c2d4ee553a1e92770019d6bc285cf3, data reload: false

query1	0.06	0.05	0.05
query2	0.10	0.05	0.05
query3	0.25	0.09	0.08
query4	1.61	0.11	0.12
query5	0.28	0.25	0.26
query6	1.15	0.65	0.66
query7	0.03	0.02	0.03
query8	0.06	0.04	0.04
query9	0.57	0.51	0.51
query10	0.56	0.56	0.55
query11	0.15	0.10	0.10
query12	0.15	0.10	0.11
query13	0.60	0.58	0.59
query14	0.95	0.94	0.95
query15	0.79	0.79	0.77
query16	0.43	0.39	0.40
query17	1.05	1.06	1.08
query18	0.23	0.22	0.21
query19	1.91	1.82	1.89
query20	0.01	0.01	0.01
query21	15.44	0.25	0.14
query22	5.39	0.05	0.05
query23	15.88	0.28	0.10
query24	0.93	0.60	0.17
query25	0.08	0.09	0.06
query26	0.14	0.14	0.13
query27	0.05	0.05	0.07
query28	3.75	1.07	0.89
query29	12.52	3.90	3.20
query30	0.28	0.13	0.12
query31	2.83	0.62	0.40
query32	3.24	0.55	0.46
query33	3.05	3.09	3.05
query34	16.35	5.11	4.41
query35	4.50	4.43	4.46
query36	0.65	0.50	0.48
query37	0.11	0.06	0.06
query38	0.08	0.04	0.04
query39	0.04	0.02	0.02
query40	0.17	0.14	0.13
query41	0.09	0.04	0.03
query42	0.04	0.02	0.02
query43	0.04	0.04	0.04
Total cold run time: 96.59 s
Total hot run time: 26.77 s

@hello-stephen
Copy link
Contributor

hello-stephen commented Jan 15, 2026

FE UT Coverage Report

Increment line coverage 0.00% (0/1) 🎉
Increment coverage report
Complete coverage report

@hello-stephen
Copy link
Contributor

hello-stephen commented Jan 15, 2026

FE Regression Coverage Report

Increment line coverage 0.00% (0/1) 🎉
Increment coverage report
Complete coverage report

sollhui
sollhui approved these changes Jan 16, 2026
View reviewed changes
Copy link
Contributor

@sollhui sollhui left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions
Copy link
Contributor

github-actions bot commented Jan 16, 2026

PR approved by anyone and no changes requested.

@JNSimba
Copy link
Member Author

JNSimba commented Jan 16, 2026

run buildall

@doris-robot
Copy link

doris-robot commented Jan 16, 2026

TPC-H: Total hot run time: 31567 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 2badad49fbc9bc623f430f9a22f21113fba0a90c, data reload: false

------ Round 1 ----------------------------------
q1	17625	4278	4054	4054
q2	2044	356	243	243
q3	10139	1311	713	713
q4	10202	810	300	300
q5	7505	2114	1792	1792
q6	194	175	138	138
q7	973	780	656	656
q8	9256	1419	1142	1142
q9	4794	4520	4440	4440
q10	6710	1795	1398	1398
q11	500	300	281	281
q12	693	724	592	592
q13	17795	3864	3090	3090
q14	283	299	276	276
q15	584	517	512	512
q16	682	682	633	633
q17	662	780	508	508
q18	6708	6299	6731	6299
q19	1157	1045	691	691
q20	404	372	263	263
q21	3299	2633	2491	2491
q22	1185	1090	1055	1055
Total cold run time: 103394 ms
Total hot run time: 31567 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4354	4362	4336	4336
q2	327	414	321	321
q3	2259	2783	2473	2473
q4	1401	2012	1377	1377
q5	4380	4455	4412	4412
q6	216	172	130	130
q7	2014	1918	1799	1799
q8	2630	2425	2591	2425
q9	7156	7202	7216	7202
q10	2542	2689	2427	2427
q11	549	470	457	457
q12	752	781	611	611
q13	3614	4045	3091	3091
q14	281	280	267	267
q15	528	478	488	478
q16	611	662	593	593
q17	1106	1257	1327	1257
q18	7451	7425	7287	7287
q19	832	785	787	785
q20	1865	1953	1809	1809
q21	4447	4211	4111	4111
q22	1050	1021	974	974
Total cold run time: 50365 ms
Total hot run time: 48622 ms

@doris-robot
Copy link

doris-robot commented Jan 16, 2026

TPC-H: Total hot run time: 31324 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 2badad49fbc9bc623f430f9a22f21113fba0a90c, data reload: false

------ Round 1 ----------------------------------
q1	17585	4271	4055	4055
q2	2079	347	256	256
q3	10140	1269	701	701
q4	10195	777	295	295
q5	7538	2059	1813	1813
q6	185	164	137	137
q7	925	769	654	654
q8	9267	1354	1133	1133
q9	4858	4609	4619	4609
q10	6744	1794	1382	1382
q11	506	291	274	274
q12	681	744	611	611
q13	17809	3784	3103	3103
q14	288	303	269	269
q15	578	507	511	507
q16	696	669	621	621
q17	659	718	606	606
q18	6667	6474	6231	6231
q19	1236	961	590	590
q20	391	361	252	252
q21	2940	2475	2267	2267
q22	1048	1004	958	958
Total cold run time: 103015 ms
Total hot run time: 31324 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4069	4028	4016	4016
q2	338	405	320	320
q3	2087	2576	2196	2196
q4	1339	1753	1333	1333
q5	4110	4058	4036	4036
q6	210	172	127	127
q7	1867	1850	1689	1689
q8	2802	2540	2463	2463
q9	7343	7546	7233	7233
q10	2511	2781	2437	2437
q11	572	519	452	452
q12	715	772	642	642
q13	3620	4181	3532	3532
q14	345	326	296	296
q15	541	519	495	495
q16	651	682	664	664
q17	1129	1337	1371	1337
q18	8319	7928	7876	7876
q19	844	844	819	819
q20	2000	2070	1841	1841
q21	4495	4229	4161	4161
q22	1076	1038	958	958
Total cold run time: 50983 ms
Total hot run time: 48923 ms

@doris-robot
Copy link

doris-robot commented Jan 16, 2026

TPC-DS: Total hot run time: 173858 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 2badad49fbc9bc623f430f9a22f21113fba0a90c, data reload: false

query5	4420	589	500	500
query6	327	225	213	213
query7	4219	465	253	253
query8	366	251	243	243
query9	8768	2849	2865	2849
query10	528	380	326	326
query11	15362	15037	15015	15015
query12	174	113	118	113
query13	1264	466	371	371
query14	6097	3045	2742	2742
query14_1	2647	2620	2638	2620
query15	200	191	178	178
query16	999	446	464	446
query17	1100	667	558	558
query18	2441	437	337	337
query19	222	222	198	198
query20	128	114	111	111
query21	218	138	114	114
query22	3861	4287	4143	4143
query23	15951	15634	15285	15285
query23_1	15532	15607	15715	15607
query24	7139	1538	1150	1150
query24_1	1151	1163	1170	1163
query25	544	458	417	417
query26	1247	273	156	156
query27	2755	444	276	276
query28	4532	2136	2126	2126
query29	750	538	453	453
query30	316	241	207	207
query31	807	604	583	583
query32	81	78	75	75
query33	536	359	322	322
query34	878	858	525	525
query35	723	762	675	675
query36	872	858	880	858
query37	141	101	95	95
query38	2734	2731	2673	2673
query39	761	764	752	752
query39_1	729	710	734	710
query40	224	137	121	121
query41	71	106	62	62
query42	104	96	99	96
query43	432	449	409	409
query44	1289	724	710	710
query45	181	187	172	172
query46	821	929	568	568
query47	1447	1488	1391	1391
query48	308	318	236	236
query49	609	414	331	331
query50	609	263	199	199
query51	3907	3764	3758	3758
query52	109	107	97	97
query53	292	319	267	267
query54	308	270	260	260
query55	82	77	77	77
query56	297	310	308	308
query57	1024	964	926	926
query58	265	268	256	256
query59	2103	2198	1966	1966
query60	328	338	315	315
query61	151	148	150	148
query62	419	349	318	318
query63	293	263	268	263
query64	4872	1230	916	916
query65	3851	3774	3690	3690
query66	1456	417	306	306
query67	15479	15480	15421	15421
query68	2437	1093	735	735
query69	441	357	314	314
query70	999	953	862	862
query71	323	307	285	285
query72	5281	3120	3206	3120
query73	595	715	307	307
query74	8675	8776	8522	8522
query75	2720	2802	2444	2444
query76	2275	1039	666	666
query77	354	374	296	296
query78	9716	9861	9229	9229
query79	2368	904	584	584
query80	1711	556	486	486
query81	542	268	232	232
query82	994	141	110	110
query83	316	248	242	242
query84	247	117	87	87
query85	917	484	423	423
query86	408	318	296	296
query87	2864	2904	2724	2724
query88	3437	2555	2545	2545
query89	393	345	313	313
query90	1980	171	156	156
query91	171	161	129	129
query92	71	74	71	71
query93	1083	910	519	519
query94	660	329	291	291
query95	572	327	361	327
query96	641	499	235	235
query97	2319	2384	2306	2306
query98	214	199	196	196
query99	611	629	528	528
Total cold run time: 248129 ms
Total hot run time: 173858 ms

@doris-robot
Copy link

doris-robot commented Jan 16, 2026

ClickBench: Total hot run time: 26.93 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 2badad49fbc9bc623f430f9a22f21113fba0a90c, data reload: false

query1	0.05	0.05	0.05
query2	0.10	0.05	0.05
query3	0.26	0.08	0.08
query4	1.61	0.12	0.11
query5	0.28	0.25	0.27
query6	1.14	0.66	0.64
query7	0.03	0.03	0.03
query8	0.05	0.03	0.04
query9	0.57	0.51	0.50
query10	0.54	0.55	0.56
query11	0.14	0.10	0.09
query12	0.14	0.11	0.10
query13	0.60	0.58	0.58
query14	0.94	0.94	0.94
query15	0.79	0.76	0.78
query16	0.40	0.40	0.41
query17	1.03	1.01	1.06
query18	0.23	0.21	0.21
query19	1.91	1.89	1.77
query20	0.02	0.01	0.02
query21	15.42	0.25	0.14
query22	5.36	0.05	0.05
query23	16.03	0.28	0.10
query24	1.17	0.48	0.52
query25	0.09	0.12	0.06
query26	0.14	0.13	0.13
query27	0.08	0.07	0.05
query28	4.56	1.07	0.89
query29	12.53	3.84	3.18
query30	0.28	0.15	0.11
query31	2.82	0.61	0.40
query32	3.25	0.57	0.45
query33	2.98	2.99	3.06
query34	16.65	5.07	4.47
query35	4.48	4.43	4.45
query36	0.64	0.50	0.49
query37	0.11	0.06	0.06
query38	0.08	0.04	0.04
query39	0.05	0.03	0.03
query40	0.17	0.13	0.13
query41	0.10	0.03	0.04
query42	0.05	0.03	0.03
query43	0.04	0.03	0.04
Total cold run time: 97.91 s
Total hot run time: 26.93 s

@JNSimba
Copy link
Member Author

JNSimba commented Jan 16, 2026

run buildall

@doris-robot
Copy link

doris-robot commented Jan 16, 2026

TPC-H: Total hot run time: 31764 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 051feb7ae8a8da76e84e2b22272ab3632f839e9b, data reload: false

------ Round 1 ----------------------------------
q1	17616	4275	4079	4079
q2	2062	363	260	260
q3	10124	1335	722	722
q4	10235	902	324	324
q5	7519	2092	1887	1887
q6	214	169	135	135
q7	945	805	663	663
q8	9268	1435	1147	1147
q9	4964	4679	4634	4634
q10	6728	1806	1413	1413
q11	555	290	272	272
q12	691	741	606	606
q13	17783	3833	3061	3061
q14	285	294	286	286
q15	570	512	513	512
q16	682	678	639	639
q17	655	875	470	470
q18	6740	6604	6408	6408
q19	1090	986	687	687
q20	403	363	246	246
q21	3099	2578	2353	2353
q22	1048	1003	960	960
Total cold run time: 103276 ms
Total hot run time: 31764 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4175	4036	4014	4014
q2	334	379	328	328
q3	2104	2610	2237	2237
q4	1335	1718	1336	1336
q5	4187	3978	4043	3978
q6	210	173	130	130
q7	1883	1855	1949	1855
q8	2710	2490	2528	2490
q9	7192	7234	7141	7141
q10	2691	2703	2344	2344
q11	555	503	492	492
q12	737	772	642	642
q13	3668	4275	3345	3345
q14	315	308	311	308
q15	559	497	496	496
q16	654	702	640	640
q17	1176	1303	1356	1303
q18	8004	8218	7815	7815
q19	991	914	884	884
q20	2009	2133	2012	2012
q21	4783	4525	4457	4457
q22	1123	1105	1045	1045
Total cold run time: 51395 ms
Total hot run time: 49292 ms

@doris-robot
Copy link

doris-robot commented Jan 16, 2026

TPC-DS: Total hot run time: 175164 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 051feb7ae8a8da76e84e2b22272ab3632f839e9b, data reload: false

query5	4805	626	484	484
query6	343	237	220	220
query7	4213	461	265	265
query8	361	251	245	245
query9	8754	2905	2887	2887
query10	527	376	328	328
query11	15228	15063	14954	14954
query12	195	118	126	118
query13	1256	488	368	368
query14	6977	3044	2835	2835
query14_1	2682	2649	2649	2649
query15	200	188	176	176
query16	997	507	492	492
query17	1121	673	564	564
query18	2702	441	333	333
query19	222	220	198	198
query20	122	115	114	114
query21	215	135	127	127
query22	4111	4304	4200	4200
query23	16085	15667	15445	15445
query23_1	15663	15648	15671	15648
query24	7054	1553	1176	1176
query24_1	1163	1149	1156	1149
query25	516	424	393	393
query26	1254	257	144	144
query27	2763	448	272	272
query28	4510	2127	2131	2127
query29	749	511	424	424
query30	319	241	204	204
query31	812	630	573	573
query32	82	73	80	73
query33	523	343	295	295
query34	902	894	527	527
query35	718	765	673	673
query36	883	909	855	855
query37	129	97	87	87
query38	2784	2686	2660	2660
query39	772	733	746	733
query39_1	733	741	724	724
query40	218	133	121	121
query41	67	62	60	60
query42	105	96	96	96
query43	449	444	417	417
query44	1326	734	738	734
query45	187	193	175	175
query46	833	946	574	574
query47	1412	1501	1411	1411
query48	329	333	236	236
query49	606	416	350	350
query50	641	297	210	210
query51	3801	3887	3740	3740
query52	105	105	94	94
query53	295	328	271	271
query54	290	273	262	262
query55	87	81	81	81
query56	311	320	323	320
query57	997	982	971	971
query58	276	269	273	269
query59	2122	2165	1967	1967
query60	342	333	317	317
query61	158	151	149	149
query62	406	359	298	298
query63	298	265	265	265
query64	4812	1299	975	975
query65	3870	3742	3767	3742
query66	1381	424	322	322
query67	15762	15626	15592	15592
query68	2431	1140	804	804
query69	469	362	325	325
query70	1051	963	877	877
query71	320	316	295	295
query72	5321	3280	3323	3280
query73	635	725	324	324
query74	8769	8791	8560	8560
query75	2753	2797	2439	2439
query76	2264	1053	672	672
query77	364	385	304	304
query78	9805	10080	9244	9244
query79	1102	905	575	575
query80	665	573	483	483
query81	482	261	232	232
query82	1328	151	110	110
query83	376	264	249	249
query84	254	112	95	95
query85	838	496	443	443
query86	351	288	295	288
query87	2927	2828	2750	2750
query88	3642	2617	2709	2617
query89	404	344	328	328
query90	1878	199	181	181
query91	182	168	148	148
query92	80	77	73	73
query93	965	913	569	569
query94	457	312	329	312
query95	600	423	350	350
query96	674	521	234	234
query97	2364	2378	2382	2378
query98	218	206	199	199
query99	597	600	515	515
Total cold run time: 248840 ms
Total hot run time: 175164 ms

@doris-robot
Copy link

doris-robot commented Jan 16, 2026

ClickBench: Total hot run time: 27.11 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 051feb7ae8a8da76e84e2b22272ab3632f839e9b, data reload: false

query1	0.06	0.05	0.05
query2	0.10	0.05	0.05
query3	0.27	0.08	0.08
query4	1.61	0.12	0.11
query5	0.26	0.26	0.25
query6	1.14	0.67	0.66
query7	0.04	0.03	0.03
query8	0.05	0.04	0.05
query9	0.59	0.52	0.51
query10	0.56	0.56	0.55
query11	0.17	0.10	0.09
query12	0.15	0.12	0.11
query13	0.61	0.59	0.58
query14	0.96	0.94	0.96
query15	0.80	0.78	0.78
query16	0.40	0.40	0.41
query17	1.08	1.01	1.03
query18	0.23	0.21	0.22
query19	1.93	1.90	1.89
query20	0.02	0.02	0.02
query21	15.47	0.26	0.14
query22	5.32	0.05	0.06
query23	16.25	0.28	0.10
query24	1.67	0.37	0.66
query25	0.11	0.08	0.06
query26	0.15	0.14	0.14
query27	0.06	0.04	0.06
query28	4.37	1.08	0.90
query29	12.56	3.93	3.13
query30	0.27	0.14	0.12
query31	2.82	0.63	0.40
query32	3.24	0.57	0.46
query33	2.97	3.05	3.08
query34	16.47	5.00	4.45
query35	4.43	4.49	4.48
query36	0.65	0.50	0.49
query37	0.10	0.06	0.06
query38	0.07	0.04	0.04
query39	0.05	0.04	0.03
query40	0.18	0.14	0.14
query41	0.09	0.04	0.03
query42	0.05	0.03	0.03
query43	0.05	0.04	0.04
Total cold run time: 98.43 s
Total hot run time: 27.11 s

@JNSimba
Copy link
Member Author

JNSimba commented Jan 16, 2026

run p0

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Jan 16, 2026
@github-actions
Copy link
Contributor

github-actions bot commented Jan 16, 2026

PR approved by at least one committer and no changes requested.

@JNSimba
Copy link
Member Author

JNSimba commented Jan 16, 2026

run nonConcurrent

@hello-stephen
Copy link
Contributor

hello-stephen commented Jan 16, 2026

FE Regression Coverage Report

Increment line coverage 0.00% (0/6) 🎉
Increment coverage report
Complete coverage report

@JNSimba JNSimba merged commit eb01099 into apache:master Jan 16, 2026
27 checks passed
github-actions bot pushed a commit that referenced this pull request Jan 16, 2026
@JNSimba
[Fix](Streamingjob) fix postgres incr consumer too slow (#59919)
eaaf21c 
### What problem does this PR solve?

Issue Number: close #xxx

Related PR:  #59798

If you close the reader before committing the LSN, the commit will not
be completed.
@github-actions github-actions bot mentioned this pull request Jan 16, 2026
Merged
yiguolei pushed a commit that referenced this pull request Jan 17, 2026
@github-actions @JNSimba
branch-4.0: [Fix](Streamingjob) fix postgres incr consumer too slow #…
07cd88e 
…59919 (#59977)

Cherry-picked from #59919

Co-authored-by: wudi <wudi@selectdb.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@CalvinKirs CalvinKirs CalvinKirs approved these changes

+1 more reviewer

@sollhui sollhui sollhui approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

approved Indicates a PR has been approved by one committer. dev/4.0.3-merged reviewed

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

6 participants

@JNSimba @doris-robot @hello-stephen @CalvinKirs @sollhui @yiguolei

Comments