Skip to content

[Fix](orc-reader) Fix StringRef nullptr data by add checking string_values empty.#42061

Merged
morningman merged 1 commit intoapache:masterfrom
kaka11chen:fix_orc_reader_string_ref_nullptr
Oct 21, 2024
Merged

[Fix](orc-reader) Fix StringRef nullptr data by add checking string_values empty.#42061
morningman merged 1 commit intoapache:masterfrom
kaka11chen:fix_orc_reader_string_ref_nullptr

Conversation

@kaka11chen
Copy link
Contributor

@kaka11chen kaka11chen commented Oct 18, 2024

Proposed changes

Added solution to #40857, fix StringRef nullptr data by add checking string_values empty in orc reader.

@doris-robot
Copy link

doris-robot commented Oct 18, 2024

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@kaka11chen
Copy link
Contributor Author

kaka11chen commented Oct 18, 2024

run buildall

@github-actions
Copy link
Contributor

github-actions bot commented Oct 18, 2024

clang-tidy review says "All clean, LGTM! 👍"

@doris-robot
Copy link

doris-robot commented Oct 18, 2024

TPC-H: Total hot run time: 41455 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 59c0f98fc6900a716df3066daad36cbc38d0fec0, data reload: false

------ Round 1 ----------------------------------
q1	17569	7830	7300	7300
q2	2018	276	277	276
q3	12319	1043	1189	1043
q4	10583	825	756	756
q5	7753	3066	3077	3066
q6	239	155	155	155
q7	1022	630	604	604
q8	9366	1930	2018	1930
q9	6667	6537	6490	6490
q10	7581	2432	2432	2432
q11	565	251	270	251
q12	616	232	235	232
q13	18144	3048	3080	3048
q14	240	208	228	208
q15	575	537	531	531
q16	654	609	600	600
q17	977	597	538	538
q18	7265	6705	6858	6705
q19	1359	884	952	884
q20	457	194	186	186
q21	4162	3312	3192	3192
q22	1153	1028	1040	1028
Total cold run time: 111284 ms
Total hot run time: 41455 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7345	7380	7424	7380
q2	337	255	245	245
q3	3297	3132	3258	3132
q4	2202	1880	1931	1880
q5	5976	6006	5973	5973
q6	235	153	147	147
q7	2329	1798	1908	1798
q8	3295	3554	3582	3554
q9	9071	9048	8918	8918
q10	3577	3586	3590	3586
q11	584	501	505	501
q12	812	610	596	596
q13	13064	3217	3217	3217
q14	321	284	276	276
q15	578	527	522	522
q16	692	640	639	639
q17	1879	1637	1641	1637
q18	8530	8021	7800	7800
q19	1725	1394	1340	1340
q20	2116	1875	1894	1875
q21	5639	5564	5507	5507
q22	1218	1066	1074	1066
Total cold run time: 74822 ms
Total hot run time: 61589 ms

@doris-robot
Copy link

doris-robot commented Oct 18, 2024

TeamCity be ut coverage result:
Function Coverage: 37.47% (9711/25919)
Line Coverage: 28.72% (80601/280620)
Region Coverage: 28.17% (41712/148091)
Branch Coverage: 24.74% (21207/85704)
Coverage Report: http://coverage.selectdb-in.cc/coverage/59c0f98fc6900a716df3066daad36cbc38d0fec0_59c0f98fc6900a716df3066daad36cbc38d0fec0/report/index.html

@doris-robot
Copy link

doris-robot commented Oct 18, 2024

TPC-DS: Total hot run time: 192182 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 59c0f98fc6900a716df3066daad36cbc38d0fec0, data reload: false

query1	834	410	398	398
query2	6259	2101	2069	2069
query3	8686	195	196	195
query4	34201	23625	23546	23546
query5	3738	481	463	463
query6	277	177	181	177
query7	4189	296	289	289
query8	303	239	227	227
query9	9641	2712	2706	2706
query10	462	282	273	273
query11	17793	15337	15316	15316
query12	144	104	99	99
query13	1563	423	400	400
query14	9228	7519	7840	7519
query15	258	170	182	170
query16	7955	506	507	506
query17	1583	603	570	570
query18	2031	315	316	315
query19	364	153	157	153
query20	123	115	120	115
query21	212	112	110	110
query22	4893	4641	4458	4458
query23	35039	34272	33890	33890
query24	10931	2782	2769	2769
query25	618	404	401	401
query26	1169	159	168	159
query27	2202	283	300	283
query28	7315	2416	2431	2416
query29	801	431	433	431
query30	289	153	154	153
query31	1056	784	823	784
query32	96	56	57	56
query33	768	302	295	295
query34	919	517	528	517
query35	871	740	742	740
query36	1098	945	952	945
query37	158	87	87	87
query38	4072	4035	3896	3896
query39	1465	1426	1418	1418
query40	205	101	100	100
query41	47	44	48	44
query42	127	101	102	101
query43	533	495	494	494
query44	1267	803	800	800
query45	203	171	165	165
query46	1143	694	701	694
query47	1919	1818	1824	1818
query48	421	325	326	325
query49	957	431	436	431
query50	838	407	394	394
query51	7059	6966	6873	6873
query52	100	88	91	88
query53	255	176	177	176
query54	1070	420	415	415
query55	78	76	77	76
query56	272	252	274	252
query57	1299	1166	1183	1166
query58	243	247	245	245
query59	3345	3029	3072	3029
query60	303	255	261	255
query61	104	118	103	103
query62	850	691	701	691
query63	209	187	181	181
query64	4005	633	632	632
query65	3288	3244	3207	3207
query66	823	308	301	301
query67	16008	15710	15788	15710
query68	4096	572	558	558
query69	565	293	285	285
query70	1187	1088	1121	1088
query71	395	282	279	279
query72	7788	3936	3969	3936
query73	776	368	360	360
query74	10347	9063	9093	9063
query75	3672	2655	2709	2655
query76	3497	847	957	847
query77	620	293	292	292
query78	10504	9599	9550	9550
query79	2046	601	605	601
query80	1150	447	470	447
query81	582	240	237	237
query82	718	139	142	139
query83	298	135	135	135
query84	272	69	71	69
query85	1296	294	277	277
query86	439	304	298	298
query87	4500	4484	4233	4233
query88	3874	2189	2165	2165
query89	394	294	285	285
query90	2016	185	183	183
query91	136	99	110	99
query92	74	50	49	49
query93	2549	537	537	537
query94	1035	302	297	297
query95	343	247	255	247
query96	627	280	277	277
query97	3249	3120	3125	3120
query98	217	202	194	194
query99	1564	1300	1302	1300
Total cold run time: 301401 ms
Total hot run time: 192182 ms

@doris-robot
Copy link

doris-robot commented Oct 18, 2024

ClickBench: Total hot run time: 33.22 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 59c0f98fc6900a716df3066daad36cbc38d0fec0, data reload: false

query1	0.03	0.03	0.03
query2	0.06	0.03	0.04
query3	0.23	0.07	0.07
query4	1.64	0.10	0.10
query5	0.53	0.48	0.52
query6	1.13	0.72	0.72
query7	0.02	0.03	0.01
query8	0.03	0.03	0.03
query9	0.56	0.50	0.51
query10	0.56	0.55	0.53
query11	0.14	0.11	0.11
query12	0.15	0.11	0.11
query13	0.60	0.59	0.60
query14	2.70	2.75	2.75
query15	0.90	0.84	0.86
query16	0.39	0.39	0.40
query17	1.05	1.07	1.08
query18	0.20	0.20	0.20
query19	1.88	1.84	1.96
query20	0.01	0.02	0.01
query21	15.35	0.62	0.61
query22	2.44	2.49	2.10
query23	16.81	0.92	0.83
query24	2.89	1.34	0.80
query25	0.32	0.24	0.25
query26	0.21	0.14	0.14
query27	0.05	0.04	0.04
query28	10.96	1.09	1.08
query29	12.52	3.32	3.32
query30	0.24	0.06	0.06
query31	2.86	0.38	0.37
query32	3.29	0.45	0.45
query33	3.00	2.99	3.07
query34	17.05	4.47	4.56
query35	4.54	4.54	4.57
query36	0.68	0.50	0.48
query37	0.08	0.06	0.06
query38	0.05	0.03	0.04
query39	0.03	0.02	0.02
query40	0.15	0.12	0.13
query41	0.08	0.02	0.02
query42	0.03	0.02	0.02
query43	0.03	0.02	0.02
Total cold run time: 106.47 s
Total hot run time: 33.22 s

@github-actions
Copy link
Contributor

github-actions bot commented Oct 18, 2024

PR approved by at least one committer and no changes requested.

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Oct 18, 2024
@github-actions
Copy link
Contributor

github-actions bot commented Oct 18, 2024

PR approved by anyone and no changes requested.

@morningman morningman merged commit 2dd1704 into apache:master Oct 21, 2024
morningman pushed a commit to morningman/doris that referenced this pull request Oct 21, 2024
@kaka11chen @morningman
[Fix](orc-reader) Fix StringRef nullptr data by add checking string_v…
9ec6460 
…alues empty. (apache#42061)

## Proposed changes

Added solution to apache#40857, fix StringRef nullptr data by add checking
string_values empty in orc reader.
@morningman morningman mentioned this pull request Oct 21, 2024
Merged
morningman pushed a commit to morningman/doris that referenced this pull request Oct 21, 2024
@kaka11chen @morningman
[Fix](orc-reader) Fix StringRef nullptr data by add checking string_v…
56a65c7 
…alues empty. (apache#42061)

## Proposed changes

Added solution to apache#40857, fix StringRef nullptr data by add checking
string_values empty in orc reader.
@morningman morningman mentioned this pull request Oct 21, 2024
Merged
morningman added a commit that referenced this pull request Oct 21, 2024
@morningman @kaka11chen
[Fix](orc-reader) Fix StringRef nullptr data by add checking string_v…
b9e2738 
…alues empty. #42061 (#42154)

cherry pick from #42061

Co-authored-by: Qi Chen <kaka11.chen@gmail.com>
morningman added a commit that referenced this pull request Oct 21, 2024
@morningman @kaka11chen
[Fix](orc-reader) Fix StringRef nullptr data by add checking string_v…
ddaf450 
…alues empty. #42061 (#42169)

cherry pick from #42061

Co-authored-by: Qi Chen <kaka11.chen@gmail.com>
GoGoWen pushed a commit to GoGoWen/incubator-doris that referenced this pull request Sep 26, 2025
fix(orc-reader): fix StringRef nullptr data by add checking string_va…
ffff537 
…lues empty. apache#42061 (apache#42154)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@morningman morningman morningman approved these changes

+1 more reviewer

@suxiaogang223 suxiaogang223 suxiaogang223 approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

approved Indicates a PR has been approved by one committer. dev/2.1.7-merged dev/3.0.3-merged reviewed

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@kaka11chen @doris-robot @morningman @suxiaogang223

Comments