Skip to content

[Test](segcompaction) refine segment compaction inverted index ut case#43219

Merged
airborne12 merged 1 commit intoapache:masterfrom
airborne12:fix-case
Nov 5, 2024
Merged

[Test](segcompaction) refine segment compaction inverted index ut case#43219
airborne12 merged 1 commit intoapache:masterfrom
airborne12:fix-case

Conversation

@airborne12
Copy link
Member

@airborne12 airborne12 commented Nov 4, 2024 •
edited
Loading

What problem does this PR solve?

Related PR: #43114

Problem Summary:
remove useless code and add size check

Check List (For Committer)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No colde files have been changed.
      • Other reason

  • Behavior changed:

    • No.
    • Yes.

  • Does this need documentation?

    • No.
    • Yes.

  • Release note

    None

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@doris-robot
Copy link

doris-robot commented Nov 4, 2024

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@airborne12
Copy link
Member Author

airborne12 commented Nov 4, 2024

run buildall

@github-actions
Copy link
Contributor

github-actions bot commented Nov 4, 2024

clang-tidy review says "All clean, LGTM! 👍"

@doris-robot
Copy link

doris-robot commented Nov 4, 2024

TPC-H: Total hot run time: 41298 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 2b00ce854e4c03ae30cda5e963bef4192fed3a82, data reload: false

------ Round 1 ----------------------------------
q1	17557	7434	7292	7292
q2	2059	179	167	167
q3	10534	1071	1194	1071
q4	10233	829	850	829
q5	7719	3094	3042	3042
q6	237	145	145	145
q7	1008	615	605	605
q8	9352	1977	2065	1977
q9	6573	6410	6458	6410
q10	7061	2409	2428	2409
q11	483	262	248	248
q12	462	223	210	210
q13	17789	3039	3005	3005
q14	240	207	206	206
q15	567	509	522	509
q16	668	606	598	598
q17	976	547	507	507
q18	7284	6771	6787	6771
q19	1324	1028	1019	1019
q20	477	180	183	180
q21	4130	3219	3078	3078
q22	1103	1020	1029	1020
Total cold run time: 107836 ms
Total hot run time: 41298 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7361	7293	7290	7290
q2	324	229	223	223
q3	2941	2922	2959	2922
q4	2067	1856	1821	1821
q5	5736	5716	5774	5716
q6	223	146	144	144
q7	2194	1846	1808	1808
q8	3373	3593	3451	3451
q9	8915	8916	8881	8881
q10	3577	3566	3534	3534
q11	588	501	505	501
q12	804	667	659	659
q13	9750	3191	3152	3152
q14	304	282	277	277
q15	563	514	527	514
q16	686	651	642	642
q17	1891	1631	1595	1595
q18	8311	7646	7607	7607
q19	1705	1566	1561	1561
q20	2111	1898	1892	1892
q21	5668	5454	5229	5229
q22	1158	1049	1038	1038
Total cold run time: 70250 ms
Total hot run time: 60457 ms

@doris-robot
Copy link

doris-robot commented Nov 4, 2024

TeamCity be ut coverage result:
Function Coverage: 37.81% (9821/25975)
Line Coverage: 28.98% (81667/281816)
Region Coverage: 28.22% (42103/149198)
Branch Coverage: 24.80% (21363/86140)
Coverage Report: http://coverage.selectdb-in.cc/coverage/2b00ce854e4c03ae30cda5e963bef4192fed3a82_2b00ce854e4c03ae30cda5e963bef4192fed3a82/report/index.html

@doris-robot
Copy link

doris-robot commented Nov 4, 2024

TPC-DS: Total hot run time: 196294 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 2b00ce854e4c03ae30cda5e963bef4192fed3a82, data reload: false

query1	1223	910	921	910
query2	6246	2058	2038	2038
query3	10793	3930	3872	3872
query4	67215	28102	23943	23943
query5	4869	484	447	447
query6	401	190	178	178
query7	5512	294	293	293
query8	317	223	222	222
query9	8599	2656	2657	2656
query10	430	263	253	253
query11	17246	15354	16030	15354
query12	153	103	101	101
query13	1484	437	437	437
query14	9638	7125	7565	7125
query15	213	194	189	189
query16	7168	480	466	466
query17	1027	560	560	560
query18	1741	292	295	292
query19	187	148	145	145
query20	126	111	110	110
query21	201	99	101	99
query22	4429	4363	4314	4314
query23	34307	33873	34201	33873
query24	6006	2795	2783	2783
query25	511	405	401	401
query26	648	155	155	155
query27	1678	288	295	288
query28	4171	2447	2411	2411
query29	683	425	424	424
query30	226	160	155	155
query31	996	812	835	812
query32	70	53	59	53
query33	435	269	272	269
query34	917	515	509	509
query35	874	742	728	728
query36	1077	961	927	927
query37	121	75	72	72
query38	4352	4283	4254	4254
query39	1485	1447	1411	1411
query40	196	99	97	97
query41	50	45	47	45
query42	106	97	98	97
query43	525	470	497	470
query44	1179	813	833	813
query45	184	168	162	162
query46	1132	696	705	696
query47	1958	1834	1839	1834
query48	414	327	322	322
query49	731	419	418	418
query50	805	389	393	389
query51	7391	7108	7129	7108
query52	99	86	89	86
query53	249	178	178	178
query54	501	410	394	394
query55	74	74	71	71
query56	236	230	222	222
query57	1299	1161	1146	1146
query58	208	207	204	204
query59	3215	2958	3075	2958
query60	267	247	245	245
query61	107	106	106	106
query62	807	668	678	668
query63	214	185	181	181
query64	1337	649	637	637
query65	3297	3207	3221	3207
query66	703	292	307	292
query67	16117	15604	15772	15604
query68	2952	607	595	595
query69	417	260	252	252
query70	1142	1121	1140	1121
query71	367	251	254	251
query72	6197	4017	4016	4016
query73	756	348	363	348
query74	10154	9063	9023	9023
query75	3402	2648	2652	2648
query76	1718	1147	1066	1066
query77	490	278	262	262
query78	10431	9381	9563	9381
query79	1456	586	590	586
query80	888	412	429	412
query81	499	238	244	238
query82	1200	116	115	115
query83	152	157	152	152
query84	275	67	69	67
query85	880	292	287	287
query86	328	301	289	289
query87	4803	4820	4598	4598
query88	3609	2225	2166	2166
query89	413	298	280	280
query90	2011	185	184	184
query91	132	100	103	100
query92	62	45	48	45
query93	1826	552	568	552
query94	849	288	292	288
query95	342	253	246	246
query96	629	292	298	292
query97	2927	2769	2733	2733
query98	225	206	199	199
query99	1664	1380	1323	1323
Total cold run time: 314372 ms
Total hot run time: 196294 ms

@doris-robot
Copy link

doris-robot commented Nov 4, 2024

ClickBench: Total hot run time: 33.09 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 2b00ce854e4c03ae30cda5e963bef4192fed3a82, data reload: false

query1	0.04	0.04	0.03
query2	0.06	0.04	0.03
query3	0.22	0.07	0.07
query4	1.64	0.10	0.10
query5	0.42	0.41	0.40
query6	1.17	0.67	0.66
query7	0.04	0.02	0.02
query8	0.04	0.04	0.03
query9	0.57	0.48	0.50
query10	0.54	0.54	0.56
query11	0.13	0.11	0.11
query12	0.14	0.11	0.11
query13	0.62	0.60	0.60
query14	2.71	2.72	2.71
query15	0.93	0.84	0.82
query16	0.38	0.38	0.38
query17	1.07	1.05	1.04
query18	0.20	0.20	0.19
query19	1.98	1.89	2.02
query20	0.01	0.01	0.01
query21	15.37	0.59	0.57
query22	2.57	2.08	2.54
query23	17.02	1.10	0.78
query24	2.93	1.59	1.29
query25	0.31	0.08	0.07
query26	0.49	0.14	0.14
query27	0.05	0.04	0.04
query28	10.08	1.11	1.08
query29	12.57	3.22	3.21
query30	0.25	0.06	0.06
query31	2.88	0.39	0.38
query32	3.27	0.46	0.46
query33	2.98	3.00	3.08
query34	17.05	4.47	4.45
query35	4.53	4.47	4.50
query36	0.68	0.49	0.47
query37	0.09	0.06	0.06
query38	0.04	0.04	0.04
query39	0.03	0.03	0.02
query40	0.16	0.13	0.13
query41	0.08	0.02	0.02
query42	0.03	0.02	0.02
query43	0.04	0.03	0.03
Total cold run time: 106.41 s
Total hot run time: 33.09 s

eldenmoon
eldenmoon approved these changes Nov 5, 2024
View reviewed changes
Copy link
Member

@eldenmoon eldenmoon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@github-actions
Copy link
Contributor

github-actions bot commented Nov 5, 2024

PR approved by at least one committer and no changes requested.

@github-actions github-actions bot added approved Indicates a PR has been approved by one committer. reviewed labels Nov 5, 2024
@github-actions
Copy link
Contributor

github-actions bot commented Nov 5, 2024

PR approved by anyone and no changes requested.

csun5285
csun5285 approved these changes Nov 5, 2024
View reviewed changes
Copy link
Contributor

@csun5285 csun5285 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@airborne12 airborne12 merged commit 4b385b5 into apache:master Nov 5, 2024
@airborne12 airborne12 deleted the fix-case branch November 5, 2024 06:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@eldenmoon eldenmoon eldenmoon approved these changes

+1 more reviewer

@csun5285 csun5285 csun5285 approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

approved Indicates a PR has been approved by one committer. reviewed

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@airborne12 @doris-robot @eldenmoon @csun5285