Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[fix](external) record not found file number #38253

Merged
merged 2 commits into from
Jul 24, 2024

Conversation

morningman
Copy link
Contributor

followup #37042

  1. The previous PR miss one place to record "not found" file num
  2. Catch "NoSuchKey" error and treat it as "not found"

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@morningman
Copy link
Contributor Author

run buildall

Copy link
Contributor

clang-tidy review says "All clean, LGTM! 👍"

1 similar comment
Copy link
Contributor

clang-tidy review says "All clean, LGTM! 👍"

@doris-robot
Copy link

TPC-H: Total hot run time: 39945 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 21640132ebacd8db92605c4d80886cf63db751d4, data reload: false

------ Round 1 ----------------------------------
q1	17642	4318	4282	4282
q2	2013	193	189	189
q3	10456	1163	1104	1104
q4	10177	848	757	757
q5	7521	2656	2775	2656
q6	231	136	141	136
q7	976	608	610	608
q8	9225	2059	2073	2059
q9	8839	6531	6980	6531
q10	8811	3800	3775	3775
q11	452	239	246	239
q12	478	222	229	222
q13	17868	3002	3011	3002
q14	279	237	237	237
q15	523	494	503	494
q16	524	397	382	382
q17	1010	664	670	664
q18	8014	7426	7360	7360
q19	3658	1460	1369	1369
q20	652	329	317	317
q21	4921	3275	4066	3275
q22	343	290	287	287
Total cold run time: 114613 ms
Total hot run time: 39945 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4345	4236	4229	4229
q2	377	274	271	271
q3	3015	2779	2905	2779
q4	1990	1758	1708	1708
q5	5688	5551	5539	5539
q6	221	128	130	128
q7	2165	1862	1905	1862
q8	3307	3403	3451	3403
q9	8977	8953	9085	8953
q10	4229	4036	3819	3819
q11	590	507	517	507
q12	833	619	621	619
q13	15942	3250	3234	3234
q14	321	278	284	278
q15	529	484	491	484
q16	472	467	445	445
q17	1802	1568	1513	1513
q18	8206	8122	7757	7757
q19	3183	1633	1408	1408
q20	2151	1865	1882	1865
q21	5115	4980	4799	4799
q22	649	507	513	507
Total cold run time: 74107 ms
Total hot run time: 56107 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 174530 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 21640132ebacd8db92605c4d80886cf63db751d4, data reload: false

query1	909	366	368	366
query2	6444	1890	1742	1742
query3	6634	209	216	209
query4	28586	17680	17341	17341
query5	3588	479	483	479
query6	254	171	160	160
query7	4587	295	283	283
query8	235	201	196	196
query9	8512	2433	2395	2395
query10	429	275	265	265
query11	12537	10310	10267	10267
query12	120	83	84	83
query13	1638	372	360	360
query14	10103	7835	7578	7578
query15	251	167	161	161
query16	7743	481	483	481
query17	1526	536	522	522
query18	1957	282	287	282
query19	199	155	159	155
query20	91	112	79	79
query21	208	127	124	124
query22	4556	4114	4216	4114
query23	34129	33831	33759	33759
query24	8984	3008	2876	2876
query25	641	381	377	377
query26	701	158	159	158
query27	2225	287	295	287
query28	6124	2108	2098	2098
query29	897	626	638	626
query30	240	150	151	150
query31	996	781	753	753
query32	97	54	59	54
query33	645	345	327	327
query34	891	490	501	490
query35	866	772	765	765
query36	1166	966	985	966
query37	149	86	90	86
query38	2971	2917	2835	2835
query39	891	872	852	852
query40	211	126	129	126
query41	49	49	50	49
query42	116	102	137	102
query43	521	469	460	460
query44	1071	736	736	736
query45	198	170	162	162
query46	1074	727	737	727
query47	1833	1735	1770	1735
query48	357	287	292	287
query49	812	406	408	406
query50	775	387	385	385
query51	6893	6812	6637	6637
query52	106	94	94	94
query53	362	294	300	294
query54	793	453	448	448
query55	75	73	74	73
query56	293	264	277	264
query57	1132	1052	1031	1031
query58	247	247	260	247
query59	2893	2540	2495	2495
query60	303	277	304	277
query61	99	95	92	92
query62	803	651	642	642
query63	326	286	294	286
query64	9127	2214	1671	1671
query65	3178	3154	3134	3134
query66	703	327	337	327
query67	15786	15128	15054	15054
query68	4508	557	545	545
query69	446	353	346	346
query70	1218	1102	1031	1031
query71	414	284	276	276
query72	6895	5477	5912	5477
query73	745	326	325	325
query74	6172	5770	5699	5699
query75	3379	2756	2732	2732
query76	2086	975	946	946
query77	436	304	306	304
query78	11330	9209	12142	9209
query79	2097	529	521	521
query80	1807	469	470	469
query81	573	221	217	217
query82	284	135	132	132
query83	269	165	164	164
query84	276	86	85	85
query85	1044	340	300	300
query86	465	303	327	303
query87	3310	3128	3116	3116
query88	2986	2376	2368	2368
query89	475	381	383	381
query90	1783	197	191	191
query91	126	100	98	98
query92	57	48	48	48
query93	992	517	511	511
query94	1080	288	297	288
query95	410	314	322	314
query96	588	271	276	271
query97	3210	3016	3064	3016
query98	290	205	199	199
query99	1572	1306	1317	1306
Total cold run time: 275179 ms
Total hot run time: 174530 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.47 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 21640132ebacd8db92605c4d80886cf63db751d4, data reload: false

query1	0.04	0.04	0.04
query2	0.08	0.04	0.04
query3	0.23	0.05	0.04
query4	1.68	0.07	0.07
query5	0.50	0.48	0.47
query6	1.14	0.72	0.73
query7	0.02	0.01	0.02
query8	0.05	0.04	0.05
query9	0.56	0.48	0.49
query10	0.55	0.55	0.55
query11	0.15	0.11	0.12
query12	0.14	0.12	0.12
query13	0.59	0.59	0.59
query14	0.76	0.77	0.77
query15	0.85	0.81	0.82
query16	0.35	0.37	0.37
query17	1.02	1.05	0.99
query18	0.23	0.22	0.21
query19	1.78	1.67	1.76
query20	0.01	0.00	0.01
query21	15.42	0.76	0.65
query22	4.32	7.45	1.79
query23	18.22	1.39	1.27
query24	2.18	0.22	0.22
query25	0.15	0.08	0.09
query26	0.29	0.22	0.21
query27	0.45	0.23	0.24
query28	13.27	1.03	0.99
query29	12.63	3.36	3.30
query30	0.25	0.06	0.06
query31	2.84	0.39	0.39
query32	3.27	0.47	0.48
query33	2.89	2.90	2.90
query34	17.22	4.36	4.34
query35	4.42	4.42	4.44
query36	0.66	0.49	0.47
query37	0.19	0.16	0.15
query38	0.15	0.16	0.14
query39	0.05	0.04	0.03
query40	0.16	0.13	0.12
query41	0.10	0.05	0.06
query42	0.06	0.05	0.05
query43	0.05	0.04	0.04
Total cold run time: 109.97 s
Total hot run time: 30.47 s

Copy link
Contributor

@kaka11chen kaka11chen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Copy link
Contributor

PR approved by at least one committer and no changes requested.

@github-actions github-actions bot added approved Indicates a PR has been approved by one committer. reviewed labels Jul 24, 2024
Copy link
Contributor

PR approved by anyone and no changes requested.

@morningman morningman merged commit 124de2a into apache:master Jul 24, 2024
27 of 29 checks passed
morningman added a commit to morningman/doris that referenced this pull request Jul 24, 2024
followup apache#37042
1. The previous PR miss one place to record "not found" file num
2. Catch "NoSuchKey" error and treat it as "not found"
dataroaring pushed a commit that referenced this pull request Jul 24, 2024
followup #37042
1. The previous PR miss one place to record "not found" file num
2. Catch "NoSuchKey" error and treat it as "not found"
@yiguolei yiguolei mentioned this pull request Sep 5, 2024
3 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by one committer. dev/2.1.6-merged dev/3.0.1-merged reviewed
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants