Stabilize credit_cards_bench inlining

[bantonsson]

What does this PR do?

Tells the credit_cards_bench to not inline the is_credit_card method to try to stabilize results a bit.

Motivation

The results fluctuate due to external factors like dependency changes that seem to affect the benchmarking harness and the inlining decisions.

Additional Notes

The benchmark results will differ from main since this PR affects the generated code, but hopefully (and according to my tests), it should stabilize them a bit going forward.

[pr-commenter]

Benchmarks

Comparison

Benchmark execution time: 2024-10-21 11:58:01

Comparing candidate commit ebcd997 in PR branch ban/stabilize-credit-cards-bench with baseline commit 2597236 in branch main.

Found 1 performance improvements and 22 performance regressions! Performance is the same for 28 metrics, 2 unstable metrics.

scenario:credit_card/is_card_number/

• 🟥 execution_time [+3.398µs; +3.399µs] or [+279.597%; +279.651%]
• 🟥 throughput [-606101454.892op/s; -606009209.807op/s] or [-73.664%; -73.653%]

scenario:credit_card/is_card_number/37828224631

• 🟥 execution_time [+3.400µs; +3.402µs] or [+279.792%; +279.939%]
• 🟥 throughput [-606348081.547op/s; -606118374.224op/s] or [-73.689%; -73.661%]

scenario:credit_card/is_card_number/378282246310005

• 🟥 execution_time [+2.215µs; +2.470µs] or [+2.078%; +2.317%]
• 🟥 throughput [-212424.207op/s; -190648.099op/s] or [-2.265%; -2.032%]

scenario:credit_card/is_card_number/37828224631000521389798

• 🟥 execution_time [+5.913µs; +6.087µs] or [+5.237%; +5.390%]
• 🟥 throughput [-453413.600op/s; -440323.177op/s] or [-5.120%; -4.972%]

scenario:credit_card/is_card_number/x371413321323331

• 🟥 execution_time [+2.995µs; +3.152µs] or [+12.449%; +13.098%]
• 🟥 throughput [-4828734.497op/s; -4588727.499op/s] or [-11.616%; -11.038%]

scenario:credit_card/is_card_number_no_luhn/

• 🟥 execution_time [+3.400µs; +3.402µs] or [+279.752%; +279.915%]
• 🟥 throughput [-606269580.198op/s; -606018715.659op/s] or [-73.688%; -73.657%]

scenario:credit_card/is_card_number_no_luhn/ 378282246310005

• 🟥 execution_time [+2.267µs; +2.485µs] or [+2.418%; +2.650%]
• 🟥 throughput [-274581.474op/s; -251091.826op/s] or [-2.575%; -2.355%]

scenario:credit_card/is_card_number_no_luhn/37828224631

• 🟥 execution_time [+3.401µs; +3.401µs] or [+279.856%; +279.918%]
• 🟥 throughput [-606395558.365op/s; -606285268.976op/s] or [-73.683%; -73.670%]

scenario:credit_card/is_card_number_no_luhn/37828224631000521389798

• 🟥 execution_time [+6.041µs; +6.223µs] or [+5.360%; +5.522%]
• 🟥 throughput [-464791.748op/s; -451101.687op/s] or [-5.238%; -5.084%]

scenario:credit_card/is_card_number_no_luhn/x371413321323331

• 🟥 execution_time [+2.830µs; +2.958µs] or [+11.683%; +12.209%]
• 🟥 throughput [-4497338.555op/s; -4308661.040op/s] or [-10.893%; -10.436%]

scenario:normalization/normalize_service/normalize_service/Data🐨dog🐶 繋がっ⛰てて

• 🟥 execution_time [+18.703µs; +18.884µs] or [+4.047%; +4.086%]
• 🟥 throughput [-84960.891op/s; -84138.587op/s] or [-3.927%; -3.889%]

scenario:redis/obfuscate_redis_string

• 🟩 execution_time [-1.364µs; -0.986µs] or [-3.516%; -2.541%]

Candidate

Candidate benchmark details

Group 1

cpu_model	git_commit_sha	git_commit_date	git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	ebcd997	1729511244	ban/stabilize-credit-cards-bench

scenario	metric	min	mean ± sd	median ± mad	p75	p95	p99	max	peak_to_median_ratio	skewness	kurtosis	cv	sem	runs	sample_size
redis/obfuscate_redis_string	execution_time	37.092µs	37.627µs ± 0.910µs	37.201µs ± 0.047µs	37.288µs	39.567µs	39.677µs	40.543µs	8.98%	1.701	1.022	2.41%	0.064µs	1	200

scenario	metric	95% CI mean	Shapiro-Wilk pvalue	Ljung-Box pvalue (lag=1)	Dip test pvalue
redis/obfuscate_redis_string	execution_time	[37.501µs; 37.753µs] or [-0.335%; +0.335%]	None	None	None

Group 2

cpu_model	git_commit_sha	git_commit_date	git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	ebcd997	1729511244	ban/stabilize-credit-cards-bench

scenario	metric	min	mean ± sd	median ± mad	p75	p95	p99	max	peak_to_median_ratio	skewness	kurtosis	cv	sem	runs	sample_size
normalization/normalize_trace/test_trace	execution_time	255.312ns	269.631ns ± 15.858ns	261.565ns ± 4.688ns	278.409ns	304.959ns	312.064ns	312.714ns	19.56%	1.208	0.313	5.87%	1.121ns	1	200

scenario	metric	95% CI mean	Shapiro-Wilk pvalue	Ljung-Box pvalue (lag=1)	Dip test pvalue
normalization/normalize_trace/test_trace	execution_time	[267.434ns; 271.829ns] or [-0.815%; +0.815%]	None	None	None

Group 3

cpu_model	git_commit_sha	git_commit_date	git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	ebcd997	1729511244	ban/stabilize-credit-cards-bench

scenario	metric	min	mean ± sd	median ± mad	p75	p95	p99	max	peak_to_median_ratio	skewness	kurtosis	cv	sem	runs	sample_size
normalization/normalize_name/normalize_name/Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Lo...	execution_time	186.948µs	190.838µs ± 1.477µs	190.723µs ± 0.955µs	191.817µs	193.414µs	194.626µs	194.905µs	2.19%	0.116	0.146	0.77%	0.104µs	1	200
normalization/normalize_name/normalize_name/Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Lo...	throughput	5130711.452op/s	5240359.517op/s ± 40524.549op/s	5243207.526op/s ± 26136.357op/s	5267046.163op/s	5309519.354op/s	5341484.353op/s	5349070.757op/s	2.02%	-0.067	0.146	0.77%	2865.518op/s	1	200
normalization/normalize_name/normalize_name/bad-name	execution_time	18.146µs	18.182µs ± 0.026µs	18.190µs ± 0.028µs	18.200µs	18.230µs	18.242µs	18.249µs	0.32%	0.362	-0.870	0.14%	0.002µs	1	200
normalization/normalize_name/normalize_name/bad-name	throughput	54798490.162op/s	54999193.256op/s ± 78799.718op/s	54975363.583op/s ± 85268.469op/s	55076430.800op/s	55093000.019op/s	55102369.439op/s	55108255.030op/s	0.24%	-0.358	-0.879	0.14%	5571.981op/s	1	200
normalization/normalize_name/normalize_name/good	execution_time	11.261µs	11.300µs ± 0.024µs	11.295µs ± 0.012µs	11.309µs	11.341µs	11.381µs	11.409µs	1.01%	1.401	2.657	0.22%	0.002µs	1	200
normalization/normalize_name/normalize_name/good	throughput	87650219.907op/s	88496846.069op/s ± 190319.326op/s	88537362.768op/s ± 90512.835op/s	88623248.145op/s	88724181.350op/s	88774816.122op/s	88799126.051op/s	0.30%	-1.384	2.581	0.21%	13457.609op/s	1	200

scenario	metric	95% CI mean	Shapiro-Wilk pvalue	Ljung-Box pvalue (lag=1)	Dip test pvalue
normalization/normalize_name/normalize_name/Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Lo...	execution_time	[190.633µs; 191.043µs] or [-0.107%; +0.107%]	None	None	None
normalization/normalize_name/normalize_name/Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Lo...	throughput	[5234743.204op/s; 5245975.829op/s] or [-0.107%; +0.107%]	None	None	None
normalization/normalize_name/normalize_name/bad-name	execution_time	[18.179µs; 18.186µs] or [-0.020%; +0.020%]	None	None	None
normalization/normalize_name/normalize_name/bad-name	throughput	[54988272.373op/s; 55010114.139op/s] or [-0.020%; +0.020%]	None	None	None
normalization/normalize_name/normalize_name/good	execution_time	[11.297µs; 11.303µs] or [-0.030%; +0.030%]	None	None	None
normalization/normalize_name/normalize_name/good	throughput	[88470469.641op/s; 88523222.497op/s] or [-0.030%; +0.030%]	None	None	None

Group 4

cpu_model	git_commit_sha	git_commit_date	git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	ebcd997	1729511244	ban/stabilize-credit-cards-bench

scenario	metric	min	mean ± sd	median ± mad	p75	p95	p99	max	peak_to_median_ratio	skewness	kurtosis	cv	sem	runs	sample_size
two way interface	execution_time	18.346µs	24.329µs ± 9.556µs	19.892µs ± 0.419µs	21.841µs	37.933µs	40.228µs	98.006µs	392.70%	3.373	18.682	39.18%	0.676µs	1	200

scenario	metric	95% CI mean	Shapiro-Wilk pvalue	Ljung-Box pvalue (lag=1)	Dip test pvalue
two way interface	execution_time	[23.005µs; 25.653µs] or [-5.444%; +5.444%]	None	None	None

Group 5

cpu_model	git_commit_sha	git_commit_date	git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	ebcd997	1729511244	ban/stabilize-credit-cards-bench

scenario	metric	min	mean ± sd	median ± mad	p75	p95	p99	max	peak_to_median_ratio	skewness	kurtosis	cv	sem	runs	sample_size
tags/replace_trace_tags	execution_time	2.685µs	2.737µs ± 0.017µs	2.740µs ± 0.005µs	2.745µs	2.757µs	2.766µs	2.774µs	1.24%	-1.361	1.847	0.61%	0.001µs	1	200

scenario	metric	95% CI mean	Shapiro-Wilk pvalue	Ljung-Box pvalue (lag=1)	Dip test pvalue
tags/replace_trace_tags	execution_time	[2.735µs; 2.739µs] or [-0.085%; +0.085%]	None	None	None

Group 6

cpu_model	git_commit_sha	git_commit_date	git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	ebcd997	1729511244	ban/stabilize-credit-cards-bench

scenario	metric	min	mean ± sd	median ± mad	p75	p95	p99	max	peak_to_median_ratio	skewness	kurtosis	cv	sem	runs	sample_size
write only interface	execution_time	1.374µs	3.195µs ± 1.395µs	3.048µs ± 0.024µs	3.069µs	3.099µs	13.794µs	14.815µs	386.02%	7.595	57.809	43.54%	0.099µs	1	200

scenario	metric	95% CI mean	Shapiro-Wilk pvalue	Ljung-Box pvalue (lag=1)	Dip test pvalue
write only interface	execution_time	[3.002µs; 3.389µs] or [-6.050%; +6.050%]	None	None	None

Group 7

cpu_model	git_commit_sha	git_commit_date	git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	ebcd997	1729511244	ban/stabilize-credit-cards-bench

scenario	metric	min	mean ± sd	median ± mad	p75	p95	p99	max	peak_to_median_ratio	skewness	kurtosis	cv	sem	runs	sample_size
normalization/normalize_service/normalize_service/A0000000000000000000000000000000000000000000000000...	execution_time	622.268µs	624.375µs ± 0.505µs	624.370µs ± 0.338µs	624.705µs	625.126µs	625.343µs	626.742µs	0.38%	0.089	2.634	0.08%	0.036µs	1	200
normalization/normalize_service/normalize_service/A0000000000000000000000000000000000000000000000000...	throughput	1595552.875op/s	1601602.223op/s ± 1294.764op/s	1601615.435op/s ± 867.854op/s	1602485.730op/s	1603462.260op/s	1604583.233op/s	1607024.257op/s	0.34%	-0.078	2.621	0.08%	91.554op/s	1	200
normalization/normalize_service/normalize_service/Data🐨dog🐶 繋がっ⛰てて	execution_time	480.046µs	480.960µs ± 0.409µs	480.895µs ± 0.254µs	481.177µs	481.690µs	482.066µs	482.681µs	0.37%	0.844	1.261	0.08%	0.029µs	1	200
normalization/normalize_service/normalize_service/Data🐨dog🐶 繋がっ⛰てて	throughput	2071760.777op/s	2079176.396op/s ± 1765.806op/s	2079457.266op/s ± 1099.824op/s	2080424.560op/s	2081509.046op/s	2082385.877op/s	2083132.786op/s	0.18%	-0.838	1.242	0.08%	124.861op/s	1	200
normalization/normalize_service/normalize_service/Test Conversion 0f Weird !@#$%^&**() Characters	execution_time	178.101µs	178.387µs ± 0.208µs	178.364µs ± 0.059µs	178.409µs	178.611µs	178.919µs	180.253µs	1.06%	5.772	44.993	0.12%	0.015µs	1	200
normalization/normalize_service/normalize_service/Test Conversion 0f Weird !@#$%^&**() Characters	throughput	5547764.630op/s	5605801.846op/s ± 6498.226op/s	5606510.225op/s ± 1849.021op/s	5609112.126op/s	5610989.509op/s	5613365.147op/s	5614776.229op/s	0.15%	-5.724	44.435	0.12%	459.494op/s	1	200
normalization/normalize_service/normalize_service/[empty string]	execution_time	46.611µs	46.737µs ± 0.064µs	46.733µs ± 0.042µs	46.773µs	46.858µs	46.918µs	46.977µs	0.52%	0.588	0.677	0.14%	0.005µs	1	200
normalization/normalize_service/normalize_service/[empty string]	throughput	21287009.602op/s	21396243.749op/s ± 29306.198op/s	21398176.439op/s ± 19323.050op/s	21417705.823op/s	21440140.585op/s	21452800.543op/s	21454221.526op/s	0.26%	-0.579	0.656	0.14%	2072.261op/s	1	200
normalization/normalize_service/normalize_service/test_ASCII	execution_time	48.721µs	49.320µs ± 0.203µs	49.305µs ± 0.139µs	49.450µs	49.695µs	49.797µs	49.883µs	1.17%	0.213	-0.111	0.41%	0.014µs	1	200
normalization/normalize_service/normalize_service/test_ASCII	throughput	20047031.268op/s	20276195.520op/s ± 83587.009op/s	20281727.343op/s ± 57201.995op/s	20336201.155op/s	20411280.665op/s	20433169.538op/s	20525064.564op/s	1.20%	-0.190	-0.117	0.41%	5910.494op/s	1	200

scenario	metric	95% CI mean	Shapiro-Wilk pvalue	Ljung-Box pvalue (lag=1)	Dip test pvalue
normalization/normalize_service/normalize_service/A0000000000000000000000000000000000000000000000000...	execution_time	[624.305µs; 624.445µs] or [-0.011%; +0.011%]	None	None	None
normalization/normalize_service/normalize_service/A0000000000000000000000000000000000000000000000000...	throughput	[1601422.781op/s; 1601781.664op/s] or [-0.011%; +0.011%]	None	None	None
normalization/normalize_service/normalize_service/Data🐨dog🐶 繋がっ⛰てて	execution_time	[480.903µs; 481.017µs] or [-0.012%; +0.012%]	None	None	None
normalization/normalize_service/normalize_service/Data🐨dog🐶 繋がっ⛰てて	throughput	[2078931.672op/s; 2079421.120op/s] or [-0.012%; +0.012%]	None	None	None
normalization/normalize_service/normalize_service/Test Conversion 0f Weird !@#$%^&**() Characters	execution_time	[178.358µs; 178.416µs] or [-0.016%; +0.016%]	None	None	None
normalization/normalize_service/normalize_service/Test Conversion 0f Weird !@#$%^&**() Characters	throughput	[5604901.254op/s; 5606702.437op/s] or [-0.016%; +0.016%]	None	None	None
normalization/normalize_service/normalize_service/[empty string]	execution_time	[46.728µs; 46.746µs] or [-0.019%; +0.019%]	None	None	None
normalization/normalize_service/normalize_service/[empty string]	throughput	[21392182.192op/s; 21400305.306op/s] or [-0.019%; +0.019%]	None	None	None
normalization/normalize_service/normalize_service/test_ASCII	execution_time	[49.292µs; 49.348µs] or [-0.057%; +0.057%]	None	None	None
normalization/normalize_service/normalize_service/test_ASCII	throughput	[20264611.164op/s; 20287779.875op/s] or [-0.057%; +0.057%]	None	None	None

Group 8

cpu_model	git_commit_sha	git_commit_date	git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	ebcd997	1729511244	ban/stabilize-credit-cards-bench

scenario	metric	min	mean ± sd	median ± mad	p75	p95	p99	max	peak_to_median_ratio	skewness	kurtosis	cv	sem	runs	sample_size
sql/obfuscate_sql_string	execution_time	72.296µs	72.439µs ± 0.132µs	72.415µs ± 0.040µs	72.466µs	72.565µs	72.774µs	73.927µs	2.09%	7.580	79.055	0.18%	0.009µs	1	200

scenario	metric	95% CI mean	Shapiro-Wilk pvalue	Ljung-Box pvalue (lag=1)	Dip test pvalue
sql/obfuscate_sql_string	execution_time	[72.420µs; 72.457µs] or [-0.025%; +0.025%]	None	None	None

Group 9

cpu_model	git_commit_sha	git_commit_date	git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	ebcd997	1729511244	ban/stabilize-credit-cards-bench

scenario	metric	min	mean ± sd	median ± mad	p75	p95	p99	max	peak_to_median_ratio	skewness	kurtosis	cv	sem	runs	sample_size
benching string interning on wordpress profile	execution_time	143.013µs	143.838µs ± 0.759µs	143.751µs ± 0.175µs	143.938µs	144.352µs	145.377µs	153.141µs	6.53%	9.455	111.480	0.53%	0.054µs	1	200

scenario	metric	95% CI mean	Shapiro-Wilk pvalue	Ljung-Box pvalue (lag=1)	Dip test pvalue
benching string interning on wordpress profile	execution_time	[143.732µs; 143.943µs] or [-0.073%; +0.073%]	None	None	None

Group 10

cpu_model	git_commit_sha	git_commit_date	git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	ebcd997	1729511244	ban/stabilize-credit-cards-bench

scenario	metric	min	mean ± sd	median ± mad	p75	p95	p99	max	peak_to_median_ratio	skewness	kurtosis	cv	sem	runs	sample_size
concentrator/add_spans_to_concentrator	execution_time	9.065ms	9.101ms ± 0.015ms	9.103ms ± 0.009ms	9.111ms	9.124ms	9.130ms	9.137ms	0.38%	-0.362	-0.041	0.16%	0.001ms	1	200

scenario	metric	95% CI mean	Shapiro-Wilk pvalue	Ljung-Box pvalue (lag=1)	Dip test pvalue
concentrator/add_spans_to_concentrator	execution_time	[9.099ms; 9.103ms] or [-0.022%; +0.022%]	None	None	None

Group 11

cpu_model	git_commit_sha	git_commit_date	git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	ebcd997	1729511244	ban/stabilize-credit-cards-bench

scenario	metric	min	mean ± sd	median ± mad	p75	p95	p99	max	peak_to_median_ratio	skewness	kurtosis	cv	sem	runs	sample_size
benching deserializing traces from msgpack to their internal representation	execution_time	1.148µs	1.220µs ± 0.028µs	1.223µs ± 0.018µs	1.244µs	1.248µs	1.249µs	1.251µs	2.27%	-1.016	0.201	2.25%	0.002µs	1	200

scenario	metric	95% CI mean	Shapiro-Wilk pvalue	Ljung-Box pvalue (lag=1)	Dip test pvalue
benching deserializing traces from msgpack to their internal representation	execution_time	[1.216µs; 1.224µs] or [-0.313%; +0.313%]	None	None	None

Group 12

cpu_model	git_commit_sha	git_commit_date	git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	ebcd997	1729511244	ban/stabilize-credit-cards-bench

scenario	metric	min	mean ± sd	median ± mad	p75	p95	p99	max	peak_to_median_ratio	skewness	kurtosis	cv	sem	runs	sample_size
credit_card/is_card_number/	execution_time	4.604µs	4.614µs ± 0.002µs	4.614µs ± 0.001µs	4.615µs	4.618µs	4.619µs	4.622µs	0.18%	0.197	1.299	0.05%	0.000µs	1	200
credit_card/is_card_number/	throughput	216356620.378op/s	216739466.160op/s ± 109354.876op/s	216744935.949op/s ± 63233.324op/s	216811279.608op/s	216900376.073op/s	216935409.578op/s	217188354.497op/s	0.20%	-0.192	1.304	0.05%	7732.557op/s	1	200
credit_card/is_card_number/ 3782-8224-6310-005	execution_time	118.895µs	120.759µs ± 0.822µs	120.703µs ± 0.578µs	121.296µs	122.157µs	122.659µs	122.931µs	1.85%	0.204	-0.387	0.68%	0.058µs	1	200
credit_card/is_card_number/ 3782-8224-6310-005	throughput	8134638.260op/s	8281325.102op/s ± 56284.363op/s	8284782.098op/s ± 39492.063op/s	8322605.029op/s	8362490.555op/s	8394120.660op/s	8410812.454op/s	1.52%	-0.172	-0.407	0.68%	3979.905op/s	1	200
credit_card/is_card_number/ 378282246310005	execution_time	111.039µs	112.518µs ± 0.646µs	112.448µs ± 0.423µs	112.949µs	113.547µs	114.452µs	114.966µs	2.24%	0.643	1.004	0.57%	0.046µs	1	200
credit_card/is_card_number/ 378282246310005	throughput	8698205.971op/s	8887735.368op/s ± 50835.089op/s	8892998.608op/s ± 33556.642op/s	8922308.900op/s	8963300.131op/s	8985496.596op/s	9005852.995op/s	1.27%	-0.599	0.894	0.57%	3594.584op/s	1	200
credit_card/is_card_number/37828224631	execution_time	4.603µs	4.616µs ± 0.006µs	4.616µs ± 0.002µs	4.618µs	4.620µs	4.623µs	4.684µs	1.48%	8.090	77.779	0.14%	0.000µs	1	200
credit_card/is_card_number/37828224631	throughput	213483840.426op/s	216616232.325op/s ± 294348.154op/s	216646589.854op/s ± 76638.993op/s	216716588.107op/s	216799336.151op/s	216879118.654op/s	217245915.556op/s	0.28%	-8.031	76.951	0.14%	20813.558op/s	1	200
credit_card/is_card_number/378282246310005	execution_time	107.388µs	108.955µs ± 0.728µs	108.902µs ± 0.471µs	109.416µs	110.080µs	110.653µs	111.773µs	2.64%	0.442	0.601	0.67%	0.051µs	1	200
credit_card/is_card_number/378282246310005	throughput	8946700.131op/s	9178524.168op/s ± 61128.675op/s	9182573.771op/s ± 39798.369op/s	9220489.081op/s	9273705.719op/s	9297534.017op/s	9312002.909op/s	1.41%	-0.395	0.488	0.66%	4322.450op/s	1	200
credit_card/is_card_number/37828224631000521389798	execution_time	117.835µs	118.922µs ± 0.399µs	118.965µs ± 0.284µs	119.192µs	119.574µs	119.707µs	119.744µs	0.65%	-0.244	-0.310	0.33%	0.028µs	1	200
credit_card/is_card_number/37828224631000521389798	throughput	8351155.682op/s	8408949.369op/s ± 28238.358op/s	8405817.024op/s ± 20099.863op/s	8431087.662op/s	8458119.116op/s	8482166.926op/s	8486457.311op/s	0.96%	0.261	-0.295	0.33%	1996.753op/s	1	200
credit_card/is_card_number/x371413321323331	execution_time	26.457µs	27.135µs ± 0.423µs	27.112µs ± 0.319µs	27.392µs	27.902µs	28.478µs	28.614µs	5.54%	0.829	0.633	1.56%	0.030µs	1	200
credit_card/is_card_number/x371413321323331	throughput	34947336.832op/s	36861817.785op/s ± 567783.160op/s	36884469.422op/s ± 435183.022op/s	37360653.597op/s	37592473.783op/s	37714028.415op/s	37797289.031op/s	2.47%	-0.742	0.383	1.54%	40148.332op/s	1	200
credit_card/is_card_number_no_luhn/	execution_time	4.601µs	4.617µs ± 0.007µs	4.616µs ± 0.002µs	4.618µs	4.620µs	4.622µs	4.709µs	2.00%	11.147	143.755	0.15%	0.000µs	1	200
credit_card/is_card_number_no_luhn/	throughput	212381748.107op/s	216609295.255op/s ± 324834.734op/s	216627451.467op/s ± 73488.048op/s	216703483.511op/s	216797024.258op/s	216897440.750op/s	217360711.354op/s	0.34%	-11.048	142.107	0.15%	22969.284op/s	1	200
credit_card/is_card_number_no_luhn/ 3782-8224-6310-005	execution_time	103.149µs	104.608µs ± 0.525µs	104.609µs ± 0.342µs	104.953µs	105.522µs	105.772µs	105.927µs	1.26%	-0.059	-0.076	0.50%	0.037µs	1	200
credit_card/is_card_number_no_luhn/ 3782-8224-6310-005	throughput	9440453.504op/s	9559739.169op/s ± 48011.178op/s	9559433.591op/s ± 31215.787op/s	9589636.514op/s	9635731.657op/s	9668979.102op/s	9694679.207op/s	1.41%	0.088	-0.061	0.50%	3394.903op/s	1	200
credit_card/is_card_number_no_luhn/ 378282246310005	execution_time	94.558µs	96.157µs ± 0.774µs	96.158µs ± 0.538µs	96.679µs	97.530µs	98.101µs	98.281µs	2.21%	0.377	-0.266	0.80%	0.055µs	1	200
credit_card/is_card_number_no_luhn/ 378282246310005	throughput	10174941.155op/s	10400323.087op/s ± 83477.489op/s	10399588.851op/s ± 58413.387op/s	10458683.504op/s	10529905.016op/s	10560367.240op/s	10575563.458op/s	1.69%	-0.339	-0.302	0.80%	5902.750op/s	1	200
credit_card/is_card_number_no_luhn/37828224631	execution_time	4.599µs	4.616µs ± 0.003µs	4.616µs ± 0.002µs	4.618µs	4.620µs	4.622µs	4.624µs	0.18%	-0.856	7.286	0.06%	0.000µs	1	200
credit_card/is_card_number_no_luhn/37828224631	throughput	216245301.006op/s	216637579.711op/s ± 123382.052op/s	216644051.674op/s ± 74560.528op/s	216713121.980op/s	216811611.366op/s	216841738.401op/s	217421142.445op/s	0.36%	0.870	7.375	0.06%	8724.429op/s	1	200
credit_card/is_card_number_no_luhn/378282246310005	execution_time	90.401µs	91.733µs ± 0.603µs	91.723µs ± 0.499µs	92.229µs	92.667µs	92.927µs	93.069µs	1.47%	-0.025	-0.852	0.66%	0.043µs	1	200
credit_card/is_card_number_no_luhn/378282246310005	throughput	10744709.989op/s	10901703.544op/s ± 71694.849op/s	10902365.501op/s ± 59035.343op/s	10956815.905op/s	11021647.908op/s	11039966.268op/s	11061844.561op/s	1.46%	0.048	-0.851	0.66%	5069.591op/s	1	200
credit_card/is_card_number_no_luhn/37828224631000521389798	execution_time	117.699µs	118.832µs ± 0.421µs	118.902µs ± 0.245µs	119.086µs	119.428µs	119.809µs	119.919µs	0.86%	-0.336	-0.000	0.35%	0.030µs	1	200
credit_card/is_card_number_no_luhn/37828224631000521389798	throughput	8338950.679op/s	8415356.599op/s ± 29875.248op/s	8410313.264op/s ± 17323.045op/s	8431671.003op/s	8471649.078op/s	8484447.166op/s	8496251.454op/s	1.02%	0.356	0.005	0.35%	2112.499op/s	1	200
credit_card/is_card_number_no_luhn/x371413321323331	execution_time	26.498µs	27.118µs ± 0.379µs	27.088µs ± 0.287µs	27.389µs	27.793µs	28.032µs	28.140µs	3.88%	0.451	-0.519	1.39%	0.027µs	1	200
credit_card/is_card_number_no_luhn/x371413321323331	throughput	35536383.076op/s	36883247.661op/s ± 512063.862op/s	36916860.698op/s ± 393837.069op/s	37284169.002op/s	37613883.388op/s	37707159.948op/s	37739165.552op/s	2.23%	-0.399	-0.587	1.38%	36208.383op/s	1	200

scenario	metric	95% CI mean	Shapiro-Wilk pvalue	Ljung-Box pvalue (lag=1)	Dip test pvalue
credit_card/is_card_number/	execution_time	[4.614µs; 4.614µs] or [-0.007%; +0.007%]	None	None	None
credit_card/is_card_number/	throughput	[216724310.626op/s; 216754621.694op/s] or [-0.007%; +0.007%]	None	None	None
credit_card/is_card_number/ 3782-8224-6310-005	execution_time	[120.645µs; 120.873µs] or [-0.094%; +0.094%]	None	None	None
credit_card/is_card_number/ 3782-8224-6310-005	throughput	[8273524.631op/s; 8289125.574op/s] or [-0.094%; +0.094%]	None	None	None
credit_card/is_card_number/ 378282246310005	execution_time	[112.429µs; 112.608µs] or [-0.080%; +0.080%]	None	None	None
credit_card/is_card_number/ 378282246310005	throughput	[8880690.114op/s; 8894780.623op/s] or [-0.079%; +0.079%]	None	None	None
credit_card/is_card_number/37828224631	execution_time	[4.616µs; 4.617µs] or [-0.019%; +0.019%]	None	None	None
credit_card/is_card_number/37828224631	throughput	[216575438.501op/s; 216657026.148op/s] or [-0.019%; +0.019%]	None	None	None
credit_card/is_card_number/378282246310005	execution_time	[108.854µs; 109.056µs] or [-0.093%; +0.093%]	None	None	None
credit_card/is_card_number/378282246310005	throughput	[9170052.322op/s; 9186996.015op/s] or [-0.092%; +0.092%]	None	None	None
credit_card/is_card_number/37828224631000521389798	execution_time	[118.867µs; 118.978µs] or [-0.047%; +0.047%]	None	None	None
credit_card/is_card_number/37828224631000521389798	throughput	[8405035.804op/s; 8412862.933op/s] or [-0.047%; +0.047%]	None	None	None
credit_card/is_card_number/x371413321323331	execution_time	[27.076µs; 27.193µs] or [-0.216%; +0.216%]	None	None	None
credit_card/is_card_number/x371413321323331	throughput	[36783128.500op/s; 36940507.070op/s] or [-0.213%; +0.213%]	None	None	None
credit_card/is_card_number_no_luhn/	execution_time	[4.616µs; 4.618µs] or [-0.021%; +0.021%]	None	None	None
credit_card/is_card_number_no_luhn/	throughput	[216564276.285op/s; 216654314.225op/s] or [-0.021%; +0.021%]	None	None	None
credit_card/is_card_number_no_luhn/ 3782-8224-6310-005	execution_time	[104.535µs; 104.681µs] or [-0.070%; +0.070%]	None	None	None
credit_card/is_card_number_no_luhn/ 3782-8224-6310-005	throughput	[9553085.281op/s; 9566393.056op/s] or [-0.070%; +0.070%]	None	None	None
credit_card/is_card_number_no_luhn/ 378282246310005	execution_time	[96.050µs; 96.264µs] or [-0.112%; +0.112%]	None	None	None
credit_card/is_card_number_no_luhn/ 378282246310005	throughput	[10388753.910op/s; 10411892.264op/s] or [-0.111%; +0.111%]	None	None	None
credit_card/is_card_number_no_luhn/37828224631	execution_time	[4.616µs; 4.616µs] or [-0.008%; +0.008%]	None	None	None
credit_card/is_card_number_no_luhn/37828224631	throughput	[216620480.146op/s; 216654679.277op/s] or [-0.008%; +0.008%]	None	None	None
credit_card/is_card_number_no_luhn/378282246310005	execution_time	[91.649µs; 91.816µs] or [-0.091%; +0.091%]	None	None	None
credit_card/is_card_number_no_luhn/378282246310005	throughput	[10891767.328op/s; 10911639.761op/s] or [-0.091%; +0.091%]	None	None	None
credit_card/is_card_number_no_luhn/37828224631000521389798	execution_time	[118.773µs; 118.890µs] or [-0.049%; +0.049%]	None	None	None
credit_card/is_card_number_no_luhn/37828224631000521389798	throughput	[8411216.177op/s; 8419497.021op/s] or [-0.049%; +0.049%]	None	None	None
credit_card/is_card_number_no_luhn/x371413321323331	execution_time	[27.065µs; 27.170µs] or [-0.194%; +0.194%]	None	None	None
credit_card/is_card_number_no_luhn/x371413321323331	throughput	[36812280.534op/s; 36954214.787op/s] or [-0.192%; +0.192%]	None	None	None

Baseline

Omitted due to size.

[codecov-commenter]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 71.80%. Comparing base (2597236) to head (ebcd997).

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #683      +/-   ##
==========================================
+ Coverage   71.78%   71.80%   +0.01%     
==========================================
  Files         271      271              
  Lines       40984    40992       +8     
==========================================
+ Hits        29422    29434      +12     
+ Misses      11562    11558       -4     

Components	Coverage Δ
crashtracker	20.33% <ø> (ø)
crashtracker-ffi	9.07% <ø> (ø)
datadog-alloc	98.73% <ø> (ø)
data-pipeline	91.85% <ø> (+0.25%)	⬆️
data-pipeline-ffi	0.00% <ø> (ø)
ddcommon	82.73% <ø> (ø)
ddcommon-ffi	69.12% <ø> (ø)
ddtelemetry	59.10% <ø> (ø)
ddtelemetry-ffi	22.13% <ø> (ø)
dogstatsd	88.86% <ø> (ø)
dogstatsd-client	79.77% <ø> (ø)
ipc	82.65% <ø> (ø)
profiling	84.30% <ø> (ø)
profiling-ffi	77.46% <ø> (ø)
serverless	0.00% <ø> (ø)
sidecar	36.09% <ø> (ø)
sidecar-ffi	0.00% <ø> (ø)
spawn-worker	50.36% <ø> (ø)
tinybytes	94.77% <ø> (ø)
trace-mini-agent	72.45% <ø> (ø)
trace-normalization	98.25% <ø> (ø)
trace-obfuscation	95.73% <ø> (ø)
trace-protobuf	77.67% <ø> (ø)
trace-utils	93.08% <ø> (-0.42%)	⬇️

[VianneyRuhlmann]

🤞