Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Lock down nightly version for reproducible builds #686

Merged
merged 1 commit into from
Oct 23, 2024
Merged

Lock down nightly version for reproducible builds #686

merged 1 commit into from
Oct 23, 2024

Conversation

bantonsson
Copy link
Contributor

What does this PR do?

Locks down the nightly version of rust so that builds are reproducible and don't break accidentally.

  • Use nightly-2024-10-21
  • Fix clippy lints
  • Change nextest version to 0.9.81 in all places

Motivation

Changes in the nightly version of rust can break formatting, clippy linting and coverage, among other things. That is not a good developer experience.

@bantonsson bantonsson requested review from a team as code owners October 22, 2024 14:55
morrisonlevi
morrisonlevi reviewed Oct 22, 2024
View reviewed changes
Copy link
Contributor

@morrisonlevi morrisonlevi left a comment •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Was a specific strategy for bumping the nightly version also chosen? In practice, it needs to be bumped periodically or it will break for a different reasons :-)

@pr-commenter
Copy link

pr-commenter bot commented Oct 22, 2024 •
edited
Loading

Benchmarks

Comparison

Benchmark execution time: 2024-10-23 07:38:48

Comparing candidate commit 54be47b in PR branch ban/lock-down-nightly-version-for-reproducible-builds with baseline commit 8d6b9cc in branch main.

Found 1 performance improvements and 0 performance regressions! Performance is the same for 50 metrics, 2 unstable metrics.

scenario:benching deserializing traces from msgpack to their internal representation

  • 🟩 execution_time [-39.010ns; -30.154ns] or [-3.208%; -2.480%]

Candidate

Candidate benchmark details

Group 1

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 54be47b 1729668267 ban/lock-down-nightly-version-for-reproducible-builds
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
two way interface execution_time 18.794µs 25.634µs ± 11.632µs 20.029µs ± 0.208µs 23.157µs 47.363µs 50.089µs 102.403µs 411.29% 3.001 13.345 45.26% 0.823µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
two way interface execution_time [24.022µs; 27.246µs] or [-6.289%; +6.289%] None None None

Group 2

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 54be47b 1729668267 ban/lock-down-nightly-version-for-reproducible-builds
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
tags/replace_trace_tags execution_time 2.651µs 2.681µs ± 0.022µs 2.678µs ± 0.015µs 2.691µs 2.731µs 2.736µs 2.741µs 2.37% 1.015 0.209 0.82% 0.002µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
tags/replace_trace_tags execution_time [2.678µs; 2.684µs] or [-0.114%; +0.114%] None None None

Group 3

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 54be47b 1729668267 ban/lock-down-nightly-version-for-reproducible-builds
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
normalization/normalize_name/normalize_name/Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Lo... execution_time 190.619µs 192.814µs ± 0.744µs 192.889µs ± 0.502µs 193.312µs 193.875µs 194.272µs 194.331µs 0.75% -0.393 -0.032 0.38% 0.053µs 1 200
normalization/normalize_name/normalize_name/Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Lo... throughput 5145862.868op/s 5186417.510op/s ± 20046.143op/s 5184339.067op/s ± 13509.590op/s 5198847.362op/s 5220022.007op/s 5241071.161op/s 5246075.452op/s 1.19% 0.414 -0.003 0.39% 1417.476op/s 1 200
normalization/normalize_name/normalize_name/bad-name execution_time 18.195µs 18.218µs ± 0.024µs 18.209µs ± 0.005µs 18.215µs 18.278µs 18.298µs 18.307µs 0.54% 2.082 3.475 0.13% 0.002µs 1 200
normalization/normalize_name/normalize_name/bad-name throughput 54623459.202op/s 54891863.691op/s ± 71264.947op/s 54918817.887op/s ± 14224.105op/s 54929855.625op/s 54942652.675op/s 54948719.387op/s 54958643.621op/s 0.07% -2.078 3.453 0.13% 5039.193op/s 1 200
normalization/normalize_name/normalize_name/good execution_time 11.301µs 11.356µs ± 0.027µs 11.353µs ± 0.020µs 11.375µs 11.400µs 11.424µs 11.466µs 1.00% 0.507 0.434 0.24% 0.002µs 1 200
normalization/normalize_name/normalize_name/good throughput 87214907.753op/s 88059593.367op/s ± 210868.572op/s 88084566.693op/s ± 157753.541op/s 88222335.706op/s 88356748.106op/s 88441610.921op/s 88487377.414op/s 0.46% -0.491 0.390 0.24% 14910.660op/s 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
normalization/normalize_name/normalize_name/Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Lo... execution_time [192.711µs; 192.917µs] or [-0.053%; +0.053%] None None None
normalization/normalize_name/normalize_name/Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Lo... throughput [5183639.308op/s; 5189195.713op/s] or [-0.054%; +0.054%] None None None
normalization/normalize_name/normalize_name/bad-name execution_time [18.214µs; 18.221µs] or [-0.018%; +0.018%] None None None
normalization/normalize_name/normalize_name/bad-name throughput [54881987.054op/s; 54901740.327op/s] or [-0.018%; +0.018%] None None None
normalization/normalize_name/normalize_name/good execution_time [11.352µs; 11.360µs] or [-0.033%; +0.033%] None None None
normalization/normalize_name/normalize_name/good throughput [88030369.011op/s; 88088817.723op/s] or [-0.033%; +0.033%] None None None

Group 4

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 54be47b 1729668267 ban/lock-down-nightly-version-for-reproducible-builds
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
normalization/normalize_trace/test_trace execution_time 258.932ns 271.870ns ± 18.193ns 264.638ns ± 3.607ns 271.021ns 320.919ns 330.310ns 334.064ns 26.23% 2.165 3.683 6.67% 1.286ns 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
normalization/normalize_trace/test_trace execution_time [269.349ns; 274.392ns] or [-0.927%; +0.927%] None None None

Group 5

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 54be47b 1729668267 ban/lock-down-nightly-version-for-reproducible-builds
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
benching string interning on wordpress profile execution_time 143.715µs 144.444µs ± 0.335µs 144.416µs ± 0.171µs 144.621µs 144.877µs 145.656µs 145.912µs 1.04% 0.857 3.186 0.23% 0.024µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
benching string interning on wordpress profile execution_time [144.398µs; 144.491µs] or [-0.032%; +0.032%] None None None

Group 6

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 54be47b 1729668267 ban/lock-down-nightly-version-for-reproducible-builds
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
write only interface execution_time 1.399µs 3.197µs ± 1.386µs 3.050µs ± 0.028µs 3.076µs 3.125µs 13.826µs 14.847µs 386.84% 7.583 57.701 43.25% 0.098µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
write only interface execution_time [3.005µs; 3.389µs] or [-6.009%; +6.009%] None None None

Group 7

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 54be47b 1729668267 ban/lock-down-nightly-version-for-reproducible-builds
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
redis/obfuscate_redis_string execution_time 38.068µs 38.498µs ± 0.660µs 38.225µs ± 0.045µs 38.261µs 39.868µs 39.898µs 41.457µs 8.45% 1.853 2.151 1.71% 0.047µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
redis/obfuscate_redis_string execution_time [38.407µs; 38.589µs] or [-0.238%; +0.238%] None None None

Group 8

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 54be47b 1729668267 ban/lock-down-nightly-version-for-reproducible-builds
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
concentrator/add_spans_to_concentrator execution_time 9.084ms 9.112ms ± 0.013ms 9.111ms ± 0.008ms 9.119ms 9.136ms 9.142ms 9.152ms 0.45% 0.289 0.093 0.14% 0.001ms 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
concentrator/add_spans_to_concentrator execution_time [9.110ms; 9.114ms] or [-0.020%; +0.020%] None None None

Group 9

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 54be47b 1729668267 ban/lock-down-nightly-version-for-reproducible-builds
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
normalization/normalize_service/normalize_service/A0000000000000000000000000000000000000000000000000... execution_time 623.088µs 624.179µs ± 0.593µs 624.144µs ± 0.451µs 624.603µs 625.157µs 625.639µs 625.790µs 0.26% 0.236 -0.386 0.09% 0.042µs 1 200
normalization/normalize_service/normalize_service/A0000000000000000000000000000000000000000000000000... throughput 1597979.606op/s 1602106.181op/s ± 1522.735op/s 1602195.282op/s ± 1159.118op/s 1603255.617op/s 1604556.133op/s 1604792.909op/s 1604909.372op/s 0.17% -0.232 -0.391 0.09% 107.674op/s 1 200
normalization/normalize_service/normalize_service/Data🐨dog🐶 繋がっ⛰てて execution_time 461.059µs 461.892µs ± 0.337µs 461.863µs ± 0.210µs 462.079µs 462.491µs 462.723µs 462.876µs 0.22% 0.405 -0.010 0.07% 0.024µs 1 200
normalization/normalize_service/normalize_service/Data🐨dog🐶 繋がっ⛰てて throughput 2160406.302op/s 2165011.304op/s ± 1577.438op/s 2165142.112op/s ± 984.030op/s 2166086.939op/s 2167345.886op/s 2167828.738op/s 2168921.794op/s 0.17% -0.401 -0.014 0.07% 111.542op/s 1 200
normalization/normalize_service/normalize_service/Test Conversion 0f Weird !@#$%^&**() Characters execution_time 178.217µs 178.471µs ± 0.106µs 178.458µs ± 0.055µs 178.522µs 178.650µs 178.836µs 178.910µs 0.25% 1.030 2.153 0.06% 0.008µs 1 200
normalization/normalize_service/normalize_service/Test Conversion 0f Weird !@#$%^&**() Characters throughput 5589414.685op/s 5603148.917op/s ± 3339.383op/s 5603557.083op/s ± 1740.033op/s 5605082.497op/s 5607851.315op/s 5608370.500op/s 5611127.002op/s 0.14% -1.024 2.135 0.06% 236.130op/s 1 200
normalization/normalize_service/normalize_service/[empty string] execution_time 46.557µs 46.778µs ± 0.076µs 46.774µs ± 0.049µs 46.822µs 46.918µs 46.938µs 46.967µs 0.41% 0.237 -0.240 0.16% 0.005µs 1 200
normalization/normalize_service/normalize_service/[empty string] throughput 21291699.454op/s 21377584.724op/s ± 34728.805op/s 21379596.557op/s ± 22500.745op/s 21402100.155op/s 21429068.401op/s 21444210.908op/s 21478978.478op/s 0.46% -0.229 -0.240 0.16% 2455.697op/s 1 200
normalization/normalize_service/normalize_service/test_ASCII execution_time 49.386µs 50.036µs ± 0.295µs 50.055µs ± 0.235µs 50.271µs 50.458µs 50.559µs 50.823µs 1.53% -0.074 -0.917 0.59% 0.021µs 1 200
normalization/normalize_service/normalize_service/test_ASCII throughput 19676153.394op/s 19986437.952op/s ± 118059.143op/s 19978170.376op/s ± 93844.656op/s 20085512.904op/s 20174049.028op/s 20213248.747op/s 20248729.745op/s 1.35% 0.093 -0.922 0.59% 8348.042op/s 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
normalization/normalize_service/normalize_service/A0000000000000000000000000000000000000000000000000... execution_time [624.097µs; 624.261µs] or [-0.013%; +0.013%] None None None
normalization/normalize_service/normalize_service/A0000000000000000000000000000000000000000000000000... throughput [1601895.145op/s; 1602317.218op/s] or [-0.013%; +0.013%] None None None
normalization/normalize_service/normalize_service/Data🐨dog🐶 繋がっ⛰てて execution_time [461.845µs; 461.938µs] or [-0.010%; +0.010%] None None None
normalization/normalize_service/normalize_service/Data🐨dog🐶 繋がっ⛰てて throughput [2164792.686op/s; 2165229.922op/s] or [-0.010%; +0.010%] None None None
normalization/normalize_service/normalize_service/Test Conversion 0f Weird !@#$%^&**() Characters execution_time [178.456µs; 178.486µs] or [-0.008%; +0.008%] None None None
normalization/normalize_service/normalize_service/Test Conversion 0f Weird !@#$%^&**() Characters throughput [5602686.111op/s; 5603611.723op/s] or [-0.008%; +0.008%] None None None
normalization/normalize_service/normalize_service/[empty string] execution_time [46.768µs; 46.789µs] or [-0.023%; +0.023%] None None None
normalization/normalize_service/normalize_service/[empty string] throughput [21372771.646op/s; 21382397.803op/s] or [-0.023%; +0.023%] None None None
normalization/normalize_service/normalize_service/test_ASCII execution_time [49.995µs; 50.077µs] or [-0.082%; +0.082%] None None None
normalization/normalize_service/normalize_service/test_ASCII throughput [19970076.091op/s; 20002799.814op/s] or [-0.082%; +0.082%] None None None

Group 10

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 54be47b 1729668267 ban/lock-down-nightly-version-for-reproducible-builds
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
benching deserializing traces from msgpack to their internal representation execution_time 1.123µs 1.181µs ± 0.024µs 1.187µs ± 0.015µs 1.201µs 1.207µs 1.209µs 1.210µs 1.91% -0.890 -0.095 1.99% 0.002µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
benching deserializing traces from msgpack to their internal representation execution_time [1.178µs; 1.185µs] or [-0.277%; +0.277%] None None None

Group 11

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 54be47b 1729668267 ban/lock-down-nightly-version-for-reproducible-builds
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
sql/obfuscate_sql_string execution_time 72.583µs 72.721µs ± 0.117µs 72.708µs ± 0.025µs 72.733µs 72.788µs 72.975µs 73.957µs 1.72% 7.989 75.140 0.16% 0.008µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
sql/obfuscate_sql_string execution_time [72.705µs; 72.737µs] or [-0.022%; +0.022%] None None None

Group 12

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 54be47b 1729668267 ban/lock-down-nightly-version-for-reproducible-builds
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
credit_card/is_card_number/ execution_time 4.605µs 4.615µs ± 0.002µs 4.614µs ± 0.002µs 4.616µs 4.618µs 4.620µs 4.621µs 0.14% -0.008 0.843 0.05% 0.000µs 1 200
credit_card/is_card_number/ throughput 216407847.340op/s 216701372.518op/s ± 105096.708op/s 216709003.906op/s ± 76499.368op/s 216777574.360op/s 216838891.142op/s 216882717.586op/s 217156788.604op/s 0.21% 0.012 0.853 0.05% 7431.459op/s 1 200
credit_card/is_card_number/ 3782-8224-6310-005 execution_time 90.449µs 91.513µs ± 0.490µs 91.511µs ± 0.293µs 91.774µs 92.364µs 93.156µs 93.459µs 2.13% 0.728 1.675 0.53% 0.035µs 1 200
credit_card/is_card_number/ 3782-8224-6310-005 throughput 10699898.867op/s 10927676.228op/s ± 58336.204op/s 10927635.826op/s ± 35139.529op/s 10966022.071op/s 11015663.613op/s 11039870.957op/s 11055915.292op/s 1.17% -0.678 1.526 0.53% 4124.993op/s 1 200
credit_card/is_card_number/ 378282246310005 execution_time 82.782µs 83.563µs ± 0.496µs 83.522µs ± 0.327µs 83.846µs 84.447µs 84.942µs 85.881µs 2.82% 1.108 2.105 0.59% 0.035µs 1 200
credit_card/is_card_number/ 378282246310005 throughput 11643985.154op/s 11967474.056op/s ± 70529.061op/s 11972862.908op/s ± 46663.966op/s 12018874.612op/s 12058879.683op/s 12072081.824op/s 12079938.996op/s 0.89% -1.058 1.891 0.59% 4987.158op/s 1 200
credit_card/is_card_number/37828224631 execution_time 4.600µs 4.615µs ± 0.003µs 4.615µs ± 0.001µs 4.616µs 4.618µs 4.619µs 4.640µs 0.56% 3.925 33.814 0.07% 0.000µs 1 200
credit_card/is_card_number/37828224631 throughput 215509293.292op/s 216702118.694op/s ± 145998.530op/s 216705678.855op/s ± 48131.192op/s 216761557.339op/s 216860089.763op/s 216897876.686op/s 217368505.590op/s 0.31% -3.884 33.521 0.07% 10323.655op/s 1 200
credit_card/is_card_number/378282246310005 execution_time 79.536µs 80.441µs ± 0.405µs 80.434µs ± 0.287µs 80.748µs 81.111µs 81.293µs 81.404µs 1.21% 0.104 -0.650 0.50% 0.029µs 1 200
credit_card/is_card_number/378282246310005 throughput 12284465.370op/s 12431765.476op/s ± 62575.825op/s 12432497.773op/s ± 44457.252op/s 12476063.737op/s 12529897.358op/s 12556422.507op/s 12572941.923op/s 1.13% -0.084 -0.652 0.50% 4424.779op/s 1 200
credit_card/is_card_number/37828224631000521389798 execution_time 59.264µs 59.567µs ± 0.212µs 59.590µs ± 0.161µs 59.698µs 59.936µs 60.078µs 60.131µs 0.91% 0.343 -0.610 0.36% 0.015µs 1 200
credit_card/is_card_number/37828224631000521389798 throughput 16630417.965op/s 16788151.811op/s ± 59744.808op/s 16781422.127op/s ± 45238.557op/s 16841407.551op/s 16870537.489op/s 16872742.358op/s 16873694.172op/s 0.55% -0.330 -0.629 0.35% 4224.596op/s 1 200
credit_card/is_card_number/x371413321323331 execution_time 7.218µs 7.223µs ± 0.003µs 7.222µs ± 0.002µs 7.224µs 7.229µs 7.231µs 7.239µs 0.23% 1.406 2.819 0.04% 0.000µs 1 200
credit_card/is_card_number/x371413321323331 throughput 138146855.194op/s 138447525.507op/s ± 60260.510op/s 138462532.039op/s ± 33367.356op/s 138492559.488op/s 138516237.566op/s 138526508.018op/s 138533247.980op/s 0.05% -1.403 2.799 0.04% 4261.062op/s 1 200
credit_card/is_card_number_no_luhn/ execution_time 4.601µs 4.615µs ± 0.002µs 4.615µs ± 0.002µs 4.616µs 4.618µs 4.620µs 4.622µs 0.17% -0.529 4.923 0.05% 0.000µs 1 200
credit_card/is_card_number_no_luhn/ throughput 216345604.266op/s 216694682.437op/s ± 110265.869op/s 216704583.303op/s ± 71190.669op/s 216766894.283op/s 216842437.292op/s 216881457.366op/s 217337101.981op/s 0.29% 0.539 4.975 0.05% 7796.974op/s 1 200
credit_card/is_card_number_no_luhn/ 3782-8224-6310-005 execution_time 74.056µs 74.427µs ± 0.146µs 74.448µs ± 0.099µs 74.523µs 74.648µs 74.693µs 74.833µs 0.52% -0.234 -0.316 0.20% 0.010µs 1 200
credit_card/is_card_number_no_luhn/ 3782-8224-6310-005 throughput 13363032.921op/s 13435964.450op/s ± 26438.127op/s 13432275.809op/s ± 17945.336op/s 13453521.447op/s 13481130.532op/s 13498018.046op/s 13503327.220op/s 0.53% 0.244 -0.313 0.20% 1869.458op/s 1 200
credit_card/is_card_number_no_luhn/ 378282246310005 execution_time 66.351µs 66.820µs ± 0.272µs 66.791µs ± 0.222µs 67.029µs 67.324µs 67.462µs 67.683µs 1.34% 0.540 -0.464 0.41% 0.019µs 1 200
credit_card/is_card_number_no_luhn/ 378282246310005 throughput 14774732.607op/s 14965841.550op/s ± 60682.319op/s 14972163.006op/s ± 49816.800op/s 15021108.413op/s 15041093.698op/s 15064214.937op/s 15071265.988op/s 0.66% -0.525 -0.494 0.40% 4290.888op/s 1 200
credit_card/is_card_number_no_luhn/37828224631 execution_time 4.600µs 4.615µs ± 0.003µs 4.615µs ± 0.001µs 4.616µs 4.619µs 4.621µs 4.641µs 0.57% 3.224 30.635 0.07% 0.000µs 1 200
credit_card/is_card_number_no_luhn/37828224631 throughput 215472174.906op/s 216688797.187op/s ± 141631.798op/s 216704724.176op/s ± 59372.969op/s 216759917.862op/s 216839129.406op/s 216868731.033op/s 217386702.856op/s 0.31% -3.180 30.308 0.07% 10014.880op/s 1 200
credit_card/is_card_number_no_luhn/378282246310005 execution_time 62.301µs 62.654µs ± 0.158µs 62.636µs ± 0.110µs 62.748µs 62.925µs 63.081µs 63.374µs 1.18% 0.940 1.771 0.25% 0.011µs 1 200
credit_card/is_card_number_no_luhn/378282246310005 throughput 15779414.826op/s 15960841.911op/s ± 40134.562op/s 15965226.939op/s ± 27949.573op/s 15992993.470op/s 16006463.891op/s 16037171.498op/s 16051193.677op/s 0.54% -0.918 1.688 0.25% 2837.942op/s 1 200
credit_card/is_card_number_no_luhn/37828224631000521389798 execution_time 59.259µs 59.566µs ± 0.227µs 59.584µs ± 0.187µs 59.711µs 60.015µs 60.078µs 60.085µs 0.84% 0.481 -0.527 0.38% 0.016µs 1 200
credit_card/is_card_number_no_luhn/37828224631000521389798 throughput 16643051.260op/s 16788299.574op/s ± 63811.566op/s 16783134.838op/s ± 52561.445op/s 16850836.848op/s 16871388.656op/s 16873358.621op/s 16875129.434op/s 0.55% -0.466 -0.549 0.38% 4512.159op/s 1 200
credit_card/is_card_number_no_luhn/x371413321323331 execution_time 7.218µs 7.222µs ± 0.003µs 7.222µs ± 0.002µs 7.224µs 7.228µs 7.229µs 7.232µs 0.14% 0.871 0.302 0.04% 0.000µs 1 200
credit_card/is_card_number_no_luhn/x371413321323331 throughput 138271310.891op/s 138457293.582op/s ± 50655.004op/s 138464067.325op/s ± 33601.231op/s 138497571.539op/s 138518099.839op/s 138527165.722op/s 138532950.072op/s 0.05% -0.869 0.297 0.04% 3581.850op/s 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
credit_card/is_card_number/ execution_time [4.614µs; 4.615µs] or [-0.007%; +0.007%] None None None
credit_card/is_card_number/ throughput [216686807.125op/s; 216715937.911op/s] or [-0.007%; +0.007%] None None None
credit_card/is_card_number/ 3782-8224-6310-005 execution_time [91.445µs; 91.581µs] or [-0.074%; +0.074%] None None None
credit_card/is_card_number/ 3782-8224-6310-005 throughput [10919591.391op/s; 10935761.065op/s] or [-0.074%; +0.074%] None None None
credit_card/is_card_number/ 378282246310005 execution_time [83.494µs; 83.631µs] or [-0.082%; +0.082%] None None None
credit_card/is_card_number/ 378282246310005 throughput [11957699.406op/s; 11977248.706op/s] or [-0.082%; +0.082%] None None None
credit_card/is_card_number/37828224631 execution_time [4.614µs; 4.615µs] or [-0.009%; +0.009%] None None None
credit_card/is_card_number/37828224631 throughput [216681884.702op/s; 216722352.686op/s] or [-0.009%; +0.009%] None None None
credit_card/is_card_number/378282246310005 execution_time [80.385µs; 80.497µs] or [-0.070%; +0.070%] None None None
credit_card/is_card_number/378282246310005 throughput [12423093.069op/s; 12440437.884op/s] or [-0.070%; +0.070%] None None None
credit_card/is_card_number/37828224631000521389798 execution_time [59.537µs; 59.596µs] or [-0.049%; +0.049%] None None None
credit_card/is_card_number/37828224631000521389798 throughput [16779871.755op/s; 16796431.866op/s] or [-0.049%; +0.049%] None None None
credit_card/is_card_number/x371413321323331 execution_time [7.223µs; 7.223µs] or [-0.006%; +0.006%] None None None
credit_card/is_card_number/x371413321323331 throughput [138439173.980op/s; 138455877.034op/s] or [-0.006%; +0.006%] None None None
credit_card/is_card_number_no_luhn/ execution_time [4.614µs; 4.615µs] or [-0.007%; +0.007%] None None None
credit_card/is_card_number_no_luhn/ throughput [216679400.648op/s; 216709964.226op/s] or [-0.007%; +0.007%] None None None
credit_card/is_card_number_no_luhn/ 3782-8224-6310-005 execution_time [74.407µs; 74.448µs] or [-0.027%; +0.027%] None None None
credit_card/is_card_number_no_luhn/ 3782-8224-6310-005 throughput [13432300.380op/s; 13439628.520op/s] or [-0.027%; +0.027%] None None None
credit_card/is_card_number_no_luhn/ 378282246310005 execution_time [66.782µs; 66.858µs] or [-0.056%; +0.056%] None None None
credit_card/is_card_number_no_luhn/ 378282246310005 throughput [14957431.564op/s; 14974251.536op/s] or [-0.056%; +0.056%] None None None
credit_card/is_card_number_no_luhn/37828224631 execution_time [4.614µs; 4.615µs] or [-0.009%; +0.009%] None None None
credit_card/is_card_number_no_luhn/37828224631 throughput [216669168.382op/s; 216708425.992op/s] or [-0.009%; +0.009%] None None None
credit_card/is_card_number_no_luhn/378282246310005 execution_time [62.632µs; 62.676µs] or [-0.035%; +0.035%] None None None
credit_card/is_card_number_no_luhn/378282246310005 throughput [15955279.647op/s; 15966404.175op/s] or [-0.035%; +0.035%] None None None
credit_card/is_card_number_no_luhn/37828224631000521389798 execution_time [59.535µs; 59.598µs] or [-0.053%; +0.053%] None None None
credit_card/is_card_number_no_luhn/37828224631000521389798 throughput [16779455.904op/s; 16797143.243op/s] or [-0.053%; +0.053%] None None None
credit_card/is_card_number_no_luhn/x371413321323331 execution_time [7.222µs; 7.223µs] or [-0.005%; +0.005%] None None None
credit_card/is_card_number_no_luhn/x371413321323331 throughput [138450273.285op/s; 138464313.878op/s] or [-0.005%; +0.005%] None None None

Baseline

Omitted due to size.

@bantonsson
Copy link
Contributor Author

@morrisonlevi no strategy for bumping was chosen. Maybe move nightly when we move the supported minimum? Do you have any other ideas?

@codecov-commenter
Copy link

codecov-commenter commented Oct 22, 2024 •
edited
Loading

Codecov Report

Attention: Patch coverage is 80.00000% with 1 line in your changes missing coverage. Please review.

Project coverage is 71.81%. Comparing base (8d6b9cc) to head (54be47b).

Additional details and impacted files 
@@           Coverage Diff           @@
##             main     #686   +/-   ##
=======================================
  Coverage   71.81%   71.81%           
=======================================
  Files         271      271           
  Lines       40982    41003   +21     
=======================================
+ Hits        29430    29446   +16     
- Misses      11552    11557    +5
Components Coverage Δ
crashtracker 20.33% <ø> (ø)
crashtracker-ffi 9.07% <ø> (ø)
datadog-alloc 98.73% <ø> (ø)
data-pipeline 91.92% <ø> (ø)
data-pipeline-ffi 0.00% <ø> (ø)
ddcommon 82.74% <100.00%> (+0.01%) ⬆️
ddcommon-ffi 69.12% <ø> (ø)
ddtelemetry 59.10% <ø> (ø)
ddtelemetry-ffi 22.13% <ø> (ø)
dogstatsd 88.86% <ø> (ø)
dogstatsd-client 79.77% <ø> (ø)
ipc 82.65% <ø> (ø)
profiling 84.30% <ø> (ø)
profiling-ffi 77.46% <ø> (ø)
serverless 0.00% <ø> (ø)
sidecar 36.09% <100.00%> (ø)
sidecar-ffi 0.00% <ø> (ø)
spawn-worker 50.36% <ø> (ø)
tinybytes 94.77% <ø> (ø)
trace-mini-agent 72.45% <ø> (ø)
trace-normalization 98.25% <ø> (ø)
trace-obfuscation 95.77% <ø> (ø)
trace-protobuf 77.67% <ø> (ø)
trace-utils 93.08% <ø> (-0.42%) ⬇️

@bantonsson bantonsson force-pushed the ban/lock-down-nightly-version-for-reproducible-builds branch from f1948f1 to da3e6e6 Compare October 22, 2024 15:14
@bantonsson
Lock down nightly version for reproducible builds
54be47b 
* Use nightly-2024-10-21
* Fix clippy lints
* Change nextest version to 0.9.81 in all places
@bantonsson bantonsson force-pushed the ban/lock-down-nightly-version-for-reproducible-builds branch from da3e6e6 to 54be47b Compare October 23, 2024 07:28
@bantonsson bantonsson merged commit 6e7ce4c into main Oct 23, 2024
32 checks passed
@bantonsson bantonsson deleted the ban/lock-down-nightly-version-for-reproducible-builds branch October 23, 2024 08:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@morrisonlevi morrisonlevi morrisonlevi left review comments

@hoolioh hoolioh hoolioh approved these changes

@ekump ekump ekump approved these changes

@mellon85 mellon85 mellon85 approved these changes

@paullegranddc paullegranddc Awaiting requested review from paullegranddc paullegranddc is a code owner automatically assigned from DataDog/libdatadog-telemetry

Assignees
No one assigned
Labels
ci-build common data-pipeline profiling Relates to the profiling* modules. sidecar telemetry
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
6 participants
@bantonsson @codecov-commenter @mellon85 @morrisonlevi @ekump @hoolioh