Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

test: ignore dogstatsd tests under miri #661

Merged
merged 3 commits into from
Oct 3, 2024
Merged

test: ignore dogstatsd tests under miri #661

merged 3 commits into from
Oct 3, 2024

Conversation

astuyve
Copy link
Contributor

@astuyve astuyve commented Oct 2, 2024 •
edited by morrisonlevi
Loading

What does this PR do?

Disables miri for dogstatsd.
Also uses collect instead of for_each + push which Levi thinks could avoid memcpys.

Motivation

dogstatsd tests were really slow under miri in CI, like 20,000 times slower (higher than normal for miri).

Additional Notes

Anything else we should know when reviewing?

How to test the change?

Describe here in detail how the change can be validated.

@astuyve astuyve requested a review from a team as a code owner October 2, 2024 19:40
@astuyve astuyve mentioned this pull request Oct 2, 2024
Merged
@morrisonlevi morrisonlevi changed the title feat: Map over sketches attr instead of assigning in map, which Levi … test: ignore dogstatsd tests under miri Oct 2, 2024
@codecov-commenter
Copy link

codecov-commenter commented Oct 2, 2024 •
edited
Loading

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 71.97%. Comparing base (abc6cd6) to head (f68d2aa).

Additional details and impacted files 
@@            Coverage Diff             @@
##             main     #661      +/-   ##
==========================================
+ Coverage   71.92%   71.97%   +0.05%     
==========================================
  Files         271      271              
  Lines       40628    40629       +1     
==========================================
+ Hits        29221    29244      +23     
+ Misses      11407    11385      -22
Components Coverage Δ
crashtracker 20.41% <ø> (+0.05%) ⬆️
crashtracker-ffi 9.07% <ø> (ø)
datadog-alloc 98.73% <ø> (ø)
data-pipeline 93.33% <ø> (ø)
data-pipeline-ffi 0.00% <ø> (ø)
ddcommon 82.73% <ø> (ø)
ddcommon-ffi 69.12% <ø> (ø)
ddtelemetry 59.10% <ø> (ø)
ddtelemetry-ffi 22.13% <ø> (ø)
dogstatsd 90.90% <100.00%> (+<0.01%) ⬆️
dogstatsd-client 79.77% <ø> (ø)
ipc 82.69% <ø> (ø)
profiling 84.26% <ø> (ø)
profiling-ffi 77.42% <ø> (ø)
serverless 0.00% <ø> (ø)
sidecar 36.35% <ø> (ø)
sidecar-ffi 0.00% <ø> (ø)
spawn-worker 50.36% <ø> (ø)
tinybytes 94.77% <ø> (ø)
trace-mini-agent 72.66% <ø> (ø)
trace-normalization 98.25% <ø> (ø)
trace-obfuscation 95.73% <ø> (ø)
trace-protobuf 77.67% <ø> (ø)
trace-utils 93.36% <ø> (ø)

@pr-commenter
Copy link

pr-commenter bot commented Oct 2, 2024 •
edited
Loading

Benchmarks

Comparison

Benchmark execution time: 2024-10-02 20:28:09

Comparing candidate commit f68d2aa in PR branch aj/dsd-disable-miri with baseline commit abc6cd6 in branch main.

Found 1 performance improvements and 1 performance regressions! Performance is the same for 49 metrics, 2 unstable metrics.

scenario:benching deserializing traces from msgpack to their internal representation

  • 🟥 execution_time [+25.709ns; +35.613ns] or [+2.156%; +2.986%]

scenario:tags/replace_trace_tags

  • 🟩 execution_time [-67.574ns; -61.207ns] or [-2.450%; -2.219%]

Candidate

Candidate benchmark details

Group 1

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz f68d2aa 1727900209 aj/dsd-disable-miri
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
benching deserializing traces from msgpack to their internal representation execution_time 1.153µs 1.223µs ± 0.026µs 1.231µs ± 0.012µs 1.243µs 1.246µs 1.248µs 1.250µs 1.50% -1.290 0.604 2.12% 0.002µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
benching deserializing traces from msgpack to their internal representation execution_time [1.220µs; 1.227µs] or [-0.295%; +0.295%] None None None

Group 2

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz f68d2aa 1727900209 aj/dsd-disable-miri
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
normalization/normalize_trace/test_trace execution_time 263.122ns 276.008ns ± 14.538ns 267.580ns ± 2.792ns 282.394ns 306.848ns 313.527ns 318.384ns 18.99% 1.201 0.188 5.25% 1.028ns 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
normalization/normalize_trace/test_trace execution_time [273.993ns; 278.023ns] or [-0.730%; +0.730%] None None None

Group 3

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz f68d2aa 1727900209 aj/dsd-disable-miri
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
normalization/normalize_name/normalize_name/Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Lo... execution_time 178.718µs 180.049µs ± 0.897µs 179.853µs ± 0.336µs 180.221µs 181.360µs 184.240µs 185.616µs 3.20% 2.848 12.233 0.50% 0.063µs 1 200
normalization/normalize_name/normalize_name/Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Lo... throughput 5387476.756op/s 5554179.489op/s ± 27286.613op/s 5560104.805op/s ± 10384.549op/s 5569185.751op/s 5581884.686op/s 5590497.226op/s 5595418.248op/s 0.64% -2.759 11.575 0.49% 1929.455op/s 1 200
normalization/normalize_name/normalize_name/bad-name execution_time 20.050µs 20.159µs ± 0.057µs 20.151µs ± 0.033µs 20.188µs 20.256µs 20.341µs 20.398µs 1.23% 1.035 2.032 0.28% 0.004µs 1 200
normalization/normalize_name/normalize_name/bad-name throughput 49024709.314op/s 49607146.557op/s ± 140407.544op/s 49625523.258op/s ± 81652.805op/s 49699649.625op/s 49810643.125op/s 49857405.419op/s 49876162.738op/s 0.51% -1.010 1.951 0.28% 9928.313op/s 1 200
normalization/normalize_name/normalize_name/good execution_time 13.114µs 13.148µs ± 0.025µs 13.135µs ± 0.010µs 13.168µs 13.205µs 13.219µs 13.224µs 0.68% 1.135 0.558 0.19% 0.002µs 1 200
normalization/normalize_name/normalize_name/good throughput 75619024.039op/s 76058800.554op/s ± 143733.104op/s 76129876.341op/s ± 55403.803op/s 76166772.075op/s 76197466.225op/s 76222465.741op/s 76254337.734op/s 0.16% -1.128 0.534 0.19% 10163.465op/s 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
normalization/normalize_name/normalize_name/Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Lo... execution_time [179.925µs; 180.173µs] or [-0.069%; +0.069%] None None None
normalization/normalize_name/normalize_name/Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Lo... throughput [5550397.827op/s; 5557961.151op/s] or [-0.068%; +0.068%] None None None
normalization/normalize_name/normalize_name/bad-name execution_time [20.151µs; 20.166µs] or [-0.039%; +0.039%] None None None
normalization/normalize_name/normalize_name/bad-name throughput [49587687.422op/s; 49626605.692op/s] or [-0.039%; +0.039%] None None None
normalization/normalize_name/normalize_name/good execution_time [13.144µs; 13.151µs] or [-0.026%; +0.026%] None None None
normalization/normalize_name/normalize_name/good throughput [76038880.528op/s; 76078720.579op/s] or [-0.026%; +0.026%] None None None

Group 4

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz f68d2aa 1727900209 aj/dsd-disable-miri
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
sql/obfuscate_sql_string execution_time 72.829µs 73.054µs ± 0.151µs 73.012µs ± 0.060µs 73.104µs 73.283µs 73.357µs 74.460µs 1.98% 4.468 36.300 0.21% 0.011µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
sql/obfuscate_sql_string execution_time [73.034µs; 73.075µs] or [-0.029%; +0.029%] None None None

Group 5

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz f68d2aa 1727900209 aj/dsd-disable-miri
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
two way interface execution_time 17.402µs 22.907µs ± 11.006µs 17.812µs ± 0.132µs 18.144µs 42.893µs 47.630µs 101.248µs 468.43% 3.188 15.149 47.93% 0.778µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
two way interface execution_time [21.382µs; 24.432µs] or [-6.659%; +6.659%] None None None

Group 6

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz f68d2aa 1727900209 aj/dsd-disable-miri
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
normalization/normalize_service/normalize_service/A0000000000000000000000000000000000000000000000000... execution_time 504.525µs 505.605µs ± 0.401µs 505.610µs ± 0.245µs 505.846µs 506.247µs 506.473µs 507.106µs 0.30% 0.153 0.550 0.08% 0.028µs 1 200
normalization/normalize_service/normalize_service/A0000000000000000000000000000000000000000000000000... throughput 1971973.158op/s 1977829.624op/s ± 1567.264op/s 1977810.364op/s ± 958.344op/s 1978769.666op/s 1980372.805op/s 1981696.536op/s 1982063.029op/s 0.22% -0.147 0.543 0.08% 110.822op/s 1 200
normalization/normalize_service/normalize_service/Data🐨dog🐶 繋がっ⛰てて execution_time 468.135µs 468.854µs ± 0.305µs 468.821µs ± 0.215µs 469.069µs 469.295µs 469.669µs 470.104µs 0.27% 0.527 0.962 0.06% 0.022µs 1 200
normalization/normalize_service/normalize_service/Data🐨dog🐶 繋がっ⛰てて throughput 2127188.639op/s 2132860.283op/s ± 1385.617op/s 2133010.373op/s ± 979.121op/s 2133786.864op/s 2134904.815op/s 2135766.843op/s 2136136.664op/s 0.15% -0.522 0.948 0.06% 97.978op/s 1 200
normalization/normalize_service/normalize_service/Test Conversion 0f Weird !@#$%^&**() Characters execution_time 212.899µs 213.180µs ± 0.097µs 213.175µs ± 0.053µs 213.236µs 213.351µs 213.424µs 213.460µs 0.13% 0.320 0.084 0.05% 0.007µs 1 200
normalization/normalize_service/normalize_service/Test Conversion 0f Weird !@#$%^&**() Characters throughput 4684711.743op/s 4690868.588op/s ± 2134.699op/s 4690981.894op/s ± 1167.824op/s 4691900.795op/s 4693934.541op/s 4694496.879op/s 4697073.958op/s 0.13% -0.317 0.082 0.05% 150.946op/s 1 200
normalization/normalize_service/normalize_service/[empty string] execution_time 45.804µs 45.983µs ± 0.071µs 45.974µs ± 0.051µs 46.036µs 46.094µs 46.149µs 46.299µs 0.71% 0.592 1.071 0.15% 0.005µs 1 200
normalization/normalize_service/normalize_service/[empty string] throughput 21598851.125op/s 21747009.950op/s ± 33578.183op/s 21751226.746op/s ± 24098.539op/s 21770177.944op/s 21799498.004op/s 21802767.896op/s 21832063.125op/s 0.37% -0.579 1.030 0.15% 2374.336op/s 1 200
normalization/normalize_service/normalize_service/test_ASCII execution_time 54.502µs 54.585µs ± 0.060µs 54.577µs ± 0.055µs 54.627µs 54.685µs 54.719µs 54.826µs 0.46% 0.750 0.829 0.11% 0.004µs 1 200
normalization/normalize_service/normalize_service/test_ASCII throughput 18239361.069op/s 18319992.618op/s ± 20206.641op/s 18322694.214op/s ± 18542.057op/s 18341500.735op/s 18344212.078op/s 18346273.656op/s 18348096.551op/s 0.14% -0.743 0.801 0.11% 1428.825op/s 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
normalization/normalize_service/normalize_service/A0000000000000000000000000000000000000000000000000... execution_time [505.550µs; 505.661µs] or [-0.011%; +0.011%] None None None
normalization/normalize_service/normalize_service/A0000000000000000000000000000000000000000000000000... throughput [1977612.416op/s; 1978046.832op/s] or [-0.011%; +0.011%] None None None
normalization/normalize_service/normalize_service/Data🐨dog🐶 繋がっ⛰てて execution_time [468.812µs; 468.896µs] or [-0.009%; +0.009%] None None None
normalization/normalize_service/normalize_service/Data🐨dog🐶 繋がっ⛰てて throughput [2132668.249op/s; 2133052.316op/s] or [-0.009%; +0.009%] None None None
normalization/normalize_service/normalize_service/Test Conversion 0f Weird !@#$%^&**() Characters execution_time [213.167µs; 213.194µs] or [-0.006%; +0.006%] None None None
normalization/normalize_service/normalize_service/Test Conversion 0f Weird !@#$%^&**() Characters throughput [4690572.739op/s; 4691164.437op/s] or [-0.006%; +0.006%] None None None
normalization/normalize_service/normalize_service/[empty string] execution_time [45.974µs; 45.993µs] or [-0.021%; +0.021%] None None None
normalization/normalize_service/normalize_service/[empty string] throughput [21742356.337op/s; 21751663.563op/s] or [-0.021%; +0.021%] None None None
normalization/normalize_service/normalize_service/test_ASCII execution_time [54.577µs; 54.594µs] or [-0.015%; +0.015%] None None None
normalization/normalize_service/normalize_service/test_ASCII throughput [18317192.171op/s; 18322793.064op/s] or [-0.015%; +0.015%] None None None

Group 7

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz f68d2aa 1727900209 aj/dsd-disable-miri
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
credit_card/is_card_number/ execution_time 1.214µs 1.215µs ± 0.000µs 1.215µs ± 0.000µs 1.215µs 1.216µs 1.216µs 1.216µs 0.11% 0.831 0.020 0.04% 0.000µs 1 200
credit_card/is_card_number/ throughput 822117374.786op/s 822993225.344op/s ± 300021.984op/s 823022591.971op/s ± 207830.224op/s 823234395.413op/s 823353173.227op/s 823393020.311op/s 823423981.008op/s 0.05% -0.829 0.017 0.04% 21214.758op/s 1 200
credit_card/is_card_number/ 3782-8224-6310-005 execution_time 116.907µs 117.498µs ± 0.525µs 117.348µs ± 0.323µs 117.804µs 118.528µs 119.174µs 119.317µs 1.68% 1.194 0.985 0.45% 0.037µs 1 200
credit_card/is_card_number/ 3782-8224-6310-005 throughput 8381041.927op/s 8510934.401op/s ± 37834.701op/s 8521659.110op/s ± 23415.980op/s 8541385.383op/s 8549238.597op/s 8552675.558op/s 8553809.429op/s 0.38% -1.174 0.913 0.44% 2675.317op/s 1 200
credit_card/is_card_number/ 378282246310005 execution_time 107.859µs 108.398µs ± 0.428µs 108.201µs ± 0.080µs 108.581µs 109.342µs 109.849µs 109.951µs 1.62% 1.693 2.379 0.39% 0.030µs 1 200
credit_card/is_card_number/ 378282246310005 throughput 9094953.367op/s 9225441.489op/s ± 36150.405op/s 9242061.982op/s ± 6797.835op/s 9247231.368op/s 9258028.821op/s 9261742.435op/s 9271394.965op/s 0.32% -1.676 2.301 0.39% 2556.220op/s 1 200
credit_card/is_card_number/37828224631 execution_time 1.214µs 1.215µs ± 0.001µs 1.215µs ± 0.000µs 1.215µs 1.216µs 1.216µs 1.221µs 0.48% 5.930 55.155 0.05% 0.000µs 1 200
credit_card/is_card_number/37828224631 throughput 819050363.240op/s 822991800.173op/s ± 386205.123op/s 822978701.028op/s ± 188781.804op/s 823245134.415op/s 823338436.125op/s 823364767.695op/s 823476688.167op/s 0.06% -5.899 54.729 0.05% 27308.826op/s 1 200
credit_card/is_card_number/378282246310005 execution_time 103.965µs 104.376µs ± 0.342µs 104.209µs ± 0.089µs 104.532µs 105.064µs 105.440µs 106.022µs 1.74% 1.599 2.794 0.33% 0.024µs 1 200
credit_card/is_card_number/378282246310005 throughput 9432038.966op/s 9580890.327op/s ± 31246.407op/s 9596063.216op/s ± 8226.540op/s 9600962.977op/s 9611752.532op/s 9618507.374op/s 9618647.672op/s 0.24% -1.578 2.672 0.33% 2209.455op/s 1 200
credit_card/is_card_number/37828224631000521389798 execution_time 102.900µs 104.367µs ± 0.578µs 104.352µs ± 0.418µs 104.792µs 105.411µs 105.624µs 105.692µs 1.28% 0.148 -0.485 0.55% 0.041µs 1 200
credit_card/is_card_number/37828224631000521389798 throughput 9461453.104op/s 9581832.490op/s ± 53013.338op/s 9582965.325op/s ± 38527.166op/s 9619331.972op/s 9664718.142op/s 9692584.510op/s 9718159.211op/s 1.41% -0.123 -0.488 0.55% 3748.609op/s 1 200
credit_card/is_card_number/x371413321323331 execution_time 23.134µs 23.861µs ± 0.315µs 23.840µs ± 0.205µs 24.071µs 24.389µs 24.594µs 24.992µs 4.83% 0.294 0.225 1.32% 0.022µs 1 200
credit_card/is_card_number/x371413321323331 throughput 40012654.362op/s 41916417.578op/s ± 550823.483op/s 41946598.391op/s ± 360822.101op/s 42287542.139op/s 42839743.906op/s 43154921.881op/s 43225514.871op/s 3.05% -0.211 0.132 1.31% 38949.102op/s 1 200
credit_card/is_card_number_no_luhn/ execution_time 1.214µs 1.215µs ± 0.000µs 1.215µs ± 0.000µs 1.215µs 1.216µs 1.216µs 1.217µs 0.16% 1.082 1.421 0.04% 0.000µs 1 200
credit_card/is_card_number_no_luhn/ throughput 821746501.338op/s 823049745.119op/s ± 299315.802op/s 823066418.904op/s ± 208143.853op/s 823289644.770op/s 823394828.876op/s 823461745.922op/s 823478305.934op/s 0.05% -1.080 1.411 0.04% 21164.823op/s 1 200
credit_card/is_card_number_no_luhn/ 3782-8224-6310-005 execution_time 95.591µs 96.097µs ± 0.358µs 96.033µs ± 0.282µs 96.342µs 96.788µs 96.932µs 97.113µs 1.12% 0.650 -0.482 0.37% 0.025µs 1 200
credit_card/is_card_number_no_luhn/ 3782-8224-6310-005 throughput 10297334.545op/s 10406259.978op/s ± 38667.661op/s 10413051.209op/s ± 30643.121op/s 10442472.932op/s 10449170.138op/s 10458328.076op/s 10461190.642op/s 0.46% -0.638 -0.508 0.37% 2734.217op/s 1 200
credit_card/is_card_number_no_luhn/ 378282246310005 execution_time 86.500µs 87.501µs ± 0.522µs 87.410µs ± 0.348µs 87.856µs 88.515µs 88.723µs 88.826µs 1.62% 0.281 -0.431 0.60% 0.037µs 1 200
credit_card/is_card_number_no_luhn/ 378282246310005 throughput 11257961.704op/s 11428827.805op/s ± 68099.987op/s 11440357.992op/s ± 45547.358op/s 11473734.328op/s 11537253.909op/s 11559396.154op/s 11560694.804op/s 1.05% -0.255 -0.451 0.59% 4815.396op/s 1 200
credit_card/is_card_number_no_luhn/37828224631 execution_time 1.214µs 1.215µs ± 0.000µs 1.215µs ± 0.000µs 1.215µs 1.216µs 1.216µs 1.219µs 0.32% 2.716 18.562 0.04% 0.000µs 1 200
credit_card/is_card_number_no_luhn/37828224631 throughput 820314127.484op/s 822959606.468op/s ± 334172.995op/s 822926800.661op/s ± 162646.005op/s 823231204.486op/s 823357847.804op/s 823404554.302op/s 823465669.836op/s 0.07% -2.701 18.400 0.04% 23629.599op/s 1 200
credit_card/is_card_number_no_luhn/378282246310005 execution_time 82.394µs 83.530µs ± 0.546µs 83.492µs ± 0.385µs 83.920µs 84.359µs 84.955µs 85.018µs 1.83% 0.173 -0.245 0.65% 0.039µs 1 200
credit_card/is_card_number_no_luhn/378282246310005 throughput 11762225.437op/s 11972219.035op/s ± 78219.210op/s 11977239.075op/s ± 55217.516op/s 12023000.000op/s 12100967.467op/s 12120713.782op/s 12136758.205op/s 1.33% -0.140 -0.276 0.65% 5530.933op/s 1 200
credit_card/is_card_number_no_luhn/37828224631000521389798 execution_time 102.923µs 104.271µs ± 0.606µs 104.254µs ± 0.452µs 104.718µs 105.266µs 105.611µs 105.881µs 1.56% 0.056 -0.463 0.58% 0.043µs 1 200
credit_card/is_card_number_no_luhn/37828224631000521389798 throughput 9444561.408op/s 9590718.479op/s ± 55687.862op/s 9591941.828op/s ± 41789.238op/s 9631422.917op/s 9684217.829op/s 9708587.134op/s 9715971.782op/s 1.29% -0.030 -0.469 0.58% 3937.726op/s 1 200
credit_card/is_card_number_no_luhn/x371413321323331 execution_time 23.005µs 23.815µs ± 0.368µs 23.802µs ± 0.220µs 24.021µs 24.463µs 24.772µs 24.915µs 4.68% 0.247 0.097 1.54% 0.026µs 1 200
credit_card/is_card_number_no_luhn/x371413321323331 throughput 40137200.088op/s 42000610.841op/s ± 647605.353op/s 42013718.008op/s ± 390981.514op/s 42407582.144op/s 43079144.105op/s 43360236.713op/s 43469656.464op/s 3.47% -0.154 0.017 1.54% 45792.614op/s 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
credit_card/is_card_number/ execution_time [1.215µs; 1.215µs] or [-0.005%; +0.005%] None None None
credit_card/is_card_number/ throughput [822951645.183op/s; 823034805.506op/s] or [-0.005%; +0.005%] None None None
credit_card/is_card_number/ 3782-8224-6310-005 execution_time [117.425µs; 117.571µs] or [-0.062%; +0.062%] None None None
credit_card/is_card_number/ 3782-8224-6310-005 throughput [8505690.875op/s; 8516177.927op/s] or [-0.062%; +0.062%] None None None
credit_card/is_card_number/ 378282246310005 execution_time [108.338µs; 108.457µs] or [-0.055%; +0.055%] None None None
credit_card/is_card_number/ 378282246310005 throughput [9220431.391op/s; 9230451.588op/s] or [-0.054%; +0.054%] None None None
credit_card/is_card_number/37828224631 execution_time [1.215µs; 1.215µs] or [-0.007%; +0.007%] None None None
credit_card/is_card_number/37828224631 throughput [822938275.858op/s; 823045324.489op/s] or [-0.007%; +0.007%] None None None
credit_card/is_card_number/378282246310005 execution_time [104.328µs; 104.423µs] or [-0.045%; +0.045%] None None None
credit_card/is_card_number/378282246310005 throughput [9576559.876op/s; 9585220.779op/s] or [-0.045%; +0.045%] None None None
credit_card/is_card_number/37828224631000521389798 execution_time [104.287µs; 104.447µs] or [-0.077%; +0.077%] None None None
credit_card/is_card_number/37828224631000521389798 throughput [9574485.352op/s; 9589179.629op/s] or [-0.077%; +0.077%] None None None
credit_card/is_card_number/x371413321323331 execution_time [23.818µs; 23.905µs] or [-0.183%; +0.183%] None None None
credit_card/is_card_number/x371413321323331 throughput [41840078.740op/s; 41992756.415op/s] or [-0.182%; +0.182%] None None None
credit_card/is_card_number_no_luhn/ execution_time [1.215µs; 1.215µs] or [-0.005%; +0.005%] None None None
credit_card/is_card_number_no_luhn/ throughput [823008262.828op/s; 823091227.411op/s] or [-0.005%; +0.005%] None None None
credit_card/is_card_number_no_luhn/ 3782-8224-6310-005 execution_time [96.048µs; 96.147µs] or [-0.052%; +0.052%] None None None
credit_card/is_card_number_no_luhn/ 3782-8224-6310-005 throughput [10400901.012op/s; 10411618.944op/s] or [-0.051%; +0.051%] None None None
credit_card/is_card_number_no_luhn/ 378282246310005 execution_time [87.429µs; 87.574µs] or [-0.083%; +0.083%] None None None
credit_card/is_card_number_no_luhn/ 378282246310005 throughput [11419389.802op/s; 11438265.808op/s] or [-0.083%; +0.083%] None None None
credit_card/is_card_number_no_luhn/37828224631 execution_time [1.215µs; 1.215µs] or [-0.006%; +0.006%] None None None
credit_card/is_card_number_no_luhn/37828224631 throughput [822913293.305op/s; 823005919.631op/s] or [-0.006%; +0.006%] None None None
credit_card/is_card_number_no_luhn/378282246310005 execution_time [83.455µs; 83.606µs] or [-0.091%; +0.091%] None None None
credit_card/is_card_number_no_luhn/378282246310005 throughput [11961378.605op/s; 11983059.465op/s] or [-0.091%; +0.091%] None None None
credit_card/is_card_number_no_luhn/37828224631000521389798 execution_time [104.187µs; 104.355µs] or [-0.080%; +0.080%] None None None
credit_card/is_card_number_no_luhn/37828224631000521389798 throughput [9583000.677op/s; 9598436.281op/s] or [-0.080%; +0.080%] None None None
credit_card/is_card_number_no_luhn/x371413321323331 execution_time [23.764µs; 23.866µs] or [-0.214%; +0.214%] None None None
credit_card/is_card_number_no_luhn/x371413321323331 throughput [41910858.967op/s; 42090362.714op/s] or [-0.214%; +0.214%] None None None

Group 8

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz f68d2aa 1727900209 aj/dsd-disable-miri
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
write only interface execution_time 1.468µs 3.161µs ± 1.552µs 3.004µs ± 0.019µs 3.021µs 3.073µs 13.783µs 17.941µs 497.24% 8.030 65.162 48.97% 0.110µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
write only interface execution_time [2.946µs; 3.376µs] or [-6.804%; +6.804%] None None None

Group 9

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz f68d2aa 1727900209 aj/dsd-disable-miri
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
benching string interning on wordpress profile execution_time 146.480µs 147.243µs ± 0.718µs 147.158µs ± 0.146µs 147.305µs 147.755µs 148.977µs 155.805µs 5.88% 8.901 100.251 0.49% 0.051µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
benching string interning on wordpress profile execution_time [147.144µs; 147.343µs] or [-0.068%; +0.068%] None None None

Group 10

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz f68d2aa 1727900209 aj/dsd-disable-miri
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
concentrator/add_spans_to_concentrator execution_time 9.081ms 9.109ms ± 0.024ms 9.106ms ± 0.007ms 9.114ms 9.128ms 9.148ms 9.408ms 3.32% 9.470 113.697 0.27% 0.002ms 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
concentrator/add_spans_to_concentrator execution_time [9.106ms; 9.112ms] or [-0.037%; +0.037%] None None None

Group 11

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz f68d2aa 1727900209 aj/dsd-disable-miri
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
tags/replace_trace_tags execution_time 2.651µs 2.694µs ± 0.014µs 2.694µs ± 0.007µs 2.702µs 2.720µs 2.724µs 2.732µs 1.39% -0.735 2.115 0.53% 0.001µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
tags/replace_trace_tags execution_time [2.692µs; 2.696µs] or [-0.073%; +0.073%] None None None

Group 12

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz f68d2aa 1727900209 aj/dsd-disable-miri
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
redis/obfuscate_redis_string execution_time 39.613µs 40.247µs ± 1.128µs 39.731µs ± 0.043µs 39.790µs 42.693µs 42.737µs 43.110µs 8.51% 1.702 0.925 2.80% 0.080µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
redis/obfuscate_redis_string execution_time [40.090µs; 40.403µs] or [-0.389%; +0.389%] None None None

Baseline

Omitted due to size.

@astuyve astuyve merged commit 488ba28 into main Oct 3, 2024
30 checks passed
@astuyve astuyve deleted the aj/dsd-disable-miri branch October 3, 2024 00:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@morrisonlevi morrisonlevi morrisonlevi approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@astuyve @codecov-commenter @morrisonlevi