Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[crashtracker] Implement RFC 0005 #738

Merged
merged 5 commits into from
Nov 14, 2024
Merged

Conversation

danielsn
Copy link
Contributor

@danielsn danielsn commented Nov 13, 2024

What does this PR do?

  • Adds a new datatype which implements RFC 0005.
  • Adds a translation from the old to the new data-format.
  • the file:// output type now outputs RFC0005 format as well as the old format.

Motivation

This is step 1 in implementing the RFC.

Additional Notes

This is step 1 of a multi-step program to move crashtracker over to RFC5.
For now, both dataformats will co-exist.
Future PRs will switch various parts of the API over to use the new format.
Once this is done, we can retire the old format.

How to test the change?

Describe here in detail how the change can be validated.

@danielsn danielsn requested review from a team as code owners November 13, 2024 21:00
@danielsn danielsn force-pushed the dsn/crashtracker-implement-rfc5 branch from 2247727 to fd84622 Compare November 13, 2024 21:05
@pr-commenter
Copy link

pr-commenter bot commented Nov 13, 2024

Benchmarks

Comparison

Benchmark execution time: 2024-11-14 18:48:42

Comparing candidate commit 47c063d in PR branch dsn/crashtracker-implement-rfc5 with baseline commit b57f46a in branch main.

Found 0 performance improvements and 0 performance regressions! Performance is the same for 51 metrics, 2 unstable metrics.

Candidate

Candidate benchmark details

Group 1

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 47c063d 1731609456 dsn/crashtracker-implement-rfc5
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
normalization/normalize_service/normalize_service/A0000000000000000000000000000000000000000000000000... execution_time 506.687µs 507.700µs ± 0.370µs 507.653µs ± 0.207µs 507.888µs 508.410µs 508.737µs 508.994µs 0.26% 0.675 1.136 0.07% 0.026µs 1 200
normalization/normalize_service/normalize_service/A0000000000000000000000000000000000000000000000000... throughput 1964659.474op/s 1969668.021op/s ± 1435.853op/s 1969847.658op/s ± 802.293op/s 1970519.178op/s 1971645.042op/s 1972426.605op/s 1973606.611op/s 0.19% -0.669 1.127 0.07% 101.530op/s 1 200
normalization/normalize_service/normalize_service/Data🐨dog🐶 繋がっ⛰てて execution_time 467.990µs 468.875µs ± 0.308µs 468.878µs ± 0.217µs 469.118µs 469.368µs 469.486µs 469.530µs 0.14% -0.201 -0.266 0.07% 0.022µs 1 200
normalization/normalize_service/normalize_service/Data🐨dog🐶 繋がっ⛰てて throughput 2129788.409op/s 2132763.259op/s ± 1401.291op/s 2132750.591op/s ± 985.592op/s 2133650.162op/s 2135157.636op/s 2135997.322op/s 2136796.593op/s 0.19% 0.204 -0.263 0.07% 99.086op/s 1 200
normalization/normalize_service/normalize_service/Test Conversion 0f Weird !@#$%^&**() Characters execution_time 179.826µs 180.291µs ± 0.191µs 180.293µs ± 0.129µs 180.402µs 180.611µs 180.716µs 180.769µs 0.26% 0.060 -0.302 0.11% 0.013µs 1 200
normalization/normalize_service/normalize_service/Test Conversion 0f Weird !@#$%^&**() Characters throughput 5531932.405op/s 5546579.905op/s ± 5869.103op/s 5546513.161op/s ± 3965.737op/s 5550517.033op/s 5556235.668op/s 5560196.089op/s 5560922.071op/s 0.26% -0.054 -0.303 0.11% 415.008op/s 1 200
normalization/normalize_service/normalize_service/[empty string] execution_time 44.852µs 45.043µs ± 0.065µs 45.043µs ± 0.045µs 45.087µs 45.149µs 45.179µs 45.190µs 0.33% -0.196 -0.096 0.14% 0.005µs 1 200
normalization/normalize_service/normalize_service/[empty string] throughput 22128862.487op/s 22200835.480op/s ± 32167.455op/s 22201069.105op/s ± 22094.461op/s 22223264.211op/s 22250921.177op/s 22286782.659op/s 22295567.436op/s 0.43% 0.204 -0.087 0.14% 2274.583op/s 1 200
normalization/normalize_service/normalize_service/test_ASCII execution_time 48.991µs 49.170µs ± 0.082µs 49.160µs ± 0.048µs 49.219µs 49.318µs 49.382µs 49.419µs 0.53% 0.444 0.103 0.17% 0.006µs 1 200
normalization/normalize_service/normalize_service/test_ASCII throughput 20235238.215op/s 20337670.172op/s ± 33809.908op/s 20341833.747op/s ± 19888.738op/s 20359658.152op/s 20391247.833op/s 20407398.223op/s 20412083.954op/s 0.35% -0.435 0.093 0.17% 2390.722op/s 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
normalization/normalize_service/normalize_service/A0000000000000000000000000000000000000000000000000... execution_time [507.649µs; 507.751µs] or [-0.010%; +0.010%] None None None
normalization/normalize_service/normalize_service/A0000000000000000000000000000000000000000000000000... throughput [1969469.026op/s; 1969867.016op/s] or [-0.010%; +0.010%] None None None
normalization/normalize_service/normalize_service/Data🐨dog🐶 繋がっ⛰てて execution_time [468.833µs; 468.918µs] or [-0.009%; +0.009%] None None None
normalization/normalize_service/normalize_service/Data🐨dog🐶 繋がっ⛰てて throughput [2132569.054op/s; 2132957.465op/s] or [-0.009%; +0.009%] None None None
normalization/normalize_service/normalize_service/Test Conversion 0f Weird !@#$%^&**() Characters execution_time [180.265µs; 180.318µs] or [-0.015%; +0.015%] None None None
normalization/normalize_service/normalize_service/Test Conversion 0f Weird !@#$%^&**() Characters throughput [5545766.503op/s; 5547393.306op/s] or [-0.015%; +0.015%] None None None
normalization/normalize_service/normalize_service/[empty string] execution_time [45.034µs; 45.052µs] or [-0.020%; +0.020%] None None None
normalization/normalize_service/normalize_service/[empty string] throughput [22196377.381op/s; 22205293.580op/s] or [-0.020%; +0.020%] None None None
normalization/normalize_service/normalize_service/test_ASCII execution_time [49.159µs; 49.181µs] or [-0.023%; +0.023%] None None None
normalization/normalize_service/normalize_service/test_ASCII throughput [20332984.444op/s; 20342355.900op/s] or [-0.023%; +0.023%] None None None

Group 2

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 47c063d 1731609456 dsn/crashtracker-implement-rfc5
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
write only interface execution_time 1.388µs 3.271µs ± 1.239µs 3.142µs ± 0.026µs 3.162µs 3.203µs 8.558µs 15.166µs 382.65% 8.271 71.793 37.80% 0.088µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
write only interface execution_time [3.099µs; 3.443µs] or [-5.252%; +5.252%] None None None

Group 3

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 47c063d 1731609456 dsn/crashtracker-implement-rfc5
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
benching deserializing traces from msgpack to their internal representation execution_time 59.716ms 60.462ms ± 0.199ms 60.488ms ± 0.078ms 60.551ms 60.679ms 61.149ms 61.244ms 1.25% -0.091 4.046 0.33% 0.014ms 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
benching deserializing traces from msgpack to their internal representation execution_time [60.434ms; 60.489ms] or [-0.046%; +0.046%] None None None

Group 4

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 47c063d 1731609456 dsn/crashtracker-implement-rfc5
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
two way interface execution_time 18.205µs 24.602µs ± 10.452µs 20.467µs ± 0.884µs 22.560µs 38.197µs 40.211µs 100.971µs 393.34% 4.017 24.194 42.38% 0.739µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
two way interface execution_time [23.154µs; 26.051µs] or [-5.888%; +5.888%] None None None

Group 5

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 47c063d 1731609456 dsn/crashtracker-implement-rfc5
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
sql/obfuscate_sql_string execution_time 70.429µs 70.589µs ± 0.140µs 70.567µs ± 0.040µs 70.609µs 70.778µs 70.988µs 72.015µs 2.05% 6.185 55.243 0.20% 0.010µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
sql/obfuscate_sql_string execution_time [70.569µs; 70.608µs] or [-0.027%; +0.027%] None None None

Group 6

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 47c063d 1731609456 dsn/crashtracker-implement-rfc5
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
normalization/normalize_trace/test_trace execution_time 294.700ns 307.698ns ± 14.998ns 301.277ns ± 5.854ns 314.140ns 338.460ns 348.542ns 349.798ns 16.11% 1.273 0.520 4.86% 1.061ns 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
normalization/normalize_trace/test_trace execution_time [305.620ns; 309.777ns] or [-0.676%; +0.676%] None None None

Group 7

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 47c063d 1731609456 dsn/crashtracker-implement-rfc5
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
concentrator/add_spans_to_concentrator execution_time 9.007ms 9.039ms ± 0.016ms 9.038ms ± 0.010ms 9.049ms 9.062ms 9.074ms 9.122ms 0.93% 0.851 3.076 0.17% 0.001ms 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
concentrator/add_spans_to_concentrator execution_time [9.037ms; 9.041ms] or [-0.024%; +0.024%] None None None

Group 8

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 47c063d 1731609456 dsn/crashtracker-implement-rfc5
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
credit_card/is_card_number/ execution_time 4.593µs 4.612µs ± 0.008µs 4.612µs ± 0.006µs 4.618µs 4.625µs 4.630µs 4.632µs 0.43% 0.148 -0.741 0.18% 0.001µs 1 200
credit_card/is_card_number/ throughput 215883405.046op/s 216824562.940op/s ± 383064.172op/s 216817338.433op/s ± 282394.628op/s 217111836.019op/s 217378678.720op/s 217410374.986op/s 217738270.568op/s 0.42% -0.141 -0.746 0.18% 27086.727op/s 1 200
credit_card/is_card_number/ 3782-8224-6310-005 execution_time 93.023µs 93.541µs ± 0.338µs 93.543µs ± 0.106µs 93.601µs 93.736µs 94.114µs 97.635µs 4.37% 9.149 106.667 0.36% 0.024µs 1 200
credit_card/is_card_number/ 3782-8224-6310-005 throughput 10242247.337op/s 10690580.905op/s ± 37409.898op/s 10690265.989op/s ± 12096.558op/s 10705248.784op/s 10716794.718op/s 10737989.933op/s 10750080.324op/s 0.56% -8.881 102.258 0.35% 2645.279op/s 1 200
credit_card/is_card_number/ 378282246310005 execution_time 85.176µs 85.366µs ± 0.300µs 85.322µs ± 0.025µs 85.349µs 85.523µs 86.074µs 89.226µs 4.58% 10.976 135.779 0.35% 0.021µs 1 200
credit_card/is_card_number/ 378282246310005 throughput 11207494.509op/s 11714453.418op/s ± 39673.429op/s 11720360.735op/s ± 3444.061op/s 11723735.634op/s 11727647.045op/s 11730989.380op/s 11740455.930op/s 0.17% -10.795 132.309 0.34% 2805.335op/s 1 200
credit_card/is_card_number/37828224631 execution_time 4.600µs 4.611µs ± 0.008µs 4.611µs ± 0.006µs 4.617µs 4.626µs 4.631µs 4.631µs 0.44% 0.430 -0.614 0.18% 0.001µs 1 200
credit_card/is_card_number/37828224631 throughput 215928899.633op/s 216855022.802op/s ± 380715.531op/s 216880961.676op/s ± 303023.652op/s 217140754.653op/s 217384936.135op/s 217401167.491op/s 217413933.465op/s 0.25% -0.424 -0.623 0.18% 26920.653op/s 1 200
credit_card/is_card_number/378282246310005 execution_time 81.633µs 82.132µs ± 0.293µs 82.096µs ± 0.158µs 82.263µs 82.697µs 83.002µs 83.633µs 1.87% 1.361 3.330 0.36% 0.021µs 1 200
credit_card/is_card_number/378282246310005 throughput 11957063.557op/s 12175657.770op/s ± 43239.364op/s 12180860.215op/s ± 23468.717op/s 12202708.393op/s 12231878.846op/s 12243815.910op/s 12249909.808op/s 0.57% -1.325 3.149 0.35% 3057.485op/s 1 200
credit_card/is_card_number/37828224631000521389798 execution_time 58.863µs 58.895µs ± 0.020µs 58.897µs ± 0.015µs 58.907µs 58.935µs 58.951µs 58.953µs 0.10% 0.594 0.050 0.03% 0.001µs 1 200
credit_card/is_card_number/37828224631000521389798 throughput 16962573.777op/s 16979317.118op/s ± 5839.257op/s 16978776.529op/s ± 4456.453op/s 16984580.549op/s 16987292.571op/s 16988116.099op/s 16988461.437op/s 0.06% -0.593 0.046 0.03% 412.898op/s 1 200
credit_card/is_card_number/x371413321323331 execution_time 6.819µs 6.827µs ± 0.015µs 6.823µs ± 0.003µs 6.826µs 6.853µs 6.901µs 6.938µs 1.68% 4.100 21.162 0.22% 0.001µs 1 200
credit_card/is_card_number/x371413321323331 throughput 144135429.453op/s 146469802.022op/s ± 315755.582op/s 146561350.582op/s ± 56192.703op/s 146615705.453op/s 146636360.084op/s 146642410.604op/s 146651579.072op/s 0.06% -4.059 20.713 0.22% 22327.291op/s 1 200
credit_card/is_card_number_no_luhn/ execution_time 4.599µs 4.613µs ± 0.008µs 4.613µs ± 0.006µs 4.619µs 4.628µs 4.631µs 4.631µs 0.40% 0.257 -0.680 0.18% 0.001µs 1 200
credit_card/is_card_number_no_luhn/ throughput 215916964.391op/s 216775507.858op/s ± 385692.988op/s 216774041.345op/s ± 263737.272op/s 217020324.459op/s 217379420.500op/s 217394575.904op/s 217416724.280op/s 0.30% -0.250 -0.685 0.18% 27272.613op/s 1 200
credit_card/is_card_number_no_luhn/ 3782-8224-6310-005 execution_time 73.398µs 73.717µs ± 0.144µs 73.737µs ± 0.081µs 73.815µs 73.899µs 73.952µs 74.614µs 1.19% 0.768 6.423 0.19% 0.010µs 1 200
credit_card/is_card_number_no_luhn/ 3782-8224-6310-005 throughput 13402373.955op/s 13565431.606op/s ± 26389.984op/s 13561744.384op/s ± 14871.927op/s 13578537.429op/s 13610790.431op/s 13623791.404op/s 13624397.208op/s 0.46% -0.723 6.135 0.19% 1866.054op/s 1 200
credit_card/is_card_number_no_luhn/ 378282246310005 execution_time 65.275µs 65.534µs ± 0.121µs 65.534µs ± 0.097µs 65.636µs 65.721µs 65.748µs 65.756µs 0.34% -0.118 -0.793 0.18% 0.009µs 1 200
credit_card/is_card_number_no_luhn/ 378282246310005 throughput 15207736.185op/s 15259371.792op/s ± 28148.018op/s 15259206.045op/s ± 22517.014op/s 15280294.934op/s 15305209.600op/s 15318127.366op/s 15319911.289op/s 0.40% 0.125 -0.789 0.18% 1990.365op/s 1 200
credit_card/is_card_number_no_luhn/37828224631 execution_time 4.593µs 4.614µs ± 0.009µs 4.614µs ± 0.006µs 4.620µs 4.629µs 4.631µs 4.633µs 0.43% 0.134 -0.882 0.20% 0.001µs 1 200
credit_card/is_card_number_no_luhn/37828224631 throughput 215826285.811op/s 216748868.941op/s ± 432776.570op/s 216754330.259op/s ± 301381.331op/s 217083832.038op/s 217385505.538op/s 217411369.151op/s 217733199.635op/s 0.45% -0.127 -0.885 0.20% 30601.925op/s 1 200
credit_card/is_card_number_no_luhn/378282246310005 execution_time 62.145µs 62.559µs ± 0.131µs 62.530µs ± 0.059µs 62.638µs 62.776µs 62.867µs 62.915µs 0.62% -0.048 0.941 0.21% 0.009µs 1 200
credit_card/is_card_number_no_luhn/378282246310005 throughput 15894558.069op/s 15985050.052op/s ± 33374.765op/s 15992423.763op/s ± 14986.519op/s 16003469.870op/s 16038444.056op/s 16088552.465op/s 16091527.811op/s 0.62% 0.066 0.963 0.21% 2359.952op/s 1 200
credit_card/is_card_number_no_luhn/37828224631000521389798 execution_time 58.863µs 58.896µs ± 0.021µs 58.896µs ± 0.017µs 58.909µs 58.938µs 58.949µs 58.957µs 0.10% 0.625 -0.390 0.04% 0.001µs 1 200
credit_card/is_card_number_no_luhn/37828224631000521389798 throughput 16961378.940op/s 16979218.033op/s ± 6110.780op/s 16978949.498op/s ± 4892.647op/s 16984378.618op/s 16986569.654op/s 16987675.119op/s 16988692.326op/s 0.06% -0.624 -0.392 0.04% 432.097op/s 1 200
credit_card/is_card_number_no_luhn/x371413321323331 execution_time 6.819µs 6.829µs ± 0.018µs 6.823µs ± 0.002µs 6.825µs 6.867µs 6.906µs 6.912µs 1.30% 3.107 9.396 0.26% 0.001µs 1 200
credit_card/is_card_number_no_luhn/x371413321323331 throughput 144679145.070op/s 146437044.467op/s ± 380019.099op/s 146554264.322op/s ± 52591.158op/s 146610087.669op/s 146634465.150op/s 146643016.787op/s 146647584.104op/s 0.06% -3.094 9.303 0.26% 26871.408op/s 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
credit_card/is_card_number/ execution_time [4.611µs; 4.613µs] or [-0.024%; +0.024%] None None None
credit_card/is_card_number/ throughput [216771473.930op/s; 216877651.950op/s] or [-0.024%; +0.024%] None None None
credit_card/is_card_number/ 3782-8224-6310-005 execution_time [93.495µs; 93.588µs] or [-0.050%; +0.050%] None None None
credit_card/is_card_number/ 3782-8224-6310-005 throughput [10685396.253op/s; 10695765.557op/s] or [-0.048%; +0.048%] None None None
credit_card/is_card_number/ 378282246310005 execution_time [85.324µs; 85.407µs] or [-0.049%; +0.049%] None None None
credit_card/is_card_number/ 378282246310005 throughput [11708955.062op/s; 11719951.774op/s] or [-0.047%; +0.047%] None None None
credit_card/is_card_number/37828224631 execution_time [4.610µs; 4.613µs] or [-0.024%; +0.024%] None None None
credit_card/is_card_number/37828224631 throughput [216802259.291op/s; 216907786.313op/s] or [-0.024%; +0.024%] None None None
credit_card/is_card_number/378282246310005 execution_time [82.092µs; 82.173µs] or [-0.049%; +0.049%] None None None
credit_card/is_card_number/378282246310005 throughput [12169665.210op/s; 12181650.330op/s] or [-0.049%; +0.049%] None None None
credit_card/is_card_number/37828224631000521389798 execution_time [58.892µs; 58.898µs] or [-0.005%; +0.005%] None None None
credit_card/is_card_number/37828224631000521389798 throughput [16978507.853op/s; 16980126.383op/s] or [-0.005%; +0.005%] None None None
credit_card/is_card_number/x371413321323331 execution_time [6.825µs; 6.829µs] or [-0.030%; +0.030%] None None None
credit_card/is_card_number/x371413321323331 throughput [146426041.335op/s; 146513562.709op/s] or [-0.030%; +0.030%] None None None
credit_card/is_card_number_no_luhn/ execution_time [4.612µs; 4.614µs] or [-0.025%; +0.025%] None None None
credit_card/is_card_number_no_luhn/ throughput [216722054.519op/s; 216828961.197op/s] or [-0.025%; +0.025%] None None None
credit_card/is_card_number_no_luhn/ 3782-8224-6310-005 execution_time [73.697µs; 73.737µs] or [-0.027%; +0.027%] None None None
credit_card/is_card_number_no_luhn/ 3782-8224-6310-005 throughput [13561774.208op/s; 13569089.004op/s] or [-0.027%; +0.027%] None None None
credit_card/is_card_number_no_luhn/ 378282246310005 execution_time [65.517µs; 65.550µs] or [-0.026%; +0.026%] None None None
credit_card/is_card_number_no_luhn/ 378282246310005 throughput [15255470.747op/s; 15263272.837op/s] or [-0.026%; +0.026%] None None None
credit_card/is_card_number_no_luhn/37828224631 execution_time [4.612µs; 4.615µs] or [-0.028%; +0.028%] None None None
credit_card/is_card_number_no_luhn/37828224631 throughput [216688890.270op/s; 216808847.611op/s] or [-0.028%; +0.028%] None None None
credit_card/is_card_number_no_luhn/378282246310005 execution_time [62.541µs; 62.577µs] or [-0.029%; +0.029%] None None None
credit_card/is_card_number_no_luhn/378282246310005 throughput [15980424.630op/s; 15989675.473op/s] or [-0.029%; +0.029%] None None None
credit_card/is_card_number_no_luhn/37828224631000521389798 execution_time [58.893µs; 58.898µs] or [-0.005%; +0.005%] None None None
credit_card/is_card_number_no_luhn/37828224631000521389798 throughput [16978371.137op/s; 16980064.928op/s] or [-0.005%; +0.005%] None None None
credit_card/is_card_number_no_luhn/x371413321323331 execution_time [6.826µs; 6.831µs] or [-0.036%; +0.036%] None None None
credit_card/is_card_number_no_luhn/x371413321323331 throughput [146384377.475op/s; 146489711.460op/s] or [-0.036%; +0.036%] None None None

Group 9

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 47c063d 1731609456 dsn/crashtracker-implement-rfc5
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
redis/obfuscate_redis_string execution_time 36.985µs 37.474µs ± 0.841µs 37.094µs ± 0.044µs 37.175µs 39.266µs 39.329µs 40.317µs 8.69% 1.734 1.151 2.24% 0.059µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
redis/obfuscate_redis_string execution_time [37.358µs; 37.591µs] or [-0.311%; +0.311%] None None None

Group 10

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 47c063d 1731609456 dsn/crashtracker-implement-rfc5
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
tags/replace_trace_tags execution_time 2.695µs 2.761µs ± 0.023µs 2.765µs ± 0.005µs 2.771µs 2.794µs 2.807µs 2.809µs 1.57% -1.025 0.975 0.82% 0.002µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
tags/replace_trace_tags execution_time [2.758µs; 2.764µs] or [-0.114%; +0.114%] None None None

Group 11

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 47c063d 1731609456 dsn/crashtracker-implement-rfc5
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
benching string interning on wordpress profile execution_time 141.224µs 142.814µs ± 0.444µs 142.816µs ± 0.163µs 142.962µs 143.403µs 144.076µs 145.618µs 1.96% 1.102 9.921 0.31% 0.031µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
benching string interning on wordpress profile execution_time [142.752µs; 142.875µs] or [-0.043%; +0.043%] None None None

Group 12

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 47c063d 1731609456 dsn/crashtracker-implement-rfc5
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
normalization/normalize_name/normalize_name/Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Lo... execution_time 271.537µs 273.438µs ± 0.903µs 273.359µs ± 0.647µs 274.052µs 274.876µs 275.980µs 276.567µs 1.17% 0.562 0.336 0.33% 0.064µs 1 200
normalization/normalize_name/normalize_name/Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Lo... throughput 3615763.863op/s 3657180.355op/s ± 12058.718op/s 3658190.179op/s ± 8662.341op/s 3666205.220op/s 3674259.009op/s 3678658.969op/s 3682738.013op/s 0.67% -0.542 0.292 0.33% 852.680op/s 1 200
normalization/normalize_name/normalize_name/bad-name execution_time 25.782µs 25.876µs ± 0.064µs 25.856µs ± 0.042µs 25.922µs 25.997µs 26.058µs 26.105µs 0.96% 0.861 0.247 0.25% 0.005µs 1 200
normalization/normalize_name/normalize_name/bad-name throughput 38306330.354op/s 38645396.818op/s ± 95111.987op/s 38675840.432op/s ± 62786.432op/s 38717800.363op/s 38759486.342op/s 38780666.832op/s 38786202.117op/s 0.29% -0.850 0.213 0.25% 6725.433op/s 1 200
normalization/normalize_name/normalize_name/good execution_time 15.455µs 15.514µs ± 0.047µs 15.502µs ± 0.022µs 15.530µs 15.602µs 15.659µs 15.765µs 1.70% 1.780 4.387 0.30% 0.003µs 1 200
normalization/normalize_name/normalize_name/good throughput 63429774.782op/s 64458640.603op/s ± 194384.793op/s 64506574.432op/s ± 91361.293op/s 64584950.294op/s 64670543.864op/s 64693715.025op/s 64704153.737op/s 0.31% -1.751 4.210 0.30% 13745.081op/s 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
normalization/normalize_name/normalize_name/Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Lo... execution_time [273.312µs; 273.563µs] or [-0.046%; +0.046%] None None None
normalization/normalize_name/normalize_name/Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Lo... throughput [3655509.133op/s; 3658851.578op/s] or [-0.046%; +0.046%] None None None
normalization/normalize_name/normalize_name/bad-name execution_time [25.868µs; 25.885µs] or [-0.034%; +0.034%] None None None
normalization/normalize_name/normalize_name/bad-name throughput [38632215.211op/s; 38658578.424op/s] or [-0.034%; +0.034%] None None None
normalization/normalize_name/normalize_name/good execution_time [15.507µs; 15.520µs] or [-0.042%; +0.042%] None None None
normalization/normalize_name/normalize_name/good throughput [64431700.740op/s; 64485580.466op/s] or [-0.042%; +0.042%] None None None

Baseline

Omitted due to size.

@codecov-commenter
Copy link

codecov-commenter commented Nov 13, 2024

Codecov Report

Attention: Patch coverage is 0% with 260 lines in your changes missing coverage. Please review.

Project coverage is 70.60%. Comparing base (b57f46a) to head (47c063d).

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #738      +/-   ##
==========================================
- Coverage   71.02%   70.60%   -0.43%     
==========================================
  Files         287      295       +8     
  Lines       42836    43096     +260     
==========================================
+ Hits        30423    30426       +3     
- Misses      12413    12670     +257     
Components Coverage Δ
crashtracker 37.14% <0.00%> (-3.59%) ⬇️
crashtracker-ffi 8.67% <ø> (ø)
datadog-alloc 98.73% <ø> (ø)
data-pipeline 91.95% <ø> (+0.06%) ⬆️
data-pipeline-ffi 0.00% <ø> (ø)
ddcommon 83.46% <ø> (ø)
ddcommon-ffi 69.12% <ø> (ø)
ddtelemetry 59.10% <ø> (ø)
ddtelemetry-ffi 22.13% <ø> (ø)
dogstatsd 89.45% <ø> (ø)
dogstatsd-client 79.77% <ø> (ø)
ipc 82.86% <ø> (ø)
profiling 84.30% <ø> (ø)
profiling-ffi 77.46% <ø> (ø)
serverless 0.00% <ø> (ø)
sidecar 37.35% <ø> (ø)
sidecar-ffi 0.00% <ø> (ø)
spawn-worker 50.36% <ø> (ø)
tinybytes 94.77% <ø> (ø)
trace-mini-agent 72.18% <ø> (ø)
trace-normalization 98.25% <ø> (ø)
trace-obfuscation 95.77% <ø> (ø)
trace-protobuf 77.67% <ø> (ø)
trace-utils 93.14% <ø> (ø)

Copy link
Contributor

@gleocadie gleocadie left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall looks good to me

crashtracker/src/rfc5_crash_info/error_data.rs Outdated Show resolved Hide resolved
crashtracker/src/rfc5_crash_info/stacktrace.rs Outdated Show resolved Hide resolved
@danielsn danielsn enabled auto-merge (squash) November 14, 2024 18:38
@danielsn danielsn merged commit 0ded706 into main Nov 14, 2024
32 checks passed
@danielsn danielsn deleted the dsn/crashtracker-implement-rfc5 branch November 14, 2024 19:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants