Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Perf] Linux/x64: 53 Improvements on 10/23/2022 8:43:59 AM #9221

Closed
performanceautofiler bot opened this issue Oct 25, 2022 · 3 comments
Closed

[Perf] Linux/x64: 53 Improvements on 10/23/2022 8:43:59 AM #9221

performanceautofiler bot opened this issue Oct 25, 2022 · 3 comments
Labels
arch-x64 branch-refs/heads/main kind-micro_mono mono-interpreter os-linux perf-improvement runtime-mono untriaged

Comments

@performanceautofiler
Copy link

performanceautofiler bot commented Oct 25, 2022 •
edited
Loading

Run Information

Architecture x64
OS ubuntu 18.04
Baseline ebf4ea7601f82cdc3f7cb9e7539a84e423d0d23a
Compare 33cb335386d39da8e4e89e75449628be7e3b375a
Diff Diff

Improvements in System.Text.Json.Tests.Perf_Guids

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
WriteGuids - Duration of single invocation 100.27 ms 92.01 ms 0.92 0.02 False
WriteGuids - Duration of single invocation 95.23 ms 87.39 ms 0.92 0.06 False
WriteGuids - Duration of single invocation 93.97 ms 84.91 ms 0.90 0.04 False
WriteGuids - Duration of single invocation 103.47 ms 95.41 ms 0.92 0.04 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Json.Tests.Perf_Guids*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Text.Json.Tests.Perf_Guids.WriteGuids(Formatted: True, SkipValidation: True)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 92.00629617307693 < 94.45351976462912.
IsChangePoint: Marked as a change because one of 9/28/2022 11:19:50 AM, 10/22/2022 11:54:51 PM, 10/25/2022 2:56:59 AM falls between 10/16/2022 1:15:28 PM and 10/25/2022 2:56:59 AM.
IsImprovementStdDev: Marked as improvement because 19.452543478065195 (T) = (0 -93200484.17844933) / Math.Sqrt((6746790200771.927 / (299)) + (1188307159026.9854 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.0892813358581669 = (102337294.54339427 - 93200484.17844933) / 102337294.54339427 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Guids.WriteGuids(Formatted: False, SkipValidation: False)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 87.39030684615383 < 90.71476787458333.
IsChangePoint: Marked as a change because one of 9/28/2022 3:03:11 PM, 10/22/2022 11:54:51 PM, 10/25/2022 2:56:59 AM falls between 10/16/2022 1:15:28 PM and 10/25/2022 2:56:59 AM.
IsImprovementStdDev: Marked as improvement because 30.446826180033803 (T) = (0 -87171621.66285105) / Math.Sqrt((7277185416620.339 / (299)) + (407080734256.679 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.09587911355144928 = (96415891.91160841 - 87171621.66285105) / 96415891.91160841 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Guids.WriteGuids(Formatted: False, SkipValidation: True)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 84.91342175 < 89.25739553208334.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/22/2022 11:54:51 PM, 10/25/2022 2:56:59 AM falls between 10/16/2022 1:15:28 PM and 10/25/2022 2:56:59 AM.
IsImprovementStdDev: Marked as improvement because 17.15999221039687 (T) = (0 -86054962.78055556) / Math.Sqrt((7470901387661.225 / (299)) + (1495622514357.8147 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.09455448215438074 = (95041569.13307306 - 86054962.78055556) / 95041569.13307306 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Guids.WriteGuids(Formatted: True, SkipValidation: False)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 95.4075750576923 < 96.96958035429488.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/22/2022 11:54:51 PM, 10/25/2022 2:56:59 AM falls between 10/16/2022 1:15:28 PM and 10/25/2022 2:56:59 AM.
IsImprovementStdDev: Marked as improvement because 21.45200013490899 (T) = (0 -94503868.89957266) / Math.Sqrt((6469993525886.372 / (299)) + (928843700511.0417 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.08705043915066789 = (103514885.10674581 - 94503868.89957266) / 103514885.10674581 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS ubuntu 18.04
Baseline ebf4ea7601f82cdc3f7cb9e7539a84e423d0d23a
Compare 33cb335386d39da8e4e89e75449628be7e3b375a
Diff Diff

Improvements in System.Collections.IterateForEach<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ReadOnlySpan - Duration of single invocation 3.69 μs 3.10 μs 0.84 0.06 False
Span - Duration of single invocation 3.74 μs 3.10 μs 0.83 0.08 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.IterateForEach&lt;Int32&gt;*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Collections.IterateForEach<Int32>.ReadOnlySpan(Size: 512)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 3.0957746989218533 < 3.545299397659925.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/22/2022 11:54:51 PM, 10/25/2022 2:56:59 AM falls between 10/16/2022 1:15:28 PM and 10/25/2022 2:56:59 AM.
IsImprovementStdDev: Marked as improvement because 63.63741465737455 (T) = (0 -3081.3933527051527) / Math.Sqrt((36037.44350850276 / (299)) + (531.5361332140604 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.2299681233207716 = (4001.643887774748 - 3081.3933527051527) / 4001.643887774748 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.IterateForEach&lt;Int32&gt;.Span(Size: 512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 3.096745567217137 < 3.5474390195766343.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/22/2022 11:54:51 PM, 10/25/2022 2:56:59 AM falls between 10/16/2022 1:15:28 PM and 10/25/2022 2:56:59 AM.
IsImprovementStdDev: Marked as improvement because 18.15550824264181 (T) = (0 -3132.1543399827733) / Math.Sqrt((108415.98520673197 / (299)) + (12302.615206510458 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.22163169302899371 = (4024.0003503886805 - 3132.1543399827733) / 4024.0003503886805 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline ebf4ea7601f82cdc3f7cb9e7539a84e423d0d23a
Compare 33cb335386d39da8e4e89e75449628be7e3b375a
Diff Diff

Improvements in System.Globalization.Tests.StringSearch

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
IsPrefix_FirstHalf - Duration of single invocation 3.32 μs 2.48 μs 0.75 0.10 False
IsSuffix_SecondHalf - Duration of single invocation 3.34 μs 2.51 μs 0.75 0.08 False
IsPrefix_FirstHalf - Duration of single invocation 3.39 μs 2.44 μs 0.72 0.10 False
IsPrefix_FirstHalf - Duration of single invocation 3.39 μs 2.48 μs 0.73 0.07 False
IsPrefix_FirstHalf - Duration of single invocation 3.38 μs 2.48 μs 0.74 0.10 False
IsSuffix_SecondHalf - Duration of single invocation 3.41 μs 2.51 μs 0.74 0.09 False
IsSuffix_SecondHalf - Duration of single invocation 3.38 μs 2.51 μs 0.74 0.09 False
IsPrefix_FirstHalf - Duration of single invocation 3.37 μs 2.48 μs 0.74 0.05 False
IsSuffix_SecondHalf - Duration of single invocation 3.39 μs 2.48 μs 0.73 0.11 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Globalization.Tests.StringSearch*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Globalization.Tests.StringSearch.IsPrefix_FirstHalf(Options: (en-US, IgnoreNonSpace, False))

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.483025418326693 < 3.211349695992968.
IsChangePoint: Marked as a change because one of 10/22/2022 11:54:51 PM, 10/25/2022 2:56:59 AM falls between 10/16/2022 1:15:28 PM and 10/25/2022 2:56:59 AM.
IsImprovementStdDev: Marked as improvement because 79.56874026832267 (T) = (0 -2462.591240537343) / Math.Sqrt((12213.419831149808 / (299)) + (557.9597034679513 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.27209437209900267 = (3383.1188359382772 - 2462.591240537343) / 3383.1188359382772 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Globalization.Tests.StringSearch.IsSuffix_SecondHalf(Options: (, IgnoreCase, False))

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.514222949406671 < 3.2145748419723694.
IsChangePoint: Marked as a change because one of 10/22/2022 11:54:51 PM, 10/25/2022 2:56:59 AM falls between 10/16/2022 1:15:28 PM and 10/25/2022 2:56:59 AM.
IsImprovementStdDev: Marked as improvement because 94.24768476669756 (T) = (0 -2518.18477175313) / Math.Sqrt((12451.574587078312 / (299)) + (264.68508808028116 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.25738566883177605 = (3390.9724954967064 - 2518.18477175313) / 3390.9724954967064 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Globalization.Tests.StringSearch.IsPrefix_FirstHalf(Options: (en-US, IgnoreCase, False))

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.4400910064002086 < 3.222055891614802.
IsChangePoint: Marked as a change because one of 10/22/2022 11:54:51 PM, 10/25/2022 2:56:59 AM falls between 10/16/2022 1:15:28 PM and 10/25/2022 2:56:59 AM.
IsImprovementStdDev: Marked as improvement because 80.19138602540781 (T) = (0 -2475.331732883871) / Math.Sqrt((7252.965487290677 / (299)) + (621.4890545105402 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.2680913197662099 = (3382.02264808936 - 2475.331732883871) / 3382.02264808936 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Globalization.Tests.StringSearch.IsPrefix_FirstHalf(Options: (, IgnoreCase, False))

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.4795277901726425 < 3.213200534709138.
IsChangePoint: Marked as a change because one of 10/22/2022 11:54:51 PM, 10/25/2022 2:56:59 AM falls between 10/16/2022 1:15:28 PM and 10/25/2022 2:56:59 AM.
IsImprovementStdDev: Marked as improvement because 18.269035491026962 (T) = (0 -2500.227750417052) / Math.Sqrt((3846.3631106079806 / (299)) + (13800.68074212425 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.260035499937744 = (3378.8482423233795 - 2500.227750417052) / 3378.8482423233795 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Globalization.Tests.StringSearch.IsPrefix_FirstHalf(Options: (, None, False))

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.4837094417372314 < 3.214512385273907.
IsChangePoint: Marked as a change because one of 10/22/2022 11:54:51 PM, 10/25/2022 2:56:59 AM falls between 10/16/2022 1:15:28 PM and 10/25/2022 2:56:59 AM.
IsImprovementStdDev: Marked as improvement because 110.53496329226851 (T) = (0 -2473.9986126813023) / Math.Sqrt((14936.942310567054 / (299)) + (112.49271020526854 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.2702514315108231 = (3390.2068733115916 - 2473.9986126813023) / 3390.2068733115916 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Globalization.Tests.StringSearch.IsSuffix_SecondHalf(Options: (en-US, IgnoreNonSpace, False))

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.5118583900908114 < 3.2239543436133924.
IsChangePoint: Marked as a change because one of 10/22/2022 11:54:51 PM, 10/25/2022 2:56:59 AM falls between 10/16/2022 1:15:28 PM and 10/25/2022 2:56:59 AM.
IsImprovementStdDev: Marked as improvement because 159.83129582034655 (T) = (0 -2514.534663474556) / Math.Sqrt((8361.830946492382 / (299)) + (13.217022525826739 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.2587799230094557 = (3392.426543117819 - 2514.534663474556) / 3392.426543117819 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Globalization.Tests.StringSearch.IsSuffix_SecondHalf(Options: (en-US, None, False))

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.5110272455456952 < 3.2227740170608286.
IsChangePoint: Marked as a change because one of 10/22/2022 11:54:51 PM, 10/25/2022 2:56:59 AM falls between 10/16/2022 1:15:28 PM and 10/25/2022 2:56:59 AM.
IsImprovementStdDev: Marked as improvement because 117.49374572209007 (T) = (0 -2509.5725524016975) / Math.Sqrt((7469.787908057397 / (299)) + (187.6671074757739 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.259900266368524 = (3390.8572566130792 - 2509.5725524016975) / 3390.8572566130792 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Globalization.Tests.StringSearch.IsPrefix_FirstHalf(Options: (en-US, None, False))

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.480454630514413 < 3.2123018344223846.
IsChangePoint: Marked as a change because one of 10/22/2022 11:54:51 PM, 10/25/2022 2:56:59 AM falls between 10/16/2022 1:15:28 PM and 10/25/2022 2:56:59 AM.
IsImprovementStdDev: Marked as improvement because 36.010643855843995 (T) = (0 -2488.905798371936) / Math.Sqrt((5239.900202340528 / (299)) + (3508.909129526181 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.26204426114433327 = (3372.7033578347564 - 2488.905798371936) / 3372.7033578347564 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Globalization.Tests.StringSearch.IsSuffix_SecondHalf(Options: (, None, False))

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.4813703845950794 < 3.218766042532956.
IsChangePoint: Marked as a change because one of 10/22/2022 11:54:51 PM, 10/25/2022 2:56:59 AM falls between 10/16/2022 1:15:28 PM and 10/25/2022 2:56:59 AM.
IsImprovementStdDev: Marked as improvement because 101.68284200089333 (T) = (0 -2504.67209252973) / Math.Sqrt((7234.553930649038 / (299)) + (312.02035749101 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.2616560577462447 = (3392.2836623868716 - 2504.67209252973) / 3392.2836623868716 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline ebf4ea7601f82cdc3f7cb9e7539a84e423d0d23a
Compare 33cb335386d39da8e4e89e75449628be7e3b375a
Diff Diff

Improvements in System.Tests.Perf_UInt64

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
TryParseHex - Duration of single invocation 223.27 ns 201.05 ns 0.90 0.17 False
TryParseHex - Duration of single invocation 573.95 ns 472.60 ns 0.82 0.11 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_UInt64*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Tests.Perf_UInt64.TryParseHex(value: "3039")

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 201.05340852627864 < 212.11795695686078.
IsChangePoint: Marked as a change because one of 10/5/2022 3:28:28 PM, 10/22/2022 11:54:51 PM, 10/25/2022 2:56:59 AM falls between 10/16/2022 1:15:28 PM and 10/25/2022 2:56:59 AM.
IsImprovementStdDev: Marked as improvement because 21.58607457261086 (T) = (0 -200.26613411008188) / Math.Sqrt((61.71601270603863 / (299)) + (12.491044498458049 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.140190898765434 = (232.91930013595766 - 200.26613411008188) / 232.91930013595766 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_UInt64.TryParseHex(value: "FFFFFFFFFFFFFFFF")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 472.6002496544997 < 547.6610412921339.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/22/2022 11:54:51 PM, 10/25/2022 2:56:59 AM falls between 10/16/2022 1:15:28 PM and 10/25/2022 2:56:59 AM.
IsImprovementStdDev: Marked as improvement because 28.68030358738252 (T) = (0 -481.19398304832947) / Math.Sqrt((370.3264700545112 / (299)) + (100.062566679631 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.20145541417036733 = (602.5887490657797 - 481.19398304832947) / 602.5887490657797 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline ebf4ea7601f82cdc3f7cb9e7539a84e423d0d23a
Compare 33cb335386d39da8e4e89e75449628be7e3b375a
Diff Diff

Improvements in System.Collections.ContainsKeyTrue<Int32, Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ConcurrentDictionary - Duration of single invocation 131.62 μs 110.35 μs 0.84 0.09 False
Dictionary - Duration of single invocation 102.48 μs 96.24 μs 0.94 0.08 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.ContainsKeyTrue&lt;Int32, Int32&gt;*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Collections.ContainsKeyTrue<Int32, Int32>.ConcurrentDictionary(Size: 512)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 110.35327072047669 < 121.90367022935777.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 9/28/2022 6:40:13 PM, 10/11/2022 6:23:40 AM, 10/22/2022 11:54:51 PM, 10/25/2022 2:56:59 AM falls between 10/16/2022 1:15:28 PM and 10/25/2022 2:56:59 AM.
IsImprovementStdDev: Marked as improvement because 11.648162718619442 (T) = (0 -112736.26434151844) / Math.Sqrt((21765555.333405733 / (299)) + (6961229.322193111 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.10292140654562919 = (125670.44310733816 - 112736.26434151844) / 125670.44310733816 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.ContainsKeyTrue&lt;Int32, Int32&gt;.Dictionary(Size: 512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 96.24053933899178 < 97.83539057536353.
IsChangePoint: Marked as a change because one of 7/21/2022 9:14:58 PM, 9/28/2022 3:03:11 PM, 10/11/2022 6:23:40 AM, 10/22/2022 11:54:51 PM, 10/25/2022 2:56:59 AM falls between 10/16/2022 1:15:28 PM and 10/25/2022 2:56:59 AM.
IsImprovementStdDev: Marked as improvement because 24.449826269054338 (T) = (0 -96105.99321643228) / Math.Sqrt((7354770.210693584 / (299)) + (338019.2691184447 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.06749087745045076 = (103061.71906787502 - 96105.99321643228) / 103061.71906787502 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline ebf4ea7601f82cdc3f7cb9e7539a84e423d0d23a
Compare 33cb335386d39da8e4e89e75449628be7e3b375a
Diff Diff

Improvements in System.Net.NetworkInformation.Tests.PhysicalAddressTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
PALong - Duration of single invocation 8.95 μs 7.73 μs 0.86 0.12 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Net.NetworkInformation.Tests.PhysicalAddressTests*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Net.NetworkInformation.Tests.PhysicalAddressTests.PALong

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 7.732033804182308 < 8.466132836321075.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 10/11/2022 6:23:40 AM, 10/22/2022 11:54:51 PM, 10/25/2022 2:56:59 AM falls between 10/16/2022 1:15:28 PM and 10/25/2022 2:56:59 AM.
IsImprovementStdDev: Marked as improvement because 68.36392584585278 (T) = (0 -7707.6486775882195) / Math.Sqrt((165787.73776236436 / (299)) + (479.79030731344244 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.1826116546352006 = (9429.604326139866 - 7707.6486775882195) / 9429.604326139866 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline ebf4ea7601f82cdc3f7cb9e7539a84e423d0d23a
Compare 33cb335386d39da8e4e89e75449628be7e3b375a
Diff Diff

Improvements in System.Collections.IterateForEach<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Span - Duration of single invocation 3.70 μs 3.17 μs 0.86 0.05 False
ReadOnlySpan - Duration of single invocation 3.74 μs 3.14 μs 0.84 0.08 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.IterateForEach&lt;String&gt;*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Collections.IterateForEach<String>.Span(Size: 512)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 3.1671148239627898 < 3.5515548023861783.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/22/2022 11:54:51 PM, 10/25/2022 2:56:59 AM falls between 10/16/2022 1:15:28 PM and 10/25/2022 2:56:59 AM.
IsImprovementStdDev: Marked as improvement because 8.147090540197262 (T) = (0 -3248.876214757345) / Math.Sqrt((77043.54982204967 / (299)) + (51704.031495181116 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.19109570578591006 = (4016.3913555606323 - 3248.876214757345) / 4016.3913555606323 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.IterateForEach&lt;String&gt;.ReadOnlySpan(Size: 512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 3.143074115992395 < 3.549148571134323.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/22/2022 11:54:51 PM, 10/25/2022 2:56:59 AM falls between 10/16/2022 1:15:28 PM and 10/25/2022 2:56:59 AM.
IsImprovementStdDev: Marked as improvement because 55.537657962255814 (T) = (0 -3136.9396818091705) / Math.Sqrt((48800.87510354592 / (299)) + (517.2173050552884 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.2185084678856676 = (4014.041807109735 - 3136.9396818091705) / 4014.041807109735 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
@performanceautofiler
Copy link
Author

performanceautofiler bot commented Oct 25, 2022 •
edited
Loading

Run Information

Architecture x64
OS ubuntu 18.04
Baseline ebf4ea7601f82cdc3f7cb9e7539a84e423d0d23a
Compare 33cb335386d39da8e4e89e75449628be7e3b375a
Diff Diff

Improvements in System.Tests.Perf_UInt32

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
TryParseHex - Duration of single invocation 230.66 ns 192.88 ns 0.84 0.19 False
TryParseHex - Duration of single invocation 338.71 ns 292.09 ns 0.86 0.12 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_UInt32*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Tests.Perf_UInt32.TryParseHex(value: "3039")

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 192.88315907397745 < 213.7732737518746.
IsChangePoint: Marked as a change because one of 10/22/2022 11:54:51 PM, 10/25/2022 2:56:59 AM falls between 10/16/2022 1:15:28 PM and 10/25/2022 2:56:59 AM.
IsImprovementStdDev: Marked as improvement because 15.439941776324543 (T) = (0 -195.88038849109728) / Math.Sqrt((50.65966847865702 / (299)) + (28.841033260881268 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.1495409226681121 = (230.32312043234953 - 195.88038849109728) / 230.32312043234953 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_UInt32.TryParseHex(value: "FFFFFFFF")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 292.09455084291415 < 324.10563163372467.
IsChangePoint: Marked as a change because one of 10/7/2022 12:48:42 AM, 10/22/2022 11:54:51 PM, 10/25/2022 2:56:59 AM falls between 10/16/2022 1:15:28 PM and 10/25/2022 2:56:59 AM.
IsImprovementStdDev: Marked as improvement because 50.77284438604697 (T) = (0 -286.972936580089) / Math.Sqrt((119.42674430728883 / (299)) + (8.619238739418789 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.1933726838750063 = (355.76892927293346 - 286.972936580089) / 355.76892927293346 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS ubuntu 18.04
Baseline ebf4ea7601f82cdc3f7cb9e7539a84e423d0d23a
Compare 33cb335386d39da8e4e89e75449628be7e3b375a
Diff Diff

Improvements in System.Collections.Sort<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Array_ComparerStruct - Duration of single invocation 161.44 μs 148.51 μs 0.92 0.04 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.Sort&lt;Int32&gt;*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Collections.Sort<Int32>.Array_ComparerStruct(Size: 512)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 148.50799085999998 < 155.15168196346156.
IsChangePoint: Marked as a change because one of 8/1/2022 8:26:50 AM, 8/2/2022 2:48:04 AM, 10/22/2022 11:54:51 PM, 10/25/2022 2:56:59 AM falls between 10/16/2022 1:15:28 PM and 10/25/2022 2:56:59 AM.
IsImprovementStdDev: Marked as improvement because 37.372297023108054 (T) = (0 -148048.78962341268) / Math.Sqrt((10697149.348011719 / (299)) + (692321.9424735417 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.08937346044306567 = (162579.0411241977 - 148048.78962341268) / 162579.0411241977 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline ebf4ea7601f82cdc3f7cb9e7539a84e423d0d23a
Compare 33cb335386d39da8e4e89e75449628be7e3b375a
Diff Diff

Improvements in System.Perf_Convert

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ToHexString - Duration of single invocation 2.47 μs 2.15 μs 0.87 0.10 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Perf_Convert*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Perf_Convert.ToHexString

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.1530228362245847 < 2.3434483092476923.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 10/11/2022 6:23:40 AM, 10/22/2022 11:54:51 PM, 10/25/2022 2:56:59 AM falls between 10/16/2022 1:15:28 PM and 10/25/2022 2:56:59 AM.
IsImprovementStdDev: Marked as improvement because 43.739497754814735 (T) = (0 -2152.9058892108906) / Math.Sqrt((12009.622890750097 / (299)) + (375.30170241346934 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.17074774534585943 = (2596.201429815601 - 2152.9058892108906) / 2596.201429815601 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline ebf4ea7601f82cdc3f7cb9e7539a84e423d0d23a
Compare 33cb335386d39da8e4e89e75449628be7e3b375a
Diff Diff

Improvements in System.Text.Json.Tests.Perf_Strings

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
WriteStringsUtf16 - Duration of single invocation 629.10 ms 552.75 ms 0.88 0.09 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Json.Tests.Perf_Strings*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Text.Json.Tests.Perf_Strings.WriteStringsUtf16(Formatted: False, SkipValidation: True, Escaped: AllEscaped)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 552.7494080666668 < 586.5001211745238.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 9/28/2022 6:40:13 PM, 10/22/2022 11:54:51 PM, 10/25/2022 2:56:59 AM falls between 10/16/2022 1:15:28 PM and 10/25/2022 2:56:59 AM.
IsImprovementStdDev: Marked as improvement because 38.74877291762433 (T) = (0 -552614006.2179488) / Math.Sqrt((1012340255144486.2 / (299)) + (22302496370873.273 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.15745152125993994 = (655883928.5358667 - 552614006.2179488) / 655883928.5358667 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline ebf4ea7601f82cdc3f7cb9e7539a84e423d0d23a
Compare 33cb335386d39da8e4e89e75449628be7e3b375a
Diff Diff

Improvements in System.Buffers.Text.Tests.Utf8ParserTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
TryParseUInt32Hex - Duration of single invocation 163.55 ns 144.33 ns 0.88 0.19 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Buffers.Text.Tests.Utf8ParserTests*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Buffers.Text.Tests.Utf8ParserTests.TryParseUInt32Hex(value: FFFFFFFFFFFFFFFF)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 144.33182615436874 < 155.34741773789386.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/22/2022 11:54:51 PM, 10/25/2022 2:56:59 AM falls between 10/16/2022 1:15:28 PM and 10/25/2022 2:56:59 AM.
IsImprovementStdDev: Marked as improvement because 49.563416560994796 (T) = (0 -144.6533908570205) / Math.Sqrt((135.6720150596549 / (299)) + (0.6085076374782187 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.2033758105996382 = (181.58297573904275 - 144.6533908570205) / 181.58297573904275 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline ebf4ea7601f82cdc3f7cb9e7539a84e423d0d23a
Compare 33cb335386d39da8e4e89e75449628be7e3b375a
Diff Diff

Improvements in System.Collections.Sort<IntClass>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Array_ComparerStruct - Duration of single invocation 171.43 μs 160.40 μs 0.94 0.03 False
Array_ComparerClass - Duration of single invocation 172.29 μs 157.81 μs 0.92 0.09 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.Sort&lt;IntClass&gt;*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Collections.Sort<IntClass>.Array_ComparerStruct(Size: 512)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 160.39931120666665 < 162.81628867714286.
IsChangePoint: Marked as a change because one of 10/5/2022 3:28:28 PM, 10/22/2022 11:54:51 PM, 10/25/2022 2:56:59 AM falls between 10/16/2022 1:15:28 PM and 10/25/2022 2:56:59 AM.
IsImprovementStdDev: Marked as improvement because 19.857094840620306 (T) = (0 -158110.75616920635) / Math.Sqrt((22094721.313998286 / (299)) + (5089709.77520927 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.10762421653480374 = (177179.5684047413 - 158110.75616920635) / 177179.5684047413 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.Sort&lt;IntClass&gt;.Array_ComparerClass(Size: 512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 157.81207726000002 < 160.6730274767857.
IsChangePoint: Marked as a change because one of 9/13/2022 8:43:59 AM, 10/5/2022 3:28:28 PM, 10/23/2022 8:43:59 AM, 10/25/2022 2:56:59 AM falls between 10/16/2022 1:15:28 PM and 10/25/2022 2:56:59 AM.
IsImprovementStdDev: Marked as improvement because 26.72824247554405 (T) = (0 -154737.80191135532) / Math.Sqrt((58436576.70648864 / (299)) + (3409627.7746018753 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.13926272640564152 = (179773.55769105328 - 154737.80191135532) / 179773.55769105328 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline ebf4ea7601f82cdc3f7cb9e7539a84e423d0d23a
Compare 33cb335386d39da8e4e89e75449628be7e3b375a
Diff Diff

Improvements in System.Collections.CreateAddAndClear<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Span - Duration of single invocation 5.28 μs 4.77 μs 0.90 0.03 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.CreateAddAndClear&lt;Int32&gt;*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Collections.CreateAddAndClear<Int32>.Span(Size: 512)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 4.766982283661191 < 4.994329291317645.
IsChangePoint: Marked as a change because one of 10/22/2022 11:54:51 PM, 10/25/2022 2:56:59 AM falls between 10/16/2022 1:15:28 PM and 10/25/2022 2:56:59 AM.
IsImprovementStdDev: Marked as improvement because 40.219370334991886 (T) = (0 -4755.389377400072) / Math.Sqrt((11284.59197436295 / (299)) + (583.048938152261 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.08945079867167711 = (5222.550709465055 - 4755.389377400072) / 5222.550709465055 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline ebf4ea7601f82cdc3f7cb9e7539a84e423d0d23a
Compare 33cb335386d39da8e4e89e75449628be7e3b375a
Diff Diff

Improvements in System.Tests.Perf_Int32

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ParseHex - Duration of single invocation 367.80 ns 306.58 ns 0.83 0.11 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_Int32*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Tests.Perf_Int32.ParseHex(value: "80000000")

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 306.5754866782769 < 342.9065599670697.
IsChangePoint: Marked as a change because one of 10/22/2022 11:54:51 PM, 10/25/2022 2:56:59 AM falls between 10/16/2022 1:15:28 PM and 10/25/2022 2:56:59 AM.
IsImprovementStdDev: Marked as improvement because 58.12011698852839 (T) = (0 -302.72576588338717) / Math.Sqrt((101.49045732762052 / (299)) + (5.652350844492679 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.1785352176976332 = (368.51946961733427 - 302.72576588338717) / 368.51946961733427 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline ebf4ea7601f82cdc3f7cb9e7539a84e423d0d23a
Compare 33cb335386d39da8e4e89e75449628be7e3b375a
Diff Diff

Improvements in System.Tests.Perf_DateTime

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ToString - Duration of single invocation 550.17 ns 506.15 ns 0.92 0.03 False
ParseO - Duration of single invocation 669.61 ns 610.18 ns 0.91 0.12 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_DateTime*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Tests.Perf_DateTime.ToString(format: "r")

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 506.1543315472427 < 533.1951339754484.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 10/22/2022 11:54:51 PM, 10/25/2022 2:56:59 AM falls between 10/16/2022 1:15:28 PM and 10/25/2022 2:56:59 AM.
IsImprovementStdDev: Marked as improvement because 19.974884034507387 (T) = (0 -512.7997797198899) / Math.Sqrt((417.43903892180873 / (299)) + (61.94253151180878 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.11766119595675312 = (581.182395435887 - 512.7997797198899) / 581.182395435887 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_DateTime.ParseO

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 610.1751586935292 < 614.1435356038761.
IsChangePoint: Marked as a change because one of 8/1/2022 12:48:33 PM, 10/22/2022 11:54:51 PM, 10/25/2022 2:56:59 AM falls between 10/16/2022 1:15:28 PM and 10/25/2022 2:56:59 AM.
IsImprovementStdDev: Marked as improvement because 17.494550186903066 (T) = (0 -604.3420482394475) / Math.Sqrt((299.0103428730615 / (299)) + (29.09631147474888 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.0654315883638856 = (646.653621837537 - 604.3420482394475) / 646.653621837537 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented Oct 25, 2022 •
edited
Loading

Run Information

Architecture x64
OS ubuntu 18.04
Baseline ebf4ea7601f82cdc3f7cb9e7539a84e423d0d23a
Compare 33cb335386d39da8e4e89e75449628be7e3b375a
Diff Diff

Improvements in BenchmarksGame.PiDigits_3

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
RunBench - Duration of single invocation 6.23 secs 5.73 secs 0.92 0.04 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'BenchmarksGame.PiDigits_3*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

BenchmarksGame.PiDigits_3.RunBench(n: 3000, expected: "8649423196\t:3000")

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 5.7340236492 < 5.9131021284634615.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/22/2022 11:54:51 PM, 10/25/2022 2:56:59 AM falls between 10/16/2022 1:15:28 PM and 10/25/2022 2:56:59 AM.
IsImprovementStdDev: Marked as improvement because 37.532181066980186 (T) = (0 -5768581538.063492) / Math.Sqrt((15095847017652304 / (299)) + (1932798032528229.2 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.11158006319070185 = (6493079791.501498 - 5768581538.063492) / 6493079791.501498 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS ubuntu 18.04
Baseline ebf4ea7601f82cdc3f7cb9e7539a84e423d0d23a
Compare 33cb335386d39da8e4e89e75449628be7e3b375a
Diff Diff

Improvements in System.Security.Cryptography.X509Certificates.Tests.X509Certificate2Tests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
CertProp - Duration of single invocation 1.01 μs 923.21 ns 0.91 0.06 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Security.Cryptography.X509Certificates.Tests.X509Certificate2Tests*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Security.Cryptography.X509Certificates.Tests.X509Certificate2Tests.CertProp

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 923.2148424482838 < 0.958246908442509.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 10/22/2022 11:54:51 PM, 10/25/2022 2:56:59 AM falls between 10/16/2022 1:15:28 PM and 10/25/2022 2:56:59 AM.
IsImprovementStdDev: Marked as improvement because 43.86236855012894 (T) = (0 -918.8485609925877) / Math.Sqrt((1160.0783008583285 / (299)) + (38.47859986805169 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.13281047152502334 = (1059.570636892327 - 918.8485609925877) / 1059.570636892327 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline ebf4ea7601f82cdc3f7cb9e7539a84e423d0d23a
Compare 33cb335386d39da8e4e89e75449628be7e3b375a
Diff Diff

Improvements in Span.IndexerBench

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Indexer4 - Duration of single invocation 75.64 μs 65.17 μs 0.86 0.03 False
CoveredIndex2 - Duration of single invocation 13.82 μs 11.98 μs 0.87 0.08 False
WriteViaIndexer1 - Duration of single invocation 12.92 μs 10.92 μs 0.85 0.08 False
Indexer1 - Duration of single invocation 7.65 μs 6.58 μs 0.86 0.04 False
SameIndex2 - Duration of single invocation 14.16 μs 11.91 μs 0.84 0.06 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'Span.IndexerBench*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

Span.IndexerBench.Indexer4(length: 1024)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 65.17048289262821 < 71.97416941231701.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/22/2022 11:54:51 PM, 10/25/2022 2:56:59 AM falls between 10/16/2022 1:15:28 PM and 10/25/2022 2:56:59 AM.
IsImprovementStdDev: Marked as improvement because 67.06825216282098 (T) = (0 -65185.45594996439) / Math.Sqrt((14576166.522578837 / (299)) + (3211.449686991897 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.18594246929468544 = (80074.75330826618 - 65185.45594996439) / 80074.75330826618 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### Span.IndexerBench.CoveredIndex2(length: 1024)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 11.979106356755235 < 13.215794575456657.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/22/2022 11:54:51 PM, 10/25/2022 2:56:59 AM falls between 10/16/2022 1:15:28 PM and 10/25/2022 2:56:59 AM.
IsImprovementStdDev: Marked as improvement because 10.35641418781497 (T) = (0 -12237.0942916016) / Math.Sqrt((808280.9367248596 / (299)) + (347033.3425853474 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.1723485324587702 = (14785.32301520024 - 12237.0942916016) / 14785.32301520024 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### Span.IndexerBench.WriteViaIndexer1(length: 1024)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 10.91773116883117 < 12.338697407912077.
IsChangePoint: Marked as a change because one of 10/12/2022 11:17:15 AM, 10/22/2022 11:54:51 PM, 10/25/2022 2:56:59 AM falls between 10/16/2022 1:15:28 PM and 10/25/2022 2:56:59 AM.
IsImprovementStdDev: Marked as improvement because 47.17018323861654 (T) = (0 -10891.00673116297) / Math.Sqrt((995165.0853559205 / (299)) + (1129.4614596261752 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.20435212005970926 = (13688.224408993943 - 10891.00673116297) / 13688.224408993943 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### Span.IndexerBench.Indexer1(length: 1024)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 6.576320090289489 < 7.24826910598902.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/22/2022 11:54:51 PM, 10/25/2022 2:56:59 AM falls between 10/16/2022 1:15:28 PM and 10/25/2022 2:56:59 AM.
IsImprovementStdDev: Marked as improvement because 69.26327887124957 (T) = (0 -6562.277066908723) / Math.Sqrt((108242.93234865041 / (299)) + (603.2114141959466 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.18500504100514206 = (8051.9235051491 - 6562.277066908723) / 8051.9235051491 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### Span.IndexerBench.SameIndex2(length: 1024)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 11.913478879183794 < 13.498108604871002.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/22/2022 11:54:51 PM, 10/25/2022 2:56:59 AM falls between 10/16/2022 1:15:28 PM and 10/25/2022 2:56:59 AM.
IsImprovementStdDev: Marked as improvement because 83.81310421044611 (T) = (0 -11932.97221775467) / Math.Sqrt((384738.86368552636 / (299)) + (734.4607771370431 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.20864772033754306 = (15079.216329350254 - 11932.97221775467) / 15079.216329350254 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline ebf4ea7601f82cdc3f7cb9e7539a84e423d0d23a
Compare 33cb335386d39da8e4e89e75449628be7e3b375a
Diff Diff

Improvements in System.Collections.IndexerSetReverse<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Span - Duration of single invocation 3.55 μs 3.24 μs 0.91 0.07 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.IndexerSetReverse&lt;String&gt;*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Collections.IndexerSetReverse<String>.Span(Size: 512)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 3.2404171716645847 < 3.380598576216048.
IsChangePoint: Marked as a change because one of 7/14/2022 1:35:19 AM, 7/27/2022 9:27:06 AM, 8/1/2022 12:35:11 AM, 10/22/2022 11:54:51 PM, 10/25/2022 2:56:59 AM falls between 10/16/2022 1:15:28 PM and 10/25/2022 2:56:59 AM.
IsImprovementStdDev: Marked as improvement because 47.207426303018366 (T) = (0 -3235.636052350001) / Math.Sqrt((12255.533281001133 / (299)) + (42.05613323013189 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.09179999319376282 = (3562.691068158424 - 3235.636052350001) / 3562.691068158424 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline ebf4ea7601f82cdc3f7cb9e7539a84e423d0d23a
Compare 33cb335386d39da8e4e89e75449628be7e3b375a
Diff Diff

Improvements in System.Collections.ContainsKeyFalse<Int32, Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ConcurrentDictionary - Duration of single invocation 70.96 μs 66.11 μs 0.93 0.03 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.ContainsKeyFalse&lt;Int32, Int32&gt;*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Collections.ContainsKeyFalse<Int32, Int32>.ConcurrentDictionary(Size: 512)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 66.11347442375887 < 68.87256708256942.
IsChangePoint: Marked as a change because one of 8/1/2022 11:22:56 PM, 10/11/2022 6:23:40 AM, 10/14/2022 1:19:37 PM, 10/22/2022 11:54:51 PM, 10/25/2022 2:56:59 AM falls between 10/16/2022 1:15:28 PM and 10/25/2022 2:56:59 AM.
IsImprovementStdDev: Marked as improvement because 26.718486898964255 (T) = (0 -66314.88502924859) / Math.Sqrt((3107742.3609358887 / (299)) + (176909.5034381897 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.07446690622537658 = (71650.47417029145 - 66314.88502924859) / 71650.47417029145 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline ebf4ea7601f82cdc3f7cb9e7539a84e423d0d23a
Compare 33cb335386d39da8e4e89e75449628be7e3b375a
Diff Diff

Improvements in System.Security.Cryptography.Primitives.Tests.Performance.Perf_FixedTimeEquals

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
FixedTimeEquals_256Bit_FirstBitDifferent - Duration of single invocation 438.22 ns 344.86 ns 0.79 0.08 False
FixedTimeEquals_256Bit_LastBitDifferent - Duration of single invocation 385.37 ns 345.32 ns 0.90 0.09 False
FixedTimeEquals_256Bit_Equal - Duration of single invocation 387.39 ns 351.46 ns 0.91 0.07 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Security.Cryptography.Primitives.Tests.Performance.Perf_FixedTimeEquals*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Security.Cryptography.Primitives.Tests.Performance.Perf_FixedTimeEquals.FixedTimeEquals_256Bit_FirstBitDifferent

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 344.85905615165694 < 369.20262429957637.
IsChangePoint: Marked as a change because one of 7/9/2022 11:03:07 AM, 7/9/2022 7:52:53 PM, 10/22/2022 11:54:51 PM, 10/25/2022 2:56:59 AM falls between 10/16/2022 1:15:28 PM and 10/25/2022 2:56:59 AM.
IsImprovementStdDev: Marked as improvement because 22.850420970426363 (T) = (0 -347.0673928675191) / Math.Sqrt((513.3618889420299 / (299)) + (14.331540816557858 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.11770102093754525 = (393.3671024263437 - 347.0673928675191) / 393.3671024263437 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Security.Cryptography.Primitives.Tests.Performance.Perf_FixedTimeEquals.FixedTimeEquals_256Bit_LastBitDifferent

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 345.32324062646677 < 367.85386895118125.
IsChangePoint: Marked as a change because one of 7/13/2022 11:09:36 AM, 7/13/2022 9:19:11 PM, 8/1/2022 7:13:37 AM, 10/22/2022 11:54:51 PM, 10/25/2022 2:56:59 AM falls between 10/16/2022 1:15:28 PM and 10/25/2022 2:56:59 AM.
IsImprovementStdDev: Marked as improvement because 36.40527866604408 (T) = (0 -345.6714251146691) / Math.Sqrt((444.19501461994804 / (299)) + (1.1352870216067577 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.11994795769650378 = (392.7852087132141 - 345.6714251146691) / 392.7852087132141 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Security.Cryptography.Primitives.Tests.Performance.Perf_FixedTimeEquals.FixedTimeEquals_256Bit_Equal

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 351.4619186023614 < 367.661836401765.
IsChangePoint: Marked as a change because one of 7/8/2022 6:19:54 AM, 10/22/2022 11:54:51 PM, 10/25/2022 2:56:59 AM falls between 10/16/2022 1:15:28 PM and 10/25/2022 2:56:59 AM.
IsImprovementStdDev: Marked as improvement because 25.104266607773507 (T) = (0 -346.2727431813519) / Math.Sqrt((657.6077439792267 / (299)) + (8.367238763354235 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.12083247457794788 = (393.8643468605379 - 346.2727431813519) / 393.8643468605379 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline ebf4ea7601f82cdc3f7cb9e7539a84e423d0d23a
Compare 33cb335386d39da8e4e89e75449628be7e3b375a
Diff Diff

Improvements in System.Collections.Tests.DictionarySequentialKeys

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
TryGetValue_3k_Int_32ByteValue - Duration of single invocation 206.82 ns 194.71 ns 0.94 0.09 False
TryGetValue_17_Int_32ByteRefsValue - Duration of single invocation 245.64 ns 215.79 ns 0.88 0.05 False
TryGetValue_3k_Int_32ByteRefsValue - Duration of single invocation 232.56 ns 219.52 ns 0.94 0.07 False
TryGetValue_17_Int_Int - Duration of single invocation 200.28 ns 183.55 ns 0.92 0.06 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.Tests.DictionarySequentialKeys*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Collections.Tests.DictionarySequentialKeys.TryGetValue_3k_Int_32ByteValue

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 194.71431322016463 < 196.18309200669646.
IsChangePoint: Marked as a change because one of 7/21/2022 9:14:58 PM, 9/28/2022 10:18:57 AM, 10/11/2022 1:45:10 PM, 10/22/2022 11:54:51 PM, 10/25/2022 2:56:59 AM falls between 10/16/2022 1:15:28 PM and 10/25/2022 2:56:59 AM.
IsImprovementStdDev: Marked as improvement because 23.472820105679066 (T) = (0 -194.7739969283717) / Math.Sqrt((33.898232529720765 / (299)) + (1.0439035338059945 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.06068162464657002 = (207.35674084421655 - 194.7739969283717) / 207.35674084421655 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.Tests.DictionarySequentialKeys.TryGetValue_17_Int_32ByteRefsValue

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 215.78875134897987 < 224.19659496029104.
IsChangePoint: Marked as a change because one of 7/21/2022 9:14:58 PM, 9/28/2022 3:03:11 PM, 10/10/2022 9:53:50 PM, 10/22/2022 11:54:51 PM, 10/25/2022 2:56:59 AM falls between 10/16/2022 1:15:28 PM and 10/25/2022 2:56:59 AM.
IsImprovementStdDev: Marked as improvement because 7.162790306044017 (T) = (0 -219.84128016896875) / Math.Sqrt((37.831379375094386 / (299)) + (17.911898879685136 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.05435150280089627 = (232.47674037458103 - 219.84128016896875) / 232.47674037458103 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.Tests.DictionarySequentialKeys.TryGetValue_3k_Int_32ByteRefsValue

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 219.51744427083335 < 220.80763944310897.
IsChangePoint: Marked as a change because one of 7/18/2022 6:27:40 PM, 9/28/2022 6:40:13 PM, 10/10/2022 9:53:50 PM, 10/22/2022 11:54:51 PM, 10/25/2022 2:56:59 AM falls between 10/16/2022 1:15:28 PM and 10/25/2022 2:56:59 AM.
IsImprovementStdDev: Marked as improvement because 15.737862394303356 (T) = (0 -219.03126942329948) / Math.Sqrt((31.952547523880003 / (299)) + (3.008922971638993 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.053068360963332666 = (231.30631652156475 - 219.03126942329948) / 231.30631652156475 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.Tests.DictionarySequentialKeys.TryGetValue_17_Int_Int

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 183.553793987343 < 187.96716051592762.
IsChangePoint: Marked as a change because one of 7/21/2022 11:11:59 AM, 9/28/2022 6:40:13 PM, 10/11/2022 6:23:40 AM, 10/22/2022 11:54:51 PM, 10/25/2022 2:56:59 AM falls between 10/16/2022 1:15:28 PM and 10/25/2022 2:56:59 AM.
IsImprovementStdDev: Marked as improvement because 10.256908084611794 (T) = (0 -183.06489533154442) / Math.Sqrt((34.30509291324227 / (299)) + (6.896705039165745 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.05926313604326373 = (194.59734421544198 - 183.06489533154442) / 194.59734421544198 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline ebf4ea7601f82cdc3f7cb9e7539a84e423d0d23a
Compare 33cb335386d39da8e4e89e75449628be7e3b375a
Diff Diff

Improvements in System.Buffers.Text.Tests.Utf8FormatterTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
FormatterDateTimeOffsetNow - Duration of single invocation 518.95 ns 483.25 ns 0.93 0.08 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Buffers.Text.Tests.Utf8FormatterTests*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Buffers.Text.Tests.Utf8FormatterTests.FormatterDateTimeOffsetNow(value: 12/30/2017 3:45:22 AM -08:00)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 483.25156090497836 < 499.5556050486842.
IsChangePoint: Marked as a change because one of 9/29/2022 2:21:18 AM, 10/11/2022 6:23:40 AM, 10/22/2022 11:54:51 PM, 10/25/2022 2:56:59 AM falls between 10/16/2022 1:15:28 PM and 10/25/2022 2:56:59 AM.
IsImprovementStdDev: Marked as improvement because 30.163078768978032 (T) = (0 -478.345428312833) / Math.Sqrt((467.9627202201854 / (299)) + (39.51624279527948 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.1525633681775468 = (564.461589634293 - 478.345428312833) / 564.461589634293 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline ebf4ea7601f82cdc3f7cb9e7539a84e423d0d23a
Compare 33cb335386d39da8e4e89e75449628be7e3b375a
Diff Diff

Improvements in System.Collections.CtorFromCollection<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
SortedList - Duration of single invocation 271.84 μs 243.14 μs 0.89 0.01 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.CtorFromCollection&lt;Int32&gt;*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Collections.CtorFromCollection<Int32>.SortedList(Size: 512)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 243.144380124328 < 255.244941920753.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/22/2022 11:54:51 PM, 10/25/2022 2:56:59 AM falls between 10/16/2022 1:15:28 PM and 10/25/2022 2:56:59 AM.
IsImprovementStdDev: Marked as improvement because 33.986092662934595 (T) = (0 -241736.30963667252) / Math.Sqrt((20011585.77271071 / (299)) + (4134090.6403472675 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.10892292265197119 = (271285.5215130367 - 241736.30963667252) / 271285.5215130367 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline ebf4ea7601f82cdc3f7cb9e7539a84e423d0d23a
Compare 33cb335386d39da8e4e89e75449628be7e3b375a
Diff Diff

Improvements in Span.Sorting

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
QuickSortSpan - Duration of single invocation 98.93 μs 89.35 μs 0.90 0.02 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'Span.Sorting*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

Span.Sorting.QuickSortSpan(Size: 512)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 89.35157146666666 < 94.28712975333332.
IsChangePoint: Marked as a change because one of 9/26/2022 10:49:59 PM, 10/22/2022 11:54:51 PM, 10/25/2022 2:56:59 AM falls between 10/16/2022 1:15:28 PM and 10/25/2022 2:56:59 AM.
IsImprovementStdDev: Marked as improvement because 35.140111218435834 (T) = (0 -88659.10258888888) / Math.Sqrt((2994170.936687426 / (299)) + (447559.67899474816 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.10337048807934622 = (98880.41985030565 - 88659.10258888888) / 98880.41985030565 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@kotlarmilos
Copy link
Member

Improvements in dotnet/runtime#77331

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
arch-x64 branch-refs/heads/main kind-micro_mono mono-interpreter os-linux perf-improvement runtime-mono untriaged
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@kotlarmilos