Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Perf] Linux/x64: 332 Improvements on 12/18/2022 12:43:31 PM #11155

Closed
performanceautofiler bot opened this issue Dec 27, 2022 · 17 comments
Closed

[Perf] Linux/x64: 332 Improvements on 12/18/2022 12:43:31 PM #11155

performanceautofiler bot opened this issue Dec 27, 2022 · 17 comments
Labels
arch-x64 branch-refs/heads/main kind-micro_mono mono-interpreter os-linux perf-improvement runtime-mono untriaged

Comments

@performanceautofiler
Copy link

No description provided.

@performanceautofiler
Copy link
Author

performanceautofiler bot commented Dec 27, 2022 •
edited
Loading

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 01b57ccc4cc532034f717226f4c7cc07a1ee9518
Compare fc65281072f9259f0b03a291b9c549bc9d792752
Diff Diff

Improvements in System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int16>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
EqualsAnyBenchmark - Duration of single invocation 134.33 ns 115.13 ns 0.86 0.16 False
InequalityOperatorBenchmark - Duration of single invocation 29.53 ns 24.61 ns 0.83 0.21 False
GreaterThanOrEqualAnyBenchmark - Duration of single invocation 30.27 ns 26.14 ns 0.86 0.28 False
GreaterThanAnyBenchmark - Duration of single invocation 28.99 ns 25.90 ns 0.89 0.23 False
GreaterThanOrEqualAllBenchmark - Duration of single invocation 139.96 ns 129.17 ns 0.92 0.12 False
EqualityOperatorBenchmark - Duration of single invocation 28.86 ns 24.76 ns 0.86 0.18 False
LessThanAllBenchmark - Duration of single invocation 31.58 ns 25.82 ns 0.82 0.28 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int16&gt;*'

Payloads

Baseline
Compare

Histogram

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int16>.EqualsAnyBenchmark

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 115.13308181703466 < 127.50337171058949.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 17.874129037668034 (T) = (0 -118.4177232047602) / Math.Sqrt((35.91148973094871 / (166)) + (39.668676962773425 / (33))) is greater than 1.9720790337782532 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (166) + (33) - 2, .975) and 0.15237498190558715 = (139.70531859828873 - 118.4177232047602) / 139.70531859828873 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int16&gt;.InequalityOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 24.614356320566852 < 27.65055708138445.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 12.552226318027495 (T) = (0 -25.270424987870232) / Math.Sqrt((1.8453545707910854 / (166)) + (3.564427105832992 / (33))) is greater than 1.9720790337782532 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (166) + (33) - 2, .975) and 0.1463514553081182 = (29.60284433799557 - 25.270424987870232) / 29.60284433799557 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int16&gt;.GreaterThanOrEqualAnyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 26.14014572357284 < 29.083597748543458.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 18.606296815690243 (T) = (0 -26.6999498845949) / Math.Sqrt((3.1003052259698913 / (166)) + (0.6805677818367528 / (33))) is greater than 1.9720790337782532 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (166) + (33) - 2, .975) and 0.12137966612512026 = (30.388495297898622 - 26.6999498845949) / 30.388495297898622 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int16&gt;.GreaterThanAnyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 25.89501230445821 < 27.666973174324873.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 20.839523655380734 (T) = (0 -25.989577781142383) / Math.Sqrt((2.6638688483650643 / (165)) + (0.5560243088641825 / (33))) is greater than 1.9721412216619414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (165) + (33) - 2, .975) and 0.12713173414730397 = (29.774914265846785 - 25.989577781142383) / 29.774914265846785 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int16&gt;.GreaterThanOrEqualAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 129.1665550574493 < 132.40922494206922.
IsChangePoint: Marked as a change because one of 11/22/2022 11:24:47 AM, 12/1/2022 11:48:13 PM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 30.286404515991975 (T) = (0 -129.54527308472075) / Math.Sqrt((37.10235558852038 / (166)) + (0.825331032971406 / (33))) is greater than 1.9720790337782532 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (166) + (33) - 2, .975) and 0.10438252627693813 = (144.64353017388544 - 129.54527308472075) / 144.64353017388544 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int16&gt;.EqualityOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 24.755217062009162 < 27.325112421559616.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 28.423694414647596 (T) = (0 -25.28120764686905) / Math.Sqrt((1.3571716207182323 / (166)) + (0.3955631735082628 / (33))) is greater than 1.9720790337782532 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (166) + (33) - 2, .975) and 0.13766700242751156 = (29.31722167426846 - 25.28120764686905) / 29.31722167426846 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int16&gt;.LessThanAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 25.819431331111684 < 27.9584772739223.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 20.221078799532396 (T) = (0 -26.034430487676367) / Math.Sqrt((3.2988838599751142 / (166)) + (0.3876201440239042 / (33))) is greater than 1.9720790337782532 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (166) + (33) - 2, .975) and 0.12135134334180252 = (29.6300805679192 - 26.034430487676367) / 29.6300805679192 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS ubuntu 18.04
Baseline 01b57ccc4cc532034f717226f4c7cc07a1ee9518
Compare fc65281072f9259f0b03a291b9c549bc9d792752
Diff Diff

Improvements in System.Numerics.Tests.Perf_VectorOf<SByte>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
EqualsAllBenchmark - Duration of single invocation 19.93 ns 18.46 ns 0.93 0.26 False
EqualityOperatorBenchmark - Duration of single invocation 19.97 ns 18.47 ns 0.92 0.31 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;SByte&gt;*'

Payloads

Baseline
Compare

Histogram

System.Numerics.Tests.Perf_VectorOf<SByte>.EqualsAllBenchmark

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 18.46195507538925 < 18.933594851074147.
IsChangePoint: Marked as a change because one of 11/28/2022 4:21:11 AM, 11/30/2022 12:23:56 PM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 10.744674689019003 (T) = (0 -18.61736112706463) / Math.Sqrt((2.799344410104408 / (288)) + (0.890277487589132 / (33))) is greater than 1.9674283869023508 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (288) + (33) - 2, .975) and 0.09955309390226014 = (20.675690039012462 - 18.61736112706463) / 20.675690039012462 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;SByte&gt;.EqualityOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 18.46849041764006 < 19.500536049918008.
IsChangePoint: Marked as a change because one of 11/22/2022 2:20:42 AM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 17.77032052416074 (T) = (0 -18.392579576837196) / Math.Sqrt((1.616744480322363 / (299)) + (0.35452067524244185 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.10935676444363976 = (20.650894592319965 - 18.392579576837196) / 20.650894592319965 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 01b57ccc4cc532034f717226f4c7cc07a1ee9518
Compare fc65281072f9259f0b03a291b9c549bc9d792752
Diff Diff

Improvements in System.Numerics.Tests.Perf_VectorOf<UInt64>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
EqualsAllBenchmark - Duration of single invocation 21.27 ns 19.17 ns 0.90 0.28 False
EqualsStaticBenchmark - Duration of single invocation 46.16 ns 41.84 ns 0.91 0.21 False
DivisionOperatorBenchmark - Duration of single invocation 51.67 ns 47.93 ns 0.93 0.15 False
MaxBenchmark - Duration of single invocation 51.80 ns 47.32 ns 0.91 0.22 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;UInt64&gt;*'

Payloads

Baseline
Compare

Histogram

System.Numerics.Tests.Perf_VectorOf<UInt64>.EqualsAllBenchmark

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 19.16517525287567 < 20.6186324036669.
IsChangePoint: Marked as a change because one of 10/15/2022 9:29:34 PM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 13.350561074869226 (T) = (0 -19.438516901144297) / Math.Sqrt((2.61414937920987 / (299)) + (1.305150298040133 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.13113817251177198 = (22.372391427689536 - 19.438516901144297) / 22.372391427689536 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt64&gt;.EqualsStaticBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 41.83556818351054 < 43.84899026357486.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 12.34608194308043 (T) = (0 -42.4658714379843) / Math.Sqrt((5.118600727070588 / (299)) + (5.20098604029684 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.1083570673049548 = (47.6265440804074 - 42.4658714379843) / 47.6265440804074 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt64&gt;.DivisionOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 47.92836627630024 < 49.270826326728915.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 12/18/2022 12:43:31 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 16.976082275843982 (T) = (0 -48.36598208173312) / Math.Sqrt((5.099793216395001 / (299)) + (1.2778892925429848 / (32))) is greater than 1.9672006833568965 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (32) - 2, .975) and 0.07731290907251337 = (52.41861792291421 - 48.36598208173312) / 52.41861792291421 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt64&gt;.MaxBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 47.31984836155486 < 50.093453409211456.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 12.450187377563115 (T) = (0 -47.99088198937359) / Math.Sqrt((8.27860461863973 / (299)) + (5.904098933295801 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.10548042683784811 = (53.64989590973898 - 47.99088198937359) / 53.64989590973898 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 01b57ccc4cc532034f717226f4c7cc07a1ee9518
Compare fc65281072f9259f0b03a291b9c549bc9d792752
Diff Diff

Improvements in System.Collections.IterateForEach<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ImmutableQueue - Duration of single invocation 36.02 μs 31.64 μs 0.88 0.14 False
ImmutableStack - Duration of single invocation 35.31 μs 30.78 μs 0.87 0.12 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.IterateForEach&lt;String&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.IterateForEach<String>.ImmutableQueue(Size: 512)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 31.639736887254905 < 34.22651470656127.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 12/2/2022 5:21:30 AM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 15.969253501522576 (T) = (0 -31336.079953528057) / Math.Sqrt((6662711.624789523 / (299)) + (2563437.062847697 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.13876549922003742 = (36385.07273587979 - 31336.079953528057) / 36385.07273587979 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.IterateForEach&lt;String&gt;.ImmutableStack(Size: 512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 30.77528947238659 < 33.518946476571664.
IsChangePoint: Marked as a change because one of 11/15/2022 5:29:43 PM, 12/2/2022 5:21:30 AM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 25.82853954747722 (T) = (0 -31131.199879023465) / Math.Sqrt((5969052.431533905 / (299)) + (478934.29790086584 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.13348752703544914 = (35927.00722762365 - 31131.199879023465) / 35927.00722762365 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented Dec 27, 2022 •
edited
Loading

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 01b57ccc4cc532034f717226f4c7cc07a1ee9518
Compare fc65281072f9259f0b03a291b9c549bc9d792752
Diff Diff

Improvements in System.Collections.ContainsFalse<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
List - Duration of single invocation 2.23 ms 724.35 μs 0.32 0.06 False
Span - Duration of single invocation 1.51 ms 691.27 μs 0.46 0.07 False
ICollection - Duration of single invocation 2.19 ms 727.96 μs 0.33 0.02 False
Queue - Duration of single invocation 2.20 ms 746.86 μs 0.34 0.06 False
ImmutableArray - Duration of single invocation 2.21 ms 755.63 μs 0.34 0.05 False
Array - Duration of single invocation 2.16 ms 733.51 μs 0.34 0.04 False
Stack - Duration of single invocation 12.00 ms 730.52 μs 0.06 0.07 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.ContainsFalse&lt;Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.ContainsFalse<Int32>.List(Size: 512)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 724.350807426948 < 2.120391598013393.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 388.33014016574094 (T) = (0 -725999.0682493447) / Math.Sqrt((3689195880.0178924 / (299)) + (22194719.651611485 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.6586323135887735 = (2126736.35247003 - 725999.0682493447) / 2126736.35247003 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.ContainsFalse&lt;Int32&gt;.Span(Size: 512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 691.2707575698759 < 1.433100928351799.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 203.42837901669492 (T) = (0 -689979.6287102987) / Math.Sqrt((3708642779.746599 / (299)) + (93104607.96199411 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.5349718166523102 = (1483737.230167872 - 689979.6287102987) / 1483737.230167872 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.ContainsFalse&lt;Int32&gt;.ICollection(Size: 512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 727.9580801445579 < 2.079729056321615.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 490.8207862124228 (T) = (0 -727327.5297545796) / Math.Sqrt((2212385301.522625 / (299)) + (22408973.974072833 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.6573023360942181 = (2122359.1706611225 - 727327.5297545796) / 2122359.1706611225 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.ContainsFalse&lt;Int32&gt;.Queue(Size: 512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 746.8573995535713 < 2.092275113394002.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 440.5653366846799 (T) = (0 -746790.4021944279) / Math.Sqrt((2852594855.4130898 / (299)) + (17381291.243752997 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.6517892471679846 = (2144650.6063375245 - 746790.4021944279) / 2144650.6063375245 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.ContainsFalse&lt;Int32&gt;.ImmutableArray(Size: 512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 755.628939781746 < 2.0995207948214283.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 471.3821051987532 (T) = (0 -755376.872406302) / Math.Sqrt((2422167733.153595 / (299)) + (23621215.426022496 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.6494848280074306 = (2155047.5778615233 - 755376.872406302) / 2155047.5778615233 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.ContainsFalse&lt;Int32&gt;.Array(Size: 512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 733.5122059948978 < 2.077206547050781.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 370.1352493475196 (T) = (0 -734244.2189870985) / Math.Sqrt((2819714428.12922 / (299)) + (159745016.46252102 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.6556909661690302 = (2132515.115323863 - 734244.2189870985) / 2132515.115323863 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.ContainsFalse&lt;Int32&gt;.Stack(Size: 512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 730.5183104395605 < 11.472601066625172.
IsChangePoint: Marked as a change because one of 12/14/2022 5:20:21 PM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 16.57043577522249 (T) = (0 -731310.3059560104) / Math.Sqrt((5577882885722.341 / (299)) + (2424072.66776418 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.7557876603803614 = (2994567.3797443165 - 731310.3059560104) / 2994567.3797443165 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS ubuntu 18.04
Baseline 01b57ccc4cc532034f717226f4c7cc07a1ee9518
Compare fc65281072f9259f0b03a291b9c549bc9d792752
Diff Diff

Improvements in System.Text.Json.Serialization.Tests.WriteJson<Dictionary<String, String>>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
SerializeToUtf8Bytes - Duration of single invocation 215.55 μs 196.93 μs 0.91 0.02 False
SerializeToUtf8Bytes - Duration of single invocation 233.37 μs 217.40 μs 0.93 0.03 False
SerializeToStream - Duration of single invocation 220.14 μs 200.08 μs 0.91 0.04 False
SerializeToStream - Duration of single invocation 258.45 μs 234.13 μs 0.91 0.03 False
SerializeToWriter - Duration of single invocation 234.00 μs 213.73 μs 0.91 0.04 False
SerializeToWriter - Duration of single invocation 215.27 μs 193.00 μs 0.90 0.02 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.WriteJson&lt;Dictionary&lt;String, String&gt;&gt;*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Serialization.Tests.WriteJson<Dictionary<String, String>>.SerializeToUtf8Bytes(Mode: SourceGen)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 196.92777033333334 < 204.8264796727617.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 41.08937820930262 (T) = (0 -194341.9068079344) / Math.Sqrt((20381241.026845727 / (112)) + (6281851.715088462 / (33))) is greater than 1.9766921979294603 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (112) + (33) - 2, .975) and 0.11426973944346215 = (219414.32449854657 - 194341.9068079344) / 219414.32449854657 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Serialization.Tests.WriteJson&lt;Dictionary&lt;String, String&gt;&gt;.SerializeToUtf8Bytes(Mode: Reflection)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 217.3981309086134 < 220.12826149954316.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 32.42664654060329 (T) = (0 -213784.7585798594) / Math.Sqrt((23959471.9998858 / (112)) + (10801545.812688336 / (33))) is greater than 1.9766921979294603 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (112) + (33) - 2, .975) and 0.1003869714059001 = (237640.79863756377 - 213784.7585798594) / 237640.79863756377 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Serialization.Tests.WriteJson&lt;Dictionary&lt;String, String&gt;&gt;.SerializeToStream(Mode: SourceGen)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 200.07619295886076 < 209.2897025079945.
IsChangePoint: Marked as a change because one of 12/5/2022 3:14:07 PM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 23.80915064126381 (T) = (0 -197308.60619284472) / Math.Sqrt((247550508.04316247 / (112)) + (8355816.463830365 / (33))) is greater than 1.9766921979294603 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (112) + (33) - 2, .975) and 0.1592375876941757 = (234678.19600987874 - 197308.60619284472) / 234678.19600987874 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Serialization.Tests.WriteJson&lt;Dictionary&lt;String, String&gt;&gt;.SerializeToStream(Mode: Reflection)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 234.1323986213235 < 235.80026176374716.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 26.416015507362758 (T) = (0 -230384.25827255528) / Math.Sqrt((33789191.55039915 / (112)) + (15755383.87600708 / (33))) is greater than 1.9766921979294603 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (112) + (33) - 2, .975) and 0.09190696945970256 = (253701.1633438935 - 230384.25827255528) / 253701.1633438935 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Serialization.Tests.WriteJson&lt;Dictionary&lt;String, String&gt;&gt;.SerializeToWriter(Mode: Reflection)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 213.72672648401826 < 220.23012428274689.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 25.80457819596711 (T) = (0 -212926.57316209367) / Math.Sqrt((25099769.5434832 / (112)) + (21730436.865361195 / (33))) is greater than 1.9766921979294603 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (112) + (33) - 2, .975) and 0.10221655389123407 = (237169.19050465288 - 212926.57316209367) / 237169.19050465288 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Serialization.Tests.WriteJson&lt;Dictionary&lt;String, String&gt;&gt;.SerializeToWriter(Mode: SourceGen)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 193.0025108281893 < 201.7841108098118.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 39.45472436322164 (T) = (0 -192674.0390782008) / Math.Sqrt((18492690.269205295 / (112)) + (6966054.774458456 / (33))) is greater than 1.9766921979294603 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (112) + (33) - 2, .975) and 0.11158469490668667 = (216873.84039153127 - 192674.0390782008) / 216873.84039153127 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 01b57ccc4cc532034f717226f4c7cc07a1ee9518
Compare fc65281072f9259f0b03a291b9c549bc9d792752
Diff Diff

Improvements in System.Text.Json.Serialization.Tests.WriteJson<ImmutableSortedDictionary<String, String>>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
SerializeToWriter - Duration of single invocation 246.64 μs 218.10 μs 0.88 0.05 False
SerializeToWriter - Duration of single invocation 279.50 μs 265.06 μs 0.95 0.01 False
SerializeToStream - Duration of single invocation 245.91 μs 224.33 μs 0.91 0.03 False
SerializeToStream - Duration of single invocation 290.45 μs 273.00 μs 0.94 0.01 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.WriteJson&lt;ImmutableSortedDictionary&lt;String, String&gt;&gt;*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Serialization.Tests.WriteJson<ImmutableSortedDictionary<String, String>>.SerializeToWriter(Mode: SourceGen)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 218.0985732266865 < 228.31188539741507.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 40.75241572633285 (T) = (0 -217556.58751800406) / Math.Sqrt((26387920.979699865 / (112)) + (7150055.703008524 / (33))) is greater than 1.9766921979294603 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (112) + (33) - 2, .975) and 0.11188034404433447 = (244963.1488945048 - 217556.58751800406) / 244963.1488945048 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Serialization.Tests.WriteJson&lt;ImmutableSortedDictionary&lt;String, String&gt;&gt;.SerializeToWriter(Mode: Reflection)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 265.06382425595234 < 265.7118985675754.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 34.10719556582198 (T) = (0 -261854.36676108424) / Math.Sqrt((35843643.8342184 / (112)) + (10673981.915177567 / (33))) is greater than 1.9766921979294603 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (112) + (33) - 2, .975) and 0.0946010403615369 = (289214.3446527108 - 261854.36676108424) / 289214.3446527108 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Serialization.Tests.WriteJson&lt;ImmutableSortedDictionary&lt;String, String&gt;&gt;.SerializeToStream(Mode: SourceGen)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 224.33031269409935 < 233.35597286946614.
IsChangePoint: Marked as a change because one of 12/5/2022 3:14:07 PM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 19.247533323088117 (T) = (0 -223814.61012044537) / Math.Sqrt((520655643.29419434 / (112)) + (15852859.856325397 / (33))) is greater than 1.9766921979294603 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (112) + (33) - 2, .975) and 0.16301417394514434 = (267405.4961902983 - 223814.61012044537) / 267405.4961902983 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Serialization.Tests.WriteJson&lt;ImmutableSortedDictionary&lt;String, String&gt;&gt;.SerializeToStream(Mode: Reflection)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 272.99548028017244 < 277.82556733926197.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 29.63521342523715 (T) = (0 -267248.9485651556) / Math.Sqrt((29069365.645766463 / (112)) + (24385765.315966908 / (33))) is greater than 1.9766921979294603 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (112) + (33) - 2, .975) and 0.09975384524345775 = (296862.08283492085 - 267248.9485651556) / 296862.08283492085 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented Dec 27, 2022 •
edited
Loading

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 01b57ccc4cc532034f717226f4c7cc07a1ee9518
Compare fc65281072f9259f0b03a291b9c549bc9d792752
Diff Diff

Improvements in System.Memory.Span<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
LastIndexOfAnyValues - Duration of single invocation 328.52 ns 274.06 ns 0.83 0.04 False
IndexOfAnyThreeValues - Duration of single invocation 393.48 ns 334.47 ns 0.85 0.10 False
Fill - Duration of single invocation 1.75 μs 1.32 μs 0.76 0.17 False
IndexOfAnyThreeValues - Duration of single invocation 110.76 ns 102.20 ns 0.92 0.14 False
IndexOfValue - Duration of single invocation 2.15 μs 689.44 ns 0.32 0.08 False
LastIndexOfValue - Duration of single invocation 11.85 μs 683.97 ns 0.06 0.04 False
IndexOfAnyTwoValues - Duration of single invocation 320.14 ns 272.08 ns 0.85 0.04 False
IndexOfValue - Duration of single invocation 162.38 ns 72.33 ns 0.45 0.07 False
IndexOfValue - Duration of single invocation 54.14 ns 35.31 ns 0.65 0.24 False
SequenceEqual - Duration of single invocation 62.69 ns 53.40 ns 0.85 0.23 False
Fill - Duration of single invocation 142.34 ns 120.42 ns 0.85 0.25 False
IndexOfAnyTwoValues - Duration of single invocation 3.89 μs 3.18 μs 0.82 0.07 False
LastIndexOfValue - Duration of single invocation 163.26 ns 33.35 ns 0.20 0.15 False
LastIndexOfAnyValues - Duration of single invocation 4.02 μs 3.19 μs 0.79 0.11 False
LastIndexOfValue - Duration of single invocation 856.52 ns 74.64 ns 0.09 0.09 False
IndexOfAnyThreeValues - Duration of single invocation 5.11 μs 4.09 μs 0.80 0.11 False
IndexOfAnyTwoValues - Duration of single invocation 110.48 ns 91.88 ns 0.83 0.18 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Memory.Span&lt;Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Memory.Span<Int32>.LastIndexOfAnyValues(Size: 33)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 274.05716198527824 < 310.7029208840075.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 45.37348040313501 (T) = (0 -275.26215116696096) / Math.Sqrt((42.1859312519928 / (70)) + (10.957383870110391 / (33))) is greater than 1.9837310029553383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (70) + (33) - 2, .975) and 0.13745841051423277 = (319.1291347830168 - 275.26215116696096) / 319.1291347830168 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Span&lt;Int32&gt;.IndexOfAnyThreeValues(Size: 33)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 334.46746202165906 < 376.62725640407365.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 35.58217813931166 (T) = (0 -336.01804566633604) / Math.Sqrt((92.70501289030433 / (70)) + (49.26291820343771 / (33))) is greater than 1.9837310029553383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (70) + (33) - 2, .975) and 0.15091365742613266 = (395.74072602292944 - 336.01804566633604) / 395.74072602292944 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Span&lt;Int32&gt;.Fill(Size: 512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.321068254441228 < 1.6587747516433844.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 34.58138984725709 (T) = (0 -1340.0621866948647) / Math.Sqrt((17827.497587562273 / (299)) + (2316.0870121958505 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.2272110490668969 = (1734.0597132979299 - 1340.0621866948647) / 1734.0597132979299 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Span&lt;Int32&gt;.IndexOfAnyThreeValues(Size: 4)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 102.20261933470502 < 105.96415439260278.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 20.939987549372727 (T) = (0 -102.62231495627059) / Math.Sqrt((13.195857103717142 / (70)) + (4.513165200355102 / (33))) is greater than 1.9837310029553383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (70) + (33) - 2, .975) and 0.1042436768180379 = (114.56499083559815 - 102.62231495627059) / 114.56499083559815 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Span&lt;Int32&gt;.IndexOfValue(Size: 512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 689.4382336801809 < 2.0463305282522044.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 313.807657354245 (T) = (0 -693.5687184579529) / Math.Sqrt((2541.7673134056968 / (288)) + (376.22411681492474 / (33))) is greater than 1.9674283869023508 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (288) + (33) - 2, .975) and 0.6704944167930297 = (2104.876984807586 - 693.5687184579529) / 2104.876984807586 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Span&lt;Int32&gt;.LastIndexOfValue(Size: 512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 683.9661445774165 < 11.207191404279056.
IsChangePoint: Marked as a change because one of 12/14/2022 5:20:21 PM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 16.54925481390424 (T) = (0 -685.6787939659382) / Math.Sqrt((5522694.692220391 / (290)) + (19.908245294489102 / (33))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (290) + (33) - 2, .975) and 0.7690927392828526 = (2969.498628308048 - 685.6787939659382) / 2969.498628308048 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Span&lt;Int32&gt;.IndexOfAnyTwoValues(Size: 33)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 272.07756011950585 < 306.0057749661562.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 55.5791335200384 (T) = (0 -273.37191961078486) / Math.Sqrt((27.984672300800316 / (70)) + (9.793583948539238 / (33))) is greater than 1.9837310029553383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (70) + (33) - 2, .975) and 0.14506683838094064 = (319.75823594569346 - 273.37191961078486) / 319.75823594569346 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Span&lt;Int32&gt;.IndexOfValue(Size: 33)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 72.33228129853651 < 154.05138049418787.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 144.08151358103382 (T) = (0 -72.62822474781315) / Math.Sqrt((23.42041266375031 / (70)) + (1.319018649103449 / (33))) is greater than 1.9837310029553383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (70) + (33) - 2, .975) and 0.5483502857806413 = (160.80653316330637 - 72.62822474781315) / 160.80653316330637 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Span&lt;Int32&gt;.IndexOfValue(Size: 4)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 35.31061349125307 < 48.366768060338295.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 37.67543295130887 (T) = (0 -35.86783620639303) / Math.Sqrt((7.950889932344679 / (70)) + (2.5381890504085183 / (33))) is greater than 1.9837310029553383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (70) + (33) - 2, .975) and 0.3143442410908042 = (52.31172602335446 - 35.86783620639303) / 52.31172602335446 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Span&lt;Int32&gt;.SequenceEqual(Size: 4)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 53.39547212554892 < 58.531961095646885.
IsChangePoint: Marked as a change because one of 12/12/2022 1:34:22 PM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 9.340653869160917 (T) = (0 -54.388348648792366) / Math.Sqrt((13.150257507653794 / (70)) + (5.7078154504082335 / (33))) is greater than 1.9837310029553383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (70) + (33) - 2, .975) and 0.09351480786166264 = (59.999158420331085 - 54.388348648792366) / 59.999158420331085 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Span&lt;Int32&gt;.Fill(Size: 33)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 120.41578119009357 < 137.08315811158815.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 19.991862085799607 (T) = (0 -119.6088872279557) / Math.Sqrt((44.90680931046178 / (70)) + (15.010051767772216 / (33))) is greater than 1.9837310029553383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (70) + (33) - 2, .975) and 0.14894536087275517 = (140.5419602090583 - 119.6088872279557) / 140.5419602090583 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Span&lt;Int32&gt;.IndexOfAnyTwoValues(Size: 512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 3.1809899952342042 < 3.7002880761707764.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 71.12794654366971 (T) = (0 -3192.1666354035196) / Math.Sqrt((20789.74978249397 / (282)) + (1256.6543861335501 / (33))) is greater than 1.9675720194858106 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (282) + (33) - 2, .975) and 0.19067875045415514 = (3944.251602432065 - 3192.1666354035196) / 3944.251602432065 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Span&lt;Int32&gt;.LastIndexOfValue(Size: 4)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 33.35090573852355 < 158.60769595665207.
IsChangePoint: Marked as a change because one of 12/14/2022 5:20:21 PM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 17.879889229796273 (T) = (0 -34.01862688949728) / Math.Sqrt((1221.578200561554 / (70)) + (1.7184910895503034 / (33))) is greater than 1.9837310029553383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (70) + (33) - 2, .975) and 0.6873931354036453 = (108.82239241106538 - 34.01862688949728) / 108.82239241106538 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Span&lt;Int32&gt;.LastIndexOfAnyValues(Size: 512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 3.19197168485971 < 3.700062887209513.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 12/2/2022 5:21:30 AM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 45.22140925149635 (T) = (0 -3210.489525294022) / Math.Sqrt((21601.38129005949 / (299)) + (4335.951982217902 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.16736248684786156 = (3855.8069683168424 - 3210.489525294022) / 3855.8069683168424 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Span&lt;Int32&gt;.LastIndexOfValue(Size: 33)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 74.64014684298783 < 813.0358271632322.
IsChangePoint: Marked as a change because one of 12/14/2022 5:20:21 PM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 9.376823798340324 (T) = (0 -73.7797852164986) / Math.Sqrt((82415.55365570127 / (70)) + (2.6572685723981206 / (33))) is greater than 1.9837310029553383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (70) + (33) - 2, .975) and 0.8134685810145225 = (395.53543106988695 - 73.7797852164986) / 395.53543106988695 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Span&lt;Int32&gt;.IndexOfAnyThreeValues(Size: 512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 4.0918223438047665 < 4.8539258549639674.
IsChangePoint: Marked as a change because one of 10/11/2022 1:45:10 PM, 10/12/2022 11:17:15 AM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 44.32487138285212 (T) = (0 -4161.1412860326445) / Math.Sqrt((50475.83338792725 / (289)) + (9950.482607517644 / (33))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (289) + (33) - 2, .975) and 0.18860596909912694 = (5128.385380667171 - 4161.1412860326445) / 5128.385380667171 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Span&lt;Int32&gt;.IndexOfAnyTwoValues(Size: 4)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 91.88359084788789 < 97.1325431835849.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 16.457582088006568 (T) = (0 -93.71329967218074) / Math.Sqrt((8.523659430766655 / (70)) + (6.175811161780361 / (33))) is greater than 1.9837310029553383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (70) + (33) - 2, .975) and 0.08892736845144113 = (102.86040478781077 - 93.71329967218074) / 102.86040478781077 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS ubuntu 18.04
Baseline 01b57ccc4cc532034f717226f4c7cc07a1ee9518
Compare fc65281072f9259f0b03a291b9c549bc9d792752
Diff Diff

Improvements in System.Text.Encodings.Web.Tests.Perf_Encoders

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
EncodeUtf16 - Duration of single invocation 979.96 ns 833.85 ns 0.85 0.07 False
EncodeUtf16 - Duration of single invocation 865.30 ns 764.38 ns 0.88 0.04 False
EncodeUtf16 - Duration of single invocation 10.23 μs 8.24 μs 0.81 0.06 False
EncodeUtf16 - Duration of single invocation 10.38 μs 8.36 μs 0.80 0.05 False
EncodeUtf16 - Duration of single invocation 10.67 μs 8.88 μs 0.83 0.03 False
EncodeUtf16 - Duration of single invocation 10.85 μs 8.84 μs 0.81 0.02 False
EncodeUtf16 - Duration of single invocation 10.22 μs 8.32 μs 0.81 0.06 False
EncodeUtf16 - Duration of single invocation 946.89 ns 815.34 ns 0.86 0.06 False
EncodeUtf16 - Duration of single invocation 10.11 μs 8.17 μs 0.81 0.04 False
EncodeUtf8 - Duration of single invocation 2.49 μs 2.24 μs 0.90 0.14 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Encodings.Web.Tests.Perf_Encoders*'

Payloads

Baseline
Compare

Histogram

System.Text.Encodings.Web.Tests.Perf_Encoders.EncodeUtf16(arguments: UnsafeRelaxed,hello "there",16)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 833.8479173376995 < 918.4271131469633.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/3/2022 2:55:34 AM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 42.77577337460166 (T) = (0 -845.5057686018263) / Math.Sqrt((3941.893682159887 / (299)) + (452.91480107499825 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.20788096681738202 = (1067.3973647681564 - 845.5057686018263) / 1067.3973647681564 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Encodings.Web.Tests.Perf_Encoders.EncodeUtf16(arguments: JavaScript,no escaping required,16)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 764.380357790527 < 817.0539519818118.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/3/2022 2:55:34 AM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 41.80451083092956 (T) = (0 -764.090795388209) / Math.Sqrt((3312.9583955659555 / (299)) + (140.30638166796072 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.17643112353446347 = (927.7800767161258 - 764.090795388209) / 927.7800767161258 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Encodings.Web.Tests.Perf_Encoders.EncodeUtf16(arguments: UnsafeRelaxed,no &lt;escaping /&gt; required,512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 8.237281500770042 < 9.878237148925603.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 74.80553924350995 (T) = (0 -8266.51887658673) / Math.Sqrt((288831.96569551295 / (299)) + (5227.356205889884 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.23279865644552233 = (10774.901459749253 - 8266.51887658673) / 10774.901459749253 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Encodings.Web.Tests.Perf_Encoders.EncodeUtf16(arguments: Url,�2020,512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 8.355048413708012 < 9.965143668324316.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 71.19751441832605 (T) = (0 -8424.159411607841) / Math.Sqrt((274794.756746676 / (299)) + (8275.395351699928 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.22424450678868194 = (10859.29714365951 - 8424.159411607841) / 10859.29714365951 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Encodings.Web.Tests.Perf_Encoders.EncodeUtf16(arguments: JavaScript,&Hello+&lt;World&gt;!,512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 8.87583967629672 < 10.105033967794922.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 76.88681755263208 (T) = (0 -8732.871958986376) / Math.Sqrt((241266.63517020727 / (299)) + (8119.587697893494 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.22220978980892414 = (11227.79876188081 - 8732.871958986376) / 11227.79876188081 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Encodings.Web.Tests.Perf_Encoders.EncodeUtf16(arguments: Url,&lorem ipsum=dolor sit amet,512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 8.842063761040434 < 10.335737018524613.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 69.04776616257818 (T) = (0 -8925.083091637644) / Math.Sqrt((214145.9531667033 / (299)) + (21533.616241262567 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.22252724304135413 = (11479.60878597367 - 8925.083091637644) / 11479.60878597367 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Encodings.Web.Tests.Perf_Encoders.EncodeUtf16(arguments: UnsafeRelaxed,hello "there",512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 8.315751832356078 < 9.986930229888737.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 65.19139277303655 (T) = (0 -8329.130275498754) / Math.Sqrt((296209.1846426404 / (299)) + (15341.176782649354 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.22994581151367718 = (10816.291113059364 - 8329.130275498754) / 10816.291113059364 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Encodings.Web.Tests.Perf_Encoders.EncodeUtf16(arguments: UnsafeRelaxed,no &lt;escaping /&gt; required,16)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 815.3393512638692 < 889.9397890275237.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/3/2022 2:55:34 AM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 40.99434228795981 (T) = (0 -821.3034932926766) / Math.Sqrt((3853.369962023905 / (299)) + (232.82362110442202 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.18227271367610554 = (1004.3733491942761 - 821.3034932926766) / 1004.3733491942761 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Encodings.Web.Tests.Perf_Encoders.EncodeUtf16(arguments: JavaScript,no escaping required,512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 8.168968822064988 < 9.603620284697188.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 53.99941061162308 (T) = (0 -8254.130202693548) / Math.Sqrt((255511.3888719379 / (299)) + (39748.54369160098 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.2289067189508903 = (10704.450947184243 - 8254.130202693548) / 10704.450947184243 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Encodings.Web.Tests.Perf_Encoders.EncodeUtf8(arguments: Url,&lorem ipsum=dolor sit amet,16)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.2414072976074557 < 2.241841565602815.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 23.57872383286161 (T) = (0 -2162.6412428974995) / Math.Sqrt((30717.658942066224 / (284)) + (6165.61233644422 / (33))) is greater than 1.9675235316906419 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (284) + (33) - 2, .975) and 0.15772449925285928 = (2567.617413754915 - 2162.6412428974995) / 2567.617413754915 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 01b57ccc4cc532034f717226f4c7cc07a1ee9518
Compare fc65281072f9259f0b03a291b9c549bc9d792752
Diff Diff

Improvements in System.Runtime.Intrinsics.Tests.Perf_Vector128Float

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
LessThanOrEqualAllBenchmark - Duration of single invocation 31.38 ns 29.11 ns 0.93 0.21 False
GreaterThanOrEqualAllBenchmark - Duration of single invocation 31.68 ns 27.98 ns 0.88 0.24 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Float*'

Payloads

Baseline
Compare

Histogram

System.Runtime.Intrinsics.Tests.Perf_Vector128Float.LessThanOrEqualAllBenchmark

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 29.109817117097844 < 30.341929871057577.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 15.451533423936437 (T) = (0 -29.387268449996704) / Math.Sqrt((2.963235078178618 / (166)) + (0.3217174789816682 / (33))) is greater than 1.9720790337782532 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (166) + (33) - 2, .975) and 0.08033336957079545 = (31.954261987609346 - 29.387268449996704) / 31.954261987609346 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Float.GreaterThanOrEqualAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 27.975355270530212 < 30.058536000690818.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 7.559853213565697 (T) = (0 -29.955005895001133) / Math.Sqrt((1.7303529321928708 / (165)) + (1.7384548450233006 / (33))) is greater than 1.9721412216619414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (165) + (33) - 2, .975) and 0.05964608198253871 = (31.855033855928383 - 29.955005895001133) / 31.855033855928383 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented Dec 27, 2022 •
edited
Loading

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 01b57ccc4cc532034f717226f4c7cc07a1ee9518
Compare fc65281072f9259f0b03a291b9c549bc9d792752
Diff Diff

Improvements in System.Memory.SequenceReader

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
TryReadTo - Duration of single invocation 5.50 μs 2.01 μs 0.37 0.03 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Memory.SequenceReader*'

Payloads

Baseline
Compare

Histogram

System.Memory.SequenceReader.TryReadTo

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.0110055606460047 < 5.244237410122393.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 541.972837947119 (T) = (0 -2013.6764884294562) / Math.Sqrt((11220.52409378932 / (299)) + (37.261665828570344 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.6259425435625467 = (5383.334709078754 - 2013.6764884294562) / 5383.334709078754 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS ubuntu 18.04
Baseline 01b57ccc4cc532034f717226f4c7cc07a1ee9518
Compare fc65281072f9259f0b03a291b9c549bc9d792752
Diff Diff

Improvements in System.Globalization.Tests.StringEquality

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Compare_Same - Duration of single invocation 2.28 μs 1.95 μs 0.86 0.09 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Globalization.Tests.StringEquality*'

Payloads

Baseline
Compare

Histogram

System.Globalization.Tests.StringEquality.Compare_Same(Count: 1024, Options: (en-US, Ordinal))

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.9548500186635898 < 2.168841927999546.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 12/2/2022 5:21:30 AM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 58.66632003090082 (T) = (0 -1929.3844088384394) / Math.Sqrt((12382.777673856921 / (299)) + (96.18920357809955 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.1683630645012212 = (2319.9840296670814 - 1929.3844088384394) / 2319.9840296670814 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented Dec 27, 2022 •
edited
Loading

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 01b57ccc4cc532034f717226f4c7cc07a1ee9518
Compare fc65281072f9259f0b03a291b9c549bc9d792752
Diff Diff

Improvements in System.MathBenchmarks.Single

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
CopySign - Duration of single invocation 110.69 μs 99.04 μs 0.89 0.14 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.MathBenchmarks.Single*'

Payloads

Baseline
Compare

Histogram

System.MathBenchmarks.Single.CopySign

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 99.04271017932491 < 103.10372876555267.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 23.02843319665819 (T) = (0 -99790.52784671038) / Math.Sqrt((45841584.95697731 / (299)) + (2564872.706216804 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.09984694600562673 = (110859.51150628897 - 99790.52784671038) / 110859.51150628897 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS ubuntu 18.04
Baseline 01b57ccc4cc532034f717226f4c7cc07a1ee9518
Compare fc65281072f9259f0b03a291b9c549bc9d792752
Diff Diff

Improvements in System.Memory.ReadOnlySpan

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
IndexOfString - Duration of single invocation 1.63 μs 1.37 μs 0.84 0.04 False
IndexOfString - Duration of single invocation 223.24 ns 190.18 ns 0.85 0.10 False
IndexOfString - Duration of single invocation 640.56 ns 552.95 ns 0.86 0.05 False
IndexOfString - Duration of single invocation 229.23 ns 193.20 ns 0.84 0.13 False
IndexOfString - Duration of single invocation 1.63 μs 1.37 μs 0.84 0.03 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Memory.ReadOnlySpan*'

Payloads

Baseline
Compare

Histogram

System.Memory.ReadOnlySpan.IndexOfString(input: "AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAXAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA", value: "X", comparisonType: Ordinal)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.3745551064072046 < 1.547215464820379.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 77.16555697374295 (T) = (0 -1369.9291562607223) / Math.Sqrt((2958.2620523273436 / (299)) + (8.01032353705191 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.15206637296733666 = (1615.6089493169152 - 1369.9291562607223) / 1615.6089493169152 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.ReadOnlySpan.IndexOfString(input: "だだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだすだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだ", value: "す", comparisonType: Ordinal)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 190.17506953562673 < 213.08420962698222.
IsChangePoint: Marked as a change because one of 9/28/2022 9:30:44 PM, 10/11/2022 10:51:47 AM, 11/15/2022 5:29:43 PM, 12/2/2022 10:04:21 AM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 32.9935099082157 (T) = (0 -192.33848219004392) / Math.Sqrt((130.18854524649595 / (299)) + (13.607011873819365 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.13639829121208766 = (222.71665309694214 - 192.33848219004392) / 222.71665309694214 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.ReadOnlySpan.IndexOfString(input: "Hello WorldbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbareallyreallylongHello WorldbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbareallyreallylongHello Worldbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbareallyreallylong!xyz", value: "~", comparisonType: Ordinal)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 552.9502227761197 < 611.2327902454924.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 69.8037653506965 (T) = (0 -543.4984122919589) / Math.Sqrt((554.1198784664265 / (288)) + (11.497928463979243 / (33))) is greater than 1.9674283869023508 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (288) + (33) - 2, .975) and 0.16220537887560518 = (648.7251154257062 - 543.4984122919589) / 648.7251154257062 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.ReadOnlySpan.IndexOfString(input: "Hello Worldbbbbbbbbbbbbbbcbbbbbbbbbbbbbbbbbbba!", value: "y", comparisonType: Ordinal)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 193.19923627902838 < 206.86686539895936.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 40.13119860107363 (T) = (0 -188.83625130302994) / Math.Sqrt((118.56361461473327 / (289)) + (12.96194598644853 / (33))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (289) + (33) - 2, .975) and 0.1599767646392168 = (224.79884288191897 - 188.83625130302994) / 224.79884288191897 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.ReadOnlySpan.IndexOfString(input: "だだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだxだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだ", value: "x", comparisonType: Ordinal)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.3704946488398293 < 1.546324701649155.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 65.51937072199891 (T) = (0 -1372.4787289613737) / Math.Sqrt((1173.966687884611 / (299)) + (318.6256413435416 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.14960929798035727 = (1613.9390114470837 - 1372.4787289613737) / 1613.9390114470837 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 01b57ccc4cc532034f717226f4c7cc07a1ee9518
Compare fc65281072f9259f0b03a291b9c549bc9d792752
Diff Diff

Improvements in System.Text.Json.Tests.Perf_Strings

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
WriteStringsUtf16 - Duration of single invocation 209.25 ms 191.83 ms 0.92 0.03 False
WriteStringsUtf16 - Duration of single invocation 615.35 ms 578.57 ms 0.94 0.09 False
WriteStringsUtf16 - Duration of single invocation 215.11 ms 199.92 ms 0.93 0.03 False
WriteStringsUtf16 - Duration of single invocation 612.86 ms 556.90 ms 0.91 0.11 False
WriteStringsUtf16 - Duration of single invocation 604.03 ms 567.02 ms 0.94 0.12 False
WriteStringsUtf16 - Duration of single invocation 173.37 ms 149.96 ms 0.86 0.05 False
WriteStringsUtf16 - Duration of single invocation 610.45 ms 565.92 ms 0.93 0.12 False
WriteStringsUtf16 - Duration of single invocation 204.94 ms 187.13 ms 0.91 0.04 False
WriteStringsUtf16 - Duration of single invocation 165.93 ms 145.82 ms 0.88 0.04 False
WriteStringsUtf16 - Duration of single invocation 175.29 ms 154.79 ms 0.88 0.03 False
WriteStringsUtf16 - Duration of single invocation 168.82 ms 146.88 ms 0.87 0.06 False
WriteStringsUtf16 - Duration of single invocation 222.42 ms 197.84 ms 0.89 0.05 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Tests.Perf_Strings*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Tests.Perf_Strings.WriteStringsUtf16(Formatted: False, SkipValidation: False, Escaped: OneEscaped)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 191.8262897857143 < 198.41612624846152.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 35.7933604846182 (T) = (0 -188803689.4942557) / Math.Sqrt((94370719678884.53 / (283)) + (7779873319123.767 / (33))) is greater than 1.9675476980854363 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (283) + (33) - 2, .975) and 0.1251334884822962 = (215808568.51718122 - 188803689.4942557) / 215808568.51718122 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Strings.WriteStringsUtf16(Formatted: True, SkipValidation: False, Escaped: AllEscaped)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 578.5663695 < 584.6660293516666.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/18/2022 9:26:37 AM, 10/22/2022 11:54:51 PM, 11/15/2022 2:18:08 PM, 12/2/2022 5:21:30 AM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 18.97846786783459 (T) = (0 -569703419.0726941) / Math.Sqrt((630551033071514.8 / (299)) + (68041099101829.64 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.06369908233400036 = (608461882.6315415 - 569703419.0726941) / 608461882.6315415 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Strings.WriteStringsUtf16(Formatted: True, SkipValidation: False, Escaped: OneEscaped)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 199.91976792857145 < 208.87635977321426.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 38.980992162112926 (T) = (0 -198273284.95067155) / Math.Sqrt((38549618387244.41 / (299)) + (7778645263011.1455 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.10612126790553107 = (221812285.97540587 - 198273284.95067155) / 221812285.97540587 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Strings.WriteStringsUtf16(Formatted: False, SkipValidation: False, Escaped: AllEscaped)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 556.8993952000001 < 577.8736371961538.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 12/2/2022 10:04:21 AM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 26.869697412507485 (T) = (0 -560115502.4308358) / Math.Sqrt((505048181568918.9 / (286)) + (35578539082577.05 / (33))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (286) + (33) - 2, .975) and 0.07484575924811743 = (605429319.5214931 - 560115502.4308358) / 605429319.5214931 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Strings.WriteStringsUtf16(Formatted: False, SkipValidation: True, Escaped: AllEscaped)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 567.0221276 < 575.0011618525641.
IsChangePoint: Marked as a change because one of 10/18/2022 8:30:48 AM, 10/22/2022 11:54:51 PM, 11/15/2022 8:48:03 PM, 12/2/2022 5:21:30 AM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 18.77545791540986 (T) = (0 -560114423.31786) / Math.Sqrt((707218022662848.4 / (299)) + (94706019953829.55 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.07123363681781808 = (603073545.2119199 - 560114423.31786) / 603073545.2119199 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Strings.WriteStringsUtf16(Formatted: True, SkipValidation: True, Escaped: NoneEscaped)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 149.95640880000002 < 166.11278090906595.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 11/8/2022 11:25:18 AM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 42.12890278969421 (T) = (0 -151071104.42426464) / Math.Sqrt((45711019364152.29 / (299)) + (6603204835978.686 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.1421321160931804 = (176100664.5175608 - 151071104.42426464) / 176100664.5175608 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Strings.WriteStringsUtf16(Formatted: True, SkipValidation: True, Escaped: AllEscaped)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 565.9160202 < 583.9776557235715.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/18/2022 9:26:37 AM, 10/22/2022 11:54:51 PM, 11/16/2022 11:42:33 AM, 12/2/2022 5:21:30 AM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 22.633610889759513 (T) = (0 -566660450.985437) / Math.Sqrt((693011430907217.2 / (299)) + (29013677571753.21 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.06665635319339 = (607129488.6125151 - 566660450.985437) / 607129488.6125151 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Strings.WriteStringsUtf16(Formatted: False, SkipValidation: True, Escaped: OneEscaped)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 187.1264692307692 < 194.79661277464285.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 35.58653374064512 (T) = (0 -186844701.89604843) / Math.Sqrt((91165386291746.08 / (285)) + (7924002388607.642 / (33))) is greater than 1.9674995188240387 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (285) + (33) - 2, .975) and 0.12474759006574669 = (213475221.29083166 - 186844701.89604843) / 213475221.29083166 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Strings.WriteStringsUtf16(Formatted: False, SkipValidation: True, Escaped: NoneEscaped)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 145.81619807142857 < 158.20484963333334.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 47.764498970906466 (T) = (0 -144917004.05107117) / Math.Sqrt((64981968598322.94 / (290)) + (4323833562538.296 / (33))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (290) + (33) - 2, .975) and 0.16416557414823443 = (173380037.4438897 - 144917004.05107117) / 173380037.4438897 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Strings.WriteStringsUtf16(Formatted: True, SkipValidation: False, Escaped: NoneEscaped)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 154.7922015 < 166.68588652083335.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 11/7/2022 8:57:30 PM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 46.30578063349357 (T) = (0 -152393362.41800976) / Math.Sqrt((62048228485040.36 / (299)) + (4649528617262.659 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.1520799142085916 = (179726090.9036882 - 152393362.41800976) / 179726090.9036882 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Strings.WriteStringsUtf16(Formatted: False, SkipValidation: False, Escaped: NoneEscaped)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 146.882584 < 160.65826003666666.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 47.38444441671332 (T) = (0 -146065214.0967144) / Math.Sqrt((72499887305143.8 / (286)) + (4382983717173.4365 / (33))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (286) + (33) - 2, .975) and 0.16779834672026034 = (175516611.29374787 - 146065214.0967144) / 175516611.29374787 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Strings.WriteStringsUtf16(Formatted: True, SkipValidation: True, Escaped: OneEscaped)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 197.84434713333332 < 206.80323311678572.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 34.69683046726755 (T) = (0 -195965442.5359418) / Math.Sqrt((34169592023062.504 / (299)) + (10688746148001.623 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.10490716451902901 = (218933092.48826838 - 195965442.5359418) / 218933092.48826838 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented Dec 27, 2022 •
edited
Loading

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 01b57ccc4cc532034f717226f4c7cc07a1ee9518
Compare fc65281072f9259f0b03a291b9c549bc9d792752
Diff Diff

Improvements in System.Memory.Span<Char>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
IndexOfValue - Duration of single invocation 824.94 ns 687.65 ns 0.83 0.06 False
SequenceCompareTo - Duration of single invocation 1.13 μs 940.84 ns 0.83 0.05 False
SequenceCompareToDifferent - Duration of single invocation 74.05 ns 68.94 ns 0.93 0.21 False
LastIndexOfAnyValues - Duration of single invocation 325.60 ns 96.12 ns 0.30 0.14 False
IndexOfValue - Duration of single invocation 87.81 ns 70.96 ns 0.81 0.19 False
SequenceCompareToDifferent - Duration of single invocation 75.34 ns 69.74 ns 0.93 0.25 False
LastIndexOfValue - Duration of single invocation 88.06 ns 82.41 ns 0.94 0.12 False
EndsWith - Duration of single invocation 106.94 ns 96.06 ns 0.90 0.32 False
IndexOfAnyFiveValues - Duration of single invocation 291.72 ns 239.84 ns 0.82 0.12 False
SequenceCompareTo - Duration of single invocation 132.43 ns 117.02 ns 0.88 0.22 False
SequenceCompareToDifferent - Duration of single invocation 76.31 ns 69.07 ns 0.91 0.22 False
IndexOfAnyFiveValues - Duration of single invocation 3.53 μs 2.62 μs 0.74 0.06 False
Fill - Duration of single invocation 138.50 ns 117.45 ns 0.85 0.13 False
LastIndexOfAnyValues - Duration of single invocation 3.87 μs 1.00 μs 0.26 0.10 False
LastIndexOfAnyValues - Duration of single invocation 88.28 ns 40.02 ns 0.45 0.20 False
LastIndexOfValue - Duration of single invocation 48.34 ns 36.47 ns 0.75 0.16 False
Fill - Duration of single invocation 1.70 μs 1.33 μs 0.78 0.16 False
IndexOfAnyFiveValues - Duration of single invocation 90.68 ns 73.47 ns 0.81 0.31 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Memory.Span&lt;Char&gt;*'

Payloads

Baseline
Compare

Histogram

System.Memory.Span<Char>.IndexOfValue(Size: 512)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 687.6492240617678 < 781.1798383999935.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 51.73692342207876 (T) = (0 -690.2678245557353) / Math.Sqrt((939.1530082793757 / (299)) + (97.43058764991059 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.15613084518355472 = (817.9796839545336 - 690.2678245557353) / 817.9796839545336 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Span&lt;Char&gt;.SequenceCompareTo(Size: 512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 940.8440704589628 < 1.0733394581525961.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 12/2/2022 5:21:30 AM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 64.3633313562217 (T) = (0 -942.8234675515832) / Math.Sqrt((2980.221460169992 / (299)) + (24.403674883242154 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.18259032886793822 = (1153.428324680611 - 942.8234675515832) / 1153.428324680611 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Span&lt;Char&gt;.SequenceCompareToDifferent(Size: 512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 68.94386956895825 < 73.73326045725665.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 8.26582170850037 (T) = (0 -70.598581885241) / Math.Sqrt((13.936129294149131 / (299)) + (18.74767761489328 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.08407889503260295 = (77.0793264860449 - 70.598581885241) / 77.0793264860449 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Span&lt;Char&gt;.LastIndexOfAnyValues(Size: 33)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 96.12176917470285 < 306.8206400822162.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 257.9986755130615 (T) = (0 -98.56575670996273) / Math.Sqrt((29.117482367964065 / (70)) + (11.13570527960465 / (33))) is greater than 1.9837310029553383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (70) + (33) - 2, .975) and 0.6943758820056348 = (322.5064741512972 - 98.56575670996273) / 322.5064741512972 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Span&lt;Char&gt;.IndexOfValue(Size: 33)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 70.96493663070062 < 82.80347982133819.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 24.877359640495214 (T) = (0 -72.51184316230629) / Math.Sqrt((9.635282410323951 / (70)) + (5.902191252470683 / (33))) is greater than 1.9837310029553383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (70) + (33) - 2, .975) and 0.16178505509158053 = (86.50745683164679 - 72.51184316230629) / 86.50745683164679 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Span&lt;Char&gt;.SequenceCompareToDifferent(Size: 4)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 69.73757737333709 < 72.18485694576961.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 8.372072907926745 (T) = (0 -70.76985023220206) / Math.Sqrt((11.938927847672222 / (69)) + (12.295560971174075 / (33))) is greater than 1.9839715185231144 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (69) + (33) - 2, .975) and 0.08036137286386658 = (76.95397751243658 - 70.76985023220206) / 76.95397751243658 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Span&lt;Char&gt;.LastIndexOfValue(Size: 33)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 82.40648838229973 < 83.75068111404138.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 20.287816944658132 (T) = (0 -73.74382309500444) / Math.Sqrt((5.443063106093785 / (70)) + (9.476013736310533 / (33))) is greater than 1.9837310029553383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (70) + (33) - 2, .975) and 0.14250613409873927 = (85.99924270886382 - 73.74382309500444) / 85.99924270886382 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Span&lt;Char&gt;.EndsWith(Size: 33)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 96.06208942405866 < 98.07813480954681.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 6.027637460616515 (T) = (0 -97.55970663732158) / Math.Sqrt((15.606079330576746 / (70)) + (25.31760184785878 / (33))) is greater than 1.9837310029553383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (70) + (33) - 2, .975) and 0.05791812277209193 = (103.55756648710054 - 97.55970663732158) / 103.55756648710054 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Span&lt;Char&gt;.IndexOfAnyFiveValues(Size: 33)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 239.8406254489922 < 277.67849715006145.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 59.73390313958748 (T) = (0 -232.62829250537672) / Math.Sqrt((49.161991684362555 / (70)) + (15.410364606523078 / (33))) is greater than 1.9837310029553383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (70) + (33) - 2, .975) and 0.2173219310295683 = (297.220915888682 - 232.62829250537672) / 297.220915888682 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Span&lt;Char&gt;.SequenceCompareTo(Size: 33)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 117.02352464877971 < 125.68911113681584.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 16.225658798359444 (T) = (0 -119.45566233218118) / Math.Sqrt((19.97056379228267 / (70)) + (17.121470458886353 / (33))) is greater than 1.9837310029553383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (70) + (33) - 2, .975) and 0.108577825496537 = (134.0057110411461 - 119.45566233218118) / 134.0057110411461 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Span&lt;Char&gt;.SequenceCompareToDifferent(Size: 33)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 69.07326131968384 < 72.23202126374669.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 15.539972874198305 (T) = (0 -69.6959458426096) / Math.Sqrt((14.893552321175152 / (70)) + (1.5552911294990663 / (33))) is greater than 1.9837310029553383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (70) + (33) - 2, .975) and 0.10206707201943604 = (77.61820919002783 - 69.6959458426096) / 77.61820919002783 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Span&lt;Char&gt;.IndexOfAnyFiveValues(Size: 512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.619543422048436 < 3.356559780505736.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 58.2938759778954 (T) = (0 -2618.546214956178) / Math.Sqrt((37231.22242962986 / (299)) + (1614.0640669122545 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.22670868102351469 = (3386.23511049114 - 2618.546214956178) / 3386.23511049114 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Span&lt;Char&gt;.Fill(Size: 33)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 117.4512553952412 < 132.02964198982187.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 23.170059859103784 (T) = (0 -119.16194130747706) / Math.Sqrt((17.029402770113684 / (70)) + (15.425894076033504 / (33))) is greater than 1.9837310029553383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (70) + (33) - 2, .975) and 0.14083707165823808 = (138.6953945248395 - 119.16194130747706) / 138.6953945248395 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Span&lt;Char&gt;.LastIndexOfAnyValues(Size: 512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.0009803528969727 < 3.672669977528444.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 295.619975859346 (T) = (0 -1002.8416642714179) / Math.Sqrt((28105.41880596666 / (299)) + (8.12731227479744 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.7410488719377956 = (3872.706297037324 - 1002.8416642714179) / 3872.706297037324 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Span&lt;Char&gt;.LastIndexOfAnyValues(Size: 4)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 40.01679531139545 < 85.47226840103566.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 97.85987805506663 (T) = (0 -40.47061115289133) / Math.Sqrt((10.164990768391144 / (70)) + (3.791616853195167 / (33))) is greater than 1.9837310029553383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (70) + (33) - 2, .975) and 0.5522184949769539 = (90.38026514920132 - 40.47061115289133) / 90.38026514920132 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Span&lt;Char&gt;.LastIndexOfValue(Size: 4)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 36.468080499477956 < 46.50543222611974.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 40.586937221048245 (T) = (0 -35.02549447584111) / Math.Sqrt((6.053923951535513 / (70)) + (0.7948989723484188 / (33))) is greater than 1.9837310029553383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (70) + (33) - 2, .975) and 0.27814695678720813 = (48.52164135784643 - 35.02549447584111) / 48.52164135784643 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Span&lt;Char&gt;.Fill(Size: 512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.3274989172155836 < 1.613916182943305.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 42.513789417691896 (T) = (0 -1324.271049711297) / Math.Sqrt((23560.59872022372 / (287)) + (302.06358130047715 / (33))) is greater than 1.967451947860886 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (287) + (33) - 2, .975) and 0.23469121556120406 = (1730.3748194689683 - 1324.271049711297) / 1730.3748194689683 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Span&lt;Char&gt;.IndexOfAnyFiveValues(Size: 4)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 73.4691455170581 < 85.88873236206602.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 23.57124018448692 (T) = (0 -75.3966136256783) / Math.Sqrt((19.985404325031638 / (70)) + (6.865165493114713 / (33))) is greater than 1.9837310029553383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (70) + (33) - 2, .975) and 0.1800794052916037 = (91.9559944124748 - 75.3966136256783) / 91.9559944124748 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS ubuntu 18.04
Baseline 01b57ccc4cc532034f717226f4c7cc07a1ee9518
Compare fc65281072f9259f0b03a291b9c549bc9d792752
Diff Diff

Improvements in System.Memory.Span<Byte>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
LastIndexOfValue - Duration of single invocation 82.60 ns 72.26 ns 0.87 0.08 False
IndexOfAnyFourValues - Duration of single invocation 153.60 ns 76.39 ns 0.50 0.29 False
IndexOfAnyFourValues - Duration of single invocation 10.30 μs 1.56 μs 0.15 0.08 False
SequenceEqual - Duration of single invocation 76.23 ns 70.87 ns 0.93 0.19 False
LastIndexOfAnyValues - Duration of single invocation 1.26 μs 1.10 μs 0.87 0.06 False
IndexOfAnyFiveValues - Duration of single invocation 288.66 ns 223.23 ns 0.77 0.10 False
IndexOfAnyFourValues - Duration of single invocation 709.67 ns 161.28 ns 0.23 0.12 False
IndexOfAnyTwoValues - Duration of single invocation 50.67 ns 41.45 ns 0.82 0.29 False
IndexOfValue - Duration of single invocation 39.17 ns 35.13 ns 0.90 0.33 False
IndexOfValue - Duration of single invocation 810.61 ns 672.42 ns 0.83 0.08 False
SequenceCompareToDifferent - Duration of single invocation 62.18 ns 57.44 ns 0.92 0.26 False
SequenceCompareTo - Duration of single invocation 90.26 ns 81.81 ns 0.91 0.23 False
LastIndexOfValue - Duration of single invocation 811.81 ns 666.73 ns 0.82 0.05 False
IndexOfValue - Duration of single invocation 80.97 ns 70.28 ns 0.87 0.31 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Memory.Span&lt;Byte&gt;*'

Payloads

Baseline
Compare

Histogram

System.Memory.Span<Byte>.LastIndexOfValue(Size: 33)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 72.26133793725221 < 78.3599718380496.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 35.3582055387094 (T) = (0 -71.0129736676595) / Math.Sqrt((5.514188808921061 / (70)) + (1.1966502617528114 / (33))) is greater than 1.9837310029553383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (70) + (33) - 2, .975) and 0.14447781365848858 = (83.0054144724567 - 71.0129736676595) / 83.0054144724567 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Span&lt;Byte&gt;.IndexOfAnyFourValues(Size: 4)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 76.39038988040873 < 142.19044230155103.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 66.38751476918772 (T) = (0 -72.88816638728099) / Math.Sqrt((44.14978904798046 / (70)) + (19.445682366621636 / (33))) is greater than 1.9837310029553383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (70) + (33) - 2, .975) and 0.5014994044769548 = (146.2148030350978 - 72.88816638728099) / 146.2148030350978 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Span&lt;Byte&gt;.IndexOfAnyFourValues(Size: 512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.5624945178535403 < 9.781666834995283.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 276.5692616639993 (T) = (0 -1566.9739988837794) / Math.Sqrt((297436.4913143401 / (299)) + (31.360610265866566 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.8477798653150095 = (10294.13094480916 - 1566.9739988837794) / 10294.13094480916 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Span&lt;Byte&gt;.SequenceEqual(Size: 33)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 70.8740238091355 < 72.91292921774773.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 10.963054343335626 (T) = (0 -70.65469757642782) / Math.Sqrt((10.27629941774543 / (70)) + (3.8844182802115483 / (33))) is greater than 1.9837310029553383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (70) + (33) - 2, .975) and 0.0739044501270024 = (76.29309695540296 - 70.65469757642782) / 76.29309695540296 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Span&lt;Byte&gt;.LastIndexOfAnyValues(Size: 512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.0997168509751272 < 1.195809256268449.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 14.973402537197222 (T) = (0 -1134.4559375136419) / Math.Sqrt((1527.6835860486785 / (288)) + (3657.0901579457586 / (33))) is greater than 1.9674283869023508 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (288) + (33) - 2, .975) and 0.12452088902072452 = (1295.8115428301714 - 1134.4559375136419) / 1295.8115428301714 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Span&lt;Byte&gt;.IndexOfAnyFiveValues(Size: 33)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 223.22981523823574 < 273.063994276671.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 56.0388914564467 (T) = (0 -225.4697877397459) / Math.Sqrt((41.86676952715213 / (70)) + (25.91274164312667 / (33))) is greater than 1.9837310029553383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (70) + (33) - 2, .975) and 0.22620016390699127 = (291.37998901390955 - 225.4697877397459) / 291.37998901390955 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Span&lt;Byte&gt;.IndexOfAnyFourValues(Size: 33)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 161.2794299909654 < 674.2633312992541.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 79.7610869507539 (T) = (0 -162.37955137901236) / Math.Sqrt((3359.2554328922533 / (70)) + (16.544067491389583 / (33))) is greater than 1.9837310029553383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (70) + (33) - 2, .975) and 0.7737809855809727 = (717.7979790780779 - 162.37955137901236) / 717.7979790780779 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Span&lt;Byte&gt;.IndexOfAnyTwoValues(Size: 4)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 41.45342667614021 < 44.57849938726196.
IsChangePoint: Marked as a change because one of 12/18/2022 12:43:31 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 5.018703568851934 (T) = (0 -42.92384773976859) / Math.Sqrt((4.154589396871492 / (71)) + (7.085441251179854 / (32))) is greater than 1.9837310029553383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (71) + (32) - 2, .975) and 0.05825776818320433 = (45.579189601554255 - 42.92384773976859) / 45.579189601554255 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Span&lt;Byte&gt;.IndexOfValue(Size: 4)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 35.13143744158122 < 36.885459890547814.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 6.7854630574408645 (T) = (0 -36.348522077583716) / Math.Sqrt((2.733534951225491 / (70)) + (3.141008332844161 / (33))) is greater than 1.9837310029553383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (70) + (33) - 2, .975) and 0.06401621750949947 = (38.83456397168145 - 36.348522077583716) / 38.83456397168145 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Span&lt;Byte&gt;.IndexOfValue(Size: 512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 672.424163536308 < 770.0937129565298.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 62.64540008324705 (T) = (0 -673.0256654779857) / Math.Sqrt((1548.8215145441839 / (299)) + (10.540751320753078 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.17917135540908086 = (819.9344283524672 - 673.0256654779857) / 819.9344283524672 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Span&lt;Byte&gt;.SequenceCompareToDifferent(Size: 4)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 57.43962398434381 < 58.97401739950065.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 8.391949691286555 (T) = (0 -57.40422471345545) / Math.Sqrt((6.965085490964678 / (70)) + (8.824349577320996 / (33))) is greater than 1.9837310029553383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (70) + (33) - 2, .975) and 0.08134807261559562 = (62.4874590715739 - 57.40422471345545) / 62.4874590715739 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Span&lt;Byte&gt;.SequenceCompareTo(Size: 4)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 81.80833250251091 < 86.10861211340699.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 12.464591097287917 (T) = (0 -81.910144414504) / Math.Sqrt((21.430602279706086 / (70)) + (3.3058886344762497 / (33))) is greater than 1.9837310029553383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (70) + (33) - 2, .975) and 0.08842445713154423 = (89.85557483997131 - 81.910144414504) / 89.85557483997131 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Span&lt;Byte&gt;.LastIndexOfValue(Size: 512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 666.7319475823266 < 772.3524457346044.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 77.3470122528298 (T) = (0 -667.7057253681061) / Math.Sqrt((919.7608057197101 / (284)) + (16.021218513044484 / (33))) is greater than 1.9675235316906419 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (284) + (33) - 2, .975) and 0.18270386333295974 = (816.9691442455991 - 667.7057253681061) / 816.9691442455991 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Span&lt;Byte&gt;.IndexOfValue(Size: 33)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 70.27617704183076 < 75.97145484256673.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 16.153668532652667 (T) = (0 -71.00222377454763) / Math.Sqrt((18.086831604820556 / (70)) + (3.8303843378818434 / (33))) is greater than 1.9837310029553383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (70) + (33) - 2, .975) and 0.12220586104407599 = (80.88710168308928 - 71.00222377454763) / 80.88710168308928 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented Dec 27, 2022 •
edited
Loading

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 01b57ccc4cc532034f717226f4c7cc07a1ee9518
Compare fc65281072f9259f0b03a291b9c549bc9d792752
Diff Diff

Improvements in System.Numerics.Tests.Perf_VectorOf<UInt16>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
EqualsStaticBenchmark - Duration of single invocation 149.32 ns 131.79 ns 0.88 0.11 False
GreaterThanOrEqualBenchmark - Duration of single invocation 156.19 ns 146.85 ns 0.94 0.12 False
UnaryNegateOperatorBenchmark - Duration of single invocation 134.56 ns 125.94 ns 0.94 0.16 False
MultiplyOperatorBenchmark - Duration of single invocation 134.80 ns 125.75 ns 0.93 0.10 False
EqualsAllBenchmark - Duration of single invocation 21.33 ns 19.15 ns 0.90 0.30 False
GreaterThanBenchmark - Duration of single invocation 149.48 ns 138.76 ns 0.93 0.11 False
MinBenchmark - Duration of single invocation 176.00 ns 162.50 ns 0.92 0.17 False
SubtractBenchmark - Duration of single invocation 135.25 ns 126.37 ns 0.93 0.11 False
DotBenchmark - Duration of single invocation 116.42 ns 100.04 ns 0.86 0.11 False
MultiplyBenchmark - Duration of single invocation 134.96 ns 124.75 ns 0.92 0.14 False
SquareRootBenchmark - Duration of single invocation 124.40 ns 111.43 ns 0.90 0.08 False
EqualsBenchmark - Duration of single invocation 39.19 ns 33.26 ns 0.85 0.26 False
DivisionOperatorBenchmark - Duration of single invocation 153.39 ns 142.09 ns 0.93 0.16 False
MaxBenchmark - Duration of single invocation 175.88 ns 163.45 ns 0.93 0.12 False
AddOperatorBenchmark - Duration of single invocation 134.64 ns 127.14 ns 0.94 0.14 False
EqualityOperatorBenchmark - Duration of single invocation 21.27 ns 17.55 ns 0.82 0.27 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;UInt16&gt;*'

Payloads

Baseline
Compare

Histogram

System.Numerics.Tests.Perf_VectorOf<UInt16>.EqualsStaticBenchmark

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 131.78564782215068 < 142.5815116399794.
IsChangePoint: Marked as a change because one of 10/11/2022 10:51:47 AM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 43.67003997887088 (T) = (0 -132.6868762843914) / Math.Sqrt((33.07909364088113 / (299)) + (2.3166495976409536 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.12277416401256346 = (151.2573739178969 - 132.6868762843914) / 151.2573739178969 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt16&gt;.GreaterThanOrEqualBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 146.85258400961072 < 148.92832428111734.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/11/2022 6:23:40 AM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 9.179935890926401 (T) = (0 -148.2157607251826) / Math.Sqrt((30.8606052461864 / (299)) + (41.43177832557275 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.06733431245801551 = (158.91627911797772 - 148.2157607251826) / 158.91627911797772 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt16&gt;.UnaryNegateOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 125.94394732373624 < 127.81756633863678.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 22.600383752942662 (T) = (0 -126.73189797340613) / Math.Sqrt((161.93817891545035 / (283)) + (6.202771381986932 / (33))) is greater than 1.9675476980854363 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (283) + (33) - 2, .975) and 0.13456262486954595 = (146.43682098234186 - 126.73189797340613) / 146.43682098234186 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt16&gt;.MultiplyOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 125.75210634833029 < 128.21318159568764.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 13.253592297185829 (T) = (0 -127.35918805919863) / Math.Sqrt((59.19771604773067 / (299)) + (25.47016563943665 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.09295548154095705 = (140.41117659314696 - 127.35918805919863) / 140.41117659314696 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt16&gt;.EqualsAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 19.147073345829416 < 20.231612467023677.
IsChangePoint: Marked as a change because one of 11/15/2022 5:29:43 PM, 12/1/2022 11:48:13 PM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 10.647052761911883 (T) = (0 -19.41516081667408) / Math.Sqrt((2.711287012800812 / (299)) + (2.4646071814279304 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.13696695651998786 = (22.496428107070198 - 19.41516081667408) / 22.496428107070198 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt16&gt;.GreaterThanBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 138.76352656601156 < 142.1670278481002.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 29.59028840585465 (T) = (0 -139.2679743907237) / Math.Sqrt((23.508785019750775 / (299)) + (2.626391051744585 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.07792624196610187 = (151.03778106393503 - 139.2679743907237) / 151.03778106393503 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt16&gt;.MinBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 162.50041531004004 < 167.28588067414734.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 11/18/2022 7:31:41 AM, 12/2/2022 5:21:30 AM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 11.395056357665442 (T) = (0 -164.730452491847) / Math.Sqrt((50.90346305392155 / (299)) + (41.50910339593523 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.07635322823926265 = (178.34788961349716 - 164.730452491847) / 178.34788961349716 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt16&gt;.SubtractBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 126.37225910606445 < 129.06507661980385.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 22.37359432487401 (T) = (0 -126.82175421683957) / Math.Sqrt((166.39258720580324 / (283)) + (7.715203142359202 / (33))) is greater than 1.9675476980854363 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (283) + (33) - 2, .975) and 0.13787433014003833 = (147.10355885521852 - 126.82175421683957) / 147.10355885521852 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt16&gt;.DotBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 100.03976433209715 < 110.86623659295088.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 21.67452868056191 (T) = (0 -102.08560340428845) / Math.Sqrt((43.7920239913741 / (299)) + (25.316865794255246 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.16870492871156265 = (122.80308993780555 - 102.08560340428845) / 122.80308993780555 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt16&gt;.MultiplyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 124.75431557208123 < 128.21092655307456.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 12.22956321745191 (T) = (0 -127.71545809857368) / Math.Sqrt((53.59825903623201 / (299)) + (30.065498435001075 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.09089905871734637 = (140.4854535937225 - 127.71545809857368) / 140.4854535937225 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt16&gt;.SquareRootBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 111.42504157275154 < 118.82130597593265.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 51.871300302005174 (T) = (0 -111.71762127375793) / Math.Sqrt((28.29435778696106 / (286)) + (0.36953961548242037 / (33))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (286) + (33) - 2, .975) and 0.13351187268410744 = (128.93150840949656 - 111.71762127375793) / 128.93150840949656 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt16&gt;.EqualsBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 33.26088728713873 < 36.38596309100309.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 11/15/2022 8:48:03 PM, 12/2/2022 5:21:30 AM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 9.282549892321608 (T) = (0 -35.137204709398894) / Math.Sqrt((5.5502194646611125 / (299)) + (5.452224097152361 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.1017318627325726 = (39.11661034341914 - 35.137204709398894) / 39.11661034341914 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt16&gt;.DivisionOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 142.09090578956705 < 145.73082584380873.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 11.29231034665572 (T) = (0 -143.75901862813257) / Math.Sqrt((90.70621264478359 / (299)) + (30.32104915232162 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.07990074815958394 = (156.2429469870566 - 143.75901862813257) / 156.2429469870566 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt16&gt;.MaxBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 163.4486619229885 < 166.98159965877795.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 27.220766097337012 (T) = (0 -159.08677902122693) / Math.Sqrt((49.30221977352638 / (299)) + (10.838491277323087 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.10728690165930661 = (178.20594244323874 - 159.08677902122693) / 178.20594244323874 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt16&gt;.AddOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 127.14353006635677 < 128.34883998056117.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/18/2022 12:43:31 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 17.652602870015365 (T) = (0 -126.8620804659525) / Math.Sqrt((63.07992180480123 / (299)) + (10.572471509753706 / (32))) is greater than 1.9672006833568965 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (32) - 2, .975) and 0.09287268400557334 = (139.8503586311714 - 126.8620804659525) / 139.8503586311714 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt16&gt;.EqualityOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 17.549311338124667 < 20.209672980601876.
IsChangePoint: Marked as a change because one of 11/15/2022 5:29:43 PM, 12/1/2022 11:48:13 PM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 10.177965284090899 (T) = (0 -19.38050955447007) / Math.Sqrt((1.9767017152845936 / (299)) + (2.4442607608318414 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.1298056946128701 = (22.27147366339994 - 19.38050955447007) / 22.27147366339994 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS ubuntu 18.04
Baseline 01b57ccc4cc532034f717226f4c7cc07a1ee9518
Compare fc65281072f9259f0b03a291b9c549bc9d792752
Diff Diff

Improvements in System.Tests.Perf_Environment

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
SetEnvironmentVariable - Duration of single invocation 1.00 μs 828.13 ns 0.83 0.05 False
GetEnvironmentVariable - Duration of single invocation 601.41 ns 554.64 ns 0.92 0.07 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Environment*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Environment.SetEnvironmentVariable

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 828.131341240876 < 949.9259279692744.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 11/15/2022 8:48:03 PM, 12/2/2022 5:21:30 AM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 70.69901991267665 (T) = (0 -818.5197174927386) / Math.Sqrt((1267.4442371481573 / (299)) + (110.93604762395545 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.19232868244421106 = (1013.4317013630985 - 818.5197174927386) / 1013.4317013630985 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Environment.GetEnvironmentVariable

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 554.6431253401053 < 575.8581966602445.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 11/15/2022 8:48:03 PM, 12/2/2022 5:21:30 AM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 18.41549562479717 (T) = (0 -566.435453706064) / Math.Sqrt((484.66470882872846 / (299)) + (180.01139385254064 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.07959757373834953 = (615.4215129644157 - 566.435453706064) / 615.4215129644157 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 01b57ccc4cc532034f717226f4c7cc07a1ee9518
Compare fc65281072f9259f0b03a291b9c549bc9d792752
Diff Diff

Improvements in System.Numerics.Tests.Perf_VectorOf<UInt32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
MaxBenchmark - Duration of single invocation 89.44 ns 82.05 ns 0.92 0.19 False
DotBenchmark - Duration of single invocation 54.43 ns 50.95 ns 0.94 0.19 False
EqualsStaticBenchmark - Duration of single invocation 75.27 ns 67.68 ns 0.90 0.23 False
EqualityOperatorBenchmark - Duration of single invocation 24.74 ns 19.16 ns 0.77 0.21 False
EqualsAllBenchmark - Duration of single invocation 21.47 ns 19.15 ns 0.89 0.25 False
DivisionOperatorBenchmark - Duration of single invocation 76.28 ns 70.92 ns 0.93 0.17 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;UInt32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Numerics.Tests.Perf_VectorOf<UInt32>.MaxBenchmark

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 82.05027721809144 < 85.02463142828951.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 21.195097627146332 (T) = (0 -82.69706272266889) / Math.Sqrt((31.197090157653342 / (299)) + (3.99566204618461 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.10848485051670018 = (92.76013174941342 - 82.69706272266889) / 92.76013174941342 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt32&gt;.DotBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 50.94619585266431 < 51.46748972910296.
IsChangePoint: Marked as a change because one of 10/3/2022 11:00:05 AM, 10/3/2022 8:45:37 PM, 10/11/2022 6:23:40 AM, 11/15/2022 9:22:46 AM, 12/2/2022 5:21:30 AM, 12/19/2022 11:10:20 AM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 9.459802804291767 (T) = (0 -51.71847671170537) / Math.Sqrt((9.863136637685521 / (299)) + (8.530024163061954 / (31))) is greater than 1.9672228266464409 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (31) - 2, .975) and 0.09217612460756228 = (56.96972520065999 - 51.71847671170537) / 56.96972520065999 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt32&gt;.EqualsStaticBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 67.67659035985831 < 72.59080317096928.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/18/2022 12:43:31 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 18.846477334453056 (T) = (0 -68.74738312689696) / Math.Sqrt((17.76535903728671 / (299)) + (8.03113427649444 / (32))) is greater than 1.9672006833568965 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (32) - 2, .975) and 0.1324948880556836 = (79.24723691001113 - 68.74738312689696) / 79.24723691001113 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt32&gt;.EqualityOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 19.15890654390352 < 20.21368788718787.
IsChangePoint: Marked as a change because one of 10/14/2022 1:19:37 PM, 12/13/2022 4:26:35 PM, 12/14/2022 7:00:38 AM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 13.442166363638488 (T) = (0 -19.48977157326337) / Math.Sqrt((1.9343048886924366 / (299)) + (1.1415560682539798 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.12262203614024068 = (22.213655204564297 - 19.48977157326337) / 22.213655204564297 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt32&gt;.EqualsAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 19.152420794043596 < 20.304928128107232.
IsChangePoint: Marked as a change because one of 10/14/2022 1:19:37 PM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 5.216292133785388 (T) = (0 -19.90247070768851) / Math.Sqrt((2.2435837165662025 / (299)) + (6.126759323922305 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.10329226416016177 = (22.195047407557226 - 19.90247070768851) / 22.195047407557226 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt32&gt;.DivisionOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 70.91736975873876 < 73.28423416377558.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 23.94876609683099 (T) = (0 -71.29664539810042) / Math.Sqrt((14.866082375095957 / (299)) + (0.7485657273982521 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.08289211153166205 = (77.740739442524 - 71.29664539810042) / 77.740739442524 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented Dec 27, 2022 •
edited
Loading

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 01b57ccc4cc532034f717226f4c7cc07a1ee9518
Compare fc65281072f9259f0b03a291b9c549bc9d792752
Diff Diff

Improvements in System.Text.Json.Node.Tests.Perf_ParseThenWrite

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ParseThenWrite - Duration of single invocation 208.57 μs 53.76 μs 0.26 0.01 False
ParseThenWrite - Duration of single invocation 1.16 ms 283.26 μs 0.24 0.00 False
ParseThenWrite - Duration of single invocation 233.93 μs 64.41 μs 0.28 0.01 False
ParseThenWrite - Duration of single invocation 23.20 μs 10.77 μs 0.46 0.00 False
ParseThenWrite - Duration of single invocation 135.57 μs 36.80 μs 0.27 0.01 False
ParseThenWrite - Duration of single invocation 24.48 μs 11.69 μs 0.48 0.00 False
ParseThenWrite - Duration of single invocation 1.03 ms 219.20 μs 0.21 0.01 False
ParseThenWrite - Duration of single invocation 75.31 ms 12.51 ms 0.17 0.01 False
ParseThenWrite - Duration of single invocation 124.75 μs 29.52 μs 0.24 0.00 False
ParseThenWrite - Duration of single invocation 672.55 μs 210.25 μs 0.31 0.00 False
ParseThenWrite - Duration of single invocation 82.09 ms 15.60 ms 0.19 0.01 False
ParseThenWrite - Duration of single invocation 597.50 μs 154.44 μs 0.26 0.00 False
ParseThenWrite - Duration of single invocation 796.50 μs 138.74 μs 0.17 0.00 False
ParseThenWrite - Duration of single invocation 869.33 μs 171.13 μs 0.20 0.00 False
ParseThenWrite - Duration of single invocation 135.34 μs 33.68 μs 0.25 0.00 False
ParseThenWrite - Duration of single invocation 144.43 μs 42.51 μs 0.29 0.01 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Node.Tests.Perf_ParseThenWrite*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Node.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented: False, TestCase: LotsOfNumbers)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 53.758345308908055 < 199.5185092103879.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 449.8784317072679 (T) = (0 -54344.884620161756) / Math.Sqrt((20675434.41297045 / (166)) + (339272.29482504254 / (33))) is greater than 1.9720790337782532 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (166) + (33) - 2, .975) and 0.7524576913980859 = (219537.76276505788 - 54344.884620161756) / 219537.76276505788 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Node.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented: True, TestCase: BroadTree)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 283.2592780032468 < 1.1015546197787887.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 465.48118111659784 (T) = (0 -287969.50119932205) / Math.Sqrt((578302495.3446633 / (166)) + (12471409.109574055 / (33))) is greater than 1.9720790337782532 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (166) + (33) - 2, .975) and 0.7605627329798599 = (1202692.9006632022 - 287969.50119932205) / 1202692.9006632022 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Node.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented: True, TestCase: LotsOfNumbers)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 64.41012045164379 < 220.39052529258575.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 460.19688514101375 (T) = (0 -65416.48232223098) / Math.Sqrt((21365237.58874781 / (166)) + (526213.5148237451 / (33))) is greater than 1.9720790337782532 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (166) + (33) - 2, .975) and 0.7279345236099154 = (240443.89310328188 - 65416.48232223098) / 240443.89310328188 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Node.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented: False, TestCase: HelloWorld)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 10.767901319920092 < 22.041338023324563.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 417.91982079520915 (T) = (0 -10879.751655556503) / Math.Sqrt((105932.27789367094 / (166)) + (9672.85383330718 / (33))) is greater than 1.9720790337782532 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (166) + (33) - 2, .975) and 0.5396420690536544 = (23633.244751950482 - 10879.751655556503) / 23633.244751950482 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Node.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented: False, TestCase: LotsOfStrings)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 36.79645800827422 < 127.55720995709075.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 367.31864863798324 (T) = (0 -37307.96468267011) / Math.Sqrt((10915315.928469494 / (166)) + (317518.96413254255 / (33))) is greater than 1.9720790337782532 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (166) + (33) - 2, .975) and 0.7299550179749674 = (138154.63039861908 - 37307.96468267011) / 138154.63039861908 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Node.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented: True, TestCase: HelloWorld)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 11.692998359611027 < 23.369173833163426.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 394.40856949998425 (T) = (0 -11752.828208657778) / Math.Sqrt((108245.04317759407 / (166)) + (15679.381610778199 / (33))) is greater than 1.9720790337782532 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (166) + (33) - 2, .975) and 0.5297869801686098 = (24994.68903024448 - 11752.828208657778) / 24994.68903024448 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Node.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented: False, TestCase: BroadTree)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 219.19959345657276 < 0.9779937429827723.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 435.61679375788265 (T) = (0 -221963.78084145294) / Math.Sqrt((572263410.9262632 / (164)) + (7248339.672024948 / (33))) is greater than 1.9722040512682435 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (164) + (33) - 2, .975) and 0.7907806918146787 = (1060914.4192602097 - 221963.78084145294) / 1060914.4192602097 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Node.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented: False, TestCase: Json400KB)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 12.507739526666667 < 71.75236260206043.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 461.31142732886826 (T) = (0 -12400626.559147919) / Math.Sqrt((3266627937327.0947 / (165)) + (27839132440.639668 / (33))) is greater than 1.9721412216619414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (165) + (33) - 2, .975) and 0.8423870202677568 = (78677698.88123682 - 12400626.559147919) / 78677698.88123682 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Node.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented: False, TestCase: Json400B)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 29.51915738513148 < 120.02653831688349.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 594.3656488029794 (T) = (0 -29339.12608827291) / Math.Sqrt((4681956.209955351 / (166)) + (40487.05337883017 / (33))) is greater than 1.9720790337782532 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (166) + (33) - 2, .975) and 0.7765592646412424 = (131306.0755961345 - 29339.12608827291) / 131306.0755961345 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Node.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented: True, TestCase: DeepTree)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 210.24616210585592 < 639.1844066100542.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 454.3664330688595 (T) = (0 -210867.44974527904) / Math.Sqrt((178549710.1816464 / (166)) + (2302932.4096089513 / (33))) is greater than 1.9720790337782532 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (166) + (33) - 2, .975) and 0.6975261808544362 = (697142.8150077354 - 210867.44974527904) / 697142.8150077354 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Node.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented: True, TestCase: Json400KB)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 15.599332395833335 < 78.14084813645832.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 462.6342458785031 (T) = (0 -15583885.7493615) / Math.Sqrt((3735216277430.699 / (166)) + (20987421624.42894 / (33))) is greater than 1.9720790337782532 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (166) + (33) - 2, .975) and 0.8186970050772693 = (85954927.30830607 - 15583885.7493615) / 85954927.30830607 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Node.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented: False, TestCase: DeepTree)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 154.44433531353135 < 562.8531841678113.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 463.3019560627805 (T) = (0 -153914.84431340412) / Math.Sqrt((156763058.4935447 / (166)) + (1372931.1687211872 / (33))) is greater than 1.9720790337782532 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (166) + (33) - 2, .975) and 0.7493051128207311 = (613952.8653543759 - 153914.84431340412) / 613952.8653543759 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Node.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented: False, TestCase: Json4KB)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 138.74274395543614 < 760.2132439765624.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 463.8763251249276 (T) = (0 -140951.25370566596) / Math.Sqrt((360791076.2951089 / (166)) + (2819539.2041125125 / (33))) is greater than 1.9720790337782532 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (166) + (33) - 2, .975) and 0.8318280245014162 = (838137.5867636935 - 140951.25370566596) / 838137.5867636935 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Node.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented: True, TestCase: Json4KB)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 171.12684056122453 < 824.9023467910879.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 507.07934319651923 (T) = (0 -174523.770484249) / Math.Sqrt((334262529.06345075 / (166)) + (3355471.1270572725 / (33))) is greater than 1.9720790337782532 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (166) + (33) - 2, .975) and 0.8086415501658969 = (912025.4194970286 - 174523.770484249) / 912025.4194970286 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Node.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented: True, TestCase: Json400B)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 33.67881415678879 < 128.42021466155876.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 593.3599024014806 (T) = (0 -33976.381322379355) / Math.Sqrt((4907166.926352361 / (166)) + (81990.60419771024 / (33))) is greater than 1.9720790337782532 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (166) + (33) - 2, .975) and 0.7576504640952013 = (140195.77630107853 - 33976.381322379355) / 140195.77630107853 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Node.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented: True, TestCase: LotsOfStrings)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 42.51332715794083 < 137.68025497080689.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 406.0994647226128 (T) = (0 -42690.85537204825) / Math.Sqrt((10235358.935498858 / (166)) + (239470.62862759692 / (33))) is greater than 1.9720790337782532 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (166) + (33) - 2, .975) and 0.7140581866858385 = (149299.09997158835 - 42690.85537204825) / 149299.09997158835 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS ubuntu 18.04
Baseline 01b57ccc4cc532034f717226f4c7cc07a1ee9518
Compare fc65281072f9259f0b03a291b9c549bc9d792752
Diff Diff

Improvements in System.IO.Tests.Perf_Path

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GetFullPathNoRedundantSegments - Duration of single invocation 3.36 μs 2.96 μs 0.88 0.05 False
GetFullPathForLegacyLength - Duration of single invocation 6.06 μs 5.14 μs 0.85 0.07 False
GetDirectoryName - Duration of single invocation 654.91 ns 552.45 ns 0.84 0.07 False
HasExtension - Duration of single invocation 66.85 ns 61.90 ns 0.93 0.16 False
GetFullPathForReallyLongPath - Duration of single invocation 29.51 μs 25.14 μs 0.85 0.05 False
GetFullPathForTypicalLongPath - Duration of single invocation 15.21 μs 13.07 μs 0.86 0.06 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.IO.Tests.Perf_Path*'

Payloads

Baseline
Compare

Histogram

System.IO.Tests.Perf_Path.GetFullPathNoRedundantSegments

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.9560311103853025 < 3.201224312198288.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/22/2022 11:54:51 PM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 56.29429869192053 (T) = (0 -2910.850324746637) / Math.Sqrt((10931.392077792088 / (299)) + (1793.7890943220723 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.1556925740679526 = (3447.619001494974 - 2910.850324746637) / 3447.619001494974 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.Perf_Path.GetFullPathForLegacyLength

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 5.139189613252449 < 5.765287343147865.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/23/2022 8:43:59 AM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 57.391921755422274 (T) = (0 -5164.046467124426) / Math.Sqrt((38876.26533265101 / (299)) + (6703.025953856073 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.1686414439090161 = (6211.57553415409 - 5164.046467124426) / 6211.57553415409 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.Perf_Path.GetDirectoryName

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 552.4469319712756 < 627.3380663232479.
IsChangePoint: Marked as a change because one of 12/8/2022 9:11:55 AM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 52.55086636679927 (T) = (0 -563.2998907876075) / Math.Sqrt((2976.210042319503 / (299)) + (281.48424337563745 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.2862666337315906 = (789.23014869361 - 563.2998907876075) / 789.23014869361 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.Perf_Path.HasExtension

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 61.89987204074224 < 64.88048820667679.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 12/5/2022 3:49:45 AM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 22.057878261818917 (T) = (0 -62.96721947302886) / Math.Sqrt((11.976073659684957 / (299)) + (1.854853115147563 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.09803172247696727 = (69.81090249199028 - 62.96721947302886) / 69.81090249199028 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.Perf_Path.GetFullPathForReallyLongPath

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 25.14127489951769 < 28.050607365469922.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/22/2022 11:54:51 PM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 73.11621783248762 (T) = (0 -25176.819008698734) / Math.Sqrt((1158087.6020586723 / (299)) + (42702.703010814614 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.1727040105509258 = (30432.661743549455 - 25176.819008698734) / 30432.661743549455 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.Perf_Path.GetFullPathForTypicalLongPath

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 13.066377949042792 < 14.43597969436479.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 12/8/2022 3:49:24 AM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 63.71098984869821 (T) = (0 -13023.1388610116) / Math.Sqrt((382620.9097147698 / (299)) + (20280.721340102704 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.17554306909981382 = (15796.020838580664 - 13023.1388610116) / 15796.020838580664 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 01b57ccc4cc532034f717226f4c7cc07a1ee9518
Compare fc65281072f9259f0b03a291b9c549bc9d792752
Diff Diff

Improvements in System.Runtime.Intrinsics.Tests.Perf_Vector128Int

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
EqualsAllBenchmark - Duration of single invocation 30.75 ns 28.18 ns 0.92 0.23 False
EqualsAnyBenchmark - Duration of single invocation 88.00 ns 79.99 ns 0.91 0.10 False
GreaterThanOrEqualAnyBenchmark - Duration of single invocation 30.31 ns 25.40 ns 0.84 0.23 False
LessThanOrEqualAllBenchmark - Duration of single invocation 29.91 ns 27.12 ns 0.91 0.21 False
EqualsBenchmark - Duration of single invocation 40.47 ns 36.87 ns 0.91 0.25 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Int*'

Payloads

Baseline
Compare

Histogram

System.Runtime.Intrinsics.Tests.Perf_Vector128Int.EqualsAllBenchmark

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 28.18187624876761 < 29.433544617672325.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 14.355704384304268 (T) = (0 -28.166975376563432) / Math.Sqrt((3.3837311663382983 / (166)) + (1.4068009404812465 / (33))) is greater than 1.9720790337782532 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (166) + (33) - 2, .975) and 0.11342740480031581 = (31.770636188251835 - 28.166975376563432) / 31.770636188251835 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Int.EqualsAnyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 79.9882709027062 < 84.23042821300166.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 19.93178459556755 (T) = (0 -82.82619432735278) / Math.Sqrt((6.953810626925045 / (166)) + (3.739534767976526 / (33))) is greater than 1.9720790337782532 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (166) + (33) - 2, .975) and 0.08659653371659354 = (90.67865120368819 - 82.82619432735278) / 90.67865120368819 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Int.GreaterThanOrEqualAnyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 25.398180428107917 < 28.616790167540575.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 18.109937358055493 (T) = (0 -26.868657879766946) / Math.Sqrt((2.279937400310883 / (166)) + (0.651765122575757 / (33))) is greater than 1.9720790337782532 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (166) + (33) - 2, .975) and 0.10979583902569659 = (30.182579522387265 - 26.868657879766946) / 30.182579522387265 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Int.LessThanOrEqualAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 27.11529916964386 < 28.202866058636502.
IsChangePoint: Marked as a change because one of 12/18/2022 12:43:31 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 13.464889587057446 (T) = (0 -27.514990214400726) / Math.Sqrt((2.7446031461728704 / (167)) + (0.6516840535527288 / (32))) is greater than 1.9720790337782532 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (167) + (32) - 2, .975) and 0.08581996700400504 = (30.09799954198001 - 27.514990214400726) / 30.09799954198001 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Int.EqualsBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 36.87490199524169 < 38.684603911001304.
IsChangePoint: Marked as a change because one of 12/2/2022 2:38:36 AM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 14.70353606716535 (T) = (0 -37.516585031585024) / Math.Sqrt((6.734504937661641 / (166)) + (2.4383066710506425 / (33))) is greater than 1.9720790337782532 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (166) + (33) - 2, .975) and 0.11707025268260525 = (42.49101941074208 - 37.516585031585024) / 42.49101941074208 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented Dec 27, 2022 •
edited
Loading

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 01b57ccc4cc532034f717226f4c7cc07a1ee9518
Compare fc65281072f9259f0b03a291b9c549bc9d792752
Diff Diff

Improvements in System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Byte>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
InequalityOperatorBenchmark - Duration of single invocation 27.75 ns 24.24 ns 0.87 0.29 False
LessThanOrEqualAllBenchmark - Duration of single invocation 28.27 ns 26.68 ns 0.94 0.29 False
GreaterThanAnyBenchmark - Duration of single invocation 27.20 ns 25.33 ns 0.93 0.26 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Byte&gt;*'

Payloads

Baseline
Compare

Histogram

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Byte>.InequalityOperatorBenchmark

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 24.239580851498 < 26.97171411010821.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 17.000263576821958 (T) = (0 -24.671018256433275) / Math.Sqrt((2.8997482974998348 / (166)) + (0.5411141308094989 / (33))) is greater than 1.9720790337782532 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (166) + (33) - 2, .975) and 0.11253790206550794 = (27.799517651349166 - 24.671018256433275) / 27.799517651349166 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Byte&gt;.LessThanOrEqualAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 26.68441125907375 < 27.140309420054344.
IsChangePoint: Marked as a change because one of 11/28/2022 10:23:06 PM, 11/29/2022 11:47:49 AM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 12.854949682771585 (T) = (0 -26.319876527410802) / Math.Sqrt((4.057833188815295 / (166)) + (0.292921434623054 / (33))) is greater than 1.9720790337782532 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (166) + (33) - 2, .975) and 0.08185721772855568 = (28.666430794452907 - 26.319876527410802) / 28.666430794452907 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Byte&gt;.GreaterThanAnyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 25.325337580459202 < 25.70310240895471.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 14.957660988606976 (T) = (0 -25.47187577938273) / Math.Sqrt((3.802533217430854 / (166)) + (0.18786616489300323 / (33))) is greater than 1.9720790337782532 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (166) + (33) - 2, .975) and 0.09033674870659815 = (28.001434314473652 - 25.47187577938273) / 28.001434314473652 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS ubuntu 18.04
Baseline 01b57ccc4cc532034f717226f4c7cc07a1ee9518
Compare fc65281072f9259f0b03a291b9c549bc9d792752
Diff Diff

Improvements in System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt64>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GreaterThanAnyBenchmark - Duration of single invocation 30.41 ns 27.09 ns 0.89 0.21 False
EqualityOperatorBenchmark - Duration of single invocation 28.78 ns 25.01 ns 0.87 0.23 False
GreaterThanAllBenchmark - Duration of single invocation 63.28 ns 57.89 ns 0.91 0.16 False
EqualsAllBenchmark - Duration of single invocation 30.44 ns 27.42 ns 0.90 0.24 False
SumBenchmark - Duration of single invocation 21.11 ns 20.01 ns 0.95 0.23 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt64&gt;*'

Payloads

Baseline
Compare

Histogram

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt64>.GreaterThanAnyBenchmark

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 27.093268561933524 < 28.28926975004219.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 6.127138920097033 (T) = (0 -26.47730148647963) / Math.Sqrt((1.5959023307420441 / (166)) + (7.39868826582678 / (33))) is greater than 1.9720790337782532 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (166) + (33) - 2, .975) and 0.10063678324460996 = (29.440053799399447 - 26.47730148647963) / 29.440053799399447 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt64&gt;.EqualityOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 25.012965133923394 < 27.204684854157424.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 20.83536235592691 (T) = (0 -25.23660448720958) / Math.Sqrt((2.8074824715846165 / (166)) + (0.7747029534922993 / (33))) is greater than 1.9720790337782532 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (166) + (33) - 2, .975) and 0.14230812996254905 = (29.42385881086599 - 25.23660448720958) / 29.42385881086599 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt64&gt;.GreaterThanAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 57.885517499033725 < 60.55028098036812.
IsChangePoint: Marked as a change because one of 12/2/2022 5:21:30 AM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 9.049742992129548 (T) = (0 -60.710122787441726) / Math.Sqrt((5.614222813269383 / (166)) + (9.836358660803162 / (33))) is greater than 1.9720790337782532 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (166) + (33) - 2, .975) and 0.07908483496486643 = (65.92368666784371 - 60.710122787441726) / 65.92368666784371 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt64&gt;.EqualsAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 27.420821613112114 < 29.27157559011772.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 17.447018310383715 (T) = (0 -28.048701741341524) / Math.Sqrt((2.9042901701929873 / (166)) + (0.9047968411096864 / (33))) is greater than 1.9720790337782532 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (166) + (33) - 2, .975) and 0.1164713331267425 = (31.746227137828818 - 28.048701741341524) / 31.746227137828818 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt64&gt;.SumBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 20.01136454923329 < 20.020098141553582.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 15.907875497909401 (T) = (0 -19.806998530569782) / Math.Sqrt((1.4735557295031707 / (166)) + (0.31114216721416343 / (33))) is greater than 1.9720790337782532 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (166) + (33) - 2, .975) and 0.09801292463755297 = (21.959293067043895 - 19.806998530569782) / 21.959293067043895 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 01b57ccc4cc532034f717226f4c7cc07a1ee9518
Compare fc65281072f9259f0b03a291b9c549bc9d792752
Diff Diff

Improvements in System.Collections.IterateForEach<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ImmutableStack - Duration of single invocation 35.39 μs 30.69 μs 0.87 0.13 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.IterateForEach&lt;Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.IterateForEach<Int32>.ImmutableStack(Size: 512)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 30.692806049836605 < 33.602894228394064.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 12/2/2022 5:21:30 AM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 20.50735525751523 (T) = (0 -31317.90560892917) / Math.Sqrt((5227277.741658132 / (299)) + (988432.8046467458 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.1248149195512159 = (35784.32300613458 - 31317.90560892917) / 35784.32300613458 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 01b57ccc4cc532034f717226f4c7cc07a1ee9518
Compare fc65281072f9259f0b03a291b9c549bc9d792752
Diff Diff

Improvements in System.Numerics.Tests.Perf_VectorConvert

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Convert_uint_float - Duration of single invocation 131.48 μs 103.08 μs 0.78 0.21 False
Narrow_long - Duration of single invocation 104.70 μs 90.06 μs 0.86 0.23 False
Narrow_short - Duration of single invocation 214.71 μs 193.98 μs 0.90 0.21 False
Narrow_int - Duration of single invocation 153.83 μs 132.95 μs 0.86 0.29 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorConvert*'

Payloads

Baseline
Compare

Histogram

System.Numerics.Tests.Perf_VectorConvert.Convert_uint_float

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 103.07843857442349 < 125.27400633907992.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 10/17/2022 12:47:34 PM, 12/3/2022 2:55:34 AM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 37.09474524650054 (T) = (0 -105463.45579952981) / Math.Sqrt((118085783.4661128 / (286)) + (9595318.74220102 / (33))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (286) + (33) - 2, .975) and 0.2278268288562722 = (136580.05709175236 - 105463.45579952981) / 136580.05709175236 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorConvert.Narrow_long

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 90.05805735111663 < 99.57763305708704.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 11/2/2022 9:35:40 AM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 5.241955021352897 (T) = (0 -91861.07356903155) / Math.Sqrt((181979346.38749775 / (299)) + (14926068.328979535 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.05551381717391662 = (97260.36784801408 - 91861.07356903155) / 97260.36784801408 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorConvert.Narrow_short

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 193.98207924382717 < 203.46219314986956.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 10/17/2022 12:47:34 PM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 16.387214435918207 (T) = (0 -197016.50194310554) / Math.Sqrt((147585894.2203391 / (287)) + (24057802.639199916 / (33))) is greater than 1.967451947860886 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (287) + (33) - 2, .975) and 0.08487219237030243 = (215288.50975843956 - 197016.50194310554) / 215288.50975843956 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorConvert.Narrow_int

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 132.95267164548025 < 140.79716049163295.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 10/17/2022 12:47:34 PM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 14.582861601195983 (T) = (0 -135173.4361851527) / Math.Sqrt((98534858.22838141 / (287)) + (15886367.852310307 / (33))) is greater than 1.967451947860886 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (287) + (33) - 2, .975) and 0.08923106254279106 = (148416.82739262682 - 135173.4361851527) / 148416.82739262682 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 01b57ccc4cc532034f717226f4c7cc07a1ee9518
Compare fc65281072f9259f0b03a291b9c549bc9d792752
Diff Diff

Improvements in System.Tests.Perf_String

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Replace_Char - Duration of single invocation 135.37 ns 102.04 ns 0.75 0.13 False
PadLeft - Duration of single invocation 7.29 μs 5.72 μs 0.78 0.08 False
CtorCharCount - Duration of single invocation 409.48 ns 343.61 ns 0.84 0.06 False
Replace_String - Duration of single invocation 198.89 ns 173.23 ns 0.87 0.22 False
ToLower - Duration of single invocation 1.22 μs 974.92 ns 0.80 0.03 False
Replace_Char - Duration of single invocation 63.01 ns 45.04 ns 0.71 0.30 False
ToLowerInvariant - Duration of single invocation 1.16 μs 923.85 ns 0.79 0.03 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_String*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_String.Replace_Char(text: "This is a very nice sentence", oldChar: 'z', newChar: 'y')

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 102.04291171520808 < 128.8375665732687.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 66.38372580244348 (T) = (0 -103.58336703273767) / Math.Sqrt((21.573737833113118 / (299)) + (6.711029726741398 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.25171679661118845 = (138.42802639913762 - 103.58336703273767) / 138.42802639913762 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.PadLeft(n: 2142)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 5.724020597482222 < 6.929605452525272.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 95.87500293568142 (T) = (0 -5723.550201036793) / Math.Sqrt((42168.93027854554 / (299)) + (5651.22550017382 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.22840398313572086 = (7417.8068262935885 - 5723.550201036793) / 7417.8068262935885 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.CtorCharCount(size: 100)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 343.61134176055845 < 391.74793464774825.
IsChangePoint: Marked as a change because one of 12/8/2022 9:11:55 AM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 57.4259012823579 (T) = (0 -344.7643634034558) / Math.Sqrt((1791.697166092711 / (299)) + (35.34068461560643 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.3068442042560241 = (497.38365533453316 - 344.7643634034558) / 497.38365533453316 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.Replace_String(text: "This is a very nice sentence", oldValue: "bad", newValue: "nice")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 173.2326166167342 < 190.1993569496726.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 11/15/2022 8:48:03 PM, 12/2/2022 5:21:30 AM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 28.4967229652607 (T) = (0 -175.44189355411575) / Math.Sqrt((95.86364547852254 / (290)) + (17.00484279748674 / (33))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (290) + (33) - 2, .975) and 0.12997068346379825 = (201.65055386017633 - 175.44189355411575) / 201.65055386017633 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.ToLower(s: "This is a much longer piece of text that might benefit more from vectorization.")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 974.9152945281095 < 1.1580073989837654.
IsChangePoint: Marked as a change because one of 12/8/2022 9:08:34 PM, 12/21/2022 9:23:17 AM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 84.67965676161647 (T) = (0 -983.2397477777324) / Math.Sqrt((2131.5287021596732 / (299)) + (123.85952024041572 / (18))) is greater than 1.9675235316906419 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (18) - 2, .975) and 0.24377487766923797 = (1300.1945039160937 - 983.2397477777324) / 1300.1945039160937 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.Replace_Char(text: "Hello", oldChar: 'a', newChar: 'b')

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 45.04470440505091 < 59.155171175807915.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 35.278050530639995 (T) = (0 -46.94488536941917) / Math.Sqrt((12.132635439081168 / (299)) + (5.9112392609582765 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.2604856041106169 = (63.480691694933824 - 46.94488536941917) / 63.480691694933824 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.ToLowerInvariant(s: "This is a much longer piece of text that might benefit more from vectorization.")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 923.8531314738726 < 1.1076880474007487.
IsChangePoint: Marked as a change because one of 12/8/2022 9:11:55 AM, 12/21/2022 9:23:17 AM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 99.02034804737838 (T) = (0 -921.5519560480839) / Math.Sqrt((1932.8660301945713 / (299)) + (74.21299025364485 / (18))) is greater than 1.9675235316906419 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (18) - 2, .975) and 0.2590518361809267 = (1243.7468652302523 - 921.5519560480839) / 1243.7468652302523 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 01b57ccc4cc532034f717226f4c7cc07a1ee9518
Compare fc65281072f9259f0b03a291b9c549bc9d792752
Diff Diff

Improvements in System.Numerics.Tests.Perf_Matrix4x4

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
MultiplyByScalarOperatorBenchmark - Duration of single invocation 90.06 ns 60.40 ns 0.67 0.21 False
MultiplyByScalarBenchmark - Duration of single invocation 89.89 ns 68.99 ns 0.77 0.22 False
LerpBenchmark - Duration of single invocation 197.83 ns 125.06 ns 0.63 0.13 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_Matrix4x4*'

Payloads

Baseline
Compare

Histogram

System.Numerics.Tests.Perf_Matrix4x4.MultiplyByScalarOperatorBenchmark

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 60.39795516780132 < 85.21018174406552.
IsChangePoint: Marked as a change because one of 10/5/2022 3:28:28 PM, 10/19/2022 7:27:15 PM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 57.35565268885392 (T) = (0 -60.1050460285119) / Math.Sqrt((77.90382656905014 / (299)) + (2.010874809185556 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.3510955887404966 = (92.62542369198857 - 60.1050460285119) / 92.62542369198857 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_Matrix4x4.MultiplyByScalarBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 68.9946463794388 < 83.1067543335148.
IsChangePoint: Marked as a change because one of 10/5/2022 3:28:28 PM, 10/19/2022 7:27:15 PM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 16.238241566955747 (T) = (0 -71.77659652582503) / Math.Sqrt((99.0788411511684 / (299)) + (60.462616713704854 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.24968179675884689 = (95.66154228402206 - 71.77659652582503) / 95.66154228402206 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_Matrix4x4.LerpBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 125.0567806683044 < 187.6891200764956.
IsChangePoint: Marked as a change because one of 10/5/2022 3:28:28 PM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 129.41184636820387 (T) = (0 -124.63702115037027) / Math.Sqrt((116.63825529972794 / (299)) + (1.4639436594227553 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.4063112826016009 = (209.93665114025015 - 124.63702115037027) / 209.93665114025015 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented Dec 27, 2022 •
edited
Loading

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 01b57ccc4cc532034f717226f4c7cc07a1ee9518
Compare fc65281072f9259f0b03a291b9c549bc9d792752
Diff Diff

Improvements in System.Text.RegularExpressions.Tests.Perf_Regex_Industry_SliceSlice

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Count - Duration of single invocation 26.90 secs 23.12 secs 0.86 0.09 False
Count - Duration of single invocation 26.83 secs 23.71 secs 0.88 0.05 False
Count - Duration of single invocation 26.73 secs 22.94 secs 0.86 0.04 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Industry_SliceSlice*'

Payloads

Baseline
Compare

Histogram

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_SliceSlice.Count(Options: NonBacktracking)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 23.122929454 < 25.65294785554167.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 11/15/2022 5:29:43 PM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 54.21965939894017 (T) = (0 -23290354114.06565) / Math.Sqrt((8.29983237589385E+17 / (299)) + (70977072496656480 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.1404520710191136 = (27096050527.024834 - 23290354114.06565) / 27096050527.024834 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_SliceSlice.Count(Options: None)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 23.710210973 < 25.10992122929167.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 39.7098120801297 (T) = (0 -23060807821.676765) / Math.Sqrt((4.072703484156316E+17 / (299)) + (2.2804238996204557E+17 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.13541020050769323 = (26672542094.78094 - 23060807821.676765) / 26672542094.78094 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_SliceSlice.Count(Options: Compiled)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 22.93722866666667 < 25.33281312900833.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 57.71302043013705 (T) = (0 -22961796416.232323) / Math.Sqrt((5.179886402794731E+17 / (299)) + (83067915321063970 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.1407816853842901 = (26724053742.38573 - 22961796416.232323) / 26724053742.38573 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS ubuntu 18.04
Baseline 01b57ccc4cc532034f717226f4c7cc07a1ee9518
Compare fc65281072f9259f0b03a291b9c549bc9d792752
Diff Diff

Improvements in System.Runtime.Intrinsics.Tests.Perf_Vector128

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ConvertUIntToFloatBenchmark - Duration of single invocation 70.01 ns 63.59 ns 0.91 0.17 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128*'

Payloads

Baseline
Compare

Histogram

System.Runtime.Intrinsics.Tests.Perf_Vector128.ConvertUIntToFloatBenchmark

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 63.593755602396755 < 66.60233219359156.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 13.70589786506004 (T) = (0 -64.60536586264838) / Math.Sqrt((7.881085610204042 / (166)) + (5.900372707143939 / (33))) is greater than 1.9720790337782532 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (166) + (33) - 2, .975) and 0.09166505024656876 = (71.12504685654295 - 64.60536586264838) / 71.12504685654295 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 01b57ccc4cc532034f717226f4c7cc07a1ee9518
Compare fc65281072f9259f0b03a291b9c549bc9d792752
Diff Diff

Improvements in System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Mariomkas

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Count - Duration of single invocation 20.20 ms 17.12 ms 0.85 0.05 False
Count - Duration of single invocation 54.74 ms 51.29 ms 0.94 0.03 False
Count - Duration of single invocation 19.88 ms 16.76 ms 0.84 0.09 False
Count - Duration of single invocation 54.55 ms 51.31 ms 0.94 0.02 False
Count - Duration of single invocation 20.19 ms 17.15 ms 0.85 0.10 False
Count - Duration of single invocation 208.56 ms 197.88 ms 0.95 0.06 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Mariomkas*'

Payloads

Baseline
Compare

Histogram

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Mariomkas.Count(Pattern: "[\w\.+-]+@[\w\.-]+\.[\w\.-]+", Options: Compiled)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 17.119652214285715 < 19.228968000083874.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 107.39703349256324 (T) = (0 -17149810.531279407) / Math.Sqrt((164332379409.9132 / (289)) + (6077961127.250822 / (33))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (289) + (33) - 2, .975) and 0.1466267393136122 = (20096493.904068917 - 17149810.531279407) / 20096493.904068917 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Mariomkas.Count(Pattern: "[\\w]+://[^/\\s?#]+[^\\s?#]+(?:\\?[^\\s#]*)?(?:#[^\\s]*)?", Options: None)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 51.29157896428572 < 52.072489231112634.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 12/2/2022 5:21:30 AM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 29.112048638038306 (T) = (0 -51780379.492317416) / Math.Sqrt((1331176995687.5684 / (290)) + (239421501498.82983 / (33))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (290) + (33) - 2, .975) and 0.05766206922732321 = (54948843.510798424 - 51780379.492317416) / 54948843.510798424 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Mariomkas.Count(Pattern: "[\\w\\.+-]+@[\\w\\.-]+\\.[\\w\\.-]+", Options: NonBacktracking)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 16.75932444444444 < 18.877370344196432.
IsChangePoint: Marked as a change because one of 11/18/2022 3:21:53 PM, 11/19/2022 1:29:52 AM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 35.88228286460425 (T) = (0 -16830388.64371174) / Math.Sqrt((873180706547.0952 / (292)) + (135897828409.31883 / (33))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (292) + (33) - 2, .975) and 0.15236421943111914 = (19855684.516309846 - 16830388.64371174) / 19855684.516309846 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Mariomkas.Count(Pattern: "[\\w]+://[^/\\s?#]+[^\\s?#]+(?:\\?[^\\s#]*)?(?:#[^\\s]*)?", Options: Compiled)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 51.30537889285714 < 51.891849009198715.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 35.64199589352466 (T) = (0 -51540093.132416196) / Math.Sqrt((900241220319.4191 / (254)) + (192749757629.20282 / (33))) is greater than 1.968322603028609 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (254) + (33) - 2, .975) and 0.06278783931417667 = (54992983.760156 - 51540093.132416196) / 54992983.760156 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Mariomkas.Count(Pattern: "[\\w\\.+-]+@[\\w\\.-]+\\.[\\w\\.-]+", Options: None)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 17.150858566326534 < 19.216068576785712.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 87.5480046491793 (T) = (0 -17151099.073040444) / Math.Sqrt((281861780567.70856 / (291)) + (6887121175.19782 / (33))) is greater than 1.9673585853226652 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (291) + (33) - 2, .975) and 0.14904110153987615 = (20155026.410883874 - 17151099.073040444) / 20155026.410883874 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Mariomkas.Count(Pattern: "(?:(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9])\\.){3}(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9])", Options: NonBacktracking)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 197.8812007142857 < 197.89907571458335.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 15.129853197766943 (T) = (0 -198661006.84532142) / Math.Sqrt((28535070262783.902 / (216)) + (14928558296095.715 / (33))) is greater than 1.9696147551832122 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (216) + (33) - 2, .975) and 0.05502139774102082 = (210228047.8842808 - 198661006.84532142) / 210228047.8842808 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 01b57ccc4cc532034f717226f4c7cc07a1ee9518
Compare fc65281072f9259f0b03a291b9c549bc9d792752
Diff Diff

Improvements in System.Collections.ContainsTrue<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Stack - Duration of single invocation 5.99 ms 383.53 μs 0.06 0.06 False
ImmutableArray - Duration of single invocation 1.14 ms 403.90 μs 0.35 0.05 False
List - Duration of single invocation 1.14 ms 377.75 μs 0.33 0.04 False
Span - Duration of single invocation 763.94 μs 346.22 μs 0.45 0.09 False
Array - Duration of single invocation 1.17 ms 386.33 μs 0.33 0.02 False
Queue - Duration of single invocation 1.10 ms 375.93 μs 0.34 0.05 False
ICollection - Duration of single invocation 1.11 ms 379.19 μs 0.34 0.04 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.ContainsTrue&lt;Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.ContainsTrue<Int32>.Stack(Size: 512)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 383.5314830792683 < 5.728429813167735.
IsChangePoint: Marked as a change because one of 12/14/2022 5:20:21 PM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 16.961798940317962 (T) = (0 -384348.5479113027) / Math.Sqrt((1383908300086.465 / (299)) + (9319072.80850351 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.7501539780051972 = (1538341.6747747848 - 384348.5479113027) / 1538341.6747747848 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.ContainsTrue&lt;Int32&gt;.ImmutableArray(Size: 512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 403.8978235042735 < 1.0903770176551053.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 415.9855427157223 (T) = (0 -406024.0631191787) / Math.Sqrt((796542877.6598607 / (299)) + (8952360.985072283 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.6370641007881592 = (1118721.1405675465 - 406024.0631191787) / 1118721.1405675465 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.ContainsTrue&lt;Int32&gt;.List(Size: 512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 377.75470962301586 < 1.0865960825512657.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 474.86469783099477 (T) = (0 -377246.3574336832) / Math.Sqrt((652353168.2571821 / (299)) + (1822476.959443469 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.6531017652278995 = (1087484.2233818814 - 377246.3574336832) / 1087484.2233818814 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.ContainsTrue&lt;Int32&gt;.Span(Size: 512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 346.2230531577797 < 726.5316950115326.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 233.30783037119113 (T) = (0 -345102.19413910335) / Math.Sqrt((785935090.6410106 / (299)) + (13587233.85882876 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.5410307572831458 = (751907.0168978679 - 345102.19413910335) / 751907.0168978679 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.ContainsTrue&lt;Int32&gt;.Array(Size: 512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 386.33220378605773 < 1.0666794065369896.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 535.9768790630222 (T) = (0 -387601.4343949136) / Math.Sqrt((467258062.5364888 / (299)) + (5920268.861093082 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.6460382861316908 = (1095037.7377230127 - 387601.4343949136) / 1095037.7377230127 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.ContainsTrue&lt;Int32&gt;.Queue(Size: 512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 375.9271681156015 < 1.0593524749590773.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 326.30361472478455 (T) = (0 -376686.3984349746) / Math.Sqrt((650340301.2925456 / (299)) + (83593348.64868958 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.6527312028798119 = (1084711.328972655 - 376686.3984349746) / 1084711.328972655 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.ContainsTrue&lt;Int32&gt;.ICollection(Size: 512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 379.19480435267866 < 1.0619872083519346.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 336.2752444921199 (T) = (0 -381962.9479368223) / Math.Sqrt((715130486.1756146 / (299)) + (66275692.70308499 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.6487204344070397 = (1087347.4729225098 - 381962.9479368223) / 1087347.4729225098 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 01b57ccc4cc532034f717226f4c7cc07a1ee9518
Compare fc65281072f9259f0b03a291b9c549bc9d792752
Diff Diff

Improvements in System.IO.Tests.BinaryWriterExtendedTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
WriteAsciiString - Duration of single invocation 38.72 μs 32.16 μs 0.83 0.11 False
WriteAsciiCharArray - Duration of single invocation 9.43 ms 8.19 ms 0.87 0.15 False
WriteAsciiString - Duration of single invocation 14.57 ms 12.82 ms 0.88 0.11 False
WriteAsciiCharArray - Duration of single invocation 38.13 μs 31.76 μs 0.83 0.12 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.IO.Tests.BinaryWriterExtendedTests*'

Payloads

Baseline
Compare

Histogram

System.IO.Tests.BinaryWriterExtendedTests.WriteAsciiString(StringLengthInChars: 8000)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 32.16124318415638 < 36.79919057548568.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 12/2/2022 5:21:30 AM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 32.441546621638906 (T) = (0 -32346.934822371386) / Math.Sqrt((11995272.363909807 / (299)) + (510916.57930347964 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.19125704792984213 = (39996.558535159034 - 32346.934822371386) / 39996.558535159034 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.BinaryWriterExtendedTests.WriteAsciiCharArray(StringLengthInChars: 2000000)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 8.193066962053571 < 9.007431640811012.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 31.190524536773626 (T) = (0 -7943248.348864157) / Math.Sqrt((619924615191.4202 / (299)) + (49958329145.30002 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.19040241727174023 = (9811353.835934434 - 7943248.348864157) / 9811353.835934434 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.BinaryWriterExtendedTests.WriteAsciiString(StringLengthInChars: 2000000)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 12.819147375 < 13.820140359121737.
IsChangePoint: Marked as a change because one of 11/16/2022 11:42:33 AM, 12/2/2022 5:21:30 AM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 36.8327993394649 (T) = (0 -12874718.730963657) / Math.Sqrt((632307797035.3252 / (299)) + (31890475765.746174 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.13703852651275963 = (14919227.713534793 - 12874718.730963657) / 14919227.713534793 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.BinaryWriterExtendedTests.WriteAsciiCharArray(StringLengthInChars: 8000)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 31.76231845098499 < 36.323742308123315.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 45.01484542646285 (T) = (0 -31871.021987325315) / Math.Sqrt((6476048.514114705 / (299)) + (189431.18297525946 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.18949074124340642 = (39322.218275728046 - 31871.021987325315) / 39322.218275728046 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented Dec 27, 2022 •
edited
Loading

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 01b57ccc4cc532034f717226f4c7cc07a1ee9518
Compare fc65281072f9259f0b03a291b9c549bc9d792752
Diff Diff

Improvements in System.IO.Tests.Perf_File

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Exists - Duration of single invocation 3.70 μs 3.23 μs 0.87 0.19 False
AppendAllTextAsync - Duration of single invocation 120.09 μs 109.68 μs 0.91 0.06 False
AppendAllText - Duration of single invocation 80.63 μs 71.00 μs 0.88 0.06 False
WriteAllText - Duration of single invocation 832.23 μs 781.88 μs 0.94 0.10 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.IO.Tests.Perf_File*'

Payloads

Baseline
Compare

Histogram

System.IO.Tests.Perf_File.Exists

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 3.2326717976634867 < 3.509846682330246.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 11/16/2022 11:42:33 AM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 8.402819535090755 (T) = (0 -3398.616646485012) / Math.Sqrt((23176.92320124354 / (299)) + (13724.422716060206 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.05206021897261721 = (3585.266400363081 - 3398.616646485012) / 3585.266400363081 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.Perf_File.AppendAllTextAsync(size: 10000)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 109.6839401 < 113.66698045065476.
IsChangePoint: Marked as a change because one of 11/15/2022 5:29:43 PM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 29.869129839009595 (T) = (0 -109339.22590229026) / Math.Sqrt((14310117.69677384 / (299)) + (2122432.224730727 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.08382527036824307 = (119343.20208355623 - 109339.22590229026) / 119343.20208355623 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.Perf_File.AppendAllText(size: 10000)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 71.00329673214284 < 75.90371146071429.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 22.338209425135606 (T) = (0 -71587.97262002026) / Math.Sqrt((5814415.083456031 / (299)) + (2973526.3864110205 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.09361262192256682 = (78981.6521627516 - 71587.97262002026) / 78981.6521627516 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.Perf_File.WriteAllText(size: 100000)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 781.8760651041666 < 791.3428528718172.
IsChangePoint: Marked as a change because one of 11/16/2022 11:42:33 AM, 12/2/2022 5:21:30 AM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 29.590603461454116 (T) = (0 -751633.8123859547) / Math.Sqrt((727251213.2873294 / (299)) + (212750146.97932634 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.10499338998090121 = (839808.1131154052 - 751633.8123859547) / 839808.1131154052 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS ubuntu 18.04
Baseline 01b57ccc4cc532034f717226f4c7cc07a1ee9518
Compare fc65281072f9259f0b03a291b9c549bc9d792752
Diff Diff

Improvements in System.Text.Json.Tests.Utf8JsonReaderCommentsTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Utf8JsonReaderCommentParsing - Duration of single invocation 11.70 μs 11.05 μs 0.94 0.08 False
Utf8JsonReaderCommentParsing - Duration of single invocation 137.86 μs 130.34 μs 0.95 0.04 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Tests.Utf8JsonReaderCommentsTests*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Tests.Utf8JsonReaderCommentsTests.Utf8JsonReaderCommentParsing(CommentHandling: Allow, SegmentSize: 0, TestCase: LongSingleLine)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 11.048714371019106 < 11.087670555711577.
IsChangePoint: Marked as a change because one of 11/16/2022 6:51:59 PM, 12/6/2022 8:34:17 AM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 30.883804307720155 (T) = (0 -11099.503635352428) / Math.Sqrt((227845.16192032196 / (299)) + (19944.883409711463 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.09326113056764662 = (12241.124770906823 - 11099.503635352428) / 12241.124770906823 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Utf8JsonReaderCommentsTests.Utf8JsonReaderCommentParsing(CommentHandling: Allow, SegmentSize: 0, TestCase: LongMultiLine)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 130.34303523065475 < 130.90286385788693.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 36.54559240071262 (T) = (0 -130173.47832032738) / Math.Sqrt((16383772.96271968 / (299)) + (105011.16862882285 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.0633189272879075 = (138973.10633535005 - 130173.47832032738) / 138973.10633535005 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 01b57ccc4cc532034f717226f4c7cc07a1ee9518
Compare fc65281072f9259f0b03a291b9c549bc9d792752
Diff Diff

Improvements in System.Text.Json.Serialization.Tests.WriteJson<Hashtable>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
SerializeToWriter - Duration of single invocation 283.92 μs 267.95 μs 0.94 0.06 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.WriteJson&lt;Hashtable&gt;*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Serialization.Tests.WriteJson<Hashtable>.SerializeToWriter(Mode: SourceGen)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 267.94952157738095 < 268.6168257986111.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 26.545489469014353 (T) = (0 -263981.33529923187) / Math.Sqrt((39330971.47837242 / (112)) + (17459064.886035897 / (33))) is greater than 1.9766921979294603 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (112) + (33) - 2, .975) and 0.08621087776342648 = (288886.493475777 - 263981.33529923187) / 288886.493475777 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 01b57ccc4cc532034f717226f4c7cc07a1ee9518
Compare fc65281072f9259f0b03a291b9c549bc9d792752
Diff Diff

Improvements in System.Globalization.Tests.StringSearch

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
IndexOf_Word_NotFound - Duration of single invocation 938.07 ns 800.61 ns 0.85 0.05 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Globalization.Tests.StringSearch*'

Payloads

Baseline
Compare

Histogram

System.Globalization.Tests.StringSearch.IndexOf_Word_NotFound(Options: (en-US, Ordinal, False))

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 800.6058230097747 < 891.1505247432372.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 56.403690565527484 (T) = (0 -804.3120863334161) / Math.Sqrt((1132.4311186853167 / (299)) + (90.23823075222964 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.15188779303092792 = (948.3557478883768 - 804.3120863334161) / 948.3557478883768 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 01b57ccc4cc532034f717226f4c7cc07a1ee9518
Compare fc65281072f9259f0b03a291b9c549bc9d792752
Diff Diff

Improvements in System.Collections.ContainsTrue<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ImmutableList - Duration of single invocation 8.32 ms 7.50 ms 0.90 0.05 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.ContainsTrue&lt;String&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.ContainsTrue<String>.ImmutableList(Size: 512)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 7.499548696428572 < 7.954445067433036.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 39.929328960571866 (T) = (0 -7549113.424757419) / Math.Sqrt((62509511301.23496 / (299)) + (9693968273.137238 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.10602910064887276 = (8444473.338267284 - 7549113.424757419) / 8444473.338267284 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 01b57ccc4cc532034f717226f4c7cc07a1ee9518
Compare fc65281072f9259f0b03a291b9c549bc9d792752
Diff Diff

Improvements in System.Numerics.Tests.Perf_VectorOf<Byte>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
EqualsStaticBenchmark - Duration of single invocation 234.99 ns 219.51 ns 0.93 0.08 False
EqualsAllBenchmark - Duration of single invocation 19.89 ns 18.40 ns 0.93 0.27 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;Byte&gt;*'

Payloads

Baseline
Compare

Histogram

System.Numerics.Tests.Perf_VectorOf<Byte>.EqualsStaticBenchmark

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 219.50737838023468 < 223.38041230365542.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/11/2022 6:23:40 AM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 23.737045360613575 (T) = (0 -220.12587371126406) / Math.Sqrt((43.96841504783483 / (299)) + (3.1756664491151003 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.05050174641467685 = (231.83389003620033 - 220.12587371126406) / 231.83389003620033 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Byte&gt;.EqualsAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 18.403367089269732 < 18.75191730570154.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/14/2022 1:19:37 PM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 7.932852889720303 (T) = (0 -18.60659483229397) / Math.Sqrt((2.2195435687685934 / (299)) + (1.718978135884437 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.09421012797624555 = (20.541844645185 - 18.60659483229397) / 20.541844645185 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 01b57ccc4cc532034f717226f4c7cc07a1ee9518
Compare fc65281072f9259f0b03a291b9c549bc9d792752
Diff Diff

Improvements in System.Collections.TryGetValueFalse<Int32, Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ImmutableDictionary - Duration of single invocation 164.22 μs 147.97 μs 0.90 0.07 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.TryGetValueFalse&lt;Int32, Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.TryGetValueFalse<Int32, Int32>.ImmutableDictionary(Size: 512)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 147.97152712301587 < 155.06294926215278.
IsChangePoint: Marked as a change because one of 11/22/2022 11:24:47 AM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 31.860090719447495 (T) = (0 -150555.1986207674) / Math.Sqrt((31540166.225856025 / (299)) + (4281739.848809637 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.09308291751223184 = (166007.67757927635 - 150555.1986207674) / 166007.67757927635 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 01b57ccc4cc532034f717226f4c7cc07a1ee9518
Compare fc65281072f9259f0b03a291b9c549bc9d792752
Diff Diff

Improvements in System.Collections.Tests.Perf_BitArray

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
BitArraySetAll - Duration of single invocation 497.50 ns 391.21 ns 0.79 0.22 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.Tests.Perf_BitArray*'

Payloads

Baseline
Compare

Histogram

System.Collections.Tests.Perf_BitArray.BitArraySetAll(Size: 512)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 391.21236897264214 < 465.81999079589053.
IsChangePoint: Marked as a change because one of 10/1/2022 4:32:12 PM, 10/3/2022 11:00:05 AM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 43.51713028094292 (T) = (0 -394.89186952074584) / Math.Sqrt((956.2402391277483 / (299)) + (73.44365353431671 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.20422911564454374 = (496.23814754242113 - 394.89186952074584) / 496.23814754242113 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 01b57ccc4cc532034f717226f4c7cc07a1ee9518
Compare fc65281072f9259f0b03a291b9c549bc9d792752
Diff Diff

Improvements in System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Single>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
LessThanOrEqualAllBenchmark - Duration of single invocation 31.51 ns 29.01 ns 0.92 0.15 False
InequalityOperatorBenchmark - Duration of single invocation 35.09 ns 28.90 ns 0.82 0.27 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Single&gt;*'

Payloads

Baseline
Compare

Histogram

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Single>.LessThanOrEqualAllBenchmark

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 29.00872602141087 < 29.85273010809865.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 9.644455042852634 (T) = (0 -29.55372094255486) / Math.Sqrt((1.5519793280908438 / (165)) + (1.4492457173433768 / (33))) is greater than 1.9721412216619414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (165) + (33) - 2, .975) and 0.070075822261745 = (31.780785627528143 - 29.55372094255486) / 31.780785627528143 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Single&gt;.InequalityOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 28.897890511289265 < 29.273599173688524.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 9.40643651673033 (T) = (0 -29.211755279362023) / Math.Sqrt((3.8329710675921005 / (166)) + (0.8940273534474167 / (33))) is greater than 1.9720790337782532 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (166) + (33) - 2, .975) and 0.06728093066027357 = (31.31892146264478 - 29.211755279362023) / 31.31892146264478 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 01b57ccc4cc532034f717226f4c7cc07a1ee9518
Compare fc65281072f9259f0b03a291b9c549bc9d792752
Diff Diff

Improvements in System.MathBenchmarks.Double

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
CopySign - Duration of single invocation 106.87 μs 98.47 μs 0.92 0.14 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.MathBenchmarks.Double*'

Payloads

Baseline
Compare

Histogram

System.MathBenchmarks.Double.CopySign

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 98.47204232180295 < 101.47267050759778.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 26.74598750958757 (T) = (0 -98682.2399576787) / Math.Sqrt((47979956.84668549 / (299)) + (1158741.7138243043 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.10703326433094744 = (110510.54425195507 - 98682.2399576787) / 110510.54425195507 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 01b57ccc4cc532034f717226f4c7cc07a1ee9518
Compare fc65281072f9259f0b03a291b9c549bc9d792752
Diff Diff

Improvements in Microsoft.Extensions.Primitives.StringSegmentBenchmark

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
LastIndexOf - Duration of single invocation 78.37 ns 68.93 ns 0.88 0.16 False
IndexOf - Duration of single invocation 107.21 ns 92.81 ns 0.87 0.28 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Microsoft.Extensions.Primitives.StringSegmentBenchmark*'

Payloads

Baseline
Compare

Histogram

Microsoft.Extensions.Primitives.StringSegmentBenchmark.LastIndexOf

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 68.93415824086372 < 74.97044069986086.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 42.31418479202355 (T) = (0 -69.31284070384923) / Math.Sqrt((11.003105659956585 / (299)) + (0.9894604550544011 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.1362656851342836 = (80.24787195658101 - 69.31284070384923) / 80.24787195658101 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### Microsoft.Extensions.Primitives.StringSegmentBenchmark.IndexOf

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 92.81380216486812 < 101.20198782040433.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 12.217001676501116 (T) = (0 -95.3214067630488) / Math.Sqrt((22.941668269322765 / (299)) + (18.38313679970486 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.09258754256248113 = (105.04749629758338 - 95.3214067630488) / 105.04749629758338 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 01b57ccc4cc532034f717226f4c7cc07a1ee9518
Compare fc65281072f9259f0b03a291b9c549bc9d792752
Diff Diff

Improvements in System.Collections.CreateAddAndClear<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Span - Duration of single invocation 5.93 μs 5.43 μs 0.91 0.06 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.CreateAddAndClear&lt;String&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.CreateAddAndClear<String>.Span(Size: 512)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 5.426272307437492 < 5.651716539413842.
IsChangePoint: Marked as a change because one of 10/22/2022 11:54:51 PM, 12/12/2022 1:34:22 PM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 22.04304789750847 (T) = (0 -5472.712023954934) / Math.Sqrt((47317.330268031896 / (299)) + (4871.61229191131 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.06580794274767834 = (5858.230094624724 - 5472.712023954934) / 5858.230094624724 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented Dec 27, 2022 •
edited
Loading

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 01b57ccc4cc532034f717226f4c7cc07a1ee9518
Compare fc65281072f9259f0b03a291b9c549bc9d792752
Diff Diff

Improvements in System.Text.Json.Document.Tests.Perf_EnumerateObject

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
EnumerateProperties - Duration of single invocation 50.37 μs 0.00 ns 0.00 0.15 False
EnumerateProperties - Duration of single invocation 53.59 μs 0.00 ns 0.00 0.15 False
EnumerateProperties - Duration of single invocation 52.53 μs 0.00 ns 0.00 0.15 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Document.Tests.Perf_EnumerateObject*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Document.Tests.Perf_EnumerateObject.EnumerateProperties(TestCase: NumericProperties)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 0 < 48.77527047189622.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 528.6692184707274 (T) = (0 -0) / Math.Sqrt((2705473.0065488024 / (299)) + (0 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 1 = (50288.65646546514 - 0) / 50288.65646546514 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Document.Tests.Perf_EnumerateObject.EnumerateProperties(TestCase: StringProperties)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 0 < 47.69189078884749.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 528.1031726330704 (T) = (0 -0) / Math.Sqrt((2713148.109716745 / (299)) + (0 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 1 = (50306.01704077589 - 0) / 50306.01704077589 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Document.Tests.Perf_EnumerateObject.EnumerateProperties(TestCase: ObjectProperties)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 0 < 49.94217960527047.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 573.9914817917912 (T) = (0 -0) / Math.Sqrt((2488903.8662649286 / (299)) + (0 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 1 = (52368.95579136021 - 0) / 52368.95579136021 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS ubuntu 18.04
Baseline 01b57ccc4cc532034f717226f4c7cc07a1ee9518
Compare fc65281072f9259f0b03a291b9c549bc9d792752
Diff Diff

Improvements in System.Text.Tests.Perf_Encoding

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GetBytes - Duration of single invocation 4.17 μs 3.79 μs 0.91 0.10 False
GetBytes - Duration of single invocation 2.74 μs 2.36 μs 0.86 0.09 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Tests.Perf_Encoding*'

Payloads

Baseline
Compare

Histogram

System.Text.Tests.Perf_Encoding.GetBytes(size: 512, encName: "utf-8")

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 3.7862651153438467 < 3.9490724188292083.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 12/2/2022 5:21:30 AM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 44.3266568399236 (T) = (0 -3765.203188582827) / Math.Sqrt((25269.45911083518 / (285)) + (1205.2067445329933 / (33))) is greater than 1.9674995188240387 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (285) + (33) - 2, .975) and 0.11638993292144466 = (4261.159224941345 - 3765.203188582827) / 4261.159224941345 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Tests.Perf_Encoding.GetBytes(size: 512, encName: "ascii")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.3583133189262964 < 2.6073675583991163.
IsChangePoint: Marked as a change because one of 10/5/2022 3:28:28 PM, 10/6/2022 8:29:49 AM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 42.48509639062082 (T) = (0 -2349.774487968605) / Math.Sqrt((34456.058824333115 / (280)) + (517.2026138015227 / (33))) is greater than 1.9676211333067681 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (280) + (33) - 2, .975) and 0.17556978137728374 = (2850.1799605236592 - 2349.774487968605) / 2850.1799605236592 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 01b57ccc4cc532034f717226f4c7cc07a1ee9518
Compare fc65281072f9259f0b03a291b9c549bc9d792752
Diff Diff

Improvements in System.Tests.Perf_Array

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
IndexOfChar - Duration of single invocation 289.59 ns 241.68 ns 0.83 0.08 False
IndexOfShort - Duration of single invocation 288.43 ns 240.09 ns 0.83 0.11 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Array*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Array.IndexOfChar

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 241.68235674451293 < 264.2962050139636.
IsChangePoint: Marked as a change because one of 9/28/2022 3:03:11 PM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 55.579889535502296 (T) = (0 -238.58289765535918) / Math.Sqrt((93.96436476931076 / (284)) + (9.403136234971171 / (33))) is greater than 1.9675235316906419 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (284) + (33) - 2, .975) and 0.15455557473076886 = (282.19820312776045 - 238.58289765535918) / 282.19820312776045 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Array.IndexOfShort

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 240.08852070350636 < 274.90717992914557.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 40.43907859702285 (T) = (0 -240.37202059108512) / Math.Sqrt((141.79947976167196 / (290)) + (21.303778741883225 / (33))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (290) + (33) - 2, .975) and 0.1519638318322508 = (283.44548217846454 - 240.37202059108512) / 283.44548217846454 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 01b57ccc4cc532034f717226f4c7cc07a1ee9518
Compare fc65281072f9259f0b03a291b9c549bc9d792752
Diff Diff

Improvements in System.Tests.Perf_Uri

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
BuilderToString - Duration of single invocation 3.55 μs 3.27 μs 0.92 0.11 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Uri*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Uri.BuilderToString

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 3.266510625524329 < 3.311686267123709.
IsChangePoint: Marked as a change because one of 12/3/2022 2:55:34 AM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 31.826329364268506 (T) = (0 -3240.6440119186177) / Math.Sqrt((51122.643431732366 / (299)) + (6339.191486777971 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.15763540760378855 = (3847.0800425029743 - 3240.6440119186177) / 3847.0800425029743 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 01b57ccc4cc532034f717226f4c7cc07a1ee9518
Compare fc65281072f9259f0b03a291b9c549bc9d792752
Diff Diff

Improvements in System.Text.RegularExpressions.Tests.Perf_Regex_Common

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
MatchesWord - Duration of single invocation 37.92 μs 33.20 μs 0.88 0.03 False
MatchesWord - Duration of single invocation 37.43 μs 34.49 μs 0.92 0.06 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Common*'

Payloads

Baseline
Compare

Histogram

System.Text.RegularExpressions.Tests.Perf_Regex_Common.MatchesWord(Options: None)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 33.19582895699501 < 35.84033776852157.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 40.13983187892303 (T) = (0 -33691.08691626898) / Math.Sqrt((1758876.6118471588 / (299)) + (185863.08792723579 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.11335425537511169 = (37998.36306722772 - 33691.08691626898) / 37998.36306722772 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.MatchesWord(Options: Compiled)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 34.48538834516179 < 35.79200049851562.
IsChangePoint: Marked as a change because one of 11/16/2022 11:42:33 AM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 40.167837091032986 (T) = (0 -33767.02275337846) / Math.Sqrt((1768308.3662038066 / (299)) + (175181.96648743254 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.1119147772173038 = (38022.27746518967 - 33767.02275337846) / 38022.27746518967 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 01b57ccc4cc532034f717226f4c7cc07a1ee9518
Compare fc65281072f9259f0b03a291b9c549bc9d792752
Diff Diff

Improvements in MicroBenchmarks.Serializers.Json_ToStream<IndexViewModel>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
SystemTextJson_SourceGen_ - Duration of single invocation 580.91 μs 538.12 μs 0.93 0.05 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Json_ToStream&lt;IndexViewModel&gt;*'

Payloads

Baseline
Compare

Histogram

MicroBenchmarks.Serializers.Json_ToStream<IndexViewModel>.SystemTextJson_SourceGen_

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 538.1212778514589 < 555.3115072810939.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 28.961198636166834 (T) = (0 -539686.1646407313) / Math.Sqrt((120732007.36591357 / (112)) + (50826119.238098405 / (33))) is greater than 1.9766921979294603 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (112) + (33) - 2, .975) and 0.07989335501862666 = (586547.4046779183 - 539686.1646407313) / 586547.4046779183 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 01b57ccc4cc532034f717226f4c7cc07a1ee9518
Compare fc65281072f9259f0b03a291b9c549bc9d792752
Diff Diff

Improvements in System.Numerics.Tests.Perf_VectorOf<Int16>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
EqualsStaticBenchmark - Duration of single invocation 148.60 ns 130.09 ns 0.88 0.17 False
NegateBenchmark - Duration of single invocation 164.26 ns 126.02 ns 0.77 0.15 False
LessThanBenchmark - Duration of single invocation 149.01 ns 139.49 ns 0.94 0.13 False
GreaterThanBenchmark - Duration of single invocation 146.47 ns 137.77 ns 0.94 0.10 False
MaxBenchmark - Duration of single invocation 172.74 ns 157.79 ns 0.91 0.12 False
EqualityOperatorBenchmark - Duration of single invocation 21.34 ns 18.91 ns 0.89 0.21 False
EqualsBenchmark - Duration of single invocation 40.55 ns 35.34 ns 0.87 0.21 False
SquareRootBenchmark - Duration of single invocation 124.85 ns 115.15 ns 0.92 0.12 False
AddBenchmark - Duration of single invocation 133.79 ns 125.83 ns 0.94 0.20 False
SubtractBenchmark - Duration of single invocation 135.22 ns 126.03 ns 0.93 0.16 False
DotBenchmark - Duration of single invocation 119.82 ns 99.91 ns 0.83 0.11 False
MinBenchmark - Duration of single invocation 174.62 ns 162.32 ns 0.93 0.15 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;Int16&gt;*'

Payloads

Baseline
Compare

Histogram

System.Numerics.Tests.Perf_VectorOf<Int16>.EqualsStaticBenchmark

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 130.0852183181968 < 140.40751974899817.
IsChangePoint: Marked as a change because one of 10/11/2022 5:13:36 PM, 11/22/2022 11:24:47 AM, 12/2/2022 2:38:36 AM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 17.59449278748498 (T) = (0 -132.40330118053) / Math.Sqrt((51.70068128566295 / (299)) + (38.60410116442415 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.13343611559331173 = (152.79116007838567 - 132.40330118053) / 152.79116007838567 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int16&gt;.NegateBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 126.0151984394995 < 127.73819338601037.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 15.32360864611269 (T) = (0 -127.92204292726039) / Math.Sqrt((138.8947518920636 / (284)) + (23.804171189084762 / (33))) is greater than 1.9675235316906419 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (284) + (33) - 2, .975) and 0.11644350339147781 = (144.78082999591012 - 127.92204292726039) / 144.78082999591012 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int16&gt;.LessThanBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 139.49125000399027 < 141.21516799576733.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 11/22/2022 11:24:47 AM, 12/2/2022 5:21:30 AM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 27.019699704269428 (T) = (0 -139.91806740594166) / Math.Sqrt((57.151139194328024 / (299)) + (0.6703210238864425 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.08155791801774072 = (152.34283157404838 - 139.91806740594166) / 152.34283157404838 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int16&gt;.GreaterThanBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 137.76919274272467 < 140.39932347406133.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 11/22/2022 11:24:47 AM, 12/2/2022 2:38:36 AM, 12/19/2022 11:10:20 AM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 4.645773698212488 (T) = (0 -141.8597496394346) / Math.Sqrt((34.894952085396824 / (299)) + (141.99334615789536 / (31))) is greater than 1.9672228266464409 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (31) - 2, .975) and 0.06627281737326368 = (151.92847790974506 - 141.8597496394346) / 151.92847790974506 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int16&gt;.MaxBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 157.79241622255637 < 165.58231950307766.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 11/22/2022 11:24:47 AM, 12/1/2022 11:48:13 PM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 24.173632597372332 (T) = (0 -158.60374575683676) / Math.Sqrt((67.12906438780551 / (286)) + (20.13288883020924 / (33))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (286) + (33) - 2, .975) and 0.12287613109368366 = (180.82251706888263 - 158.60374575683676) / 180.82251706888263 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int16&gt;.EqualityOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 18.91360084111206 < 21.042295829869893.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/14/2022 1:19:37 PM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 18.542663943668085 (T) = (0 -19.022897162913043) / Math.Sqrt((1.7228275421985597 / (286)) + (0.6899216921829516 / (33))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (286) + (33) - 2, .975) and 0.13790322581095718 = (22.065848907516795 - 19.022897162913043) / 22.065848907516795 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int16&gt;.EqualsBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 35.339097515903994 < 37.282280187823744.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 11/15/2022 5:29:43 PM, 12/2/2022 5:21:30 AM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 12.00855530086984 (T) = (0 -34.80929978991445) / Math.Sqrt((4.912995972597365 / (299)) + (3.8102841727685273 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.11133848893668227 = (39.17048207507466 - 34.80929978991445) / 39.17048207507466 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int16&gt;.SquareRootBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 115.14665700773233 < 118.84382737548388.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 6.401129757230847 (T) = (0 -116.2276446265046) / Math.Sqrt((25.888821845330945 / (287)) + (129.51409550972488 / (33))) is greater than 1.967451947860886 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (287) + (33) - 2, .975) and 0.09938531243186223 = (129.05368547824307 - 116.2276446265046) / 129.05368547824307 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int16&gt;.AddBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 125.83009606840594 < 127.23534706056591.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 14.676404403773484 (T) = (0 -126.8496074347957) / Math.Sqrt((115.14933516865526 / (299)) + (15.06431910430909 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.0959566494264446 = (140.31363358219278 - 126.8496074347957) / 140.31363358219278 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int16&gt;.SubtractBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 126.03096599939923 < 128.4019032884784.
IsChangePoint: Marked as a change because one of 10/10/2022 3:36:43 PM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 20.653717438815395 (T) = (0 -127.34399777821477) / Math.Sqrt((152.78382361077956 / (287)) + (9.471858027253596 / (33))) is greater than 1.967451947860886 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (287) + (33) - 2, .975) and 0.12801729952273977 = (146.03959196497348 - 127.34399777821477) / 146.03959196497348 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int16&gt;.DotBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 99.91147981705817 < 113.19148543892874.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 27.65147460699113 (T) = (0 -101.09437874175512) / Math.Sqrt((33.45085141570192 / (299)) + (12.535688403840897 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.16093699364914543 = (120.484847951314 - 101.09437874175512) / 120.484847951314 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int16&gt;.MinBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 162.32251298073018 < 164.74747770725605.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 11/22/2022 11:24:47 AM, 12/2/2022 5:21:30 AM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 21.729460908392568 (T) = (0 -163.92468801674696) / Math.Sqrt((88.38076879574577 / (283)) + (11.807892074472905 / (33))) is greater than 1.9675476980854363 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (283) + (33) - 2, .975) and 0.0978900499043668 = (181.71253736794412 - 163.92468801674696) / 181.71253736794412 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented Dec 27, 2022 •
edited
Loading

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 01b57ccc4cc532034f717226f4c7cc07a1ee9518
Compare fc65281072f9259f0b03a291b9c549bc9d792752
Diff Diff

Improvements in System.Numerics.Tests.Perf_VectorOf<Int64>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
MinBenchmark - Duration of single invocation 53.99 ns 49.43 ns 0.92 0.25 False
EqualsAllBenchmark - Duration of single invocation 27.56 ns 19.45 ns 0.71 0.27 False
MaxBenchmark - Duration of single invocation 51.74 ns 47.46 ns 0.92 0.21 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;Int64&gt;*'

Payloads

Baseline
Compare

Histogram

System.Numerics.Tests.Perf_VectorOf<Int64>.MinBenchmark

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 49.426148373378524 < 49.54611770369357.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 19.732868481069485 (T) = (0 -48.235110712140965) / Math.Sqrt((10.400536688858946 / (287)) + (2.001278155158826 / (33))) is greater than 1.967451947860886 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (287) + (33) - 2, .975) and 0.1129532142800192 = (54.37718899233757 - 48.235110712140965) / 54.37718899233757 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int64&gt;.EqualsAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 19.45250551285542 < 20.258063812825863.
IsChangePoint: Marked as a change because one of 10/14/2022 9:06:25 AM, 12/19/2022 11:10:20 AM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 20.970556056284888 (T) = (0 -19.15322240961754) / Math.Sqrt((2.5330403590512574 / (299)) + (0.49804347649515335 / (31))) is greater than 1.9672228266464409 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (31) - 2, .975) and 0.1463993356177633 = (22.43815311868227 - 19.15322240961754) / 22.43815311868227 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int64&gt;.MaxBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 47.4585170028924 < 49.211408216788094.
IsChangePoint: Marked as a change because one of 12/19/2022 2:44:31 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 26.715878793128237 (T) = (0 -47.96453364789154) / Math.Sqrt((6.58575696009292 / (290)) + (0.6714423440526331 / (30))) is greater than 1.967451947860886 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (290) + (30) - 2, .975) and 0.10576575527383278 = (53.63754959147115 - 47.96453364789154) / 53.63754959147115 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS ubuntu 18.04
Baseline 01b57ccc4cc532034f717226f4c7cc07a1ee9518
Compare fc65281072f9259f0b03a291b9c549bc9d792752
Diff Diff

Improvements in System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
InequalityOperatorBenchmark - Duration of single invocation 28.94 ns 24.82 ns 0.86 0.23 False
EqualityOperatorBenchmark - Duration of single invocation 28.87 ns 25.65 ns 0.89 0.15 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt32>.InequalityOperatorBenchmark

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 24.815727391343405 < 27.50164494802604.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 21.586952080939188 (T) = (0 -25.149557592756818) / Math.Sqrt((4.991729325272642 / (165)) + (0.429963748601564 / (33))) is greater than 1.9721412216619414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (165) + (33) - 2, .975) and 0.15151610304866242 = (29.64058326047312 - 25.149557592756818) / 29.64058326047312 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt32&gt;.EqualityOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 25.646487875220327 < 27.33166965820421.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 26.57465395865433 (T) = (0 -25.539574156237425) / Math.Sqrt((1.898980995464527 / (166)) + (0.2650054177793811 / (33))) is greater than 1.9720790337782532 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (166) + (33) - 2, .975) and 0.1267828334513482 = (29.247677593400212 - 25.539574156237425) / 29.247677593400212 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 01b57ccc4cc532034f717226f4c7cc07a1ee9518
Compare fc65281072f9259f0b03a291b9c549bc9d792752
Diff Diff

Improvements in System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt16>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
EqualsAnyBenchmark - Duration of single invocation 133.95 ns 117.10 ns 0.87 0.17 False
LessThanOrEqualAllBenchmark - Duration of single invocation 29.57 ns 27.59 ns 0.93 0.32 False
EqualityOperatorBenchmark - Duration of single invocation 29.76 ns 25.04 ns 0.84 0.29 False
InequalityOperatorBenchmark - Duration of single invocation 29.18 ns 24.83 ns 0.85 0.16 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt16&gt;*'

Payloads

Baseline
Compare

Histogram

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt16>.EqualsAnyBenchmark

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 117.10197244551217 < 127.6179324006133.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 31.071200148136942 (T) = (0 -117.96048969746522) / Math.Sqrt((26.735977023261093 / (166)) + (9.143138344526243 / (33))) is greater than 1.9720790337782532 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (166) + (33) - 2, .975) and 0.14846466545996168 = (138.52682902604653 - 117.96048969746522) / 138.52682902604653 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt16&gt;.LessThanOrEqualAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 27.590902834967462 < 29.143259203309473.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 15.29582735942137 (T) = (0 -26.937061268144735) / Math.Sqrt((4.298149338595892 / (166)) + (1.1459974712443006 / (33))) is greater than 1.9720790337782532 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (166) + (33) - 2, .975) and 0.12265863326255734 = (30.703056175631176 - 26.937061268144735) / 30.703056175631176 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt16&gt;.EqualityOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 25.035935189952145 < 27.334431736619223.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 21.607752736778377 (T) = (0 -25.572611293140703) / Math.Sqrt((2.3438179911987183 / (166)) + (0.7433254965674241 / (33))) is greater than 1.9720790337782532 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (166) + (33) - 2, .975) and 0.13922796735291673 = (29.70892445761592 - 25.572611293140703) / 29.70892445761592 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt16&gt;.InequalityOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 24.830996776584183 < 27.54806323565633.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 29.601312879130187 (T) = (0 -24.9635045012375) / Math.Sqrt((3.0303422912296085 / (166)) + (0.3226304049668678 / (33))) is greater than 1.9720790337782532 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (166) + (33) - 2, .975) and 0.1656459833419391 = (29.919559327138913 - 24.9635045012375) / 29.919559327138913 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 01b57ccc4cc532034f717226f4c7cc07a1ee9518
Compare fc65281072f9259f0b03a291b9c549bc9d792752
Diff Diff

Improvements in System.Text.Json.Serialization.Tests.WriteJson<ImmutableDictionary<String, String>>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
SerializeToUtf8Bytes - Duration of single invocation 271.42 μs 253.11 μs 0.93 0.03 False
SerializeToStream - Duration of single invocation 288.04 μs 252.65 μs 0.88 0.02 False
SerializeToWriter - Duration of single invocation 264.06 μs 248.63 μs 0.94 0.02 False
SerializeToWriter - Duration of single invocation 328.26 μs 295.00 μs 0.90 0.02 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.WriteJson&lt;ImmutableDictionary&lt;String, String&gt;&gt;*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Serialization.Tests.WriteJson<ImmutableDictionary<String, String>>.SerializeToUtf8Bytes(Mode: SourceGen)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 253.1119792626728 < 256.0264013335738.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 32.64133674782348 (T) = (0 -247071.05720454032) / Math.Sqrt((36391894.927262515 / (112)) + (15603477.497797007 / (33))) is greater than 1.9766921979294603 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (112) + (33) - 2, .975) and 0.10554567390597551 = (276225.4594747953 - 247071.05720454032) / 276225.4594747953 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Serialization.Tests.WriteJson&lt;ImmutableDictionary&lt;String, String&gt;&gt;.SerializeToStream(Mode: SourceGen)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 252.65228420218895 < 260.8931915699405.
IsChangePoint: Marked as a change because one of 12/5/2022 3:14:07 PM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 19.266612541193084 (T) = (0 -252673.10762281035) / Math.Sqrt((611736643.4176723 / (112)) + (11669847.32424129 / (33))) is greater than 1.9766921979294603 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (112) + (33) - 2, .975) and 0.1553221652860208 = (299135.4777391185 - 252673.10762281035) / 299135.4777391185 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Serialization.Tests.WriteJson&lt;ImmutableDictionary&lt;String, String&gt;&gt;.SerializeToWriter(Mode: SourceGen)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 248.62695965608464 < 253.76530928344434.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 38.87299887684631 (T) = (0 -245860.45287493264) / Math.Sqrt((35215258.53965385 / (112)) + (6799924.758095796 / (33))) is greater than 1.9766921979294603 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (112) + (33) - 2, .975) and 0.10238823221890316 = (273905.11321247666 - 245860.45287493264) / 273905.11321247666 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Serialization.Tests.WriteJson&lt;ImmutableDictionary&lt;String, String&gt;&gt;.SerializeToWriter(Mode: Reflection)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 294.9972192216982 < 302.1996789988672.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 24.427275045134806 (T) = (0 -293052.2314896449) / Math.Sqrt((39980775.56376865 / (112)) + (33479412.234909877 / (33))) is greater than 1.9766921979294603 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (112) + (33) - 2, .975) and 0.08893581487943798 = (321659.2598806472 - 293052.2314896449) / 321659.2598806472 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 01b57ccc4cc532034f717226f4c7cc07a1ee9518
Compare fc65281072f9259f0b03a291b9c549bc9d792752
Diff Diff

Improvements in System.Linq.Tests.Perf_Enumerable

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Contains_ElementNotFound - Duration of single invocation 920.17 ns 345.91 ns 0.38 0.04 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Linq.Tests.Perf_Enumerable*'

Payloads

Baseline
Compare

Histogram

System.Linq.Tests.Perf_Enumerable.Contains_ElementNotFound(input: ICollection)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 345.9133256920586 < 872.7112326578465.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 257.27542268328335 (T) = (0 -349.70875113142085) / Math.Sqrt((338.09335904503394 / (299)) + (113.04618093729654 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.6109510626684307 = (898.8811369850355 - 349.70875113142085) / 898.8811369850355 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 01b57ccc4cc532034f717226f4c7cc07a1ee9518
Compare fc65281072f9259f0b03a291b9c549bc9d792752
Diff Diff

Improvements in System.Numerics.Tests.Perf_VectorOf<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
MaxBenchmark - Duration of single invocation 89.25 ns 82.37 ns 0.92 0.19 False
DivisionOperatorBenchmark - Duration of single invocation 76.62 ns 71.50 ns 0.93 0.16 False
EqualsBenchmark - Duration of single invocation 39.08 ns 33.77 ns 0.86 0.31 False
EqualityOperatorBenchmark - Duration of single invocation 21.25 ns 19.14 ns 0.90 0.28 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Numerics.Tests.Perf_VectorOf<Int32>.MaxBenchmark

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 82.37014839639511 < 85.07669340058554.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 27.926355824569903 (T) = (0 -83.37998195329686) / Math.Sqrt((15.959164899155516 / (299)) + (1.9561122604597634 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.10105412503336991 = (92.75306141917835 - 83.37998195329686) / 92.75306141917835 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int32&gt;.DivisionOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 71.49878546348863 < 72.82314297039225.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 8.076059404881798 (T) = (0 -73.08702145274438) / Math.Sqrt((9.88140647155166 / (299)) + (11.101294916189461 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.0629370613118615 = (77.99585111653667 - 73.08702145274438) / 77.99585111653667 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int32&gt;.EqualsBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 33.769866086252826 < 36.07144137406188.
IsChangePoint: Marked as a change because one of 10/11/2022 5:13:36 PM, 11/15/2022 5:29:43 PM, 12/2/2022 5:21:30 AM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 11.28345738513306 (T) = (0 -34.007105546916506) / Math.Sqrt((6.534762546675937 / (299)) + (3.3489679704499786 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.10436486906998192 = (37.969820937689086 - 34.007105546916506) / 37.969820937689086 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int32&gt;.EqualityOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 19.142009457406264 < 20.22376914045586.
IsChangePoint: Marked as a change because one of 10/14/2022 1:19:37 PM, 12/18/2022 12:43:31 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 16.373832314612855 (T) = (0 -19.122435720456313) / Math.Sqrt((2.342547406988018 / (299)) + (0.9944696459250553 / (32))) is greater than 1.9672006833568965 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (32) - 2, .975) and 0.14449996071433419 = (22.3523493189122 - 19.122435720456313) / 22.3523493189122 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 01b57ccc4cc532034f717226f4c7cc07a1ee9518
Compare fc65281072f9259f0b03a291b9c549bc9d792752
Diff Diff

Improvements in System.Text.Perf_Utf8Encoding

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GetBytes - Duration of single invocation 1.25 ms 1.11 ms 0.89 0.11 False
GetString - Duration of single invocation 2.21 ms 2.01 ms 0.91 0.03 False
GetString - Duration of single invocation 1.59 ms 1.42 ms 0.90 0.01 False
GetString - Duration of single invocation 1.51 ms 1.38 ms 0.91 0.02 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Perf_Utf8Encoding*'

Payloads

Baseline
Compare

Histogram

System.Text.Perf_Utf8Encoding.GetBytes(Input: EnglishAllAscii)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.108357137276786 < 1.1844349259701237.
IsChangePoint: Marked as a change because one of 11/16/2022 11:42:33 AM, 12/2/2022 5:21:30 AM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 36.20508644617162 (T) = (0 -1108902.7555366475) / Math.Sqrt((6122008859.469288 / (299)) + (27194517.290452916 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.13094874781990948 = (1275992.3569005495 - 1108902.7555366475) / 1275992.3569005495 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Perf_Utf8Encoding.GetString(Input: Greek)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.0145167119791667 < 2.1050266204408485.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 63.93210017774061 (T) = (0 -2026406.5344115521) / Math.Sqrt((1227664868.3063555 / (299)) + (295530418.48106873 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.10235125767093323 = (2257460.4506811607 - 2026406.5344115521) / 2257460.4506811607 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Perf_Utf8Encoding.GetString(Input: Cyrillic)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.4238556621212126 < 1.4791585546266233.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 70.06996306984401 (T) = (0 -1431541.707632866) / Math.Sqrt((432153172.74796003 / (299)) + (102433998.63372943 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.09453173889051081 = (1580996.009599241 - 1431541.707632866) / 1580996.009599241 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Perf_Utf8Encoding.GetString(Input: EnglishMostlyAscii)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.3750402224702383 < 1.4370520418046537.
IsChangePoint: Marked as a change because one of 11/16/2022 11:42:33 AM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 41.317014030192915 (T) = (0 -1381456.1687559257) / Math.Sqrt((666628588.0323737 / (299)) + (195901041.93190613 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.07873686703651016 = (1499523.98975535 - 1381456.1687559257) / 1499523.98975535 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented Dec 27, 2022 •
edited
Loading

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 01b57ccc4cc532034f717226f4c7cc07a1ee9518
Compare fc65281072f9259f0b03a291b9c549bc9d792752
Diff Diff

Improvements in System.Text.Json.Serialization.Tests.WriteJson<HashSet<String>>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
SerializeToWriter - Duration of single invocation 111.20 μs 103.32 μs 0.93 0.05 False
SerializeToWriter - Duration of single invocation 123.34 μs 114.42 μs 0.93 0.04 False
SerializeToStream - Duration of single invocation 115.17 μs 106.49 μs 0.92 0.03 False
SerializeToStream - Duration of single invocation 128.57 μs 119.31 μs 0.93 0.03 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.WriteJson&lt;HashSet&lt;String&gt;&gt;*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Serialization.Tests.WriteJson<HashSet<String>>.SerializeToWriter(Mode: SourceGen)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 103.32487188888886 < 105.33947599898241.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 28.98455508079033 (T) = (0 -101633.01630428918) / Math.Sqrt((7810269.481639166 / (112)) + (4493381.3934977995 / (33))) is greater than 1.9766921979294603 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (112) + (33) - 2, .975) and 0.11457953230990596 = (114785.03153358518 - 101633.01630428918) / 114785.03153358518 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Serialization.Tests.WriteJson&lt;HashSet&lt;String&gt;&gt;.SerializeToWriter(Mode: Reflection)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 114.41960177277433 < 117.21825828892298.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 31.73115766610343 (T) = (0 -113616.3028908603) / Math.Sqrt((11346635.734130317 / (112)) + (2925892.939973952 / (33))) is greater than 1.9766921979294603 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (112) + (33) - 2, .975) and 0.10851836196481411 = (127446.59905869646 - 113616.3028908603) / 127446.59905869646 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Serialization.Tests.WriteJson&lt;HashSet&lt;String&gt;&gt;.SerializeToStream(Mode: SourceGen)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 106.48822723817568 < 110.49891059002807.
IsChangePoint: Marked as a change because one of 12/5/2022 3:14:07 PM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 21.83682398765588 (T) = (0 -105180.10142818757) / Math.Sqrt((67063347.02021593 / (112)) + (3811905.971185649 / (33))) is greater than 1.9766921979294603 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (112) + (33) - 2, .975) and 0.1492738167861587 = (123635.66974139922 - 105180.10142818757) / 123635.66974139922 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Serialization.Tests.WriteJson&lt;HashSet&lt;String&gt;&gt;.SerializeToStream(Mode: Reflection)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 119.31419322519083 < 122.32811966742506.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 30.63364644736576 (T) = (0 -119063.8074980906) / Math.Sqrt((8858783.246100493 / (112)) + (3675023.027077182 / (33))) is greater than 1.9766921979294603 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (112) + (33) - 2, .975) and 0.10094979943608712 = (132432.88019224064 - 119063.8074980906) / 132432.88019224064 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS ubuntu 18.04
Baseline 01b57ccc4cc532034f717226f4c7cc07a1ee9518
Compare fc65281072f9259f0b03a291b9c549bc9d792752
Diff Diff

Improvements in System.Buffers.Tests.RentReturnArrayPoolTests<Byte>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
SingleSerial - Duration of single invocation 25.94 μs 23.15 μs 0.89 0.05 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Buffers.Tests.RentReturnArrayPoolTests&lt;Byte&gt;*'

Payloads

Baseline
Compare

Histogram

System.Buffers.Tests.RentReturnArrayPoolTests<Byte>.SingleSerial(RentalSize: 4096, ManipulateArray: True, Async: True, UseSharedPool: True)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 23.146235855000004 < 24.696796984987497.
IsChangePoint: Marked as a change because one of 10/22/2022 11:54:51 PM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 18.044973538381768 (T) = (0 -23913.671053470982) / Math.Sqrt((1264583.100400419 / (299)) + (129474.83110889352 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.06378800911334474 = (25543.008726926408 - 23913.671053470982) / 25543.008726926408 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 01b57ccc4cc532034f717226f4c7cc07a1ee9518
Compare fc65281072f9259f0b03a291b9c549bc9d792752
Diff Diff

Improvements in System.Text.Json.Document.Tests.Perf_EnumerateArray

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
EnumerateArray - Duration of single invocation 35.13 μs 0.00 ns 0.00 0.14 False
EnumerateArray - Duration of single invocation 37.37 μs 0.00 ns 0.00 0.18 False
EnumerateArray - Duration of single invocation 35.66 μs 0.00 ns 0.00 0.12 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Document.Tests.Perf_EnumerateArray*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Document.Tests.Perf_EnumerateArray.EnumerateArray(TestCase: ArrayOfNumbers)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 0 < 33.73008571327562.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 438.6403943199348 (T) = (0 -0) / Math.Sqrt((1968389.1351422956 / (299)) + (0 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 1 = (35590.0426730004 - 0) / 35590.0426730004 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Document.Tests.Perf_EnumerateArray.EnumerateArray(TestCase: Json400KB)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 0 < 35.56879776472317.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 450.5479921785408 (T) = (0 -0) / Math.Sqrt((2098631.2275871392 / (299)) + (0 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 1 = (37746.22548381743 - 0) / 37746.22548381743 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Document.Tests.Perf_EnumerateArray.EnumerateArray(TestCase: ArrayOfStrings)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 0 < 33.77126296174715.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 455.31891844827567 (T) = (0 -0) / Math.Sqrt((1820823.9767705337 / (299)) + (0 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 1 = (35531.54427118059 - 0) / 35531.54427118059 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 01b57ccc4cc532034f717226f4c7cc07a1ee9518
Compare fc65281072f9259f0b03a291b9c549bc9d792752
Diff Diff

Improvements in System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
InequalityOperatorBenchmark - Duration of single invocation 29.01 ns 24.93 ns 0.86 0.20 False
LessThanOrEqualAllBenchmark - Duration of single invocation 29.59 ns 27.22 ns 0.92 0.19 False
GreaterThanOrEqualAnyBenchmark - Duration of single invocation 29.99 ns 26.97 ns 0.90 0.16 False
EqualsAllBenchmark - Duration of single invocation 30.36 ns 27.80 ns 0.92 0.19 False
EqualityOperatorBenchmark - Duration of single invocation 41.49 ns 25.44 ns 0.61 0.30 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int32>.InequalityOperatorBenchmark

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 24.930356115594254 < 27.86048847759152.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 26.56486132448278 (T) = (0 -25.061265212326987) / Math.Sqrt((1.9317352024858456 / (165)) + (0.4875402831174533 / (33))) is greater than 1.9721412216619414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (165) + (33) - 2, .975) and 0.14711755132806545 = (29.384196205879395 - 25.061265212326987) / 29.384196205879395 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int32&gt;.LessThanOrEqualAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 27.215763550935208 < 28.337366631810553.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 3.9669320577276053 (T) = (0 -27.969546983608605) / Math.Sqrt((2.5110376030790738 / (166)) + (9.075034822000589 / (33))) is greater than 1.9720790337782532 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (166) + (33) - 2, .975) and 0.07097287340460073 = (30.106275891112517 - 27.969546983608605) / 30.106275891112517 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int32&gt;.GreaterThanOrEqualAnyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 26.96641338205192 < 28.488588048468415.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 13.656817664701702 (T) = (0 -27.26882003259564) / Math.Sqrt((2.255162788535279 / (165)) + (0.8449287004298213 / (33))) is greater than 1.9721412216619414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (165) + (33) - 2, .975) and 0.09028726155107292 = (29.975198631481582 - 27.26882003259564) / 29.975198631481582 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int32&gt;.EqualsAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 27.79575537145332 < 29.230789711363073.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 14.658452277290968 (T) = (0 -28.27999901327384) / Math.Sqrt((1.2288631241356298 / (166)) + (1.3453097635559648 / (33))) is greater than 1.9720790337782532 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (166) + (33) - 2, .975) and 0.10214183822320592 = (31.49717874960321 - 28.27999901327384) / 31.49717874960321 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int32&gt;.EqualityOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 25.44458546554784 < 27.372926176746965.
IsChangePoint: Marked as a change because one of 12/18/2022 12:43:31 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 13.267710196630967 (T) = (0 -25.980977903662303) / Math.Sqrt((5.209195893068288 / (167)) + (1.5723420715901837 / (32))) is greater than 1.9720790337782532 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (167) + (32) - 2, .975) and 0.12643580593813247 = (29.74134938253007 - 25.980977903662303) / 29.74134938253007 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 01b57ccc4cc532034f717226f4c7cc07a1ee9518
Compare fc65281072f9259f0b03a291b9c549bc9d792752
Diff Diff

Improvements in System.Collections.ContainsFalse<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ImmutableList - Duration of single invocation 17.27 ms 15.15 ms 0.88 0.06 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.ContainsFalse&lt;String&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.ContainsFalse<String>.ImmutableList(Size: 512)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 15.145004633333333 < 16.12137774534014.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 26.126949920862828 (T) = (0 -15273439.476868762) / Math.Sqrt((312965327736.6414 / (299)) + (149634398300.44647 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.11331339065981355 = (17225296.19369604 - 15273439.476868762) / 17225296.19369604 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 01b57ccc4cc532034f717226f4c7cc07a1ee9518
Compare fc65281072f9259f0b03a291b9c549bc9d792752
Diff Diff

Improvements in System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int64>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GreaterThanOrEqualAnyBenchmark - Duration of single invocation 30.09 ns 26.27 ns 0.87 0.18 False
LessThanAllBenchmark - Duration of single invocation 29.58 ns 25.66 ns 0.87 0.23 False
InequalityOperatorBenchmark - Duration of single invocation 32.27 ns 24.96 ns 0.77 0.21 False
SumBenchmark - Duration of single invocation 21.09 ns 19.81 ns 0.94 0.25 False
EqualsAllBenchmark - Duration of single invocation 31.44 ns 27.77 ns 0.88 0.24 False
LessThanOrEqualAllBenchmark - Duration of single invocation 29.80 ns 26.65 ns 0.89 0.26 False
GreaterThanAllBenchmark - Duration of single invocation 63.78 ns 57.84 ns 0.91 0.15 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int64&gt;*'

Payloads

Baseline
Compare

Histogram

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int64>.GreaterThanOrEqualAnyBenchmark

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 26.26531972934733 < 28.688727451678076.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 14.656526524323553 (T) = (0 -26.86442215856561) / Math.Sqrt((3.0376039651041333 / (166)) + (1.0881205329340962 / (33))) is greater than 1.9720790337782532 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (166) + (33) - 2, .975) and 0.10995305752233875 = (30.18315200744579 - 26.86442215856561) / 30.18315200744579 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int64&gt;.LessThanAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 25.662778931327686 < 27.5812625684642.
IsChangePoint: Marked as a change because one of 12/19/2022 2:44:31 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 19.58801296744426 (T) = (0 -25.952977401227844) / Math.Sqrt((2.796914059845839 / (169)) + (0.5015944963619213 / (30))) is greater than 1.9720790337782532 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (169) + (30) - 2, .975) and 0.12100747311356541 = (29.525822583678185 - 25.952977401227844) / 29.525822583678185 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int64&gt;.InequalityOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 24.95571085428813 < 28.045146051833104.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 29.843026167953052 (T) = (0 -25.10749400741248) / Math.Sqrt((1.8248845901521966 / (166)) + (0.34322577520129127 / (33))) is greater than 1.9720790337782532 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (166) + (33) - 2, .975) and 0.14810571557305038 = (29.472546613341507 - 25.10749400741248) / 29.472546613341507 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int64&gt;.SumBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 19.809537244140838 < 20.03681722189147.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 10.948015057811867 (T) = (0 -20.08952016629895) / Math.Sqrt((1.63243943745448 / (166)) + (0.722929582846193 / (33))) is greater than 1.9720790337782532 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (166) + (33) - 2, .975) and 0.08849796523404642 = (22.04001680748561 - 20.08952016629895) / 22.04001680748561 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int64&gt;.EqualsAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 27.76796466151691 < 29.968056301605944.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 22.81569875333216 (T) = (0 -27.949440509027042) / Math.Sqrt((2.609366216070576 / (166)) + (0.42581944221522894 / (33))) is greater than 1.9720790337782532 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (166) + (33) - 2, .975) and 0.12134804444677845 = (31.80945576047727 - 27.949440509027042) / 31.80945576047727 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int64&gt;.LessThanOrEqualAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 26.64729051354461 < 28.27642191765336.
IsChangePoint: Marked as a change because one of 12/19/2022 8:13:55 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 14.60894800666343 (T) = (0 -26.76251294761284) / Math.Sqrt((2.0921223754866274 / (171)) + (1.0625746677839094 / (28))) is greater than 1.9720790337782532 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (171) + (28) - 2, .975) and 0.10896081012577273 = (30.0351693300835 - 26.76251294761284) / 30.0351693300835 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int64&gt;.GreaterThanAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 57.83895954869196 < 60.50833459934153.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 13.857472156541627 (T) = (0 -59.585551091835654) / Math.Sqrt((5.979795735830928 / (166)) + (6.05588613918814 / (33))) is greater than 1.9720790337782532 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (166) + (33) - 2, .975) and 0.0982598134135738 = (66.07840260219427 - 59.585551091835654) / 66.07840260219427 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 01b57ccc4cc532034f717226f4c7cc07a1ee9518
Compare fc65281072f9259f0b03a291b9c549bc9d792752
Diff Diff

Improvements in System.Text.Json.Serialization.Tests.WriteJson<IndexViewModel>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
SerializeToUtf8Bytes - Duration of single invocation 422.17 μs 375.37 μs 0.89 0.01 False
SerializeToStream - Duration of single invocation 584.33 μs 546.32 μs 0.93 0.01 False
SerializeToWriter - Duration of single invocation 438.69 μs 388.54 μs 0.89 0.01 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.WriteJson&lt;IndexViewModel&gt;*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Serialization.Tests.WriteJson<IndexViewModel>.SerializeToUtf8Bytes(Mode: SourceGen)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 375.3699370039683 < 395.0300516183036.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 37.168714527284195 (T) = (0 -374839.38298543426) / Math.Sqrt((60968634.064684644 / (112)) + (34523543.83330074 / (33))) is greater than 1.9766921979294603 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (112) + (33) - 2, .975) and 0.11115509085530847 = (421715.17114963377 - 374839.38298543426) / 421715.17114963377 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Serialization.Tests.WriteJson&lt;IndexViewModel&gt;.SerializeToStream(Mode: SourceGen)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 546.3166744260203 < 556.9822069098248.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 33.882250407922115 (T) = (0 -545528.7348623261) / Math.Sqrt((79225511.37615985 / (112)) + (37869588.26763857 / (33))) is greater than 1.9766921979294603 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (112) + (33) - 2, .975) and 0.07799257450438836 = (591674.9906532315 - 545528.7348623261) / 591674.9906532315 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Serialization.Tests.WriteJson&lt;IndexViewModel&gt;.SerializeToWriter(Mode: SourceGen)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 388.54466866289204 < 405.27514149674226.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 33.208792350938296 (T) = (0 -389343.5664470454) / Math.Sqrt((71333532.45641303 / (112)) + (39176993.30554514 / (33))) is greater than 1.9766921979294603 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (112) + (33) - 2, .975) and 0.10329783296690648 = (434194.96546468855 - 389343.5664470454) / 434194.96546468855 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 01b57ccc4cc532034f717226f4c7cc07a1ee9518
Compare fc65281072f9259f0b03a291b9c549bc9d792752
Diff Diff

Improvements in System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Count - Duration of single invocation 2.27 ms 1.95 ms 0.86 0.03 False
Count - Duration of single invocation 10.73 ms 10.00 ms 0.93 0.01 False
Count - Duration of single invocation 2.02 ms 1.72 ms 0.85 0.04 False
Count - Duration of single invocation 2.34 ms 2.05 ms 0.87 0.03 False
Count - Duration of single invocation 1.99 ms 1.67 ms 0.84 0.08 False
Count - Duration of single invocation 2.28 ms 1.97 ms 0.86 0.02 False
Count - Duration of single invocation 2.10 ms 1.78 ms 0.85 0.03 False
Count - Duration of single invocation 2.37 ms 2.00 ms 0.85 0.05 False
Count - Duration of single invocation 2.95 ms 2.66 ms 0.90 0.02 False
Count - Duration of single invocation 4.17 ms 3.80 ms 0.91 0.06 False
Count - Duration of single invocation 2.07 ms 1.76 ms 0.85 0.03 False
Count - Duration of single invocation 2.36 ms 2.04 ms 0.86 0.05 False
Count - Duration of single invocation 1.99 ms 1.68 ms 0.85 0.04 False
Count - Duration of single invocation 2.02 ms 1.71 ms 0.84 0.04 False
Count - Duration of single invocation 2.04 ms 1.74 ms 0.85 0.02 False
Count - Duration of single invocation 2.10 ms 1.79 ms 0.85 0.05 False
Count - Duration of single invocation 1.83 ms 1.53 ms 0.84 0.05 False
Count - Duration of single invocation 2.84 ms 2.52 ms 0.89 0.02 False
Count - Duration of single invocation 2.66 ms 2.24 ms 0.84 0.02 False
Count - Duration of single invocation 1.99 ms 1.68 ms 0.84 0.05 False
Count - Duration of single invocation 4.10 ms 3.75 ms 0.91 0.08 False
Count - Duration of single invocation 10.80 ms 10.01 ms 0.93 0.03 False
Count - Duration of single invocation 2.22 ms 1.89 ms 0.85 0.04 False
Count - Duration of single invocation 1.84 ms 1.53 ms 0.83 0.05 False
Count - Duration of single invocation 2.34 ms 2.05 ms 0.88 0.05 False
Count - Duration of single invocation 31.71 ms 27.93 ms 0.88 0.10 False
Count - Duration of single invocation 4.03 ms 3.80 ms 0.94 0.09 False
Count - Duration of single invocation 40.44 ms 38.14 ms 0.94 0.18 False
Count - Duration of single invocation 1.83 ms 1.53 ms 0.84 0.05 False
Count - Duration of single invocation 1.98 ms 1.68 ms 0.85 0.07 False
Count - Duration of single invocation 2.59 ms 2.25 ms 0.87 0.02 False
Count - Duration of single invocation 4.08 ms 3.67 ms 0.90 0.09 False
Count - Duration of single invocation 4.17 ms 3.68 ms 0.88 0.09 False
Count - Duration of single invocation 2.04 ms 1.73 ms 0.85 0.02 False
Count - Duration of single invocation 2.35 ms 2.06 ms 0.87 0.02 False
Count - Duration of single invocation 2.68 ms 2.35 ms 0.87 0.03 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock*'

Payloads

Baseline
Compare

Histogram

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Sherlock|Street", Options: NonBacktracking)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.954532154105737 < 2.1560741601969697.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 87.47327426569704 (T) = (0 -1948255.7643456613) / Math.Sqrt((2941700745.6930585 / (299)) + (71383568.56398919 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.13460559582337076 = (2251292.3066556104 - 1948255.7643456613) / 2251292.3066556104 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "the", Options: None)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 9.995278591666667 < 10.2235447334145.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 11/15/2022 8:48:03 PM, 12/2/2022 5:21:30 AM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 24.548674406741664 (T) = (0 -10033853.596200602) / Math.Sqrt((58042602609.71392 / (299)) + (15643130607.777143 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.059479551448593286 = (10668405.574439963 - 10033853.596200602) / 10668405.574439963 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "(?m)^Sherlock Holmes|Sherlock Holmes$", Options: Compiled)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.7186890013793104 < 1.9225049739032452.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 100.05327007394722 (T) = (0 -1720450.6848792569) / Math.Sqrt((2010248525.7076297 / (299)) + (63919952.648350276 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.14613160269002964 = (2014889.7538536033 - 1720450.6848792569) / 2014889.7538536033 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "[a-q][^u-z]{13}x", Options: NonBacktracking)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.047128902185792 < 2.2223064395047163.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 78.85392564682797 (T) = (0 -2028142.9922935516) / Math.Sqrt((3545716624.311347 / (299)) + (110479019.70789857 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.13165325639612183 = (2335637.240805671 - 2028142.9922935516) / 2335637.240805671 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Sherlock Holmes", Options: None)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.6650412826666667 < 1.888364776907552.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 55.605672600601466 (T) = (0 -1684426.7847173598) / Math.Sqrt((5155420589.301477 / (299)) + (357901170.92511386 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.14890319129149815 = (1979124.7804975274 - 1684426.7847173598) / 1979124.7804975274 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Sherlock|Street", Options: Compiled)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.9726334184476941 < 2.1730045016082062.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 107.9371658069751 (T) = (0 -1973252.9295529511) / Math.Sqrt((1085808875.2622788 / (299)) + (127528178.89952275 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.13025483464755297 = (2268771.369086403 - 1973252.9295529511) / 2268771.369086403 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "(?m)^Sherlock Holmes|Sherlock Holmes$", Options: NonBacktracking)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.7809925233333332 < 1.9947007163307695.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 127.66546238481006 (T) = (0 -1785095.6415618402) / Math.Sqrt((1160488561.6463325 / (299)) + (55973395.37994147 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.14449426327998757 = (2086596.9273401394 - 1785095.6415618402) / 2086596.9273401394 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Holmes", Options: None)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.0037863333333332 < 2.2512922186675826.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 53.60448130754445 (T) = (0 -2030325.6911278542) / Math.Sqrt((6406354979.5874815 / (299)) + (444148790.3123302 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.13490243341968647 = (2346932.611489855 - 2030325.6911278542) / 2346932.611489855 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "The", Options: NonBacktracking)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.659025323297491 < 2.836091894698795.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 50.892053437370535 (T) = (0 -2644908.29740648) / Math.Sqrt((10921262431.950003 / (285)) + (380550426.7474816 / (33))) is greater than 1.9674995188240387 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (285) + (33) - 2, .975) and 0.11960720622516055 = (3004236.649945724 - 2644908.29740648) / 3004236.649945724 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "aei", Options: Compiled)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 3.8027673971861464 < 3.8729732796762293.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 11/15/2022 8:48:03 PM, 12/2/2022 5:21:30 AM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 39.28684157262739 (T) = (0 -3634687.6078567924) / Math.Sqrt((11959211192.378506 / (299)) + (2869970488.7165804 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.10857031940096667 = (4077368.845756081 - 3634687.6078567924) / 4077368.845756081 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Sherlock", Options: NonBacktracking)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.7570234089201877 < 1.9699530612063492.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 62.50827347249583 (T) = (0 -1761858.3657533138) / Math.Sqrt((6641618671.674697 / (286)) + (61735258.658851 / (33))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (286) + (33) - 2, .975) and 0.15090419173484565 = (2074981.8201942216 - 1761858.3657533138) / 2074981.8201942216 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Holmes", Options: Compiled)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.043376803794038 < 2.241993527608955.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 67.52550377196333 (T) = (0 -2033534.6553094198) / Math.Sqrt((2197534072.1676707 / (299)) + (442017770.611694 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.1313705453261076 = (2341084.1577699725 - 2033534.6553094198) / 2341084.1577699725 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Sherlock Holmes", Options: Compiled)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.6808343954138707 < 1.8859190236607142.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 131.04683726442425 (T) = (0 -1679233.5346604236) / Math.Sqrt((1190113294.6637723 / (299)) + (34301120.226934485 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.1488247144924855 = (1972841.0390336677 - 1679233.5346604236) / 1972841.0390336677 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "(?m)^Sherlock Holmes|Sherlock Holmes$", Options: None)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.7086652059686889 < 1.9217211342317708.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 93.44656568609442 (T) = (0 -1721865.2326430217) / Math.Sqrt((2545415796.6524725 / (299)) + (42448295.157194845 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.14521772312874837 = (2014390.423425182 - 1721865.2326430217) / 2014390.423425182 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Sherlock\\s+Holmes", Options: Compiled)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.7358705675925925 < 1.9433445420731026.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 103.19230139333949 (T) = (0 -1737681.2396053728) / Math.Sqrt((1711855331.4931555 / (290)) + (70893300.93147679 / (33))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (290) + (33) - 2, .975) and 0.14420436418603327 = (2030486.2129293573 - 1737681.2396053728) / 2030486.2129293573 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Sherlock Holmes", Options: NonBacktracking)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.7884307456320658 < 1.9941635054901452.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 97.67428806965019 (T) = (0 -1790843.249445249) / Math.Sqrt((1552674651.3220832 / (299)) + (140111412.64809793 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.14351529945003952 = (2090922.6379587683 - 1790843.249445249) / 2090922.6379587683 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "zqj", Options: Compiled)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.5343574326923077 < 1.738172579734241.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 60.56808164458142 (T) = (0 -1532194.8162191033) / Math.Sqrt((6432389843.374916 / (299)) + (38038991.452906094 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.1583892590240329 = (1820550.4535770372 - 1532194.8162191033) / 1820550.4535770372 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Sherlock|Street", Options: None)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.5245328946608945 < 2.7059562545944944.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 71.64562686368717 (T) = (0 -2525039.971521528) / Math.Sqrt((2100698600.9979722 / (299)) + (345656045.00591147 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.10610352896601738 = (2824756.617061905 - 2525039.971521528) / 2824756.617061905 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "The", Options: Compiled)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.240199881818181 < 2.468890288177083.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 60.448483982935024 (T) = (0 -2252567.7996618384) / Math.Sqrt((6334033328.431536 / (290)) + (226847783.2962559 / (33))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (290) + (33) - 2, .975) and 0.12572333727718554 = (2576493.1121992045 - 2252567.7996618384) / 2576493.1121992045 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Sherlock", Options: Compiled)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.6763066375838929 < 1.884213268814174.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 69.50047394191337 (T) = (0 -1678726.20067217) / Math.Sqrt((5106759883.750167 / (284)) + (45969801.34991353 / (33))) is greater than 1.9675235316906419 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (284) + (33) - 2, .975) and 0.15414213231110604 = (1984643.3600705178 - 1678726.20067217) / 1984643.3600705178 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "aqj", Options: None)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 3.7458545243781094 < 3.842067911083452.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 11/15/2022 8:48:03 PM, 12/2/2022 10:04:21 AM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 21.801843564901933 (T) = (0 -3661791.7913445896) / Math.Sqrt((14502706475.209488 / (299)) + (10936769280.956686 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.10398306267122705 = (4086743.9428781453 - 3661791.7913445896) / 4086743.9428781453 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "the", Options: Compiled)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 10.013862 < 10.228273527345571.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 11/15/2022 8:48:03 PM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 15.630774424991122 (T) = (0 -10021202.5649198) / Math.Sqrt((58496868701.587494 / (299)) + (45271915216.25433 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.058162477880298664 = (10640054.49938548 - 10021202.5649198) / 10640054.49938548 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Sherlock\\s+Holmes", Options: NonBacktracking)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.8935078384615385 < 2.101366369719764.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 68.73862761848868 (T) = (0 -1897232.7598938614) / Math.Sqrt((5081375606.313812 / (290)) + (110511107.55043368 / (33))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (290) + (33) - 2, .975) and 0.1420136905409909 = (2211262.2765392773 - 1897232.7598938614) / 2211262.2765392773 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "zqj", Options: None)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.5321833008928574 < 1.7355988068061277.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 62.20258529786926 (T) = (0 -1531899.125847101) / Math.Sqrt((6322370034.51816 / (299)) + (11315379.635713065 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.15840804665781086 = (1820239.7489228782 - 1531899.125847101) / 1820239.7489228782 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "[a-q][^u-z]{13}x", Options: Compiled)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.0520674520807063 < 2.2365788976031036.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 67.70137706633403 (T) = (0 -2053731.573011156) / Math.Sqrt((4916195558.325835 / (299)) + (78170992.50689381 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.12508997501985297 = (2347363.173782079 - 2053731.573011156) / 2347363.173782079 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Sherlock|Holmes|Watson|Irene|Adler|John|Baker", Options: NonBacktracking)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 27.925133829629626 < 30.75185140874705.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 26.450013634263236 (T) = (0 -28435297.20214215) / Math.Sqrt((4249404807919.1855 / (285)) + (184197259290.76077 / (33))) is greater than 1.9674995188240387 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (285) + (33) - 2, .975) and 0.11750868663886777 = (32221617.110134527 - 28435297.20214215) / 32221617.110134527 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "aqj", Options: Compiled)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 3.8005640272952843 < 3.9500039947330814.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 11/16/2022 11:42:33 AM, 12/2/2022 5:21:30 AM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 31.078815242901534 (T) = (0 -3639415.1375954635) / Math.Sqrt((14814697467.638271 / (299)) + (5044668254.274406 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.10833242944110115 = (4081582.932655352 - 3639415.1375954635) / 4081582.932655352 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "\\w+\\s+Holmes", Options: NonBacktracking)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 38.13896471111111 < 38.54909937910715.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 20.397001403936258 (T) = (0 -38440116.73738853) / Math.Sqrt((1981901765277.046 / (216)) + (220536883687.1403 / (33))) is greater than 1.9696147551832122 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (216) + (33) - 2, .975) and 0.06263541361538902 = (41008714.53406511 - 38440116.73738853) / 41008714.53406511 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "zqj", Options: NonBacktracking)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.5324480101461038 < 1.7345053124379959.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 119.42245527516259 (T) = (0 -1533547.883958024) / Math.Sqrt((1286935989.255425 / (299)) + (42472290.178697616 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.15550283454888503 = (1815930.173239635 - 1533547.883958024) / 1815930.173239635 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Sherlock", Options: None)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.6799952874720356 < 1.883581275235305.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 51.009899391679795 (T) = (0 -1680301.0603125687) / Math.Sqrt((7540295528.649373 / (280)) + (329409153.09569496 / (33))) is greater than 1.9676211333067681 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (280) + (33) - 2, .975) and 0.1557174655118617 = (1990211.797205164 - 1680301.0603125687) / 1990211.797205164 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "The", Options: None)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.2520872152905196 < 2.4664007491419184.
IsChangePoint: Marked as a change because one of 9/28/2022 3:03:11 PM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 60.22625093342019 (T) = (0 -2247662.3980361954) / Math.Sqrt((6997463330.949582 / (286)) + (168769866.70764735 / (33))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (286) + (33) - 2, .975) and 0.1271969371638238 = (2575222.858100898 - 2247662.3980361954) / 2575222.858100898 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "aei", Options: None)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 3.666996997058823 < 4.002000741918103.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 11/15/2022 8:48:03 PM, 12/2/2022 5:21:30 AM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 38.33599275121043 (T) = (0 -3638649.4060898945) / Math.Sqrt((14723899072.385712 / (299)) + (2733551941.389252 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.10800532864741867 = (4079227.7386280876 - 3638649.4060898945) / 4079227.7386280876 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "aei", Options: NonBacktracking)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 3.675114739303483 < 3.929493138870057.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 11/16/2022 11:42:33 AM, 12/2/2022 5:21:30 AM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 21.108690273557084 (T) = (0 -3672057.5912553277) / Math.Sqrt((16164148960.17197 / (299)) + (11666994271.203705 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.10398857445655871 = (4098226.3022239725 - 3672057.5912553277) / 4098226.3022239725 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Sherlock\\s+Holmes", Options: None)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.7327671912037037 < 1.9424087518169066.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 114.76512180903715 (T) = (0 -1736242.7533943562) / Math.Sqrt((1072021271.7479738 / (290)) + (91547542.14009258 / (33))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (290) + (33) - 2, .975) and 0.1439402656934594 = (2028179.4410069017 - 1736242.7533943562) / 2028179.4410069017 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "[a-q][^u-z]{13}x", Options: None)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.055104667060213 < 2.2374398251264878.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 97.98791960505478 (T) = (0 -2058232.9697374874) / Math.Sqrt((1277895131.142775 / (299)) + (135223244.5110247 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.12106981146897425 = (2341747.9529033527 - 2058232.9697374874) / 2341747.9529033527 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Holmes", Options: NonBacktracking)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.3452321676190477 < 2.5549113061111104.
IsChangePoint: Marked as a change because one of 9/28/2022 3:03:11 PM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 73.98002808696977 (T) = (0 -2355590.476554653) / Math.Sqrt((2541431368.5778975 / (299)) + (364150030.7712727 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.12188951457772489 = (2682567.302930988 - 2355590.476554653) / 2682567.302930988 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@kotlarmilos
Copy link
Member

Could be dotnet/runtime@200a90a cc @BrzVlad @SamMonoRT

@lewing
Copy link
Member

lewing commented Jan 10, 2023

Could be dotnet/runtime@200a90a cc @BrzVlad @SamMonoRT

it is

@kotlarmilos kotlarmilos mentioned this issue Apr 4, 2023
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
arch-x64 branch-refs/heads/main kind-micro_mono mono-interpreter os-linux perf-improvement runtime-mono untriaged
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@lewing @kotlarmilos