Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Perf] Linux/x64: 137 Improvements on 11/2/2022 4:25:40 PM #9607

Closed
performanceautofiler bot opened this issue Nov 8, 2022 · 7 comments
Closed

[Perf] Linux/x64: 137 Improvements on 11/2/2022 4:25:40 PM #9607

performanceautofiler bot opened this issue Nov 8, 2022 · 7 comments
Labels
arch-x64 branch-refs/heads/main kind-micro_mono mono-jit os-linux perf-improvement PGO Applied if there were any profile guided optimization updates in the observed interval. runtime-mono untriaged

Comments

@performanceautofiler
Copy link

performanceautofiler bot commented Nov 8, 2022 •
edited
Loading

Run Information

Architecture x64
OS ubuntu 18.04
Baseline bdd67af67dbf43c0f5e9151b98dc100b9555898a
Compare fbc20578253e0deb7267eea5a02b680c3cfe0e2a
Diff Diff

Improvements in System.Numerics.Tests.Perf_VectorOf<UInt16>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
MultiplyBenchmark - Duration of single invocation 20.47 ns 18.02 ns 0.88 0.02 False
EqualsAnyBenchmark - Duration of single invocation 18.88 ns 13.23 ns 0.70 0.39 False
GreaterThanAllBenchmark - Duration of single invocation 18.28 ns 2.10 ns 0.11 0.46 False
GreaterThanAnyBenchmark - Duration of single invocation 19.13 ns 13.27 ns 0.69 0.38 False
LessThanAnyBenchmark - Duration of single invocation 18.86 ns 2.45 ns 0.13 0.02 False
LessThanOrEqualAllBenchmark - Duration of single invocation 19.73 ns 12.69 ns 0.64 0.13 False
MultiplyOperatorBenchmark - Duration of single invocation 21.18 ns 18.02 ns 0.85 0.01 False
GreaterThanOrEqualAnyBenchmark - Duration of single invocation 20.39 ns 12.62 ns 0.62 0.01 False
LessThanAllBenchmark - Duration of single invocation 18.20 ns 11.39 ns 0.63 0.04 False
GreaterThanOrEqualAllBenchmark - Duration of single invocation 19.73 ns 1.98 ns 0.10 0.03 False
LessThanOrEqualAnyBenchmark - Duration of single invocation 20.36 ns 2.25 ns 0.11 0.12 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;UInt16&gt;*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Numerics.Tests.Perf_VectorOf<UInt16>.MultiplyBenchmark

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 18.019622727108093 < 19.46469875215925.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 11/2/2022 9:35:40 AM, 11/8/2022 2:29:58 AM falls between 10/30/2022 12:44:54 AM and 11/8/2022 2:29:58 AM.
IsImprovementStdDev: Marked as improvement because 63.904844626452096 (T) = (0 -18.042171554420015) / Math.Sqrt((0.4041062973479435 / (299)) + (0.0006437272348136468 / (18))) is greater than 1.9675235316906419 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (18) - 2, .975) and 0.11654954887861639 = (20.422392146066233 - 18.042171554420015) / 20.422392146066233 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt16&gt;.EqualsAnyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 13.233462674720249 < 18.0754923898955.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 11/2/2022 9:35:40 AM, 11/8/2022 2:29:58 AM falls between 10/30/2022 12:44:54 AM and 11/8/2022 2:29:58 AM.
IsImprovementStdDev: Marked as improvement because 43.290594327551986 (T) = (0 -13.133061056536503) / Math.Sqrt((0.8875304804376908 / (299)) + (0.2704183456262514 / (14))) is greater than 1.9676211333067681 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (14) - 2, .975) and 0.3297883991118985 = (19.595395005299526 - 13.133061056536503) / 19.595395005299526 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt16&gt;.GreaterThanAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.097088913581886 < 17.761144969189697.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/8/2022 2:29:58 AM falls between 10/30/2022 12:44:54 AM and 11/8/2022 2:29:58 AM.
IsImprovementStdDev: Marked as improvement because 185.19080182809697 (T) = (0 -2.2409628011065688) / Math.Sqrt((1.29144348403567 / (299)) + (0.05898978427295077 / (14))) is greater than 1.9676211333067681 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (14) - 2, .975) and 0.8841735225947568 = (19.34758659081111 - 2.2409628011065688) / 19.34758659081111 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt16&gt;.GreaterThanAnyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 13.273152647593587 < 18.136454668810604.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 11/2/2022 9:35:40 AM, 11/8/2022 2:29:58 AM falls between 10/30/2022 12:44:54 AM and 11/8/2022 2:29:58 AM.
IsImprovementStdDev: Marked as improvement because 28.988368281768945 (T) = (0 -12.896704172076209) / Math.Sqrt((0.8494757010215835 / (299)) + (0.683236933350808 / (14))) is greater than 1.9676211333067681 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (14) - 2, .975) and 0.3381004595780365 = (19.484383028660982 - 12.896704172076209) / 19.484383028660982 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt16&gt;.LessThanAnyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.4463292786765085 < 17.911204416486637.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/8/2022 2:29:58 AM falls between 10/30/2022 12:44:54 AM and 11/8/2022 2:29:58 AM.
IsImprovementStdDev: Marked as improvement because 369.3551702988579 (T) = (0 -2.3384700019729947) / Math.Sqrt((0.4464918450061122 / (299)) + (0.008225395594609476 / (19))) is greater than 1.9674995188240387 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (19) - 2, .975) and 0.8739295172594729 = (18.548909714147218 - 2.3384700019729947) / 18.548909714147218 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt16&gt;.LessThanOrEqualAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 12.691208078961589 < 18.75008885094645.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 11/2/2022 9:35:40 AM, 11/8/2022 2:29:58 AM falls between 10/30/2022 12:44:54 AM and 11/8/2022 2:29:58 AM.
IsImprovementStdDev: Marked as improvement because 145.19497010825148 (T) = (0 -12.623973174260875) / Math.Sqrt((0.626288330921791 / (299)) + (0.003297320051385293 / (19))) is greater than 1.9674995188240387 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (19) - 2, .975) and 0.3539057161763674 = (19.538902433173206 - 12.623973174260875) / 19.538902433173206 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt16&gt;.MultiplyOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 18.02240989428157 < 19.465646629133253.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 11/2/2022 9:35:40 AM, 11/8/2022 2:29:58 AM falls between 10/30/2022 12:44:54 AM and 11/8/2022 2:29:58 AM.
IsImprovementStdDev: Marked as improvement because 65.16478434689043 (T) = (0 -18.04620892454189) / Math.Sqrt((0.3914970765020964 / (299)) + (0.000551347581107259 / (19))) is greater than 1.9674995188240387 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (19) - 2, .975) and 0.11668883291332566 = (20.4301831528539 - 18.04620892454189) / 20.4301831528539 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt16&gt;.GreaterThanOrEqualAnyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 12.624383603271424 < 19.36749765041214.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 11/2/2022 9:35:40 AM, 11/8/2022 2:29:58 AM falls between 10/30/2022 12:44:54 AM and 11/8/2022 2:29:58 AM.
IsImprovementStdDev: Marked as improvement because 169.50295258412777 (T) = (0 -12.617087965142298) / Math.Sqrt((0.4702759466219753 / (299)) + (0.006707434135623273 / (19))) is greater than 1.9674995188240387 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (19) - 2, .975) and 0.37089594605353593 = (20.0556456217273 - 12.617087965142298) / 20.0556456217273 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt16&gt;.LessThanAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 11.390782670244429 < 17.3154384142872.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 11/2/2022 9:35:40 AM, 11/8/2022 2:29:58 AM falls between 10/30/2022 12:44:54 AM and 11/8/2022 2:29:58 AM.
IsImprovementStdDev: Marked as improvement because 30.775778377664132 (T) = (0 -12.577409513380777) / Math.Sqrt((0.25359962783423096 / (299)) + (0.5734553551925649 / (19))) is greater than 1.9674995188240387 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (19) - 2, .975) and 0.3012032963238675 = (17.998667491153423 - 12.577409513380777) / 17.998667491153423 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt16&gt;.GreaterThanOrEqualAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.9802269198067302 < 18.747264353638375.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/8/2022 2:29:58 AM falls between 10/30/2022 12:44:54 AM and 11/8/2022 2:29:58 AM.
IsImprovementStdDev: Marked as improvement because 362.0115836410033 (T) = (0 -2.0801144907356552) / Math.Sqrt((0.46690255569494926 / (299)) + (0.013827582251400938 / (19))) is greater than 1.9674995188240387 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (19) - 2, .975) and 0.8927842554706629 = (19.40120361862041 - 2.0801144907356552) / 19.40120361862041 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt16&gt;.LessThanOrEqualAnyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.2490846642903026 < 19.353725214551726.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/8/2022 2:29:58 AM falls between 10/30/2022 12:44:54 AM and 11/8/2022 2:29:58 AM.
IsImprovementStdDev: Marked as improvement because 354.72465105460213 (T) = (0 -2.1474387365670986) / Math.Sqrt((0.5605197760050092 / (299)) + (0.013304993304858629 / (19))) is greater than 1.9674995188240387 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (19) - 2, .975) and 0.8934138025313662 = (20.14743735650244 - 2.1474387365670986) / 20.14743735650244 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS ubuntu 18.04
Baseline bdd67af67dbf43c0f5e9151b98dc100b9555898a
Compare fbc20578253e0deb7267eea5a02b680c3cfe0e2a
Diff Diff

Improvements in System.Text.Json.Serialization.Tests.WriteJson<ImmutableDictionary<String, String>>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
SerializeToString - Duration of single invocation 93.01 μs 85.24 μs 0.92 0.00 False
SerializeObjectProperty - Duration of single invocation 94.35 μs 85.10 μs 0.90 0.00 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Json.Serialization.Tests.WriteJson&lt;ImmutableDictionary&lt;String, String&gt;&gt;*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Text.Json.Serialization.Tests.WriteJson<ImmutableDictionary<String, String>>.SerializeToString

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 85.24051284530387 < 88.66177516382335.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 10/25/2022 8:17:52 AM, 11/2/2022 9:35:40 AM, 11/8/2022 2:29:58 AM falls between 10/30/2022 12:44:54 AM and 11/8/2022 2:29:58 AM.
IsImprovementStdDev: Marked as improvement because 48.57114600209831 (T) = (0 -84414.06909289694) / Math.Sqrt((2447384.788514654 / (299)) + (191892.98820258892 / (19))) is greater than 1.9674995188240387 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (19) - 2, .975) and 0.07218867272775778 = (90981.93416227588 - 84414.06909289694) / 90981.93416227588 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Serialization.Tests.WriteJson&lt;ImmutableDictionary&lt;String, String&gt;&gt;.SerializeObjectProperty

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 85.09915353260871 < 89.57653737131884.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 10/25/2022 8:17:52 AM, 11/2/2022 9:35:40 AM, 11/8/2022 2:29:58 AM falls between 10/30/2022 12:44:54 AM and 11/8/2022 2:29:58 AM.
IsImprovementStdDev: Marked as improvement because 54.9191700246249 (T) = (0 -85008.52968737227) / Math.Sqrt((2525317.0594496583 / (299)) + (110488.00317198347 / (19))) is greater than 1.9674995188240387 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (19) - 2, .975) and 0.07162435357761507 = (91566.9535440353 - 85008.52968737227) / 91566.9535440353 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline bdd67af67dbf43c0f5e9151b98dc100b9555898a
Compare fbc20578253e0deb7267eea5a02b680c3cfe0e2a
Diff Diff

Improvements in System.IO.Tests.BinaryWriterExtendedTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
WriteAsciiCharArray - Duration of single invocation 7.57 ms 6.13 ms 0.81 0.01 False
WriteAsciiString - Duration of single invocation 150.07 ns 138.99 ns 0.93 0.01 False
WriteAsciiString - Duration of single invocation 30.12 μs 24.32 μs 0.81 0.01 False
WriteAsciiString - Duration of single invocation 12.46 ms 9.13 ms 0.73 0.01 False
WriteAsciiCharArray - Duration of single invocation 30.07 μs 24.40 μs 0.81 0.01 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.IO.Tests.BinaryWriterExtendedTests*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.IO.Tests.BinaryWriterExtendedTests.WriteAsciiCharArray(StringLengthInChars: 2000000)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 6.1331909836309535 < 7.212380149592682.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 11/2/2022 9:35:40 AM, 11/8/2022 2:29:58 AM falls between 10/30/2022 12:44:54 AM and 11/8/2022 2:29:58 AM.
IsImprovementStdDev: Marked as improvement because 161.61630102181508 (T) = (0 -6133854.978860412) / Math.Sqrt((20379525725.533768 / (299)) + (94639365.32414024 / (19))) is greater than 1.9674995188240387 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (19) - 2, .975) and 0.18389678302302387 = (7516028.427851988 - 6133854.978860412) / 7516028.427851988 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.BinaryWriterExtendedTests.WriteAsciiString(StringLengthInChars: 32)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 138.99360486201027 < 142.4198608747424.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 11/2/2022 9:35:40 AM, 11/8/2022 2:29:58 AM falls between 10/30/2022 12:44:54 AM and 11/8/2022 2:29:58 AM.
IsImprovementStdDev: Marked as improvement because 99.98960397158758 (T) = (0 -139.00684283908822) / Math.Sqrt((2.6868614592397737 / (299)) + (0.024875858265206465 / (19))) is greater than 1.9674995188240387 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (19) - 2, .975) and 0.06802154001583431 = (149.15242015512885 - 139.00684283908822) / 149.15242015512885 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.BinaryWriterExtendedTests.WriteAsciiString(StringLengthInChars: 8000)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 24.321884858920242 < 28.61876707848936.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 11/2/2022 9:35:40 AM, 11/8/2022 2:29:58 AM falls between 10/30/2022 12:44:54 AM and 11/8/2022 2:29:58 AM.
IsImprovementStdDev: Marked as improvement because 161.17191960008037 (T) = (0 -24324.22373022431) / Math.Sqrt((338102.7784153541 / (299)) + (627.5494466582504 / (19))) is greater than 1.9674995188240387 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (19) - 2, .975) and 0.18436788427717607 = (29822.542861333826 - 24324.22373022431) / 29822.542861333826 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.BinaryWriterExtendedTests.WriteAsciiString(StringLengthInChars: 2000000)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 9.130138682291665 < 11.801072355905877.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 11/2/2022 9:35:40 AM, 11/8/2022 2:29:58 AM falls between 10/30/2022 12:44:54 AM and 11/8/2022 2:29:58 AM.
IsImprovementStdDev: Marked as improvement because 32.19383449600631 (T) = (0 -9998879.014622495) / Math.Sqrt((61304351666.03463 / (299)) + (93856817284.63864 / (19))) is greater than 1.9674995188240387 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (19) - 2, .975) and 0.18761593753993697 = (12308068.900741195 - 9998879.014622495) / 12308068.900741195 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.BinaryWriterExtendedTests.WriteAsciiCharArray(StringLengthInChars: 8000)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 24.398079416926677 < 28.556258583233166.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 11/2/2022 9:35:40 AM, 11/8/2022 2:29:58 AM falls between 10/30/2022 12:44:54 AM and 11/8/2022 2:29:58 AM.
IsImprovementStdDev: Marked as improvement because 162.54858798098778 (T) = (0 -24313.640143982557) / Math.Sqrt((334131.17970084987 / (299)) + (576.5758878873324 / (19))) is greater than 1.9674995188240387 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (19) - 2, .975) and 0.18467391278155235 = (29820.755799597373 - 24313.640143982557) / 29820.755799597373 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
@performanceautofiler
Copy link
Author

performanceautofiler bot commented Nov 8, 2022 •
edited
Loading

Run Information

Architecture x64
OS ubuntu 18.04
Baseline bdd67af67dbf43c0f5e9151b98dc100b9555898a
Compare fbc20578253e0deb7267eea5a02b680c3cfe0e2a
Diff Diff

Improvements in System.IO.Tests.Perf_Directory

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GetCurrentDirectory - Duration of single invocation 2.27 μs 2.08 μs 0.92 0.01 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.IO.Tests.Perf_Directory*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.IO.Tests.Perf_Directory.GetCurrentDirectory

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.07965143670124 < 2.148233334148139.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 11/2/2022 9:35:40 AM, 11/8/2022 2:29:58 AM falls between 10/30/2022 12:44:54 AM and 11/8/2022 2:29:58 AM.
IsImprovementStdDev: Marked as improvement because 32.42171127990174 (T) = (0 -2067.4796927315597) / Math.Sqrt((2392.7632335368035 / (299)) + (288.36022247430236 / (19))) is greater than 1.9674995188240387 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (19) - 2, .975) and 0.0701997394693077 = (2223.574008843067 - 2067.4796927315597) / 2223.574008843067 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented Nov 8, 2022 •
edited
Loading

Run Information

Architecture x64
OS ubuntu 18.04
Baseline bdd67af67dbf43c0f5e9151b98dc100b9555898a
Compare fbc20578253e0deb7267eea5a02b680c3cfe0e2a
Diff Diff

Improvements in System.Numerics.Tests.Perf_VectorOf<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
AbsBenchmark - Duration of single invocation 12.79 ns 11.01 ns 0.86 0.00 False
LessThanAllBenchmark - Duration of single invocation 11.29 ns 5.77 ns 0.51 0.02 False
LessThanOrEqualAllBenchmark - Duration of single invocation 11.72 ns 6.09 ns 0.52 0.00 False
GreaterThanOrEqualAnyBenchmark - Duration of single invocation 12.83 ns 5.91 ns 0.46 0.01 False
MultiplyBenchmark - Duration of single invocation 14.13 ns 11.71 ns 0.83 0.01 False
LessThanAnyBenchmark - Duration of single invocation 11.94 ns 2.05 ns 0.17 0.02 False
MultiplyOperatorBenchmark - Duration of single invocation 13.95 ns 11.69 ns 0.84 0.00 False
LessThanOrEqualAnyBenchmark - Duration of single invocation 12.40 ns 2.88 ns 0.23 0.03 False
GreaterThanOrEqualAllBenchmark - Duration of single invocation 12.18 ns 1.77 ns 0.14 0.02 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;Int32&gt;*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Numerics.Tests.Perf_VectorOf<Int32>.AbsBenchmark

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 11.008526415811781 < 12.146207377766865.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/8/2022 2:29:58 AM falls between 10/30/2022 12:44:54 AM and 11/8/2022 2:29:58 AM.
IsImprovementStdDev: Marked as improvement because 378.46720348881945 (T) = (0 -11.011370701647161) / Math.Sqrt((0.0007769541001279637 / (299)) + (0.0003621073610819536 / (19))) is greater than 1.9674995188240387 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (19) - 2, .975) and 0.13789378562717117 = (12.772638125173232 - 11.011370701647161) / 12.772638125173232 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int32&gt;.LessThanAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 5.771588357504272 < 10.734780819755976.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/8/2022 2:29:58 AM falls between 10/30/2022 12:44:54 AM and 11/8/2022 2:29:58 AM.
IsImprovementStdDev: Marked as improvement because 135.68253866596092 (T) = (0 -5.622527835422011) / Math.Sqrt((0.03136161108568679 / (299)) + (0.022313890172623283 / (14))) is greater than 1.9676211333067681 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (14) - 2, .975) and 0.49865040069129096 = (11.214784739380844 - 5.622527835422011) / 11.214784739380844 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int32&gt;.LessThanOrEqualAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 6.088425878991845 < 11.13673882675185.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/8/2022 2:29:58 AM falls between 10/30/2022 12:44:54 AM and 11/8/2022 2:29:58 AM.
IsImprovementStdDev: Marked as improvement because 394.47886291468234 (T) = (0 -6.026080828092368) / Math.Sqrt((0.00035241837946868614 / (299)) + (0.00291394877538592 / (14))) is greater than 1.9676211333067681 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (14) - 2, .975) and 0.48641344390131924 = (11.733330548735225 - 6.026080828092368) / 11.733330548735225 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int32&gt;.GreaterThanOrEqualAnyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 5.90620066855462 < 12.173367050274228.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/8/2022 2:29:58 AM falls between 10/30/2022 12:44:54 AM and 11/8/2022 2:29:58 AM.
IsImprovementStdDev: Marked as improvement because 275.3607782118815 (T) = (0 -5.941361625057965) / Math.Sqrt((0.06486929797783926 / (299)) + (0.005403414808509102 / (14))) is greater than 1.9676211333067681 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (14) - 2, .975) and 0.5322736857730485 = (12.702645637711717 - 5.941361625057965) / 12.702645637711717 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int32&gt;.MultiplyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 11.713321466519297 < 13.4411284848205.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/8/2022 2:29:58 AM falls between 10/30/2022 12:44:54 AM and 11/8/2022 2:29:58 AM.
IsImprovementStdDev: Marked as improvement because 274.59538179946395 (T) = (0 -11.701642067082677) / Math.Sqrt((0.0164014480471696 / (299)) + (0.00047405252243787273 / (19))) is greater than 1.9674995188240387 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (19) - 2, .975) and 0.17330310556945927 = (14.154694599576546 - 11.701642067082677) / 14.154694599576546 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int32&gt;.LessThanAnyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.0482909761040733 < 11.358097256456222.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/8/2022 2:29:58 AM falls between 10/30/2022 12:44:54 AM and 11/8/2022 2:29:58 AM.
IsImprovementStdDev: Marked as improvement because 322.65827121424786 (T) = (0 -1.992022658132227) / Math.Sqrt((0.02811242331680177 / (299)) + (0.01183925118772555 / (14))) is greater than 1.9676211333067681 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (14) - 2, .975) and 0.8323617665553483 = (11.88286596201769 - 1.992022658132227) / 11.88286596201769 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int32&gt;.MultiplyOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 11.688327983354874 < 13.454637723817472.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/8/2022 2:29:58 AM falls between 10/30/2022 12:44:54 AM and 11/8/2022 2:29:58 AM.
IsImprovementStdDev: Marked as improvement because 348.8533731294273 (T) = (0 -11.706365887758297) / Math.Sqrt((0.003052982953455526 / (299)) + (0.0007298175423791773 / (19))) is greater than 1.9674995188240387 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (19) - 2, .975) and 0.1720457686777678 = (14.138904597496147 - 11.706365887758297) / 14.138904597496147 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int32&gt;.LessThanOrEqualAnyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.87601366062908 < 11.775990372872567.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/8/2022 2:29:58 AM falls between 10/30/2022 12:44:54 AM and 11/8/2022 2:29:58 AM.
IsImprovementStdDev: Marked as improvement because 132.38683787360767 (T) = (0 -2.478782735383709) / Math.Sqrt((0.0007626541728745831 / (299)) + (0.10659521554213419 / (19))) is greater than 1.9674995188240387 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (19) - 2, .975) and 0.8000505095129783 = (12.397044520323998 - 2.478782735383709) / 12.397044520323998 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int32&gt;.GreaterThanOrEqualAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.7657088248861375 < 11.563235103445775.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/8/2022 2:29:58 AM falls between 10/30/2022 12:44:54 AM and 11/8/2022 2:29:58 AM.
IsImprovementStdDev: Marked as improvement because 344.57685518360586 (T) = (0 -1.837426552541198) / Math.Sqrt((0.0632530367753014 / (299)) + (0.009327750177333036 / (14))) is greater than 1.9676211333067681 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (14) - 2, .975) and 0.8474726546044549 = (12.046538591334214 - 1.837426552541198) / 12.046538591334214 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS ubuntu 18.04
Baseline bdd67af67dbf43c0f5e9151b98dc100b9555898a
Compare 803688595257517f4b47e873d11152b0560c204a
Diff Diff

Improvements in System.Text.Json.Tests.Perf_Get

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GetString - Duration of single invocation 31.77 μs 27.40 μs 0.86 0.01 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Json.Tests.Perf_Get*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Text.Json.Tests.Perf_Get.GetString

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 27.39655150968823 < 30.242097625419284.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 11/2/2022 9:35:40 AM, 11/8/2022 2:29:58 AM falls between 10/30/2022 12:44:54 AM and 11/8/2022 2:29:58 AM.
IsImprovementStdDev: Marked as improvement because 42.19885850279634 (T) = (0 -27391.482970378776) / Math.Sqrt((1348431.3694033716 / (299)) + (39923.29060793533 / (14))) is greater than 1.9676211333067681 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (14) - 2, .975) and 0.11674860264483271 = (31012.102615858406 - 27391.482970378776) / 31012.102615858406 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline bdd67af67dbf43c0f5e9151b98dc100b9555898a
Compare fbc20578253e0deb7267eea5a02b680c3cfe0e2a
Diff Diff

Improvements in System.Numerics.Tests.Perf_VectorOf<Int64>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GreaterThanAnyBenchmark - Duration of single invocation 8.92 ns 4.13 ns 0.46 0.02 False
GreaterThanAllBenchmark - Duration of single invocation 8.34 ns 2.36 ns 0.28 0.01 False
LessThanAllBenchmark - Duration of single invocation 8.32 ns 4.18 ns 0.50 0.01 False
GreaterThanOrEqualAnyBenchmark - Duration of single invocation 9.32 ns 3.52 ns 0.38 0.03 False
AbsBenchmark - Duration of single invocation 10.71 ns 9.16 ns 0.86 0.00 False
EqualsAnyBenchmark - Duration of single invocation 8.92 ns 4.15 ns 0.47 0.01 False
GreaterThanOrEqualAllBenchmark - Duration of single invocation 8.67 ns 1.89 ns 0.22 0.05 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;Int64&gt;*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Numerics.Tests.Perf_VectorOf<Int64>.GreaterThanAnyBenchmark

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 4.134132631610551 < 8.473449839237308.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/8/2022 2:29:58 AM falls between 10/30/2022 12:44:54 AM and 11/8/2022 2:29:58 AM.
IsImprovementStdDev: Marked as improvement because 199.93419115949155 (T) = (0 -4.196557650734211) / Math.Sqrt((0.04102212584123662 / (299)) + (0.007875172657092972 / (19))) is greater than 1.9674995188240387 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (19) - 2, .975) and 0.5280836513454531 = (8.89258798238029 - 4.196557650734211) / 8.89258798238029 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int64&gt;.GreaterThanAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.364469244367396 < 7.894110778205718.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/8/2022 2:29:58 AM falls between 10/30/2022 12:44:54 AM and 11/8/2022 2:29:58 AM.
IsImprovementStdDev: Marked as improvement because 261.23853108414255 (T) = (0 -2.389600373257591) / Math.Sqrt((0.00615912623021518 / (299)) + (0.00923799762022021 / (19))) is greater than 1.9674995188240387 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (19) - 2, .975) and 0.7110769087575185 = (8.270714407011848 - 2.389600373257591) / 8.270714407011848 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int64&gt;.LessThanAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 4.182419551573215 < 7.900119703700054.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/8/2022 2:29:58 AM falls between 10/30/2022 12:44:54 AM and 11/8/2022 2:29:58 AM.
IsImprovementStdDev: Marked as improvement because 108.98978104732693 (T) = (0 -4.239218710375558) / Math.Sqrt((0.013233244365700808 / (299)) + (0.024788009213825863 / (19))) is greater than 1.9674995188240387 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (19) - 2, .975) and 0.4856635975419209 = (8.242112924762456 - 4.239218710375558) / 8.242112924762456 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int64&gt;.GreaterThanOrEqualAnyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 3.517836357366907 < 8.855192149475004.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/8/2022 2:29:58 AM falls between 10/30/2022 12:44:54 AM and 11/8/2022 2:29:58 AM.
IsImprovementStdDev: Marked as improvement because 186.21868957363068 (T) = (0 -3.361105010158742) / Math.Sqrt((0.036223338641742556 / (299)) + (0.015639953038650053 / (18))) is greater than 1.9675235316906419 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (18) - 2, .975) and 0.635472446937869 = (9.22044158781563 - 3.361105010158742) / 9.22044158781563 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int64&gt;.AbsBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 9.161834519520148 < 10.125307712514171.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/8/2022 2:29:58 AM falls between 10/30/2022 12:44:54 AM and 11/8/2022 2:29:58 AM.
IsImprovementStdDev: Marked as improvement because 53.93176170092677 (T) = (0 -8.960706453452152) / Math.Sqrt((0.002506089029401812 / (299)) + (0.01401693102910113 / (14))) is greater than 1.9676211333067681 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (14) - 2, .975) and 0.1605374630771476 = (10.674337518739865 - 8.960706453452152) / 10.674337518739865 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int64&gt;.EqualsAnyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 4.151689008417188 < 8.472708663984811.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/8/2022 2:29:58 AM falls between 10/30/2022 12:44:54 AM and 11/8/2022 2:29:58 AM.
IsImprovementStdDev: Marked as improvement because 359.07263451697355 (T) = (0 -4.157605145882321) / Math.Sqrt((0.008402784647643269 / (299)) + (0.0027495591441203332 / (19))) is greater than 1.9674995188240387 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (19) - 2, .975) and 0.5316938595105352 = (8.877964191408784 - 4.157605145882321) / 8.877964191408784 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int64&gt;.GreaterThanOrEqualAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.8866384117448385 < 8.241061237704415.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/8/2022 2:29:58 AM falls between 10/30/2022 12:44:54 AM and 11/8/2022 2:29:58 AM.
IsImprovementStdDev: Marked as improvement because 176.46447530052518 (T) = (0 -2.0577814645446044) / Math.Sqrt((0.0632305015450598 / (299)) + (0.02190912266749936 / (19))) is greater than 1.9674995188240387 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (19) - 2, .975) and 0.7600655037844598 = (8.57643022158864 - 2.0577814645446044) / 8.57643022158864 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented Nov 8, 2022 •
edited
Loading

Run Information

Architecture x64
OS ubuntu 18.04
Baseline bdd67af67dbf43c0f5e9151b98dc100b9555898a
Compare 803688595257517f4b47e873d11152b0560c204a
Diff Diff

Improvements in System.Text.Json.Serialization.Tests.WriteJson<BinaryData>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
SerializeToString - Duration of single invocation 10.93 μs 9.25 μs 0.85 0.01 False
SerializeObjectProperty - Duration of single invocation 11.30 μs 9.53 μs 0.84 0.01 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Json.Serialization.Tests.WriteJson&lt;BinaryData&gt;*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Text.Json.Serialization.Tests.WriteJson<BinaryData>.SerializeToString

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 9.246338153739886 < 10.387071816427696.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 11/2/2022 9:35:40 AM, 11/8/2022 2:29:58 AM falls between 10/30/2022 12:44:54 AM and 11/8/2022 2:29:58 AM.
IsImprovementStdDev: Marked as improvement because 56.00056991658926 (T) = (0 -9137.061121504647) / Math.Sqrt((90205.4487205626 / (299)) + (7633.53055558826 / (14))) is greater than 1.9676211333067681 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (14) - 2, .975) and 0.15136748204940326 = (10766.80533474038 - 9137.061121504647) / 10766.80533474038 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Serialization.Tests.WriteJson&lt;BinaryData&gt;.SerializeObjectProperty

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 9.529394956064031 < 10.758383498652524.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 11/2/2022 9:35:40 AM, 11/8/2022 2:29:58 AM falls between 10/30/2022 12:44:54 AM and 11/8/2022 2:29:58 AM.
IsImprovementStdDev: Marked as improvement because 50.217587634652844 (T) = (0 -9551.891124316739) / Math.Sqrt((100246.08206380982 / (299)) + (13158.75022881395 / (19))) is greater than 1.9674995188240387 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (19) - 2, .975) and 0.14423854220563462 = (11161.861798421873 - 9551.891124316739) / 11161.861798421873 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS ubuntu 18.04
Baseline bdd67af67dbf43c0f5e9151b98dc100b9555898a
Compare fbc20578253e0deb7267eea5a02b680c3cfe0e2a
Diff Diff

Improvements in System.Text.Json.Serialization.Tests.WriteJson<Hashtable>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
SerializeObjectProperty - Duration of single invocation 88.21 μs 79.07 μs 0.90 0.01 False
SerializeToString - Duration of single invocation 88.07 μs 79.10 μs 0.90 0.01 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Json.Serialization.Tests.WriteJson&lt;Hashtable&gt;*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Text.Json.Serialization.Tests.WriteJson<Hashtable>.SerializeObjectProperty

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 79.07008493416305 < 83.85984514126629.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 11/2/2022 9:35:40 AM, 11/8/2022 2:29:58 AM falls between 10/30/2022 12:44:54 AM and 11/8/2022 2:29:58 AM.
IsImprovementStdDev: Marked as improvement because 46.400741318724755 (T) = (0 -79211.38463233494) / Math.Sqrt((3761687.681238385 / (299)) + (155910.5062946719 / (19))) is greater than 1.9674995188240387 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (19) - 2, .975) and 0.0778786284992906 = (85901.25668969381 - 79211.38463233494) / 85901.25668969381 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Serialization.Tests.WriteJson&lt;Hashtable&gt;.SerializeToString

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 79.09997310606062 < 83.49314863968048.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 10/25/2022 8:17:52 AM, 11/2/2022 9:35:40 AM, 11/8/2022 2:29:58 AM falls between 10/30/2022 12:44:54 AM and 11/8/2022 2:29:58 AM.
IsImprovementStdDev: Marked as improvement because 42.959147684490496 (T) = (0 -78900.36513384937) / Math.Sqrt((3659394.362591054 / (299)) + (141382.03782320797 / (14))) is greater than 1.9676211333067681 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (14) - 2, .975) and 0.07525187546024441 = (85320.9247362549 - 78900.36513384937) / 85320.9247362549 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline bdd67af67dbf43c0f5e9151b98dc100b9555898a
Compare fbc20578253e0deb7267eea5a02b680c3cfe0e2a
Diff Diff

Improvements in System.Text.Json.Serialization.Tests.ReadJson<HashSet<String>>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
DeserializeFromString - Duration of single invocation 65.93 μs 61.38 μs 0.93 0.09 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Json.Serialization.Tests.ReadJson&lt;HashSet&lt;String&gt;&gt;*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Text.Json.Serialization.Tests.ReadJson<HashSet<String>>.DeserializeFromString

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 61.37950273109244 < 62.294576638354265.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 11/2/2022 9:35:40 AM, 11/8/2022 2:29:58 AM falls between 10/30/2022 12:44:54 AM and 11/8/2022 2:29:58 AM.
IsImprovementStdDev: Marked as improvement because 37.402661084315 (T) = (0 -61736.80300542994) / Math.Sqrt((1540581.8898921206 / (299)) + (51108.07839950465 / (19))) is greater than 1.9674995188240387 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (19) - 2, .975) and 0.05091953377997497 = (65049.07139361302 - 61736.80300542994) / 65049.07139361302 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline bdd67af67dbf43c0f5e9151b98dc100b9555898a
Compare fbc20578253e0deb7267eea5a02b680c3cfe0e2a
Diff Diff

Improvements in System.Text.Perf_Utf8Encoding

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GetBytes - Duration of single invocation 1.07 ms 878.83 μs 0.82 0.01 False
GetString - Duration of single invocation 1.13 ms 965.39 μs 0.86 0.01 False
GetByteCount - Duration of single invocation 396.62 μs 315.86 μs 0.80 0.01 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Perf_Utf8Encoding*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Text.Perf_Utf8Encoding.GetBytes(Input: EnglishAllAscii)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 878.8266675925926 < 1.0137889895897816.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 11/2/2022 9:35:40 AM, 11/8/2022 2:29:58 AM falls between 10/30/2022 12:44:54 AM and 11/8/2022 2:29:58 AM.
IsImprovementStdDev: Marked as improvement because 68.29088626751945 (T) = (0 -845661.026903368) / Math.Sqrt((462030915.5439399 / (299)) + (155713216.5421148 / (19))) is greater than 1.9674995188240387 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (19) - 2, .975) and 0.2013004153409423 = (1058797.3790725793 - 845661.026903368) / 1058797.3790725793 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Perf_Utf8Encoding.GetString(Input: EnglishAllAscii)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 965.3861850490196 < 1.0708486481808033.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 11/2/2022 9:35:40 AM, 11/8/2022 2:29:58 AM falls between 10/30/2022 12:44:54 AM and 11/8/2022 2:29:58 AM.
IsImprovementStdDev: Marked as improvement because 46.391804330451386 (T) = (0 -897804.6397767806) / Math.Sqrt((1430046112.0217016 / (299)) + (314488011.71423393 / (19))) is greater than 1.9674995188240387 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (19) - 2, .975) and 0.1926844447489061 = (1112086.387952159 - 897804.6397767806) / 1112086.387952159 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Perf_Utf8Encoding.GetByteCount(Input: EnglishAllAscii)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 315.85569872448974 < 376.8945140396557.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 11/2/2022 9:35:40 AM, 11/8/2022 2:29:58 AM falls between 10/30/2022 12:44:54 AM and 11/8/2022 2:29:58 AM.
IsImprovementStdDev: Marked as improvement because 38.244152219237364 (T) = (0 -293363.09697604505) / Math.Sqrt((77419365.25034836 / (299)) + (90365371.47085975 / (14))) is greater than 1.9676211333067681 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (14) - 2, .975) and 0.2524943528916756 = (392456.02773825265 - 293363.09697604505) / 392456.02773825265 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline bdd67af67dbf43c0f5e9151b98dc100b9555898a
Compare 803688595257517f4b47e873d11152b0560c204a
Diff Diff

Improvements in System.Text.Json.Serialization.Tests.WriteJson<ArrayList>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
SerializeToString - Duration of single invocation 44.10 μs 39.78 μs 0.90 0.00 False
SerializeObjectProperty - Duration of single invocation 44.77 μs 40.36 μs 0.90 0.00 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Json.Serialization.Tests.WriteJson&lt;ArrayList&gt;*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Text.Json.Serialization.Tests.WriteJson<ArrayList>.SerializeToString

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 39.780955927453476 < 41.83042905106437.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 10/25/2022 8:17:52 AM, 11/2/2022 9:35:40 AM, 11/8/2022 2:29:58 AM falls between 10/30/2022 12:44:54 AM and 11/8/2022 2:29:58 AM.
IsImprovementStdDev: Marked as improvement because 44.48992817154189 (T) = (0 -39699.97047086795) / Math.Sqrt((848389.7171315993 / (299)) + (26795.83625036826 / (14))) is greater than 1.9676211333067681 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (14) - 2, .975) and 0.07170797988187008 = (42766.682908483825 - 39699.97047086795) / 42766.682908483825 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Serialization.Tests.WriteJson&lt;ArrayList&gt;.SerializeObjectProperty

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 40.361581876606685 < 42.34956624805342.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 10/25/2022 8:17:52 AM, 11/2/2022 9:35:40 AM, 11/8/2022 2:29:58 AM falls between 10/30/2022 12:44:54 AM and 11/8/2022 2:29:58 AM.
IsImprovementStdDev: Marked as improvement because 46.74722424905352 (T) = (0 -40224.03742444491) / Math.Sqrt((831044.1395631724 / (299)) + (21422.192411484648 / (14))) is greater than 1.9676211333067681 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (14) - 2, .975) and 0.07088533169789345 = (43292.86663609734 - 40224.03742444491) / 43292.86663609734 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline bdd67af67dbf43c0f5e9151b98dc100b9555898a
Compare 803688595257517f4b47e873d11152b0560c204a
Diff Diff

Improvements in System.Text.Tests.Perf_Encoding

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GetChars - Duration of single invocation 3.45 μs 2.80 μs 0.81 0.01 False
GetChars - Duration of single invocation 147.08 ns 135.06 ns 0.92 0.01 False
GetString - Duration of single invocation 3.46 μs 2.78 μs 0.80 0.01 False
GetChars - Duration of single invocation 2.53 μs 2.09 μs 0.83 0.01 False
GetChars - Duration of single invocation 107.43 ns 94.61 ns 0.88 0.01 False
GetString - Duration of single invocation 2.53 μs 2.10 μs 0.83 0.01 False
GetByteCount - Duration of single invocation 78.60 ns 68.90 ns 0.88 0.01 False
GetByteCount - Duration of single invocation 1.29 μs 952.27 ns 0.74 0.01 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Tests.Perf_Encoding*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Text.Tests.Perf_Encoding.GetChars(size: 512, encName: "utf-8")

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.7996003858839056 < 3.2836057346726792.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 11/2/2022 9:35:40 AM, 11/8/2022 2:29:58 AM falls between 10/30/2022 12:44:54 AM and 11/8/2022 2:29:58 AM.
IsImprovementStdDev: Marked as improvement because 61.430669825246255 (T) = (0 -2756.8895446441275) / Math.Sqrt((14654.859166204442 / (299)) + (789.2452258892961 / (14))) is greater than 1.9676211333067681 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (14) - 2, .975) and 0.18616461625768652 = (3387.5272563929802 - 2756.8895446441275) / 3387.5272563929802 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Tests.Perf_Encoding.GetChars(size: 16, encName: "utf-8")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 135.06142155619494 < 138.83531039447388.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 11/2/2022 9:35:40 AM, 11/8/2022 2:29:58 AM falls between 10/30/2022 12:44:54 AM and 11/8/2022 2:29:58 AM.
IsImprovementStdDev: Marked as improvement because 23.55070080902951 (T) = (0 -136.15345446700496) / Math.Sqrt((21.494142226717823 / (299)) + (0.4882274677640126 / (14))) is greater than 1.9676211333067681 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (14) - 2, .975) and 0.05349377428589469 = (143.84845103821903 - 136.15345446700496) / 143.84845103821903 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Tests.Perf_Encoding.GetString(size: 512, encName: "utf-8")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.7793864611192416 < 3.2814156791136533.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 11/2/2022 9:35:40 AM, 11/8/2022 2:29:58 AM falls between 10/30/2022 12:44:54 AM and 11/8/2022 2:29:58 AM.
IsImprovementStdDev: Marked as improvement because 66.20044624141188 (T) = (0 -2759.2447309073627) / Math.Sqrt((20287.49114520339 / (299)) + (169.70566033621697 / (14))) is greater than 1.9676211333067681 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (14) - 2, .975) and 0.17665440931986784 = (3351.25949800504 - 2759.2447309073627) / 3351.25949800504 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Tests.Perf_Encoding.GetChars(size: 512, encName: "ascii")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.094807615222058 < 2.4046913226700624.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 11/2/2022 9:35:40 AM, 11/8/2022 2:29:58 AM falls between 10/30/2022 12:44:54 AM and 11/8/2022 2:29:58 AM.
IsImprovementStdDev: Marked as improvement because 55.91846646255181 (T) = (0 -2086.5226719681004) / Math.Sqrt((9872.982773518643 / (299)) + (228.4822090056981 / (14))) is greater than 1.9676211333067681 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (14) - 2, .975) and 0.15842547961606704 = (2479.3082744665467 - 2086.5226719681004) / 2479.3082744665467 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Tests.Perf_Encoding.GetChars(size: 16, encName: "ascii")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 94.60540507128259 < 102.06143305295402.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 11/2/2022 9:35:40 AM, 11/8/2022 2:29:58 AM falls between 10/30/2022 12:44:54 AM and 11/8/2022 2:29:58 AM.
IsImprovementStdDev: Marked as improvement because 30.35356306802765 (T) = (0 -94.49076450290579) / Math.Sqrt((20.241065203414404 / (299)) + (0.7256447863842767 / (14))) is greater than 1.9676211333067681 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (14) - 2, .975) and 0.09995793933802677 = (104.98483196819562 - 94.49076450290579) / 104.98483196819562 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Tests.Perf_Encoding.GetString(size: 512, encName: "ascii")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.1000206973607956 < 2.4010979456982793.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 11/2/2022 9:35:40 AM, 11/8/2022 2:29:58 AM falls between 10/30/2022 12:44:54 AM and 11/8/2022 2:29:58 AM.
IsImprovementStdDev: Marked as improvement because 43.878409173020025 (T) = (0 -2084.4582897820924) / Math.Sqrt((13229.397166893003 / (299)) + (332.9418692844377 / (14))) is greater than 1.9676211333067681 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (14) - 2, .975) and 0.1479350679285497 = (2446.3608480102303 - 2084.4582897820924) / 2446.3608480102303 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Tests.Perf_Encoding.GetByteCount(size: 16, encName: "utf-8")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 68.89523516431204 < 74.65677024867654.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 11/2/2022 9:35:40 AM, 11/8/2022 2:29:58 AM falls between 10/30/2022 12:44:54 AM and 11/8/2022 2:29:58 AM.
IsImprovementStdDev: Marked as improvement because 63.267094061065244 (T) = (0 -68.6442640098323) / Math.Sqrt((5.460825447801921 / (299)) + (0.008402951165416058 / (19))) is greater than 1.9674995188240387 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (19) - 2, .975) and 0.11194454111082286 = (77.29727160924823 - 68.6442640098323) / 77.29727160924823 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Tests.Perf_Encoding.GetByteCount(size: 512, encName: "utf-8")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 952.2738550825848 < 1.2215473057604156.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 11/2/2022 9:35:40 AM, 11/8/2022 2:29:58 AM falls between 10/30/2022 12:44:54 AM and 11/8/2022 2:29:58 AM.
IsImprovementStdDev: Marked as improvement because 189.69161514210674 (T) = (0 -951.4655466404892) / Math.Sqrt((844.3747588641631 / (299)) + (0.43119013395044575 / (19))) is greater than 1.9674995188240387 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (19) - 2, .975) and 0.2517076720844175 = (1271.5158383233181 - 951.4655466404892) / 1271.5158383233181 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

Run Information

Architecture x64
OS ubuntu 18.04
Baseline bdd67af67dbf43c0f5e9151b98dc100b9555898a
Compare 803688595257517f4b47e873d11152b0560c204a
Diff Diff

Improvements in System.Numerics.Tests.Perf_VectorConvert

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Widen_int - Duration of single invocation 20.80 μs 17.69 μs 0.85 0.00 False
Widen_short - Duration of single invocation 23.63 μs 20.72 μs 0.88 0.00 False
Widen_uint - Duration of single invocation 20.81 μs 18.05 μs 0.87 0.00 False
Widen_ushort - Duration of single invocation 23.70 μs 21.03 μs 0.89 0.01 False
Widen_sbyte - Duration of single invocation 30.18 μs 26.92 μs 0.89 0.00 False
Widen_byte - Duration of single invocation 30.07 μs 27.57 μs 0.92 0.00 False
Widen_float - Duration of single invocation 32.95 μs 29.99 μs 0.91 0.00 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Numerics.Tests.Perf_VectorConvert*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Numerics.Tests.Perf_VectorConvert.Widen_int

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 17.690857634228188 < 19.903034502243383.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/8/2022 2:29:58 AM falls between 10/30/2022 12:44:54 AM and 11/8/2022 2:29:58 AM.
IsImprovementStdDev: Marked as improvement because 85.72252630368207 (T) = (0 -17604.44562324196) / Math.Sqrt((8613.850241581025 / (299)) + (20484.27194249389 / (14))) is greater than 1.9676211333067681 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (14) - 2, .975) and 0.15830896869199376 = (20915.56755200809 - 17604.44562324196) / 20915.56755200809 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorConvert.Widen_short

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 20.722240509697258 < 22.429648184608745.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 11/2/2022 9:35:40 AM, 11/8/2022 2:29:58 AM falls between 10/30/2022 12:44:54 AM and 11/8/2022 2:29:58 AM.
IsImprovementStdDev: Marked as improvement because 205.18338958208415 (T) = (0 -20716.87946413478) / Math.Sqrt((53510.90815737743 / (299)) + (224.74070409448467 / (14))) is greater than 1.9676211333067681 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (14) - 2, .975) and 0.12150540325478065 = (23582.250296006183 - 20716.87946413478) / 23582.250296006183 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorConvert.Widen_uint

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 18.047222749903327 < 19.875804205469024.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/8/2022 2:29:58 AM falls between 10/30/2022 12:44:54 AM and 11/8/2022 2:29:58 AM.
IsImprovementStdDev: Marked as improvement because 69.30054150200401 (T) = (0 -17855.840456765636) / Math.Sqrt((4389.204119536702 / (299)) + (36846.79934867938 / (19))) is greater than 1.9674995188240387 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (19) - 2, .975) and 0.14643748480965157 = (20919.194715086218 - 17855.840456765636) / 20919.194715086218 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorConvert.Widen_ushort

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 21.026244366926935 < 22.458807126250395.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 11/2/2022 9:35:40 AM, 11/8/2022 2:29:58 AM falls between 10/30/2022 12:44:54 AM and 11/8/2022 2:29:58 AM.
IsImprovementStdDev: Marked as improvement because 11.62814672898728 (T) = (0 -21206.335341107224) / Math.Sqrt((30599.377454245332 / (299)) + (562897.3620675363 / (14))) is greater than 1.9676211333067681 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (14) - 2, .975) and 0.09917205689403207 = (23540.93864805062 - 21206.335341107224) / 23540.93864805062 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorConvert.Widen_sbyte

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 26.92306762048193 < 28.662676249856638.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 11/2/2022 9:35:40 AM, 11/8/2022 2:29:58 AM falls between 10/30/2022 12:44:54 AM and 11/8/2022 2:29:58 AM.
IsImprovementStdDev: Marked as improvement because 84.77047959212629 (T) = (0 -26804.27510520042) / Math.Sqrt((169847.80902021236 / (299)) + (10922.251093787632 / (14))) is greater than 1.9676211333067681 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (14) - 2, .975) and 0.10404176293452615 = (29916.880046766786 - 26804.27510520042) / 29916.880046766786 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorConvert.Widen_byte

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 27.56619286971831 < 28.57814895126868.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 11/2/2022 9:35:40 AM, 11/8/2022 2:29:58 AM falls between 10/30/2022 12:44:54 AM and 11/8/2022 2:29:58 AM.
IsImprovementStdDev: Marked as improvement because 95.56438164251769 (T) = (0 -27566.04306683094) / Math.Sqrt((156243.67643360648 / (299)) + (1089.5382521258036 / (14))) is greater than 1.9676211333067681 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (14) - 2, .975) and 0.07829367376464255 = (29907.620553525372 - 27566.04306683094) / 29907.620553525372 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorConvert.Widen_float

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 29.98863488247863 < 31.31092073076923.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/8/2022 2:29:58 AM falls between 10/30/2022 12:44:54 AM and 11/8/2022 2:29:58 AM.
IsImprovementStdDev: Marked as improvement because 452.06699004959034 (T) = (0 -30008.16457470495) / Math.Sqrt((644.3307786322771 / (299)) + (567.0314196284814 / (14))) is greater than 1.9676211333067681 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (14) - 2, .975) and 0.08957816914855078 = (32960.726069848926 - 30008.16457470495) / 32960.726069848926 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS ubuntu 18.04
Baseline bdd67af67dbf43c0f5e9151b98dc100b9555898a
Compare fbc20578253e0deb7267eea5a02b680c3cfe0e2a
Diff Diff

Improvements in System.Buffers.Tests.RentReturnArrayPoolTests<Object>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ProducerConsumer - Duration of single invocation 4.88 μs 3.92 μs 0.80 0.32 False
SingleSerial - Duration of single invocation 6.48 μs 5.56 μs 0.86 0.12 False
MultipleSerial - Duration of single invocation 53.31 μs 46.23 μs 0.87 0.17 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Buffers.Tests.RentReturnArrayPoolTests&lt;Object&gt;*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Buffers.Tests.RentReturnArrayPoolTests<Object>.ProducerConsumer(RentalSize: 4096, ManipulateArray: True, Async: True, UseSharedPool: False)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 3.916858862142858 < 4.587862084454762.
IsChangePoint: Marked as a change because one of 8/2/2022 9:59:24 PM, 10/11/2022 10:51:47 AM, 11/2/2022 9:35:40 AM, 11/8/2022 2:29:58 AM falls between 10/30/2022 12:44:54 AM and 11/8/2022 2:29:58 AM.
IsImprovementStdDev: Marked as improvement because 4.405547310887852 (T) = (0 -4040.195674882654) / Math.Sqrt((205957.07187988135 / (299)) + (80252.81872571696 / (19))) is greater than 1.9674995188240387 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (19) - 2, .975) and 0.0710019508393292 = (4348.981871956439 - 4040.195674882654) / 4348.981871956439 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Buffers.Tests.RentReturnArrayPoolTests&lt;Object&gt;.SingleSerial(RentalSize: 4096, ManipulateArray: True, Async: False, UseSharedPool: False)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 5.561234449333333 < 6.150492921249999.
IsChangePoint: Marked as a change because one of 7/27/2022 9:27:06 AM, 9/13/2022 4:57:27 AM, 11/2/2022 9:35:40 AM, 11/8/2022 2:29:58 AM falls between 10/30/2022 12:44:54 AM and 11/8/2022 2:29:58 AM.
IsImprovementStdDev: Marked as improvement because 31.410303385176704 (T) = (0 -5562.042599830344) / Math.Sqrt((147566.67550958032 / (299)) + (34.873512179873124 / (19))) is greater than 1.9674995188240387 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (19) - 2, .975) and 0.11165635427209407 = (6261.138498123465 - 5562.042599830344) / 6261.138498123465 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Buffers.Tests.RentReturnArrayPoolTests&lt;Object&gt;.MultipleSerial(RentalSize: 4096, ManipulateArray: True, Async: False, UseSharedPool: True)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 46.234224538571425 < 50.6533420155673.
IsChangePoint: Marked as a change because one of 8/5/2022 4:49:35 PM, 11/2/2022 9:35:40 AM, 11/8/2022 2:29:58 AM falls between 10/30/2022 12:44:54 AM and 11/8/2022 2:29:58 AM.
IsImprovementStdDev: Marked as improvement because 31.177784290492866 (T) = (0 -46194.381522302196) / Math.Sqrt((9311967.989020811 / (299)) + (4616.345247201156 / (18))) is greater than 1.9675235316906419 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (18) - 2, .975) and 0.10682186778898779 = (51719.11386584287 - 46194.381522302196) / 51719.11386584287 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline bdd67af67dbf43c0f5e9151b98dc100b9555898a
Compare 803688595257517f4b47e873d11152b0560c204a
Diff Diff

Improvements in System.Numerics.Tests.Perf_VectorOf<Double>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
LessThanAllBenchmark - Duration of single invocation 9.20 ns 3.26 ns 0.35 0.01 False
AbsBenchmark - Duration of single invocation 10.81 ns 8.86 ns 0.82 0.00 False
LessThanOrEqualAnyBenchmark - Duration of single invocation 9.90 ns 2.45 ns 0.25 0.03 False
LessThanOrEqualAllBenchmark - Duration of single invocation 9.23 ns 3.78 ns 0.41 0.01 False
LessThanAnyBenchmark - Duration of single invocation 9.86 ns 2.24 ns 0.23 0.04 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;Double&gt;*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Numerics.Tests.Perf_VectorOf<Double>.LessThanAllBenchmark

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 3.2647560441063908 < 8.73081123464963.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/8/2022 2:29:58 AM falls between 10/30/2022 12:44:54 AM and 11/8/2022 2:29:58 AM.
IsImprovementStdDev: Marked as improvement because 193.6873093825916 (T) = (0 -3.332446588026728) / Math.Sqrt((0.026472608139715522 / (299)) + (0.011263159729420326 / (14))) is greater than 1.9676211333067681 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (14) - 2, .975) and 0.634623482672052 = (9.120582276051561 - 3.332446588026728) / 9.120582276051561 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Double&gt;.AbsBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 8.857459647460592 < 10.279868497854226.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/8/2022 2:29:58 AM falls between 10/30/2022 12:44:54 AM and 11/8/2022 2:29:58 AM.
IsImprovementStdDev: Marked as improvement because 98.379889591572 (T) = (0 -8.958412820998108) / Math.Sqrt((0.00033193773363477373 / (299)) + (0.0068364880297810855 / (19))) is greater than 1.9674995188240387 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (19) - 2, .975) and 0.17261928967440718 = (10.827437368551621 - 8.958412820998108) / 10.827437368551621 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Double&gt;.LessThanOrEqualAnyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.4499708597521486 < 9.430588654980847.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/8/2022 2:29:58 AM falls between 10/30/2022 12:44:54 AM and 11/8/2022 2:29:58 AM.
IsImprovementStdDev: Marked as improvement because 160.31613476324873 (T) = (0 -2.564705139078456) / Math.Sqrt((0.02680587095635243 / (299)) + (0.02837074716616924 / (14))) is greater than 1.9676211333067681 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (14) - 2, .975) and 0.7419678830912847 = (9.939480285649005 - 2.564705139078456) / 9.939480285649005 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Double&gt;.LessThanOrEqualAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 3.784667159348111 < 8.778046289719843.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/8/2022 2:29:58 AM falls between 10/30/2022 12:44:54 AM and 11/8/2022 2:29:58 AM.
IsImprovementStdDev: Marked as improvement because 53.58487130930869 (T) = (0 -4.293199393322834) / Math.Sqrt((0.026420917703319346 / (299)) + (0.11904406268855103 / (14))) is greater than 1.9676211333067681 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (14) - 2, .975) and 0.5363717030404417 = (9.260002940884611 - 4.293199393322834) / 9.260002940884611 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Double&gt;.LessThanAnyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.2443366211432676 < 9.37328062594638.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/8/2022 2:29:58 AM falls between 10/30/2022 12:44:54 AM and 11/8/2022 2:29:58 AM.
IsImprovementStdDev: Marked as improvement because 193.66001340033648 (T) = (0 -2.0772094012749465) / Math.Sqrt((0.04572254308226407 / (299)) + (0.019901599368384756 / (14))) is greater than 1.9676211333067681 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (14) - 2, .975) and 0.7872046950425317 = (9.761537744877035 - 2.0772094012749465) / 9.761537744877035 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline bdd67af67dbf43c0f5e9151b98dc100b9555898a
Compare fbc20578253e0deb7267eea5a02b680c3cfe0e2a
Diff Diff

Improvements in System.Numerics.Tests.Perf_VectorOf<Int16>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GreaterThanAnyBenchmark - Duration of single invocation 18.87 ns 11.45 ns 0.61 0.03 False
EqualsAnyBenchmark - Duration of single invocation 18.88 ns 11.56 ns 0.61 0.02 False
GreaterThanAllBenchmark - Duration of single invocation 18.23 ns 2.43 ns 0.13 0.01 False
LessThanOrEqualAllBenchmark - Duration of single invocation 19.74 ns 12.69 ns 0.64 0.05 False
GreaterThanOrEqualAnyBenchmark - Duration of single invocation 20.37 ns 16.60 ns 0.81 0.16 False
LessThanOrEqualAnyBenchmark - Duration of single invocation 20.41 ns 2.52 ns 0.12 0.04 False
GreaterThanOrEqualAllBenchmark - Duration of single invocation 19.74 ns 3.07 ns 0.16 0.18 False
GetHashCodeBenchmark - Duration of single invocation 53.15 ns 41.41 ns 0.78 0.48 False
LessThanAllBenchmark - Duration of single invocation 18.23 ns 11.35 ns 0.62 0.15 False
LessThanAnyBenchmark - Duration of single invocation 18.86 ns 1.75 ns 0.09 0.15 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;Int16&gt;*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Numerics.Tests.Perf_VectorOf<Int16>.GreaterThanAnyBenchmark

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 11.452625998113678 < 17.923000188992578.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/8/2022 2:29:58 AM falls between 10/30/2022 12:44:54 AM and 11/8/2022 2:29:58 AM.
IsImprovementStdDev: Marked as improvement because 90.8285034370247 (T) = (0 -11.184021717641375) / Math.Sqrt((0.08930478681587584 / (299)) + (0.12602039557896372 / (19))) is greater than 1.9674995188240387 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (19) - 2, .975) and 0.40338881666517784 = (18.745913636963838 - 11.184021717641375) / 18.745913636963838 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int16&gt;.EqualsAnyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 11.559732150662017 < 17.921903681154795.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/8/2022 2:29:58 AM falls between 10/30/2022 12:44:54 AM and 11/8/2022 2:29:58 AM.
IsImprovementStdDev: Marked as improvement because 105.97762768379994 (T) = (0 -11.120515590061181) / Math.Sqrt((0.1053409203223075 / (299)) + (0.09104928768865582 / (19))) is greater than 1.9674995188240387 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (19) - 2, .975) and 0.40600896301824574 = (18.721689213641742 - 11.120515590061181) / 18.721689213641742 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int16&gt;.GreaterThanAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.4267545971501705 < 17.32692476222341.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/8/2022 2:29:58 AM falls between 10/30/2022 12:44:54 AM and 11/8/2022 2:29:58 AM.
IsImprovementStdDev: Marked as improvement because 602.0907934538805 (T) = (0 -2.3581303818860078) / Math.Sqrt((0.108593002524139 / (299)) + (0.006041423787923774 / (19))) is greater than 1.9674995188240387 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (19) - 2, .975) and 0.8695153363546104 = (18.0720884432408 - 2.3581303818860078) / 18.0720884432408 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int16&gt;.LessThanOrEqualAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 12.68580739538118 < 18.74242744622705.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 11/2/2022 9:35:40 AM, 11/8/2022 2:29:58 AM falls between 10/30/2022 12:44:54 AM and 11/8/2022 2:29:58 AM.
IsImprovementStdDev: Marked as improvement because 11.562935440026184 (T) = (0 -15.12657954963372) / Math.Sqrt((0.7341844093637053 / (299)) + (2.3537657186300813 / (19))) is greater than 1.9674995188240387 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (19) - 2, .975) and 0.21365276158560473 = (19.236513858858622 - 15.12657954963372) / 19.236513858858622 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int16&gt;.GreaterThanOrEqualAnyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 16.595336616084303 < 19.35417618326401.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 11/2/2022 9:35:40 AM, 11/8/2022 2:29:58 AM falls between 10/30/2022 12:44:54 AM and 11/8/2022 2:29:58 AM.
IsImprovementStdDev: Marked as improvement because 26.07496179535897 (T) = (0 -15.337168302767722) / Math.Sqrt((0.3730398700070579 / (299)) + (0.6250930374673292 / (19))) is greater than 1.9674995188240387 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (19) - 2, .975) and 0.2390599655175854 = (20.155554456009064 - 15.337168302767722) / 20.155554456009064 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int16&gt;.LessThanOrEqualAnyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.523468473126167 < 19.37147882503521.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/8/2022 2:29:58 AM falls between 10/30/2022 12:44:54 AM and 11/8/2022 2:29:58 AM.
IsImprovementStdDev: Marked as improvement because 270.6651686257745 (T) = (0 -2.7494894861447405) / Math.Sqrt((0.7406088971622274 / (299)) + (0.029119961407455548 / (19))) is greater than 1.9674995188240387 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (19) - 2, .975) and 0.8617537652892315 = (19.888349884516394 - 2.7494894861447405) / 19.888349884516394 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int16&gt;.GreaterThanOrEqualAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 3.0701424221954565 < 18.7435306321282.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/8/2022 2:29:58 AM falls between 10/30/2022 12:44:54 AM and 11/8/2022 2:29:58 AM.
IsImprovementStdDev: Marked as improvement because 253.66196029882582 (T) = (0 -2.8786318136582936) / Math.Sqrt((0.41645998424845154 / (299)) + (0.05556099489134905 / (19))) is greater than 1.9674995188240387 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (19) - 2, .975) and 0.8527208351996629 = (19.545411040053008 - 2.8786318136582936) / 19.545411040053008 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int16&gt;.GetHashCodeBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 41.41037283054157 < 49.680002056458406.
IsChangePoint: Marked as a change because one of 9/17/2022 12:21:41 PM, 11/2/2022 9:17:08 PM, 11/8/2022 2:29:58 AM falls between 10/30/2022 12:44:54 AM and 11/8/2022 2:29:58 AM.
IsImprovementStdDev: Marked as improvement because 7.379024349539957 (T) = (0 -42.822067517743925) / Math.Sqrt((21.34106372680749 / (299)) + (7.148954019957949 / (17))) is greater than 1.9675476980854363 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (17) - 2, .975) and 0.10782521456260975 = (47.9973971655681 - 42.822067517743925) / 47.9973971655681 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int16&gt;.LessThanAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 11.353195530388101 < 17.32421673782099.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 11/2/2022 9:35:40 AM, 11/8/2022 2:29:58 AM falls between 10/30/2022 12:44:54 AM and 11/8/2022 2:29:58 AM.
IsImprovementStdDev: Marked as improvement because 25.67261116307652 (T) = (0 -12.797394542782653) / Math.Sqrt((0.412836383563778 / (299)) + (0.8172324185645647 / (19))) is greater than 1.9674995188240387 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (19) - 2, .975) and 0.2970982730427694 = (18.206520274435654 - 12.797394542782653) / 18.206520274435654 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int16&gt;.LessThanAnyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.7506965833149024 < 17.914235577463202.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/8/2022 2:29:58 AM falls between 10/30/2022 12:44:54 AM and 11/8/2022 2:29:58 AM.
IsImprovementStdDev: Marked as improvement because 346.6210572281065 (T) = (0 -1.8620732798641644) / Math.Sqrt((0.41464278790132053 / (299)) + (0.018178647035906204 / (19))) is greater than 1.9674995188240387 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (19) - 2, .975) and 0.9001142279600215 = (18.642027206025755 - 1.8620732798641644) / 18.642027206025755 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented Nov 8, 2022 •
edited
Loading

Run Information

Architecture x64
OS ubuntu 18.04
Baseline bdd67af67dbf43c0f5e9151b98dc100b9555898a
Compare 803688595257517f4b47e873d11152b0560c204a
Diff Diff

Improvements in System.Numerics.Tests.Perf_VectorOf<SByte>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GreaterThanOrEqualAllBenchmark - Duration of single invocation 36.63 ns 1.94 ns 0.05 0.48 False
GreaterThanAllBenchmark - Duration of single invocation 32.98 ns 2.27 ns 0.07 0.31 False
GreaterThanOrEqualAnyBenchmark - Duration of single invocation 37.10 ns 25.44 ns 0.69 0.38 False
LessThanAllBenchmark - Duration of single invocation 32.30 ns 22.28 ns 0.69 0.26 False
MultiplyBenchmark - Duration of single invocation 33.03 ns 24.32 ns 0.74 0.28 False
AbsBenchmark - Duration of single invocation 26.16 ns 21.13 ns 0.81 0.31 False
LessThanOrEqualAllBenchmark - Duration of single invocation 29.66 ns 23.98 ns 0.81 0.18 False
GreaterThanAnyBenchmark - Duration of single invocation 26.80 ns 18.64 ns 0.70 0.28 False
MultiplyOperatorBenchmark - Duration of single invocation 26.58 ns 24.33 ns 0.92 0.30 False
LessThanOrEqualAnyBenchmark - Duration of single invocation 30.30 ns 2.44 ns 0.08 0.15 False
EqualsAnyBenchmark - Duration of single invocation 26.82 ns 19.19 ns 0.72 0.29 False
LessThanAnyBenchmark - Duration of single invocation 26.84 ns 2.27 ns 0.08 0.29 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;SByte&gt;*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Numerics.Tests.Perf_VectorOf<SByte>.GreaterThanOrEqualAllBenchmark

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.9415523914241508 < 34.8134335493964.
IsChangePoint: Marked as a change because one of 8/4/2022 6:25:19 PM, 11/2/2022 9:35:40 AM, 11/8/2022 2:29:58 AM falls between 10/30/2022 12:44:54 AM and 11/8/2022 2:29:58 AM.
IsImprovementStdDev: Marked as improvement because 146.80119555173266 (T) = (0 -2.1635911209445706) / Math.Sqrt((12.49916186399267 / (299)) + (0.060727997648251396 / (14))) is greater than 1.9676211333067681 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (14) - 2, .975) and 0.9357930228401169 = (33.69713412230837 - 2.1635911209445706) / 33.69713412230837 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;SByte&gt;.GreaterThanAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.2676411337001983 < 24.828943805622014.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 11/2/2022 9:35:40 AM, 11/8/2022 2:29:58 AM falls between 10/30/2022 12:44:54 AM and 11/8/2022 2:29:58 AM.
IsImprovementStdDev: Marked as improvement because 145.4283997233927 (T) = (0 -2.053053794110837) / Math.Sqrt((7.913810858439054 / (299)) + (0.038917734382147066 / (14))) is greater than 1.9676211333067681 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (14) - 2, .975) and 0.9237464595005734 = (26.924045502205598 - 2.053053794110837) / 26.924045502205598 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;SByte&gt;.GreaterThanOrEqualAnyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 25.438438844645322 < 34.78734402571764.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 11/2/2022 9:35:40 AM, 11/8/2022 2:29:58 AM falls between 10/30/2022 12:44:54 AM and 11/8/2022 2:29:58 AM.
IsImprovementStdDev: Marked as improvement because 25.80745797446757 (T) = (0 -25.224169987254037) / Math.Sqrt((9.282656716010376 / (299)) + (1.3138132029285337 / (14))) is greater than 1.9676211333067681 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (14) - 2, .975) and 0.2655532304419401 = (34.34444949963117 - 25.224169987254037) / 34.34444949963117 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;SByte&gt;.LessThanAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 22.280020061825123 < 25.62473084767538.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 11/2/2022 9:35:40 AM, 11/8/2022 2:29:58 AM falls between 10/30/2022 12:44:54 AM and 11/8/2022 2:29:58 AM.
IsImprovementStdDev: Marked as improvement because 27.56622425644487 (T) = (0 -19.636803104142615) / Math.Sqrt((7.117007110338729 / (299)) + (0.7008001130712648 / (19))) is greater than 1.9674995188240387 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (19) - 2, .975) and 0.2569603090325864 = (26.427663747781942 - 19.636803104142615) / 26.427663747781942 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;SByte&gt;.MultiplyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 24.322292183275632 < 31.60987215692679.
IsChangePoint: Marked as a change because one of 9/14/2022 12:15:21 PM, 11/2/2022 9:35:40 AM, 11/8/2022 2:29:58 AM falls between 10/30/2022 12:44:54 AM and 11/8/2022 2:29:58 AM.
IsImprovementStdDev: Marked as improvement because 25.675882941788405 (T) = (0 -24.391580950817698) / Math.Sqrt((13.65498632011139 / (299)) + (0.004680128052405065 / (19))) is greater than 1.9674995188240387 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (19) - 2, .975) and 0.18404698290156732 = (29.893364494876565 - 24.391580950817698) / 29.893364494876565 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;SByte&gt;.AbsBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 21.133611038663805 < 24.69642468824434.
IsChangePoint: Marked as a change because one of 8/4/2022 9:08:56 AM, 11/2/2022 9:35:40 AM, 11/8/2022 2:29:58 AM falls between 10/30/2022 12:44:54 AM and 11/8/2022 2:29:58 AM.
IsImprovementStdDev: Marked as improvement because 19.226011311811792 (T) = (0 -21.23075043040384) / Math.Sqrt((2.9268050904018086 / (299)) + (0.5791676285738694 / (19))) is greater than 1.9674995188240387 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (19) - 2, .975) and 0.15378129049722356 = (25.08896363551057 - 21.23075043040384) / 25.08896363551057 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;SByte&gt;.LessThanOrEqualAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 23.98272012745423 < 28.174944695998406.
IsChangePoint: Marked as a change because one of 8/5/2022 4:49:35 PM, 11/2/2022 9:35:40 AM, 11/8/2022 2:29:58 AM falls between 10/30/2022 12:44:54 AM and 11/8/2022 2:29:58 AM.
IsImprovementStdDev: Marked as improvement because 16.02107403148912 (T) = (0 -23.80424907883985) / Math.Sqrt((1.8500777471771912 / (299)) + (2.735050332640835 / (19))) is greater than 1.9674995188240387 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (19) - 2, .975) and 0.20684306187735815 = (30.012029063482917 - 23.80424907883985) / 30.012029063482917 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;SByte&gt;.GreaterThanAnyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 18.644945729851113 < 25.50751083511078.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 11/2/2022 9:35:40 AM, 11/8/2022 2:29:58 AM falls between 10/30/2022 12:44:54 AM and 11/8/2022 2:29:58 AM.
IsImprovementStdDev: Marked as improvement because 50.05333606912576 (T) = (0 -18.849445013475894) / Math.Sqrt((5.63787033900065 / (299)) + (0.13456964335478888 / (14))) is greater than 1.9676211333067681 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (14) - 2, .975) and 0.3094089644159764 = (27.294656377251073 - 18.849445013475894) / 27.294656377251073 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;SByte&gt;.MultiplyOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 24.332533800628124 < 28.329700194580887.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 11/2/2022 9:35:40 AM, 11/8/2022 2:29:58 AM falls between 10/30/2022 12:44:54 AM and 11/8/2022 2:29:58 AM.
IsImprovementStdDev: Marked as improvement because 26.016981494722007 (T) = (0 -24.3853557366003) / Math.Sqrt((13.52063976649371 / (299)) + (0.006454067399019501 / (19))) is greater than 1.9674995188240387 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (19) - 2, .975) and 0.1854871703595954 = (29.938577821255528 - 24.3853557366003) / 29.938577821255528 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;SByte&gt;.LessThanOrEqualAnyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.444988064360154 < 28.784947671859708.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/8/2022 2:29:58 AM falls between 10/30/2022 12:44:54 AM and 11/8/2022 2:29:58 AM.
IsImprovementStdDev: Marked as improvement because 372.1478106019883 (T) = (0 -2.3151984168516515) / Math.Sqrt((1.4749625911193254 / (299)) + (0.015152944795268538 / (19))) is greater than 1.9674995188240387 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (19) - 2, .975) and 0.9240590725036985 = (30.486833558418216 - 2.3151984168516515) / 30.486833558418216 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;SByte&gt;.EqualsAnyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 19.193100659382324 < 25.497838046125374.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 11/2/2022 9:35:40 AM, 11/8/2022 2:29:58 AM falls between 10/30/2022 12:44:54 AM and 11/8/2022 2:29:58 AM.
IsImprovementStdDev: Marked as improvement because 51.27449497028684 (T) = (0 -19.095786567374404) / Math.Sqrt((6.102833642761798 / (299)) + (0.07596079864669426 / (14))) is greater than 1.9676211333067681 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (14) - 2, .975) and 0.3014808451695776 = (27.33752744691767 - 19.095786567374404) / 27.33752744691767 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;SByte&gt;.LessThanAnyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.2707840905264622 < 25.532500567279634.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 11/2/2022 9:35:40 AM, 11/8/2022 2:29:58 AM falls between 10/30/2022 12:44:54 AM and 11/8/2022 2:29:58 AM.
IsImprovementStdDev: Marked as improvement because 167.28164823561914 (T) = (0 -2.153151752450795) / Math.Sqrt((6.313682814153734 / (299)) + (0.017129084388641688 / (19))) is greater than 1.9674995188240387 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (19) - 2, .975) and 0.9201794689233178 = (26.97491138441938 - 2.153151752450795) / 26.97491138441938 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS ubuntu 18.04
Baseline bdd67af67dbf43c0f5e9151b98dc100b9555898a
Compare fbc20578253e0deb7267eea5a02b680c3cfe0e2a
Diff Diff

Improvements in System.IO.Tests.Perf_File

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
AppendAllTextAsync - Duration of single invocation 57.90 μs 50.49 μs 0.87 0.00 False
WriteAllText - Duration of single invocation 684.23 μs 516.82 μs 0.76 0.01 False
AppendAllText - Duration of single invocation 50.71 μs 43.66 μs 0.86 0.00 False
WriteAllText - Duration of single invocation 85.52 μs 69.21 μs 0.81 0.01 False
WriteAllTextAsync - Duration of single invocation 107.46 μs 88.99 μs 0.83 0.01 False
WriteAllTextAsync - Duration of single invocation 840.65 μs 657.40 μs 0.78 0.07 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.IO.Tests.Perf_File*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.IO.Tests.Perf_File.AppendAllTextAsync(size: 10000)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 50.49068921153846 < 55.46838726729167.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 11/2/2022 9:35:40 AM, 11/8/2022 2:29:58 AM falls between 10/30/2022 12:44:54 AM and 11/8/2022 2:29:58 AM.
IsImprovementStdDev: Marked as improvement because 57.75903834214905 (T) = (0 -51111.3784217539) / Math.Sqrt((973871.9448093527 / (299)) + (227921.07875856492 / (19))) is greater than 1.9674995188240387 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (19) - 2, .975) and 0.12247283084239055 = (58244.78172091099 - 51111.3784217539) / 58244.78172091099 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.Perf_File.WriteAllText(size: 100000)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 516.8242991666667 < 649.7639087464414.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 11/2/2022 9:35:40 AM, 11/8/2022 2:29:58 AM falls between 10/30/2022 12:44:54 AM and 11/8/2022 2:29:58 AM.
IsImprovementStdDev: Marked as improvement because 24.661436400741895 (T) = (0 -551999.4888621113) / Math.Sqrt((159034402.00827998 / (299)) + (479744184.24801743 / (19))) is greater than 1.9674995188240387 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (19) - 2, .975) and 0.18490293321114265 = (677219.3292717384 - 551999.4888621113) / 677219.3292717384 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.Perf_File.AppendAllText(size: 10000)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 43.661700011904756 < 48.387482317135984.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 11/2/2022 9:35:40 AM, 11/8/2022 2:29:58 AM falls between 10/30/2022 12:44:54 AM and 11/8/2022 2:29:58 AM.
IsImprovementStdDev: Marked as improvement because 75.25432624082619 (T) = (0 -43619.743270790845) / Math.Sqrt((513654.2130808864 / (299)) + (124116.0529336555 / (19))) is greater than 1.9674995188240387 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (19) - 2, .975) and 0.1354756600111178 = (50455.193975628026 - 43619.743270790845) / 50455.193975628026 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.Perf_File.WriteAllText(size: 10000)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 69.21332549342105 < 81.96919205949501.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 11/2/2022 9:35:40 AM, 11/8/2022 2:29:58 AM falls between 10/30/2022 12:44:54 AM and 11/8/2022 2:29:58 AM.
IsImprovementStdDev: Marked as improvement because 22.799916542347773 (T) = (0 -72646.18137331675) / Math.Sqrt((1723518.2436911513 / (299)) + (5730285.519342999 / (19))) is greater than 1.9674995188240387 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (19) - 2, .975) and 0.14820939936705183 = (85286.43227494511 - 72646.18137331675) / 85286.43227494511 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.Perf_File.WriteAllTextAsync(size: 10000)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 88.99428255208333 < 102.15866836433486.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 11/2/2022 9:35:40 AM, 11/8/2022 2:29:58 AM falls between 10/30/2022 12:44:54 AM and 11/8/2022 2:29:58 AM.
IsImprovementStdDev: Marked as improvement because 21.2757723044456 (T) = (0 -93094.42718589523) / Math.Sqrt((2057957.0507662252 / (299)) + (7864148.734706541 / (19))) is greater than 1.9674995188240387 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (19) - 2, .975) and 0.1291088052773033 = (106895.5889668143 - 93094.42718589523) / 106895.5889668143 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.Perf_File.WriteAllTextAsync(size: 100000)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 657.4000163194444 < 799.350614174107.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 11/2/2022 9:35:40 AM, 11/8/2022 2:29:58 AM falls between 10/30/2022 12:44:54 AM and 11/8/2022 2:29:58 AM.
IsImprovementStdDev: Marked as improvement because 17.281107669973792 (T) = (0 -686814.0827948465) / Math.Sqrt((11278670925.5225 / (299)) + (706423098.5166334 / (19))) is greater than 1.9674995188240387 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (19) - 2, .975) and 0.1788199423663191 = (836374.5276206238 - 686814.0827948465) / 836374.5276206238 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline bdd67af67dbf43c0f5e9151b98dc100b9555898a
Compare 803688595257517f4b47e873d11152b0560c204a
Diff Diff

Improvements in System.Numerics.Tests.Perf_VectorOf<UInt32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
LessThanOrEqualAllBenchmark - Duration of single invocation 12.18 ns 6.07 ns 0.50 0.01 False
EqualsAnyBenchmark - Duration of single invocation 12.00 ns 5.42 ns 0.45 0.01 False
MultiplyOperatorBenchmark - Duration of single invocation 14.33 ns 12.15 ns 0.85 0.01 False
LessThanOrEqualAnyBenchmark - Duration of single invocation 12.85 ns 1.98 ns 0.15 0.01 False
LessThanAnyBenchmark - Duration of single invocation 11.96 ns 2.17 ns 0.18 0.01 False
MultiplyBenchmark - Duration of single invocation 14.33 ns 11.69 ns 0.82 0.00 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;UInt32&gt;*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Numerics.Tests.Perf_VectorOf<UInt32>.LessThanOrEqualAllBenchmark

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 6.066390104482796 < 11.568306348653904.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 11/2/2022 9:35:40 AM, 11/8/2022 2:29:58 AM falls between 10/30/2022 12:44:54 AM and 11/8/2022 2:29:58 AM.
IsImprovementStdDev: Marked as improvement because 43.160301083584 (T) = (0 -7.071582513208234) / Math.Sqrt((0.06605429891140317 / (299)) + (0.1831761680157471 / (14))) is greater than 1.9676211333067681 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (14) - 2, .975) and 0.41314636632031426 = (12.049993571426056 - 7.071582513208234) / 12.049993571426056 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt32&gt;.EqualsAnyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 5.422802396005238 < 11.357235508206005.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/8/2022 2:29:58 AM falls between 10/30/2022 12:44:54 AM and 11/8/2022 2:29:58 AM.
IsImprovementStdDev: Marked as improvement because 226.47813508989958 (T) = (0 -5.617194954682239) / Math.Sqrt((0.009268232579645258 / (299)) + (0.014001446861850671 / (19))) is greater than 1.9674995188240387 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (19) - 2, .975) and 0.5276967890815281 = (11.893196626291555 - 5.617194954682239) / 11.893196626291555 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt32&gt;.MultiplyOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 12.14502618768023 < 13.55675226642215.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/8/2022 2:29:58 AM falls between 10/30/2022 12:44:54 AM and 11/8/2022 2:29:58 AM.
IsImprovementStdDev: Marked as improvement because 44.034137134525345 (T) = (0 -12.0419687826154) / Math.Sqrt((0.044895379774585235 / (299)) + (0.025759324748493146 / (14))) is greater than 1.9676211333067681 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (14) - 2, .975) and 0.14024976239588258 = (14.006357027795639 - 12.0419687826154) / 14.006357027795639 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt32&gt;.LessThanOrEqualAnyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.9776384813159578 < 12.21569660329252.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/8/2022 2:29:58 AM falls between 10/30/2022 12:44:54 AM and 11/8/2022 2:29:58 AM.
IsImprovementStdDev: Marked as improvement because 520.145911925356 (T) = (0 -2.0014535370907875) / Math.Sqrt((0.07107435078875024 / (299)) + (0.002635605391387679 / (14))) is greater than 1.9676211333067681 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (14) - 2, .975) and 0.8428592764235703 = (12.736695437941814 - 2.0014535370907875) / 12.736695437941814 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt32&gt;.LessThanAnyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.167694262548015 < 11.36241164111467.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/8/2022 2:29:58 AM falls between 10/30/2022 12:44:54 AM and 11/8/2022 2:29:58 AM.
IsImprovementStdDev: Marked as improvement because 314.07322731942224 (T) = (0 -2.0310229881702133) / Math.Sqrt((0.03687354623987745 / (299)) + (0.011994198161364445 / (14))) is greater than 1.9676211333067681 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (14) - 2, .975) and 0.8287982059834976 = (11.86332771708245 - 2.0310229881702133) / 11.86332771708245 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt32&gt;.MultiplyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 11.69195955187509 < 13.18551621725354.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/8/2022 2:29:58 AM falls between 10/30/2022 12:44:54 AM and 11/8/2022 2:29:58 AM.
IsImprovementStdDev: Marked as improvement because 46.56633528390958 (T) = (0 -12.04910420975658) / Math.Sqrt((0.043175602096688526 / (299)) + (0.022633798259109314 / (14))) is greater than 1.9676211333067681 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (14) - 2, .975) and 0.13955140799800514 = (14.003281918007538 - 12.04910420975658) / 14.003281918007538 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented Nov 8, 2022 •
edited
Loading

Run Information

Architecture x64
OS ubuntu 18.04
Baseline bdd67af67dbf43c0f5e9151b98dc100b9555898a
Compare fbc20578253e0deb7267eea5a02b680c3cfe0e2a
Diff Diff

Improvements in Burgers

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Test3 - Duration of single invocation 3.37 secs 2.79 secs 0.83 0.00 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'Burgers*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

Burgers.Test3

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.78927854325 < 3.210264663753846.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/8/2022 2:29:58 AM falls between 10/30/2022 12:44:54 AM and 11/8/2022 2:29:58 AM.
IsImprovementStdDev: Marked as improvement because 264.9043319355516 (T) = (0 -2781183078.6837773) / Math.Sqrt((22600632778882.164 / (299)) + (95823245416702.94 / (19))) is greater than 1.9674995188240387 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (19) - 2, .975) and 0.17729370247769635 = (3380529706.7248707 - 2781183078.6837773) / 3380529706.7248707 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS ubuntu 18.04
Baseline bdd67af67dbf43c0f5e9151b98dc100b9555898a
Compare fbc20578253e0deb7267eea5a02b680c3cfe0e2a
Diff Diff

Improvements in System.Text.Json.Serialization.Tests.ReadJson<IndexViewModel>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
DeserializeFromString - Duration of single invocation 224.34 μs 204.45 μs 0.91 0.00 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Json.Serialization.Tests.ReadJson&lt;IndexViewModel&gt;*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Text.Json.Serialization.Tests.ReadJson<IndexViewModel>.DeserializeFromString

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 204.4471425865801 < 211.55256543420197.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 11/2/2022 9:35:40 AM, 11/8/2022 2:29:58 AM falls between 10/30/2022 12:44:54 AM and 11/8/2022 2:29:58 AM.
IsImprovementStdDev: Marked as improvement because 61.67712478687167 (T) = (0 -204382.17684051162) / Math.Sqrt((9270183.43244476 / (299)) + (847211.3044139994 / (19))) is greater than 1.9674995188240387 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (19) - 2, .975) and 0.07661399795488187 = (221339.91243948397 - 204382.17684051162) / 221339.91243948397 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline bdd67af67dbf43c0f5e9151b98dc100b9555898a
Compare 803688595257517f4b47e873d11152b0560c204a
Diff Diff

Improvements in System.IO.Tests.Perf_StreamWriter

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
WriteString - Duration of single invocation 359.87 ms 336.11 ms 0.93 0.00 False
WriteString - Duration of single invocation 8.09 secs 6.89 secs 0.85 0.00 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.IO.Tests.Perf_StreamWriter*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.IO.Tests.Perf_StreamWriter.WriteString(writeLength: 2)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 336.1115383076923 < 341.7433799137363.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 11/2/2022 9:35:40 AM, 11/8/2022 2:29:58 AM falls between 10/30/2022 12:44:54 AM and 11/8/2022 2:29:58 AM.
IsImprovementStdDev: Marked as improvement because 88.3024281478671 (T) = (0 -336072681.79064626) / Math.Sqrt((10560547474496.22 / (299)) + (408531351519.3559 / (14))) is greater than 1.9676211333067681 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (14) - 2, .975) and 0.06255558034032616 = (358498781.093233 - 336072681.79064626) / 358498781.093233 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.Perf_StreamWriter.WriteString(writeLength: 100)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 6.885616698533334 < 7.678406729790512.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 11/2/2022 9:35:40 AM, 11/8/2022 2:29:58 AM falls between 10/30/2022 12:44:54 AM and 11/8/2022 2:29:58 AM.
IsImprovementStdDev: Marked as improvement because 144.07246581575603 (T) = (0 -6893669657.287231) / Math.Sqrt((15435328469244238 / (299)) + (119663680046863.25 / (14))) is greater than 1.9676211333067681 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (14) - 2, .975) and 0.13949980075365465 = (8011235399.276998 - 6893669657.287231) / 8011235399.276998 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline bdd67af67dbf43c0f5e9151b98dc100b9555898a
Compare fbc20578253e0deb7267eea5a02b680c3cfe0e2a
Diff Diff

Improvements in IfStatements.IfStatements

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
AndAndAnd - Duration of single invocation 87.93 μs 76.24 μs 0.87 0.36 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'IfStatements.IfStatements*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

IfStatements.IfStatements.AndAndAnd

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 76.24496313713593 < 83.39324713665246.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 11/2/2022 9:35:40 AM, 11/8/2022 2:29:58 AM falls between 10/30/2022 12:44:54 AM and 11/8/2022 2:29:58 AM.
IsImprovementStdDev: Marked as improvement because 11.011741438769166 (T) = (0 -76708.33661618312) / Math.Sqrt((28943268.53124285 / (244)) + (7356836.895487099 / (19))) is greater than 1.9690947240420478 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (244) + (19) - 2, .975) and 0.09263871012108348 = (84540.01451441633 - 76708.33661618312) / 84540.01451441633 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline bdd67af67dbf43c0f5e9151b98dc100b9555898a
Compare fbc20578253e0deb7267eea5a02b680c3cfe0e2a
Diff Diff

Improvements in Benchstone.BenchF.InvMt

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Test - Duration of single invocation 5.53 ms 4.29 ms 0.78 0.41 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'Benchstone.BenchF.InvMt*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

Benchstone.BenchF.InvMt.Test

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 4.291558169642857 < 4.666245239366987.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 11/2/2022 9:35:40 AM, 11/8/2022 2:29:58 AM falls between 10/30/2022 12:44:54 AM and 11/8/2022 2:29:58 AM.
IsImprovementStdDev: Marked as improvement because 7.882805532389524 (T) = (0 -4368524.297056644) / Math.Sqrt((378042749260.73645 / (299)) + (98480086346.65813 / (19))) is greater than 1.9674995188240387 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (19) - 2, .975) and 0.12655465458473455 = (5001485.576615787 - 4368524.297056644) / 5001485.576615787 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline bdd67af67dbf43c0f5e9151b98dc100b9555898a
Compare 803688595257517f4b47e873d11152b0560c204a
Diff Diff

Improvements in System.Text.Json.Serialization.Tests.ReadJson<Dictionary<String, String>>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
DeserializeFromString - Duration of single invocation 125.90 μs 117.70 μs 0.93 0.02 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Json.Serialization.Tests.ReadJson&lt;Dictionary&lt;String, String&gt;&gt;*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Text.Json.Serialization.Tests.ReadJson<Dictionary<String, String>>.DeserializeFromString

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 117.7047800390399 < 119.74257069260753.
IsChangePoint: Marked as a change because one of 8/5/2022 10:45:33 AM, 11/2/2022 9:35:40 AM, 11/8/2022 2:29:58 AM falls between 10/30/2022 12:44:54 AM and 11/8/2022 2:29:58 AM.
IsImprovementStdDev: Marked as improvement because 29.00597685082928 (T) = (0 -117463.22484002545) / Math.Sqrt((9814603.812482083 / (299)) + (315280.27034678066 / (14))) is greater than 1.9676211333067681 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (14) - 2, .975) and 0.05490350451051831 = (124287.01767557526 - 117463.22484002545) / 124287.01767557526 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline bdd67af67dbf43c0f5e9151b98dc100b9555898a
Compare 803688595257517f4b47e873d11152b0560c204a
Diff Diff

Improvements in System.Text.Json.Serialization.Tests.WriteJson<HashSet<String>>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
SerializeObjectProperty - Duration of single invocation 41.42 μs 37.17 μs 0.90 0.01 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Json.Serialization.Tests.WriteJson&lt;HashSet&lt;String&gt;&gt;*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Text.Json.Serialization.Tests.WriteJson<HashSet<String>>.SerializeObjectProperty

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 37.17426227943671 < 39.251521659749585.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 10/17/2022 12:47:34 PM, 11/2/2022 9:35:40 AM, 11/8/2022 2:29:58 AM falls between 10/30/2022 12:44:54 AM and 11/8/2022 2:29:58 AM.
IsImprovementStdDev: Marked as improvement because 39.04811599790285 (T) = (0 -37052.212544569156) / Math.Sqrt((1227380.9135932198 / (299)) + (8567.872571205819 / (14))) is greater than 1.9676211333067681 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (14) - 2, .975) and 0.06749439942368884 = (39734.03754537237 - 37052.212544569156) / 39734.03754537237 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline bdd67af67dbf43c0f5e9151b98dc100b9555898a
Compare fbc20578253e0deb7267eea5a02b680c3cfe0e2a
Diff Diff

Improvements in System.Buffers.Tests.RentReturnArrayPoolTests<Byte>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
SingleSerial - Duration of single invocation 5.49 μs 3.71 μs 0.68 0.12 False
MultipleSerial - Duration of single invocation 44.77 μs 30.51 μs 0.68 0.09 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Buffers.Tests.RentReturnArrayPoolTests&lt;Byte&gt;*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Buffers.Tests.RentReturnArrayPoolTests<Byte>.SingleSerial(RentalSize: 4096, ManipulateArray: True, Async: False, UseSharedPool: False)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 3.7147382684615384 < 5.212155072879167.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 10/10/2022 9:53:50 PM, 11/2/2022 9:35:40 AM, 11/8/2022 2:29:58 AM falls between 10/30/2022 12:44:54 AM and 11/8/2022 2:29:58 AM.
IsImprovementStdDev: Marked as improvement because 18.312337699865086 (T) = (0 -3715.754662085696) / Math.Sqrt((780908.6705529388 / (299)) + (6.832589267725235 / (19))) is greater than 1.9674995188240387 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (19) - 2, .975) and 0.2012004671416541 = (4651.673554176483 - 3715.754662085696) / 4651.673554176483 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Buffers.Tests.RentReturnArrayPoolTests&lt;Byte&gt;.MultipleSerial(RentalSize: 4096, ManipulateArray: True, Async: False, UseSharedPool: True)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 30.513575120666673 < 42.556699397957416.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 11/2/2022 9:35:40 AM, 11/8/2022 2:29:58 AM falls between 10/30/2022 12:44:54 AM and 11/8/2022 2:29:58 AM.
IsImprovementStdDev: Marked as improvement because 12.302706643926967 (T) = (0 -31324.80269737883) / Math.Sqrt((36089548.22132407 / (299)) + (10690976.625876775 / (19))) is greater than 1.9674995188240387 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (19) - 2, .975) and 0.24509607253265828 = (41495.085079861354 - 31324.80269737883) / 41495.085079861354 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline bdd67af67dbf43c0f5e9151b98dc100b9555898a
Compare fbc20578253e0deb7267eea5a02b680c3cfe0e2a
Diff Diff

Improvements in Layout.SearchLoops

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
LoopReturn - Duration of single invocation 117.60 ns 80.08 ns 0.68 0.49 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'Layout.SearchLoops*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

Layout.SearchLoops.LoopReturn

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 80.0794998321741 < 106.17801995240622.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 8/5/2022 8:01:14 PM, 11/2/2022 9:35:40 AM, 11/8/2022 2:29:58 AM falls between 10/30/2022 12:44:54 AM and 11/8/2022 2:29:58 AM.
IsImprovementStdDev: Marked as improvement because 3.513083981953457 (T) = (0 -93.78668014138175) / Math.Sqrt((247.01771369215976 / (299)) + (238.29473015664638 / (19))) is greater than 1.9674995188240387 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (19) - 2, .975) and 0.12045826867090297 = (106.63130218921893 - 93.78668014138175) / 106.63130218921893 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline bdd67af67dbf43c0f5e9151b98dc100b9555898a
Compare fbc20578253e0deb7267eea5a02b680c3cfe0e2a
Diff Diff

Improvements in System.Text.Json.Serialization.Tests.WriteJson<Dictionary<String, String>>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
SerializeObjectProperty - Duration of single invocation 82.38 μs 73.63 μs 0.89 0.01 False
SerializeToString - Duration of single invocation 81.78 μs 73.56 μs 0.90 0.01 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Json.Serialization.Tests.WriteJson&lt;Dictionary&lt;String, String&gt;&gt;*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Text.Json.Serialization.Tests.WriteJson<Dictionary<String, String>>.SerializeObjectProperty

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 73.63000903755868 < 78.21794192575611.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 10/25/2022 8:17:52 AM, 11/2/2022 9:35:40 AM, 11/8/2022 2:29:58 AM falls between 10/30/2022 12:44:54 AM and 11/8/2022 2:29:58 AM.
IsImprovementStdDev: Marked as improvement because 46.417814749867745 (T) = (0 -73151.73425961737) / Math.Sqrt((3500108.7815541187 / (299)) + (135295.29647796613 / (19))) is greater than 1.9674995188240387 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (19) - 2, .975) and 0.08009271330205422 = (79520.76836155876 - 73151.73425961737) / 79520.76836155876 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Serialization.Tests.WriteJson&lt;Dictionary&lt;String, String&gt;&gt;.SerializeToString

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 73.55566908730158 < 77.58807495738718.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 10/25/2022 8:17:52 AM, 11/2/2022 9:35:40 AM, 11/8/2022 2:29:58 AM falls between 10/30/2022 12:44:54 AM and 11/8/2022 2:29:58 AM.
IsImprovementStdDev: Marked as improvement because 41.86908522478476 (T) = (0 -72694.5888528981) / Math.Sqrt((3620905.3340109433 / (299)) + (189612.95013798124 / (19))) is greater than 1.9674995188240387 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (19) - 2, .975) and 0.07885246872849433 = (78917.4224378088 - 72694.5888528981) / 78917.4224378088 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented Nov 8, 2022 •
edited
Loading

Run Information

Architecture x64
OS ubuntu 18.04
Baseline bdd67af67dbf43c0f5e9151b98dc100b9555898a
Compare fbc20578253e0deb7267eea5a02b680c3cfe0e2a
Diff Diff

Improvements in System.Numerics.Tests.Perf_VectorOf<UInt64>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
EqualsAnyBenchmark - Duration of single invocation 8.92 ns 4.13 ns 0.46 0.01 False
GreaterThanAllBenchmark - Duration of single invocation 8.51 ns 2.37 ns 0.28 0.03 False
LessThanOrEqualAnyBenchmark - Duration of single invocation 9.32 ns 2.23 ns 0.24 0.04 False
GreaterThanAnyBenchmark - Duration of single invocation 9.13 ns 4.16 ns 0.45 0.01 False
LessThanOrEqualAllBenchmark - Duration of single invocation 8.67 ns 3.27 ns 0.38 0.02 False
LessThanAllBenchmark - Duration of single invocation 8.30 ns 4.14 ns 0.50 0.01 False
LessThanAnyBenchmark - Duration of single invocation 8.94 ns 2.66 ns 0.30 0.03 False
GreaterThanOrEqualAnyBenchmark - Duration of single invocation 9.32 ns 3.56 ns 0.38 0.02 False
GreaterThanOrEqualAllBenchmark - Duration of single invocation 8.69 ns 2.24 ns 0.26 0.03 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;UInt64&gt;*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Numerics.Tests.Perf_VectorOf<UInt64>.EqualsAnyBenchmark

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 4.127424071658522 < 8.473924861244361.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/8/2022 2:29:58 AM falls between 10/30/2022 12:44:54 AM and 11/8/2022 2:29:58 AM.
IsImprovementStdDev: Marked as improvement because 359.66112386463374 (T) = (0 -4.154762826018442) / Math.Sqrt((0.0052508668569487904 / (299)) + (0.0029463598472188606 / (19))) is greater than 1.9674995188240387 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (19) - 2, .975) and 0.5321395287957146 = (8.88034591878209 - 4.154762826018442) / 8.88034591878209 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt64&gt;.GreaterThanAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.3668003091636565 < 8.085242733884371.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/8/2022 2:29:58 AM falls between 10/30/2022 12:44:54 AM and 11/8/2022 2:29:58 AM.
IsImprovementStdDev: Marked as improvement because 178.13346823100196 (T) = (0 -2.4078901181215575) / Math.Sqrt((0.02638913547705421 / (299)) + (0.020043224082301147 / (19))) is greater than 1.9674995188240387 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (19) - 2, .975) and 0.7143905957777085 = (8.430710202551598 - 2.4078901181215575) / 8.430710202551598 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt64&gt;.LessThanOrEqualAnyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.2305029006011416 < 8.861299932784672.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/8/2022 2:29:58 AM falls between 10/30/2022 12:44:54 AM and 11/8/2022 2:29:58 AM.
IsImprovementStdDev: Marked as improvement because 255.15645587029698 (T) = (0 -2.0983741675410092) / Math.Sqrt((0.03648991238092063 / (299)) + (0.012478034711376557 / (19))) is greater than 1.9674995188240387 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (19) - 2, .975) and 0.7723840516734972 = (9.218924170159662 - 2.0983741675410092) / 9.218924170159662 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt64&gt;.GreaterThanAnyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 4.155170543392801 < 8.673393889339382.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/8/2022 2:29:58 AM falls between 10/30/2022 12:44:54 AM and 11/8/2022 2:29:58 AM.
IsImprovementStdDev: Marked as improvement because 281.157276394099 (T) = (0 -4.18449349037529) / Math.Sqrt((0.027858921767802955 / (299)) + (0.003903691879531713 / (19))) is greater than 1.9674995188240387 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (19) - 2, .975) and 0.5372751676669077 = (9.043157397187374 - 4.18449349037529) / 9.043157397187374 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt64&gt;.LessThanOrEqualAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 3.271785898763633 < 8.240068726460876.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/8/2022 2:29:58 AM falls between 10/30/2022 12:44:54 AM and 11/8/2022 2:29:58 AM.
IsImprovementStdDev: Marked as improvement because 152.99880106250134 (T) = (0 -3.3414470325965575) / Math.Sqrt((0.03573435365508289 / (299)) + (0.018901252973002963 / (18))) is greater than 1.9675235316906419 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (18) - 2, .975) and 0.6102758560102727 = (8.573877405667313 - 3.3414470325965575) / 8.573877405667313 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt64&gt;.LessThanAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 4.13588144380954 < 7.885906937000849.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/8/2022 2:29:58 AM falls between 10/30/2022 12:44:54 AM and 11/8/2022 2:29:58 AM.
IsImprovementStdDev: Marked as improvement because 255.9612738377574 (T) = (0 -4.176757625432435) / Math.Sqrt((0.009663024984010884 / (299)) + (0.004230195884176396 / (19))) is greater than 1.9674995188240387 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (19) - 2, .975) and 0.49457230442747 = (8.263808378567694 - 4.176757625432435) / 8.263808378567694 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt64&gt;.LessThanAnyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.6634942911175954 < 8.482782876703727.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/8/2022 2:29:58 AM falls between 10/30/2022 12:44:54 AM and 11/8/2022 2:29:58 AM.
IsImprovementStdDev: Marked as improvement because 138.10463336018526 (T) = (0 -2.329911559177801) / Math.Sqrt((0.006392365777895408 / (299)) + (0.042500445099474274 / (19))) is greater than 1.9674995188240387 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (19) - 2, .975) and 0.7379996619691407 = (8.892780737188883 - 2.329911559177801) / 8.892780737188883 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt64&gt;.GreaterThanOrEqualAnyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 3.557051027182132 < 8.85914942774964.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/8/2022 2:29:58 AM falls between 10/30/2022 12:44:54 AM and 11/8/2022 2:29:58 AM.
IsImprovementStdDev: Marked as improvement because 210.81439993903462 (T) = (0 -3.3322645260665418) / Math.Sqrt((0.027766510695193484 / (299)) + (0.013209719522917267 / (19))) is greater than 1.9674995188240387 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (19) - 2, .975) and 0.6397758636470771 = (9.250530960540127 - 3.3322645260665418) / 9.250530960540127 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt64&gt;.GreaterThanOrEqualAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.235665041698013 < 8.249929616673027.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/8/2022 2:29:58 AM falls between 10/30/2022 12:44:54 AM and 11/8/2022 2:29:58 AM.
IsImprovementStdDev: Marked as improvement because 192.6635199378738 (T) = (0 -2.0738316162237154) / Math.Sqrt((0.027057863527066833 / (299)) + (0.02009852549380535 / (19))) is greater than 1.9674995188240387 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (19) - 2, .975) and 0.7589288767694926 = (8.602571674421409 - 2.0738316162237154) / 8.602571674421409 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS ubuntu 18.04
Baseline bdd67af67dbf43c0f5e9151b98dc100b9555898a
Compare fbc20578253e0deb7267eea5a02b680c3cfe0e2a
Diff Diff

Improvements in System.IO.Tests.StreamReaderReadToEndTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ReadToEnd - Duration of single invocation 335.02 ms 290.65 ms 0.87 0.01 False
ReadToEndAsync - Duration of single invocation 338.97 ms 295.86 ms 0.87 0.01 False
ReadToEnd - Duration of single invocation 333.25 ms 293.44 ms 0.88 0.01 False
ReadToEndAsync - Duration of single invocation 339.00 ms 295.05 ms 0.87 0.01 False
ReadToEndAsync - Duration of single invocation 339.78 ms 298.90 ms 0.88 0.01 False
ReadToEndAsync - Duration of single invocation 339.22 ms 299.39 ms 0.88 0.01 False
ReadToEnd - Duration of single invocation 332.07 ms 294.47 ms 0.89 0.01 False
ReadToEndAsync - Duration of single invocation 339.07 ms 296.39 ms 0.87 0.01 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.IO.Tests.StreamReaderReadToEndTests*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.IO.Tests.StreamReaderReadToEndTests.ReadToEnd(LineLengthRange: [ 1, 1])

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 290.64559046666665 < 316.76764156273805.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 11/2/2022 9:35:40 AM, 11/8/2022 2:29:58 AM falls between 10/30/2022 12:44:54 AM and 11/8/2022 2:29:58 AM.
IsImprovementStdDev: Marked as improvement because 66.08660675858974 (T) = (0 -289602475.2477733) / Math.Sqrt((101774683317980 / (299)) + (294703603186.1594 / (19))) is greater than 1.9674995188240387 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (19) - 2, .975) and 0.11982336673807832 = (329027679.5630336 - 289602475.2477733) / 329027679.5630336 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.StreamReaderReadToEndTests.ReadToEndAsync(LineLengthRange: [  33,  128])

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 295.8558704615384 < 321.8983783375.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 11/2/2022 9:35:40 AM, 11/8/2022 2:29:58 AM falls between 10/30/2022 12:44:54 AM and 11/8/2022 2:29:58 AM.
IsImprovementStdDev: Marked as improvement because 71.19232870835161 (T) = (0 -295469853.5417059) / Math.Sqrt((80222130046629.19 / (299)) + (784534805896.3813 / (14))) is greater than 1.9676211333067681 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (14) - 2, .975) and 0.12066330905169564 = (336014471.5706812 - 295469853.5417059) / 336014471.5706812 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.StreamReaderReadToEndTests.ReadToEnd(LineLengthRange: [   0,    0])

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 293.44009271428575 < 316.6266613116666.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 11/2/2022 9:35:40 AM, 11/8/2022 2:29:58 AM falls between 10/30/2022 12:44:54 AM and 11/8/2022 2:29:58 AM.
IsImprovementStdDev: Marked as improvement because 61.935414002267066 (T) = (0 -289736605.2739011) / Math.Sqrt((99100011388222.22 / (299)) + (1238740185289.5422 / (18))) is greater than 1.9675235316906419 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (18) - 2, .975) and 0.11912899918954084 = (328920585.42888165 - 289736605.2739011) / 328920585.42888165 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.StreamReaderReadToEndTests.ReadToEndAsync(LineLengthRange: [1025, 2048])

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 295.050615 < 321.91598201166664.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 11/2/2022 9:35:40 AM, 11/8/2022 2:29:58 AM falls between 10/30/2022 12:44:54 AM and 11/8/2022 2:29:58 AM.
IsImprovementStdDev: Marked as improvement because 62.69170049574015 (T) = (0 -295998159.2144558) / Math.Sqrt((78317804073791.66 / (299)) + (2102222023934.837 / (14))) is greater than 1.9676211333067681 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (14) - 2, .975) and 0.11968889933793109 = (336242674.8814595 - 295998159.2144558) / 336242674.8814595 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.StreamReaderReadToEndTests.ReadToEndAsync(LineLengthRange: [ 129, 1024])

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 298.8962140666667 < 322.41301739807693.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 11/2/2022 9:35:40 AM, 11/8/2022 2:29:58 AM falls between 10/30/2022 12:44:54 AM and 11/8/2022 2:29:58 AM.
IsImprovementStdDev: Marked as improvement because 67.51752897117858 (T) = (0 -295290482.1361329) / Math.Sqrt((79279751807442.28 / (299)) + (1424642911704.0884 / (14))) is greater than 1.9676211333067681 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (14) - 2, .975) and 0.12165056240038463 = (336187933.29351157 - 295290482.1361329) / 336187933.29351157 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.StreamReaderReadToEndTests.ReadToEndAsync(LineLengthRange: [   9,   32])

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 299.3940668 < 321.9293524261538.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 11/2/2022 9:35:40 AM, 11/8/2022 2:29:58 AM falls between 10/30/2022 12:44:54 AM and 11/8/2022 2:29:58 AM.
IsImprovementStdDev: Marked as improvement because 65.31899590765376 (T) = (0 -295753131.46538454) / Math.Sqrt((82370450961866.8 / (299)) + (1453498616630.38 / (14))) is greater than 1.9676211333067681 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (14) - 2, .975) and 0.1197345981040019 = (335981774.1653412 - 295753131.46538454) / 335981774.1653412 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.StreamReaderReadToEndTests.ReadToEnd(LineLengthRange: [   1,    8])

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 294.4729638666667 < 317.3000463916667.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 11/2/2022 9:35:40 AM, 11/8/2022 2:29:58 AM falls between 10/30/2022 12:44:54 AM and 11/8/2022 2:29:58 AM.
IsImprovementStdDev: Marked as improvement because 58.97167786538425 (T) = (0 -290196600.2085792) / Math.Sqrt((104047364880521.19 / (299)) + (1627103480050.2583 / (19))) is greater than 1.9674995188240387 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (19) - 2, .975) and 0.11802232645474 = (329029417.5385238 - 290196600.2085792) / 329029417.5385238 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.StreamReaderReadToEndTests.ReadToEndAsync(LineLengthRange: [   0, 1024])

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 296.39204293333336 < 321.42222724285705.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 11/2/2022 9:35:40 AM, 11/8/2022 2:29:58 AM falls between 10/30/2022 12:44:54 AM and 11/8/2022 2:29:58 AM.
IsImprovementStdDev: Marked as improvement because 63.543883338814425 (T) = (0 -295728753.5784406) / Math.Sqrt((80016439734215.12 / (299)) + (1885255581199.1245 / (14))) is greater than 1.9676211333067681 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (14) - 2, .975) and 0.11993751015579249 = (336031539.7953068 - 295728753.5784406) / 336031539.7953068 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline bdd67af67dbf43c0f5e9151b98dc100b9555898a
Compare fbc20578253e0deb7267eea5a02b680c3cfe0e2a
Diff Diff

Improvements in System.Numerics.Tests.Perf_VectorOf<Single>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GreaterThanAnyBenchmark - Duration of single invocation 15.67 ns 9.59 ns 0.61 0.02 False
AbsBenchmark - Duration of single invocation 20.98 ns 19.26 ns 0.92 0.03 False
EqualsAnyBenchmark - Duration of single invocation 15.68 ns 7.41 ns 0.47 0.01 False
GreaterThanAllBenchmark - Duration of single invocation 14.90 ns 3.21 ns 0.22 0.02 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;Single&gt;*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Numerics.Tests.Perf_VectorOf<Single>.GreaterThanAnyBenchmark

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 9.590211102904972 < 14.821561293161077.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/8/2022 2:29:58 AM falls between 10/30/2022 12:44:54 AM and 11/8/2022 2:29:58 AM.
IsImprovementStdDev: Marked as improvement because 29.89201327695158 (T) = (0 -9.159593902912995) / Math.Sqrt((0.04176563404174014 / (299)) + (0.8493293131299584 / (19))) is greater than 1.9674995188240387 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (19) - 2, .975) and 0.40865600382578315 = (15.48945108460097 - 9.159593902912995) / 15.48945108460097 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Single&gt;.AbsBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 19.26333082101998 < 19.894315151032185.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/8/2022 2:29:58 AM falls between 10/30/2022 12:44:54 AM and 11/8/2022 2:29:58 AM.
IsImprovementStdDev: Marked as improvement because 91.87286523696483 (T) = (0 -19.16754844633153) / Math.Sqrt((0.029798834141876927 / (299)) + (0.004031490571418244 / (14))) is greater than 1.9676211333067681 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (14) - 2, .975) and 0.08623095049153381 = (20.976359898206358 - 19.16754844633153) / 20.976359898206358 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Single&gt;.EqualsAnyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 7.413910559186641 < 14.847379128185397.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/8/2022 2:29:58 AM falls between 10/30/2022 12:44:54 AM and 11/8/2022 2:29:58 AM.
IsImprovementStdDev: Marked as improvement because 38.79976765649203 (T) = (0 -9.3789091622628) / Math.Sqrt((0.032710367399350175 / (299)) + (0.47467474139753324 / (19))) is greater than 1.9674995188240387 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (19) - 2, .975) and 0.39588358662441275 = (15.52500305339628 - 9.3789091622628) / 15.52500305339628 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Single&gt;.GreaterThanAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 3.2051940642770873 < 14.166019153594949.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/8/2022 2:29:58 AM falls between 10/30/2022 12:44:54 AM and 11/8/2022 2:29:58 AM.
IsImprovementStdDev: Marked as improvement because 211.20361274677012 (T) = (0 -3.0012946927590543) / Math.Sqrt((0.049461252253359574 / (299)) + (0.052825098981908625 / (18))) is greater than 1.9675235316906419 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (18) - 2, .975) and 0.7966726300960666 = (14.760898614766337 - 3.0012946927590543) / 14.760898614766337 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler performanceautofiler bot added Improvement PGO Applied if there were any profile guided optimization updates in the observed interval. untriaged labels Nov 8, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
arch-x64 branch-refs/heads/main kind-micro_mono mono-jit os-linux perf-improvement PGO Applied if there were any profile guided optimization updates in the observed interval. runtime-mono untriaged
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@kotlarmilos