[Perf] Linux/x64: 86 Improvements on 12/16/2022 3:18:59 PM

[performanceautofiler]

Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	af6b1bd43f24d04cafff386bb2a1cd4510ecfe8c
Compare	97a51ccdb147992b80d427b83a77caa967d275b4
Diff	Diff

Improvements in System.Text.Json.Serialization.Tests.ReadJson<ImmutableSortedDictionary<String, String>>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
DeserializeFromString - Duration of single invocation	667.70 μs	607.90 μs	0.91	0.02	False
DeserializeFromUtf8Bytes - Duration of single invocation	623.58 μs	582.72 μs	0.93	0.02	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.ReadJson<ImmutableSortedDictionary<String, String>>*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Serialization.Tests.ReadJson<ImmutableSortedDictionary<String, String>>.DeserializeFromString(Mode: Reflection)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 607.9035018028846 < 619.4496040344831.
IsChangePoint: Marked as a change because one of 12/16/2022 11:20:37 AM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 16.53933690863864 (T) = (0 -614029.1200014484) / Math.Sqrt((159894812.15404162 / (104)) + (49959006.162301265 / (13))) is greater than 1.980807541103565 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (104) + (13) - 2, .975) and 0.05880548516617418 = (652393.4323075176 - 614029.1200014484) / 652393.4323075176 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Serialization.Tests.ReadJson&lt;ImmutableSortedDictionary&lt;String, String&gt;&gt;.DeserializeFromUtf8Bytes(Mode: Reflection)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 582.7211703042327 < 592.3933348701922.
IsChangePoint: Marked as a change because one of 12/16/2022 11:20:37 AM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 15.355456708001148 (T) = (0 -586813.0957820669) / Math.Sqrt((137154162.5705261 / (104)) + (42477971.79038732 / (13))) is greater than 1.980807541103565 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (104) + (13) - 2, .975) and 0.053065858142115965 = (619697.896445829 - 586813.0957820669) / 619697.896445829 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	af6b1bd43f24d04cafff386bb2a1cd4510ecfe8c
Compare	97a51ccdb147992b80d427b83a77caa967d275b4
Diff	Diff

Improvements in MicroBenchmarks.Serializers.Binary_FromStream<MyEventsListerViewModel>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
BinaryFormatter_ - Duration of single invocation	1.80 ms	1.48 ms	0.82	0.01	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Binary_FromStream<MyEventsListerViewModel>*'

Payloads

Baseline
Compare

Histogram

MicroBenchmarks.Serializers.Binary_FromStream<MyEventsListerViewModel>.BinaryFormatter_

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.4786778421266236 < 1.6964967797133925.
IsChangePoint: Marked as a change because one of 12/7/2022 5:17:51 AM, 12/9/2022 1:52:32 PM, 12/16/2022 11:20:37 AM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 51.39810145049001 (T) = (0 -1475227.5263853841) / Math.Sqrt((5434507344.046267 / (299)) + (177154425.02226767 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.16421594725779678 = (1765082.1663145763 - 1475227.5263853841) / 1765082.1663145763 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	af6b1bd43f24d04cafff386bb2a1cd4510ecfe8c
Compare	97a51ccdb147992b80d427b83a77caa967d275b4
Diff	Diff

Improvements in System.Collections.AddGivenSize<Int32>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
SortedList - Duration of single invocation	1.21 ms	453.68 μs	0.37	0.01	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.AddGivenSize<Int32>*'

Payloads

Baseline
Compare

Histogram

System.Collections.AddGivenSize<Int32>.SortedList(Size: 512)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 453.6831457738095 < 1.1333441616866986.
IsChangePoint: Marked as a change because one of 12/7/2022 5:17:51 AM, 12/16/2022 11:20:37 AM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 7.7149167894097745 (T) = (0 -455821.16886761197) / Math.Sqrt((77997579190.33432 / (299)) + (22837371.55838585 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2152450688753402 = (580845.2432587568 - 455821.16886761197) / 580845.2432587568 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	af6b1bd43f24d04cafff386bb2a1cd4510ecfe8c
Compare	97a51ccdb147992b80d427b83a77caa967d275b4
Diff	Diff

Improvements in System.Collections.CopyTo<String>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
ImmutableArray - Duration of single invocation	3.39 μs	1.76 μs	0.52	0.05	False
Array - Duration of single invocation	3.34 μs	1.95 μs	0.58	0.04	False
List - Duration of single invocation	2.33 μs	1.59 μs	0.68	0.02	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.CopyTo<String>*'

Payloads

Baseline
Compare

Histogram

System.Collections.CopyTo<String>.ImmutableArray(Size: 2048)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.7646624361883156 < 3.197594264855792.
IsChangePoint: Marked as a change because one of 12/7/2022 5:17:51 AM, 12/16/2022 11:20:37 AM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 5.952256626141112 (T) = (0 -1783.0673014765537) / Math.Sqrt((373979.65369660466 / (299)) + (3424.7854270465523 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.11496551866731256 = (2014.6868162600917 - 1783.0673014765537) / 2014.6868162600917 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.CopyTo&lt;String&gt;.Array(Size: 2048)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.9476845262832752 < 3.172418372940943.
IsChangePoint: Marked as a change because one of 12/7/2022 5:17:51 AM, 12/16/2022 11:20:37 AM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 5.994546338802475 (T) = (0 -1768.6630540907981) / Math.Sqrt((372829.6282253889 / (299)) + (2975.582065068595 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.115204572845991 = (1998.9513957817298 - 1768.6630540907981) / 1998.9513957817298 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.CopyTo&lt;String&gt;.List(Size: 2048)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.586407115916124 < 2.209653096133411.
IsChangePoint: Marked as a change because one of 12/7/2022 5:17:51 AM, 12/16/2022 11:20:37 AM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 6.722882190764111 (T) = (0 -1593.200576960438) / Math.Sqrt((81018.83994772314 / (299)) + (226.88823579524626 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.06687111766902108 = (1707.3746265152395 - 1593.200576960438) / 1707.3746265152395 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	af6b1bd43f24d04cafff386bb2a1cd4510ecfe8c
Compare	97a51ccdb147992b80d427b83a77caa967d275b4
Diff	Diff

Improvements in System.Collections.Tests.Perf_BitArray

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
BitArrayIntArrayCtor - Duration of single invocation	2.24 μs	612.10 ns	0.27	0.02	False
BitArrayCopyToIntArray - Duration of single invocation	912.90 ns	167.26 ns	0.18	0.17	False
BitArrayIntArrayCtor - Duration of single invocation	1.98 μs	398.10 ns	0.20	0.04	False
BitArraySetAll - Duration of single invocation	611.91 ns	398.35 ns	0.65	0.21	False
BitArrayBitArrayCtor - Duration of single invocation	1.99 μs	395.67 ns	0.20	0.06	False
BitArrayBitArrayCtor - Duration of single invocation	2.01 μs	401.42 ns	0.20	0.03	False
BitArrayCopyToIntArray - Duration of single invocation	921.72 ns	175.68 ns	0.19	0.13	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.Tests.Perf_BitArray*'

Payloads

Baseline
Compare

Histogram

System.Collections.Tests.Perf_BitArray.BitArrayIntArrayCtor(Size: 512)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 612.0972966537452 < 2.145002757692504.
IsChangePoint: Marked as a change because one of 12/7/2022 5:17:51 AM, 12/16/2022 11:20:37 AM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 7.38189712464279 (T) = (0 -613.9251918171708) / Math.Sqrt((378637.3827019777 / (299)) + (90.83573087952095 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.30024199053025863 = (877.3392851657211 - 613.9251918171708) / 877.3392851657211 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.Tests.Perf_BitArray.BitArrayCopyToIntArray(Size: 4)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 167.25522914085255 < 840.0051804920573.
IsChangePoint: Marked as a change because one of 12/7/2022 5:17:51 AM, 12/16/2022 11:20:37 AM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 7.225559864492292 (T) = (0 -171.1712116373349) / Math.Sqrt((75276.7372772946 / (299)) + (48.8135075165298 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.4029000546050997 = (286.67095510137494 - 171.1712116373349) / 286.67095510137494 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.Tests.Perf_BitArray.BitArrayIntArrayCtor(Size: 4)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 398.0990756007529 < 1.8828510799400178.
IsChangePoint: Marked as a change because one of 12/7/2022 5:17:51 AM, 12/16/2022 11:20:37 AM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 7.475483902655579 (T) = (0 -392.63330514117484) / Math.Sqrt((361127.33146252367 / (299)) + (49.712303914188205 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.3985775481161695 = (652.841116774628 - 392.63330514117484) / 652.841116774628 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.Tests.Perf_BitArray.BitArraySetAll(Size: 512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 398.34884160616576 < 494.64435797416036.
IsChangePoint: Marked as a change because one of 10/1/2022 4:32:12 PM, 10/3/2022 11:00:05 AM, 11/15/2022 8:48:03 PM, 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 43.13045200569618 (T) = (0 -395.5552837048261) / Math.Sqrt((956.2402391277483 / (299)) + (11.256010084468647 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.20289222893527775 = (496.23814754242113 - 395.5552837048261) / 496.23814754242113 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.Tests.Perf_BitArray.BitArrayBitArrayCtor(Size: 512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 395.6741349503628 < 1.8876166726161494.
IsChangePoint: Marked as a change because one of 12/7/2022 5:17:51 AM, 12/16/2022 11:20:37 AM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 7.463491124551777 (T) = (0 -402.0658440189466) / Math.Sqrt((360413.12445723335 / (299)) + (13.712161509538518 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.39200932309415853 = (661.3026470490006 - 402.0658440189466) / 661.3026470490006 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.Tests.Perf_BitArray.BitArrayBitArrayCtor(Size: 4)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 401.4204169877842 < 1.89690347752217.
IsChangePoint: Marked as a change because one of 12/7/2022 5:17:51 AM, 12/16/2022 11:20:37 AM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 7.3466693945304975 (T) = (0 -401.1142914712622) / Math.Sqrt((361564.2219592542 / (299)) + (31.33619321215615 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.3893303455552909 = (656.8433334648032 - 401.1142914712622) / 656.8433334648032 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.Tests.Perf_BitArray.BitArrayCopyToIntArray(Size: 512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 175.67888327640384 < 852.2527174062938.
IsChangePoint: Marked as a change because one of 12/7/2022 5:17:51 AM, 12/16/2022 11:20:37 AM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 7.518392867276416 (T) = (0 -180.90840172623865) / Math.Sqrt((74553.80932906877 / (299)) + (24.098244753670286 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.39711068340455263 = (300.0690122489471 - 180.90840172623865) / 300.0690122489471 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	af6b1bd43f24d04cafff386bb2a1cd4510ecfe8c
Compare	97a51ccdb147992b80d427b83a77caa967d275b4
Diff	Diff

Improvements in System.Text.Json.Serialization.Tests.ReadJson<HashSet<String>>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
DeserializeFromStream - Duration of single invocation	156.53 μs	146.36 μs	0.93	0.04	False
DeserializeFromReader - Duration of single invocation	180.05 μs	167.12 μs	0.93	0.02	False
DeserializeFromUtf8Bytes - Duration of single invocation	139.35 μs	124.82 μs	0.90	0.02	False
DeserializeFromString - Duration of single invocation	156.55 μs	144.24 μs	0.92	0.03	False
DeserializeFromString - Duration of single invocation	158.37 μs	142.35 μs	0.90	0.04	False
DeserializeFromStream - Duration of single invocation	165.42 μs	147.37 μs	0.89	0.04	False
DeserializeFromUtf8Bytes - Duration of single invocation	136.87 μs	125.90 μs	0.92	0.02	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.ReadJson<HashSet<String>>*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Serialization.Tests.ReadJson<HashSet<String>>.DeserializeFromStream(Mode: Reflection)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 146.3586362654321 < 150.4797096222643.
IsChangePoint: Marked as a change because one of 12/16/2022 11:20:37 AM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 24.176327966596503 (T) = (0 -145313.69399262482) / Math.Sqrt((11370446.986302998 / (104)) + (2303625.0070108282 / (13))) is greater than 1.980807541103565 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (104) + (13) - 2, .975) and 0.08177496345097339 = (158254.9900172169 - 145313.69399262482) / 158254.9900172169 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Serialization.Tests.ReadJson&lt;HashSet&lt;String&gt;&gt;.DeserializeFromReader(Mode: Reflection)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 167.11555552631575 < 171.41951121886973.
IsChangePoint: Marked as a change because one of 12/16/2022 11:20:37 AM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 28.0787719480679 (T) = (0 -168068.3850933109) / Math.Sqrt((12875748.624533132 / (104)) + (1654253.9537072217 / (13))) is greater than 1.980807541103565 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (104) + (13) - 2, .975) and 0.07724385947501135 = (182137.37921883765 - 168068.3850933109) / 182137.37921883765 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Serialization.Tests.ReadJson&lt;HashSet&lt;String&gt;&gt;.DeserializeFromUtf8Bytes(Mode: SourceGen)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 124.82243292493798 < 130.37537655827293.
IsChangePoint: Marked as a change because one of 12/16/2022 11:20:37 AM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 23.1542640579173 (T) = (0 -124305.99787276372) / Math.Sqrt((8099224.4924588455 / (104)) + (3202677.715552831 / (13))) is greater than 1.980807541103565 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (104) + (13) - 2, .975) and 0.09589361852671262 = (137490.4551278587 - 124305.99787276372) / 137490.4551278587 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Serialization.Tests.ReadJson&lt;HashSet&lt;String&gt;&gt;.DeserializeFromString(Mode: Reflection)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 144.2365380681818 < 149.60208155097328.
IsChangePoint: Marked as a change because one of 12/16/2022 11:20:37 AM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 22.632021530644344 (T) = (0 -144416.98774982238) / Math.Sqrt((11584172.757960599 / (104)) + (3498920.64758724 / (13))) is greater than 1.980807541103565 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (104) + (13) - 2, .975) and 0.088150535798084 = (158378.1023287889 - 144416.98774982238) / 158378.1023287889 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Serialization.Tests.ReadJson&lt;HashSet&lt;String&gt;&gt;.DeserializeFromString(Mode: SourceGen)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 142.35474244318183 < 149.82143417087545.
IsChangePoint: Marked as a change because one of 12/16/2022 11:20:37 AM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 29.3104774422206 (T) = (0 -144602.95768171435) / Math.Sqrt((9896264.95827089 / (104)) + (1681987.9808676902 / (13))) is greater than 1.980807541103565 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (104) + (13) - 2, .975) and 0.08763199609938432 = (158491.92109269317 - 144602.95768171435) / 158491.92109269317 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Serialization.Tests.ReadJson&lt;HashSet&lt;String&gt;&gt;.DeserializeFromStream(Mode: SourceGen)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 147.37295089285718 < 150.4679902530242.
IsChangePoint: Marked as a change because one of 12/16/2022 11:20:37 AM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 23.44250277113755 (T) = (0 -146613.66200920645) / Math.Sqrt((10048661.790774714 / (104)) + (2092411.134515147 / (13))) is greater than 1.980807541103565 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (104) + (13) - 2, .975) and 0.07505801986992588 = (158511.19871171622 - 146613.66200920645) / 158511.19871171622 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Serialization.Tests.ReadJson&lt;HashSet&lt;String&gt;&gt;.DeserializeFromUtf8Bytes(Mode: Reflection)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 125.8955397 < 128.79998716877054.
IsChangePoint: Marked as a change because one of 12/16/2022 11:20:37 AM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 21.283513921672796 (T) = (0 -124753.46436461278) / Math.Sqrt((8766695.773763483 / (104)) + (3222208.1616573785 / (13))) is greater than 1.980807541103565 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (104) + (13) - 2, .975) and 0.0895224335174675 = (137019.81131350168 - 124753.46436461278) / 137019.81131350168 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

[performanceautofiler]

Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	af6b1bd43f24d04cafff386bb2a1cd4510ecfe8c
Compare	97a51ccdb147992b80d427b83a77caa967d275b4
Diff	Diff

Improvements in System.Linq.Tests.Perf_Enumerable

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
ToList - Duration of single invocation	1.57 μs	375.45 ns	0.24	0.03	False
ToArray - Duration of single invocation	14.90 μs	7.63 μs	0.51	0.06	False
Contains_ElementNotFound - Duration of single invocation	878.35 ns	353.72 ns	0.40	0.03	False
SelectToList - Duration of single invocation	17.65 μs	9.21 μs	0.52	0.08	False
SelectToArray - Duration of single invocation	16.30 μs	9.13 μs	0.56	0.04	False
ToArray - Duration of single invocation	1.52 μs	343.76 ns	0.23	0.04	False
ToList - Duration of single invocation	16.21 μs	9.17 μs	0.57	0.04	False
TakeLastHalf - Duration of single invocation	23.39 μs	16.84 μs	0.72	0.03	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Linq.Tests.Perf_Enumerable*'

Payloads

Baseline
Compare

Histogram

System.Linq.Tests.Perf_Enumerable.ToList(input: ICollection)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 375.4470016295152 < 1.4777411809457204.
IsChangePoint: Marked as a change because one of 12/7/2022 5:17:51 AM, 12/16/2022 11:20:37 AM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 7.7654973010032124 (T) = (0 -379.8234586733366) / Math.Sqrt((196892.97489691814 / (299)) + (18.900641998753418 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.34435929964830647 = (579.3164739004072 - 379.8234586733366) / 579.3164739004072 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Linq.Tests.Perf_Enumerable.ToArray(input: IEnumerable)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 7.634726269026672 < 14.039707571772961.
IsChangePoint: Marked as a change because one of 12/7/2022 5:17:51 AM, 12/16/2022 11:20:37 AM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 5.828474037006199 (T) = (0 -7888.104399444932) / Math.Sqrt((7174303.830385627 / (299)) + (107268.71432816196 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.11714123621983324 = (8934.729679377213 - 7888.104399444932) / 8934.729679377213 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Linq.Tests.Perf_Enumerable.Contains_ElementNotFound(input: ICollection)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 353.72061730437844 < 835.3223292040972.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 280.72778765417513 (T) = (0 -347.63245011188025) / Math.Sqrt((338.09335904503394 / (299)) + (13.62570130222127 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.6132609353914303 = (898.8811369850355 - 347.63245011188025) / 898.8811369850355 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Linq.Tests.Perf_Enumerable.SelectToList(input: IEnumerable)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 9.209700315208826 < 16.62726058469984.
IsChangePoint: Marked as a change because one of 12/7/2022 5:17:51 AM, 12/16/2022 11:20:37 AM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 6.5647734749426805 (T) = (0 -9425.409376983087) / Math.Sqrt((9641023.223588755 / (299)) + (35696.14085344815 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1152668210834654 = (10653.392007436263 - 9425.409376983087) / 10653.392007436263 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Linq.Tests.Perf_Enumerable.SelectToArray(input: IEnumerable)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 9.133031437183973 < 15.380816967426409.
IsChangePoint: Marked as a change because one of 12/7/2022 5:17:51 AM, 12/16/2022 11:20:37 AM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 6.549259967843533 (T) = (0 -9349.278344758253) / Math.Sqrt((7036632.279757662 / (299)) + (44645.35297085708 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.10316922779800915 = (10424.79655532219 - 9349.278344758253) / 10424.79655532219 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Linq.Tests.Perf_Enumerable.ToArray(input: ICollection)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 343.76231435583856 < 1.448392742762173.
IsChangePoint: Marked as a change because one of 12/7/2022 5:17:51 AM, 12/16/2022 11:20:37 AM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 7.679822744657616 (T) = (0 -339.3958873435341) / Math.Sqrt((196692.63638228408 / (299)) + (21.653170993584904 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.36752964253341935 = (536.6194373172146 - 339.3958873435341) / 536.6194373172146 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Linq.Tests.Perf_Enumerable.ToList(input: IEnumerable)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 9.173891323362396 < 15.277474702166646.
IsChangePoint: Marked as a change because one of 12/7/2022 5:17:51 AM, 12/16/2022 11:20:37 AM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 5.119376420263617 (T) = (0 -8025.768266816505) / Math.Sqrt((9589433.687797915 / (299)) + (150097.76552608595 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.11755695931538201 = (9094.94199261852 - 8025.768266816505) / 9094.94199261852 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Linq.Tests.Perf_Enumerable.TakeLastHalf(input: IEnumerable)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 16.838715370269288 < 22.469727916755446.
IsChangePoint: Marked as a change because one of 12/7/2022 5:17:51 AM, 12/16/2022 11:20:37 AM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 6.340638998094721 (T) = (0 -17370.624549375625) / Math.Sqrt((5803697.031901001 / (299)) + (168563.5088084448 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.06163218120076667 = (18511.530554834728 - 17370.624549375625) / 18511.530554834728 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	af6b1bd43f24d04cafff386bb2a1cd4510ecfe8c
Compare	97a51ccdb147992b80d427b83a77caa967d275b4
Diff	Diff

Improvements in MicroBenchmarks.Serializers.Binary_FromStream<IndexViewModel>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
BinaryFormatter_ - Duration of single invocation	531.46 μs	457.91 μs	0.86	0.01	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Binary_FromStream<IndexViewModel>*'

Payloads

Baseline
Compare

Histogram

MicroBenchmarks.Serializers.Binary_FromStream<IndexViewModel>.BinaryFormatter_

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 457.9076769132652 < 503.3512715283764.
IsChangePoint: Marked as a change because one of 12/7/2022 5:17:51 AM, 12/9/2022 1:52:32 PM, 12/16/2022 11:20:37 AM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 53.090514134815955 (T) = (0 -457580.8658923184) / Math.Sqrt((263925334.6657103 / (299)) + (12174807.550031506 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.13531589488345142 = (529188.478410439 - 457580.8658923184) / 529188.478410439 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	af6b1bd43f24d04cafff386bb2a1cd4510ecfe8c
Compare	97a51ccdb147992b80d427b83a77caa967d275b4
Diff	Diff

Improvements in System.Text.Json.Serialization.Tests.ReadJson<Dictionary<String, String>>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
DeserializeFromUtf8Bytes - Duration of single invocation	225.27 μs	211.91 μs	0.94	0.02	False
DeserializeFromReader - Duration of single invocation	310.64 μs	294.90 μs	0.95	0.01	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.ReadJson<Dictionary<String, String>>*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Serialization.Tests.ReadJson<Dictionary<String, String>>.DeserializeFromUtf8Bytes(Mode: Reflection)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 211.9127033928572 < 213.9137854441425.
IsChangePoint: Marked as a change because one of 12/8/2022 9:11:55 AM, 12/16/2022 11:20:37 AM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 21.613909718552705 (T) = (0 -211460.46655071437) / Math.Sqrt((58519602.79056121 / (104)) + (5286157.2400018545 / (13))) is greater than 1.980807541103565 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (104) + (13) - 2, .975) and 0.09143124401227345 = (232740.19182051963 - 211460.46655071437) / 232740.19182051963 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Serialization.Tests.ReadJson&lt;Dictionary&lt;String, String&gt;&gt;.DeserializeFromReader(Mode: Reflection)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 294.8964315762363 < 295.595993778766.
IsChangePoint: Marked as a change because one of 12/8/2022 5:23:10 PM, 12/16/2022 11:20:37 AM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 16.952213422289496 (T) = (0 -295744.8247833253) / Math.Sqrt((55812215.817453966 / (104)) + (17216213.443752322 / (13))) is greater than 1.980807541103565 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (104) + (13) - 2, .975) and 0.07252416191998096 = (318870.6515477006 - 295744.8247833253) / 318870.6515477006 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	af6b1bd43f24d04cafff386bb2a1cd4510ecfe8c
Compare	97a51ccdb147992b80d427b83a77caa967d275b4
Diff	Diff

Improvements in MicroBenchmarks.Serializers.Json_ToString<CollectionsOfPrimitives>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
JsonNet_ - Duration of single invocation	4.95 ms	4.53 ms	0.91	0.01	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Json_ToString<CollectionsOfPrimitives>*'

Payloads

Baseline
Compare

Histogram

MicroBenchmarks.Serializers.Json_ToString<CollectionsOfPrimitives>.JsonNet_

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 4.5278256481481485 < 4.632746116725022.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 12/7/2022 5:17:51 AM, 12/9/2022 1:52:32 PM, 12/16/2022 11:20:37 AM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 25.404030051976 (T) = (0 -4555543.9345873995) / Math.Sqrt((20574666710.146774 / (299)) + (2327400716.2866354 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.08070579504438447 = (4955479.877965015 - 4555543.9345873995) / 4955479.877965015 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	af6b1bd43f24d04cafff386bb2a1cd4510ecfe8c
Compare	97a51ccdb147992b80d427b83a77caa967d275b4
Diff	Diff

Improvements in System.Collections.CopyTo<Int32>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
ImmutableArray - Duration of single invocation	2.14 μs	542.93 ns	0.25	0.03	False
List - Duration of single invocation	1.14 μs	371.58 ns	0.33	0.04	False
Array - Duration of single invocation	2.14 μs	516.95 ns	0.24	0.05	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.CopyTo<Int32>*'

Payloads

Baseline
Compare

Histogram

System.Collections.CopyTo<Int32>.ImmutableArray(Size: 2048)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 542.9250877993004 < 2.0512397015689463.
IsChangePoint: Marked as a change because one of 12/7/2022 5:17:51 AM, 12/16/2022 11:20:37 AM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 7.440277830713689 (T) = (0 -542.5698819981465) / Math.Sqrt((378955.12816147954 / (299)) + (62.203672527004905 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.32845971024409776 = (807.9483692562435 - 542.5698819981465) / 807.9483692562435 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.CopyTo&lt;Int32&gt;.List(Size: 2048)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 371.58223190816165 < 1.0736832354249695.
IsChangePoint: Marked as a change because one of 12/7/2022 5:17:51 AM, 12/16/2022 11:20:37 AM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 7.721721262076102 (T) = (0 -374.27159408011005) / Math.Sqrt((81487.50077804163 / (299)) + (22.17887242505428 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.25465386542434826 = (502.14467710789734 - 374.27159408011005) / 502.14467710789734 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.CopyTo&lt;Int32&gt;.Array(Size: 2048)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 516.9506768720435 < 2.028598711520455.
IsChangePoint: Marked as a change because one of 12/7/2022 5:17:51 AM, 12/16/2022 11:20:37 AM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 7.405354255781733 (T) = (0 -529.9832361726104) / Math.Sqrt((377495.24955321604 / (299)) + (108.28897055205137 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.3324956413504983 = (793.9771917667703 - 529.9832361726104) / 793.9771917667703 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	af6b1bd43f24d04cafff386bb2a1cd4510ecfe8c
Compare	97a51ccdb147992b80d427b83a77caa967d275b4
Diff	Diff

Improvements in System.Reflection.Attributes

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
GetCustomAttributesClassHit - Duration of single invocation	12.31 μs	8.75 μs	0.71	0.01	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Reflection.Attributes*'

Payloads

Baseline
Compare

Histogram

System.Reflection.Attributes.GetCustomAttributesClassHit

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 8.751571128090658 < 11.654650406984166.
IsChangePoint: Marked as a change because one of 12/7/2022 5:17:51 AM, 12/9/2022 1:52:32 PM, 12/16/2022 11:20:37 AM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 74.31910911730454 (T) = (0 -8823.163319873154) / Math.Sqrt((2683733.543776607 / (299)) + (7084.095935833083 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.4511167490309831 = (16074.754156364665 - 8823.163319873154) / 16074.754156364665 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	af6b1bd43f24d04cafff386bb2a1cd4510ecfe8c
Compare	97a51ccdb147992b80d427b83a77caa967d275b4
Diff	Diff

Improvements in System.IO.Tests.Perf_RandomAccess

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
ReadScatterAsync - Duration of single invocation	36.24 ms	33.68 ms	0.93	0.20	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.IO.Tests.Perf_RandomAccess*'

Payloads

Baseline
Compare

Histogram

System.IO.Tests.Perf_RandomAccess.ReadScatterAsync(fileSize: 104857600, buffersSize: 65536, options: None)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 33.6790866122449 < 34.246845200320514.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 11/16/2022 6:51:59 PM, 12/15/2022 8:20:21 PM, 12/16/2022 11:20:37 AM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 15.76395465966633 (T) = (0 -33803811.38727207) / Math.Sqrt((3895806618286.7173 / (299)) + (118183550352.75328 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.06485952885273553 = (36148378.163764335 - 33803811.38727207) / 36148378.163764335 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	af6b1bd43f24d04cafff386bb2a1cd4510ecfe8c
Compare	97a51ccdb147992b80d427b83a77caa967d275b4
Diff	Diff

Improvements in System.Collections.AddGivenSize<String>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
SortedList - Duration of single invocation	2.27 ms	1.48 ms	0.65	0.05	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.AddGivenSize<String>*'

Payloads

Baseline
Compare

Histogram

System.Collections.AddGivenSize<String>.SortedList(Size: 512)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.4765641696969696 < 2.1570533231579505.
IsChangePoint: Marked as a change because one of 12/7/2022 5:17:51 AM, 12/16/2022 11:20:37 AM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 8.197908342438208 (T) = (0 -1491621.831311397) / Math.Sqrt((85294860229.71133 / (299)) + (160680126.56623745 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.0866043129719111 = (1633051.0998630612 - 1491621.831311397) / 1633051.0998630612 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

[performanceautofiler]

Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	af6b1bd43f24d04cafff386bb2a1cd4510ecfe8c
Compare	97a51ccdb147992b80d427b83a77caa967d275b4
Diff	Diff

Improvements in System.Tests.Perf_Array

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
ArrayRetrieve2D - Duration of single invocation	133.73 ms	49.32 ms	0.37	0.01	False
ArrayRetrieve3D - Duration of single invocation	192.62 ms	59.52 ms	0.31	0.01	False
ArrayCopy3D - Duration of single invocation	2.49 μs	911.17 ns	0.37	0.01	False
ArrayRetrieve1D - Duration of single invocation	92.23 ms	38.36 ms	0.42	0.01	False
ArrayAssign1D - Duration of single invocation	88.79 ms	37.05 ms	0.42	0.01	False
IndexOfChar - Duration of single invocation	272.03 ns	237.59 ns	0.87	0.09	False
ArrayAssign3D - Duration of single invocation	177.17 ms	55.47 ms	0.31	0.01	False
ArrayCopy2D - Duration of single invocation	2.48 μs	900.84 ns	0.36	0.01	False
ArrayAssign2D - Duration of single invocation	135.52 ms	48.83 ms	0.36	0.01	False
IndexOfShort - Duration of single invocation	303.91 ns	238.36 ns	0.78	0.11	False
Clear - Duration of single invocation	1.29 μs	801.80 ns	0.62	0.03	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Array*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Array.ArrayRetrieve2D

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 49.32231841666666 < 128.07133625134614.
IsChangePoint: Marked as a change because one of 12/7/2022 5:17:51 AM, 12/16/2022 11:20:37 AM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 19.190089971885687 (T) = (0 -48991913.5482812) / Math.Sqrt((766541830458147.1 / (299)) + (271450863363.02463 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.3863970767723309 = (79843025.01457186 - 48991913.5482812) / 79843025.01457186 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Array.ArrayRetrieve3D

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 59.52244013333333 < 183.381171684478.
IsChangePoint: Marked as a change because one of 12/7/2022 5:17:51 AM, 12/16/2022 11:20:37 AM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 15.544126061617565 (T) = (0 -60325929.56832206) / Math.Sqrt((1978746208370983 / (299)) + (583714008835.549 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.3994371930425566 = (100448993.62640156 - 60325929.56832206) / 100448993.62640156 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Array.ArrayCopy3D

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 911.1746799181466 < 2.377243626275759.
IsChangePoint: Marked as a change because one of 12/7/2022 5:17:51 AM, 12/16/2022 11:20:37 AM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 7.279209068440374 (T) = (0 -918.5601394755748) / Math.Sqrt((364079.7446768085 / (299)) + (100.35784568978364 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.21716187288538688 = (1173.371745268982 - 918.5601394755748) / 1173.371745268982 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Array.ArrayRetrieve1D

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 38.3599103076923 < 86.31324188482142.
IsChangePoint: Marked as a change because one of 12/7/2022 5:17:51 AM, 12/16/2022 11:20:37 AM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 28.48591661915856 (T) = (0 -39014635.40551799) / Math.Sqrt((225114657644301.78 / (299)) + (513246894720.737 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.39391407780134374 = (64371459.51845784 - 39014635.40551799) / 64371459.51845784 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Array.ArrayAssign1D

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 37.04918120512821 < 84.40815545334823.
IsChangePoint: Marked as a change because one of 12/7/2022 5:17:51 AM, 12/16/2022 11:20:37 AM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 27.919681601669343 (T) = (0 -37522060.13804828) / Math.Sqrt((224538948358333.7 / (299)) + (266107967154.72763 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.39523778110440894 = (62044319.18146371 - 37522060.13804828) / 62044319.18146371 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Array.IndexOfChar

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 237.58512305497538 < 266.09978299531264.
IsChangePoint: Marked as a change because one of 9/28/2022 3:03:11 PM, 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 45.15546149261431 (T) = (0 -237.7117911248146) / Math.Sqrt((166.94210536344312 / (299)) + (2.5064554526457044 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.1635579628000279 = (284.19397944245566 - 237.7117911248146) / 284.19397944245566 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Array.ArrayAssign3D

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 55.4682535 < 166.35260324017855.
IsChangePoint: Marked as a change because one of 12/7/2022 5:17:51 AM, 12/16/2022 11:20:37 AM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 14.912739324065802 (T) = (0 -54190861.53262363) / Math.Sqrt((1644137194212300 / (299)) + (498970239198.5517 / (12))) is greater than 1.9676708853560316 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (12) - 2, .975) and 0.39310810233425764 = (89292445.22962853 - 54190861.53262363) / 89292445.22962853 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Array.ArrayCopy2D

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 900.8369291509283 < 2.3614111830622146.
IsChangePoint: Marked as a change because one of 12/7/2022 5:17:51 AM, 12/16/2022 11:20:37 AM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 7.330578232552738 (T) = (0 -910.178215838354) / Math.Sqrt((363649.5350325619 / (299)) + (100.80138559773626 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.21983000984790405 = (1166.6408953527073 - 910.178215838354) / 1166.6408953527073 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Array.ArrayAssign2D

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 48.83157235714286 < 128.44325641.
IsChangePoint: Marked as a change because one of 12/7/2022 5:17:51 AM, 12/16/2022 11:20:37 AM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 18.001871153008697 (T) = (0 -48502987.37318258) / Math.Sqrt((775060660745930.2 / (299)) + (730204431641.7057 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.3765583651009479 = (77798761.99804366 - 48502987.37318258) / 77798761.99804366 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Array.IndexOfShort

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 238.36305730588532 < 268.3800185226896.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 36.9628842064156 (T) = (0 -239.97471960163588) / Math.Sqrt((181.98221953134572 / (299)) + (4.243639540059549 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.15679054179983576 = (284.5968071964745 - 239.97471960163588) / 284.5968071964745 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Array.Clear

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 801.7970585842967 < 1.207943287954789.
IsChangePoint: Marked as a change because one of 12/7/2022 5:17:51 AM, 12/16/2022 11:20:37 AM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 7.2649446746585085 (T) = (0 -816.8431843401224) / Math.Sqrt((30695.919187226056 / (299)) + (110.14116075593488 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.08572267165687004 = (893.4304275273024 - 816.8431843401224) / 893.4304275273024 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	af6b1bd43f24d04cafff386bb2a1cd4510ecfe8c
Compare	97a51ccdb147992b80d427b83a77caa967d275b4
Diff	Diff

Improvements in System.Collections.CtorFromCollectionNonGeneric<String>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
ArrayList - Duration of single invocation	4.02 μs	1.56 μs	0.39	0.02	False
Queue - Duration of single invocation	584.21 μs	392.65 μs	0.67	0.02	False
Stack - Duration of single invocation	590.74 μs	390.61 μs	0.66	0.03	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.CtorFromCollectionNonGeneric<String>*'

Payloads

Baseline
Compare

Histogram

System.Collections.CtorFromCollectionNonGeneric<String>.ArrayList(Size: 512)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.5635280114976906 < 3.8163430691121207.
IsChangePoint: Marked as a change because one of 12/7/2022 5:17:51 AM, 12/16/2022 11:20:37 AM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 7.1246696377532315 (T) = (0 -1516.934579202128) / Math.Sqrt((895208.2776556589 / (299)) + (625.7414383505292 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2057518702402966 = (1909.9000959071448 - 1516.934579202128) / 1909.9000959071448 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.CtorFromCollectionNonGeneric&lt;String&gt;.Queue(Size: 512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 392.6472839972527 < 559.2429312422734.
IsChangePoint: Marked as a change because one of 12/7/2022 5:17:51 AM, 12/9/2022 1:52:32 PM, 12/16/2022 11:20:37 AM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 81.27179483247224 (T) = (0 -390681.98123459786) / Math.Sqrt((2259967588.2094088 / (299)) + (37721973.918216184 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.40219914000356455 = (653531.982601911 - 390681.98123459786) / 653531.982601911 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.CtorFromCollectionNonGeneric&lt;String&gt;.Stack(Size: 512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 390.60842347560975 < 550.3449169444443.
IsChangePoint: Marked as a change because one of 12/7/2022 5:17:51 AM, 12/9/2022 1:52:32 PM, 12/16/2022 11:20:37 AM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 70.93790913094502 (T) = (0 -385784.5643386281) / Math.Sqrt((2172514837.20554 / (299)) + (81535864.1233886 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.403542399979515 = (646792.9393897882 - 385784.5643386281) / 646792.9393897882 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	af6b1bd43f24d04cafff386bb2a1cd4510ecfe8c
Compare	97a51ccdb147992b80d427b83a77caa967d275b4
Diff	Diff

Improvements in System.Collections.CreateAddAndRemove<Int32>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
SortedList - Duration of single invocation	2.40 ms	895.82 μs	0.37	0.01	False
List - Duration of single invocation	520.42 μs	121.90 μs	0.23	0.06	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.CreateAddAndRemove<Int32>*'

Payloads

Baseline
Compare

Histogram

System.Collections.CreateAddAndRemove<Int32>.SortedList(Size: 512)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 895.8232949074073 < 2.272403569992559.
IsChangePoint: Marked as a change because one of 12/7/2022 5:17:51 AM, 12/16/2022 11:20:37 AM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 7.767094976119334 (T) = (0 -891004.6295215576) / Math.Sqrt((322073347082.92957 / (299)) + (55434212.79817022 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.22279839726631437 = (1146426.649646099 - 891004.6295215576) / 1146426.649646099 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.CreateAddAndRemove&lt;Int32&gt;.List(Size: 512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 121.89836220005581 < 486.58013477867814.
IsChangePoint: Marked as a change because one of 12/7/2022 5:17:51 AM, 12/16/2022 11:20:37 AM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 8.071212423551373 (T) = (0 -124605.90070390477) / Math.Sqrt((21027613439.263317 / (299)) + (5263913.233177774 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.35265079218449846 = (192486.37242392087 - 124605.90070390477) / 192486.37242392087 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	af6b1bd43f24d04cafff386bb2a1cd4510ecfe8c
Compare	97a51ccdb147992b80d427b83a77caa967d275b4
Diff	Diff

Improvements in System.Collections.CreateAddAndClear<Int32>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
ICollection - Duration of single invocation	29.55 μs	17.76 μs	0.60	0.06	False
ImmutableArray - Duration of single invocation	1.09 ms	268.56 μs	0.25	0.02	False
List - Duration of single invocation	23.53 μs	12.44 μs	0.53	0.08	False
Queue - Duration of single invocation	37.33 μs	25.20 μs	0.68	0.11	False
SortedList - Duration of single invocation	1.23 ms	455.74 μs	0.37	0.01	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.CreateAddAndClear<Int32>*'

Payloads

Baseline
Compare

Histogram

System.Collections.CreateAddAndClear<Int32>.ICollection(Size: 512)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 17.760575488244218 < 28.218928544046236.
IsChangePoint: Marked as a change because one of 12/7/2022 5:17:51 AM, 12/16/2022 11:20:37 AM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 7.112080427549112 (T) = (0 -18196.551503310016) / Math.Sqrt((18860878.226883013 / (299)) + (191751.4252591605 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.09831827846052149 = (20180.681351998897 - 18196.551503310016) / 20180.681351998897 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.CreateAddAndClear&lt;Int32&gt;.ImmutableArray(Size: 512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 268.5551656426554 < 1.035056615561756.
IsChangePoint: Marked as a change because one of 12/7/2022 5:17:51 AM, 12/16/2022 11:20:37 AM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 7.249122349930577 (T) = (0 -273994.91742666945) / Math.Sqrt((95950058893.16058 / (299)) + (16374402.277901178 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.3219770902108336 = (404108.6421576361 - 273994.91742666945) / 404108.6421576361 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.CreateAddAndClear&lt;Int32&gt;.List(Size: 512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 12.438960137559807 < 22.43132859206371.
IsChangePoint: Marked as a change because one of 12/7/2022 5:17:51 AM, 12/16/2022 11:20:37 AM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 7.150607737755745 (T) = (0 -12544.029164556634) / Math.Sqrt((18105466.04984796 / (299)) + (47152.63183471392 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.12618989789392213 = (14355.555210820654 - 12544.029164556634) / 14355.555210820654 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.CreateAddAndClear&lt;Int32&gt;.Queue(Size: 512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 25.198293762981766 < 35.268058710418096.
IsChangePoint: Marked as a change because one of 12/7/2022 5:17:51 AM, 12/16/2022 11:20:37 AM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 5.96826837101248 (T) = (0 -25995.125470318275) / Math.Sqrt((17919296.986072153 / (299)) + (854314.0549628334 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.07525812600609741 = (28110.682776856363 - 25995.125470318275) / 28110.682776856363 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.CreateAddAndClear&lt;Int32&gt;.SortedList(Size: 512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 455.7406634803921 < 1.1596450118303572.
IsChangePoint: Marked as a change because one of 12/7/2022 5:17:51 AM, 12/16/2022 11:20:37 AM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 7.685648890944333 (T) = (0 -462176.9659825411) / Math.Sqrt((82894473369.11467 / (299)) + (42436060.84170638 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.21783970419285362 = (590898.0147165357 - 462176.9659825411) / 590898.0147165357 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	af6b1bd43f24d04cafff386bb2a1cd4510ecfe8c
Compare	97a51ccdb147992b80d427b83a77caa967d275b4
Diff	Diff

Improvements in System.Collections.CreateAddAndClear<String>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
SortedList - Duration of single invocation	2.32 ms	1.53 ms	0.66	0.04	False
ImmutableArray - Duration of single invocation	1.16 ms	328.36 μs	0.28	0.01	False
Queue - Duration of single invocation	42.86 μs	29.78 μs	0.69	0.10	False
List - Duration of single invocation	28.45 μs	16.78 μs	0.59	0.10	False
ICollection - Duration of single invocation	34.68 μs	22.34 μs	0.64	0.03	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.CreateAddAndClear<String>*'

Payloads

Baseline
Compare

Histogram

System.Collections.CreateAddAndClear<String>.SortedList(Size: 512)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.5262696401785711 < 2.187116485491072.
IsChangePoint: Marked as a change because one of 12/7/2022 5:17:51 AM, 12/16/2022 11:20:37 AM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 7.079626292604104 (T) = (0 -1515137.1656627506) / Math.Sqrt((92887925423.7863 / (299)) + (465427011.39596635 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.08001445741430421 = (1646914.1041111709 - 1515137.1656627506) / 1646914.1041111709 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.CreateAddAndClear&lt;String&gt;.ImmutableArray(Size: 512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 328.36463094618057 < 1.0943411938387544.
IsChangePoint: Marked as a change because one of 12/7/2022 5:17:51 AM, 12/16/2022 11:20:37 AM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 7.3078027401745596 (T) = (0 -324100.68966535904) / Math.Sqrt((100028207316.30785 / (299)) + (32205625.822344933 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2927550806688633 = (458258.06705245905 - 324100.68966535904) / 458258.06705245905 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.CreateAddAndClear&lt;String&gt;.Queue(Size: 512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 29.776797886506422 < 40.65742467919445.
IsChangePoint: Marked as a change because one of 12/7/2022 5:17:51 AM, 12/16/2022 11:20:37 AM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 6.088564271263202 (T) = (0 -30840.60895867638) / Math.Sqrt((20878981.00853532 / (299)) + (1196487.0953000241 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.07358298676545176 = (33290.201408324334 - 30840.60895867638) / 33290.201408324334 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.CreateAddAndClear&lt;String&gt;.List(Size: 512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 16.780397652649768 < 26.9230734852377.
IsChangePoint: Marked as a change because one of 12/7/2022 5:17:51 AM, 12/16/2022 11:20:37 AM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 7.484909340039807 (T) = (0 -16772.31403248326) / Math.Sqrt((19894214.853945605 / (299)) + (87089.61647602571 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.10775493429266941 = (18797.87815826916 - 16772.31403248326) / 18797.87815826916 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.CreateAddAndClear&lt;String&gt;.ICollection(Size: 512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 22.341149678111584 < 32.722467439124706.
IsChangePoint: Marked as a change because one of 12/7/2022 5:17:51 AM, 12/16/2022 11:20:37 AM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 7.141442376674739 (T) = (0 -22938.91077810135) / Math.Sqrt((19707765.583967526 / (299)) + (159895.4042196361 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.08009282039275685 = (24936.114519614013 - 22938.91077810135) / 24936.114519614013 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	af6b1bd43f24d04cafff386bb2a1cd4510ecfe8c
Compare	97a51ccdb147992b80d427b83a77caa967d275b4
Diff	Diff

Improvements in Microsoft.Extensions.DependencyInjection.GetServiceIEnumerable

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
Transient - Duration of single invocation	53.90 μs	41.48 μs	0.77	0.01	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Microsoft.Extensions.DependencyInjection.GetServiceIEnumerable*'

Payloads

Baseline
Compare

Histogram

Microsoft.Extensions.DependencyInjection.GetServiceIEnumerable.Transient

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 41.48454257142858 < 51.04050342419518.
IsChangePoint: Marked as a change because one of 12/7/2022 5:17:51 AM, 12/9/2022 1:52:32 PM, 12/16/2022 11:20:37 AM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 71.22177184857333 (T) = (0 -40956.82091042178) / Math.Sqrt((27490422.79418947 / (299)) + (395174.29456782126 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.37819924730159055 = (65868.07869350871 - 40956.82091042178) / 65868.07869350871 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

[performanceautofiler]

Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	af6b1bd43f24d04cafff386bb2a1cd4510ecfe8c
Compare	97a51ccdb147992b80d427b83a77caa967d275b4
Diff	Diff

Improvements in System.Collections.CtorFromCollection<String>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
List - Duration of single invocation	1.94 μs	717.69 ns	0.37	0.01	False
Stack - Duration of single invocation	2.00 μs	713.01 ns	0.36	0.02	False
ImmutableList - Duration of single invocation	263.01 μs	142.33 μs	0.54	0.04	False
ImmutableArray - Duration of single invocation	1.51 μs	646.76 ns	0.43	0.01	False
Queue - Duration of single invocation	1.98 μs	738.33 ns	0.37	0.02	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.CtorFromCollection<String>*'

Payloads

Baseline
Compare

Histogram

System.Collections.CtorFromCollection<String>.List(Size: 512)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 717.6893846657294 < 1.857550475968666.
IsChangePoint: Marked as a change because one of 12/7/2022 5:17:51 AM, 12/16/2022 11:20:37 AM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 7.17088630558735 (T) = (0 -721.2269315372735) / Math.Sqrt((227298.41188261713 / (299)) + (80.88953756057798 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.21584249313775838 = (919.7475318743284 - 721.2269315372735) / 919.7475318743284 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.CtorFromCollection&lt;String&gt;.Stack(Size: 512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 713.009305959694 < 1.8839423607622978.
IsChangePoint: Marked as a change because one of 12/7/2022 5:17:51 AM, 12/16/2022 11:20:37 AM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 7.325642353030093 (T) = (0 -733.6542968938364) / Math.Sqrt((226292.4877405171 / (299)) + (79.91849677393407 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.21618439506119533 = (936.0036879479013 - 733.6542968938364) / 936.0036879479013 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.CtorFromCollection&lt;String&gt;.ImmutableList(Size: 512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 142.3307874415888 < 250.25563966025405.
IsChangePoint: Marked as a change because one of 12/7/2022 5:17:51 AM, 12/16/2022 11:20:37 AM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 6.462676032248972 (T) = (0 -143355.42236857655) / Math.Sqrt((2171077596.449101 / (299)) + (5782354.339933765 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1112250185977852 = (161295.51952779497 - 143355.42236857655) / 161295.51952779497 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.CtorFromCollection&lt;String&gt;.ImmutableArray(Size: 512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 646.7645628838479 < 1.3980658414821325.
IsChangePoint: Marked as a change because one of 12/7/2022 5:17:51 AM, 12/16/2022 11:20:37 AM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 8.16870101035691 (T) = (0 -661.1329241382763) / Math.Sqrt((92717.51180579564 / (299)) + (60.4794907458674 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.17979090539526832 = (806.0541250848772 - 661.1329241382763) / 806.0541250848772 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.CtorFromCollection&lt;String&gt;.Queue(Size: 512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 738.3276748917749 < 1.890697250448309.
IsChangePoint: Marked as a change because one of 12/7/2022 5:17:51 AM, 12/16/2022 11:20:37 AM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 7.361049543855952 (T) = (0 -737.3157148923239) / Math.Sqrt((226009.92637142158 / (299)) + (41.0524569497211 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.21572016853779483 = (940.1181635866859 - 737.3157148923239) / 940.1181635866859 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	af6b1bd43f24d04cafff386bb2a1cd4510ecfe8c
Compare	97a51ccdb147992b80d427b83a77caa967d275b4
Diff	Diff

Improvements in System.Collections.CtorFromCollection<Int32>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
Queue - Duration of single invocation	1.79 μs	547.89 ns	0.31	0.03	False
ImmutableList - Duration of single invocation	262.42 μs	144.61 μs	0.55	0.05	False
ImmutableArray - Duration of single invocation	1.27 μs	513.79 ns	0.40	0.02	False
Stack - Duration of single invocation	1.77 μs	559.05 ns	0.32	0.02	False
List - Duration of single invocation	1.72 μs	530.88 ns	0.31	0.02	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.CtorFromCollection<Int32>*'

Payloads

Baseline
Compare

Histogram

System.Collections.CtorFromCollection<Int32>.Queue(Size: 512)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 547.8938152979292 < 1.6768627229050925.
IsChangePoint: Marked as a change because one of 12/7/2022 5:17:51 AM, 12/16/2022 11:20:37 AM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 7.470801481963285 (T) = (0 -558.4468013635093) / Math.Sqrt((208911.93642902383 / (299)) + (58.570057117697466 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.26185853924825186 = (756.5579649119943 - 558.4468013635093) / 756.5579649119943 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.CtorFromCollection&lt;Int32&gt;.ImmutableList(Size: 512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 144.61432098909657 < 251.46711974991481.
IsChangePoint: Marked as a change because one of 12/7/2022 5:17:51 AM, 12/16/2022 11:20:37 AM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 5.285849791353029 (T) = (0 -145762.45477978315) / Math.Sqrt((2279551086.9802175 / (299)) + (10773387.209831715 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.09537486757020122 = (161130.2290356107 - 145762.45477978315) / 161130.2290356107 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.CtorFromCollection&lt;Int32&gt;.ImmutableArray(Size: 512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 513.7909906300773 < 1.2058115063650219.
IsChangePoint: Marked as a change because one of 12/7/2022 5:17:51 AM, 12/16/2022 11:20:37 AM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 7.822294513860342 (T) = (0 -504.40549986168696) / Math.Sqrt((83918.63566034641 / (299)) + (62.54914405990033 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.20762125855589939 = (636.5712171207598 - 504.40549986168696) / 636.5712171207598 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.CtorFromCollection&lt;Int32&gt;.Stack(Size: 512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 559.0508497323698 < 1.6694865175494145.
IsChangePoint: Marked as a change because one of 12/7/2022 5:17:51 AM, 12/16/2022 11:20:37 AM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 7.260331020425507 (T) = (0 -557.9425434463625) / Math.Sqrt((208291.6557643124 / (299)) + (81.53629124542216 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.25650316430000014 = (750.4302865271261 - 557.9425434463625) / 750.4302865271261 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.CtorFromCollection&lt;Int32&gt;.List(Size: 512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 530.8810558933856 < 1.6612591610637362.
IsChangePoint: Marked as a change because one of 12/7/2022 5:17:51 AM, 12/16/2022 11:20:37 AM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 7.293656338470467 (T) = (0 -542.2663322560686) / Math.Sqrt((208910.09625719866 / (299)) + (107.16883831685585 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.26341777841254255 = (736.1925340627884 - 542.2663322560686) / 736.1925340627884 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	af6b1bd43f24d04cafff386bb2a1cd4510ecfe8c
Compare	97a51ccdb147992b80d427b83a77caa967d275b4
Diff	Diff

Improvements in Benchstone.MDBenchI.MDGeneralArray

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
Test2 - Duration of single invocation	6.13 secs	954.97 ms	0.16	0.12	False
Test - Duration of single invocation	6.28 secs	984.50 ms	0.16	0.14	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Benchstone.MDBenchI.MDGeneralArray*'

Payloads

Baseline
Compare

Histogram

Benchstone.MDBenchI.MDGeneralArray.Test2

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 954.97216225 < 5.917894803766667.
IsChangePoint: Marked as a change because one of 12/7/2022 5:17:51 AM, 12/16/2022 11:20:37 AM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 7.436465026986591 (T) = (0 -976284232.1661173) / Math.Sqrt((4.024612109282615E+18 / (299)) + (965280017832589.1 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.4698217828913934 = (1841426525.3868895 - 976284232.1661173) / 1841426525.3868895 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### Benchstone.MDBenchI.MDGeneralArray.Test

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 984.5021473076923 < 5.999485342929999.
IsChangePoint: Marked as a change because one of 12/7/2022 5:17:51 AM, 12/16/2022 11:20:37 AM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 7.4229089042655705 (T) = (0 -977583800.3913214) / Math.Sqrt((4.050171251564861E+18 / (299)) + (1688001042170311.2 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.47032744126360154 = (1845637997.036268 - 977583800.3913214) / 1845637997.036268 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	af6b1bd43f24d04cafff386bb2a1cd4510ecfe8c
Compare	97a51ccdb147992b80d427b83a77caa967d275b4
Diff	Diff

Improvements in System.Collections.CreateAddAndRemove<String>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
SortedList - Duration of single invocation	4.33 ms	2.72 ms	0.63	0.03	False
List - Duration of single invocation	563.22 μs	166.93 μs	0.30	0.05	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.CreateAddAndRemove<String>*'

Payloads

Baseline
Compare

Histogram

System.Collections.CreateAddAndRemove<String>.SortedList(Size: 512)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.7183866131944447 < 4.013934064726563.
IsChangePoint: Marked as a change because one of 12/7/2022 5:17:51 AM, 12/16/2022 11:20:37 AM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 7.421897500976781 (T) = (0 -2699724.6221843595) / Math.Sqrt((357719914574.243 / (299)) + (1367623610.8367097 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.0902324866918282 = (2967488.487654827 - 2699724.6221843595) / 2967488.487654827 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.CreateAddAndRemove&lt;String&gt;.List(Size: 512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 166.93219160017733 < 531.0297255770158.
IsChangePoint: Marked as a change because one of 12/7/2022 5:17:51 AM, 12/16/2022 11:20:37 AM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 7.77726248026649 (T) = (0 -169673.2665817419) / Math.Sqrt((21335312526.155716 / (299)) + (4406092.563842143 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2795966439319892 = (235525.36943723456 - 169673.2665817419) / 235525.36943723456 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	af6b1bd43f24d04cafff386bb2a1cd4510ecfe8c
Compare	97a51ccdb147992b80d427b83a77caa967d275b4
Diff	Diff

Improvements in System.Collections.CtorFromCollectionNonGeneric<Int32>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
Queue - Duration of single invocation	664.02 μs	463.20 μs	0.70	0.01	False
ArrayList - Duration of single invocation	524.77 μs	161.15 μs	0.31	0.02	False
Stack - Duration of single invocation	661.52 μs	445.83 μs	0.67	0.01	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.CtorFromCollectionNonGeneric<Int32>*'

Payloads

Baseline
Compare

Histogram

System.Collections.CtorFromCollectionNonGeneric<Int32>.Queue(Size: 512)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 463.2038669117647 < 633.8964435099639.
IsChangePoint: Marked as a change because one of 12/7/2022 5:17:51 AM, 12/9/2022 1:52:32 PM, 12/16/2022 11:20:37 AM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 75.08421029203338 (T) = (0 -454243.43873230653) / Math.Sqrt((2592362599.828819 / (299)) + (27154710.674996372 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.3515678190078188 = (700525.7481781025 - 454243.43873230653) / 700525.7481781025 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.CtorFromCollectionNonGeneric&lt;Int32&gt;.ArrayList(Size: 512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 161.14655891875984 < 497.3104752528184.
IsChangePoint: Marked as a change because one of 12/7/2022 5:17:51 AM, 12/16/2022 11:20:37 AM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 5.134271510590976 (T) = (0 -161553.3518346667) / Math.Sqrt((21355422941.876507 / (299)) + (2140157.523175908 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.21191217653275685 = (204994.09713488826 - 161553.3518346667) / 204994.09713488826 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.CtorFromCollectionNonGeneric&lt;Int32&gt;.Stack(Size: 512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 445.83090183080816 < 629.1699360892211.
IsChangePoint: Marked as a change because one of 12/7/2022 5:17:51 AM, 12/9/2022 1:52:32 PM, 12/16/2022 11:20:37 AM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 71.08674798871976 (T) = (0 -447674.0405229137) / Math.Sqrt((2583587903.2750115 / (299)) + (45124283.245396055 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.3559303427811095 = (695070.8444424813 - 447674.0405229137) / 695070.8444424813 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	af6b1bd43f24d04cafff386bb2a1cd4510ecfe8c
Compare	97a51ccdb147992b80d427b83a77caa967d275b4
Diff	Diff

Improvements in System.Runtime.Serialization.Formatters.Tests.Perf_BinaryFormatter

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
DeserializeLargeList - Duration of single invocation	2.53 secs	2.28 secs	0.90	0.01	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Runtime.Serialization.Formatters.Tests.Perf_BinaryFormatter*'

Payloads

Baseline
Compare

Histogram

System.Runtime.Serialization.Formatters.Tests.Perf_BinaryFormatter.DeserializeLargeList

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.2797662964285714 < 2.402115807839487.
IsChangePoint: Marked as a change because one of 12/7/2022 5:17:51 AM, 12/9/2022 1:52:32 PM, 12/16/2022 11:20:37 AM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 53.141151993576074 (T) = (0 -2271368016.544801) / Math.Sqrt((7933943549619453 / (299)) + (493160841842043.3 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1581466541089337 = (2698056648.026568 - 2271368016.544801) / 2698056648.026568 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	af6b1bd43f24d04cafff386bb2a1cd4510ecfe8c
Compare	97a51ccdb147992b80d427b83a77caa967d275b4
Diff	Diff

Improvements in Microsoft.Extensions.Logging.FormattingOverhead

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
FourArguments_EnumerableArgument - Duration of single invocation	22.23 μs	16.79 μs	0.75	0.02	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Microsoft.Extensions.Logging.FormattingOverhead*'

Payloads

Baseline
Compare

Histogram

Microsoft.Extensions.Logging.FormattingOverhead.FourArguments_EnumerableArgument

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 16.786542524131274 < 21.129280604360392.
IsChangePoint: Marked as a change because one of 12/7/2022 5:17:51 AM, 12/9/2022 1:52:32 PM, 12/16/2022 11:20:37 AM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 41.662924283813176 (T) = (0 -16753.55024213604) / Math.Sqrt((1651296.8117917057 / (299)) + (82715.35147252497 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2132880330838968 = (21295.659589124665 - 16753.55024213604) / 21295.659589124665 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	af6b1bd43f24d04cafff386bb2a1cd4510ecfe8c
Compare	97a51ccdb147992b80d427b83a77caa967d275b4
Diff	Diff

Improvements in LinqBenchmarks

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
Where00ForX - Duration of single invocation	5.08 secs	3.36 secs	0.66	0.04	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'LinqBenchmarks*'

Payloads

Baseline
Compare

Histogram

LinqBenchmarks.Where00ForX

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 3.356687443461538 < 4.810977464050549.
IsChangePoint: Marked as a change because one of 12/7/2022 5:17:51 AM, 12/16/2022 11:20:37 AM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 6.9897720342310885 (T) = (0 -3420487386.748944) / Math.Sqrt((3.7028433324805754E+17 / (299)) + (11827448421648606 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.08651930985911219 = (3744455053.8023915 - 3420487386.748944) / 3744455053.8023915 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

[lewing]

Looks like dotnet/runtime#79686 cc @vargaz