Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Perf] Linux/x64: 87 Regressions on 1/15/2023 6:48:05 PM #11791

Closed
performanceautofiler bot opened this issue Jan 17, 2023 · 5 comments
Closed

[Perf] Linux/x64: 87 Regressions on 1/15/2023 6:48:05 PM #11791

performanceautofiler bot opened this issue Jan 17, 2023 · 5 comments
Labels
arch-x64 branch-refs/heads/main kind-micro_mono mono-interpreter os-linux perf-regression runtime-mono

Comments

@performanceautofiler
Copy link

performanceautofiler bot commented Jan 17, 2023 •
edited
Loading

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 2eca2d319bf8d1247edf38facfdd501e0e2f0030
Compare 51a8dd5323721b363e61069575511f783e7ea6d3
Diff Diff

Regressions in MicroBenchmarks.Serializers.Json_ToStream<CollectionsOfPrimitives>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
SystemTextJson_Reflection_ - Duration of single invocation 2.18 ms 2.52 ms 1.16 0.02 False
SystemTextJson_SourceGen_ - Duration of single invocation 2.30 ms 2.62 ms 1.14 0.02 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Json_ToStream&lt;CollectionsOfPrimitives&gt;*'

Payloads

Baseline
Compare

Histogram

MicroBenchmarks.Serializers.Json_ToStream<CollectionsOfPrimitives>.SystemTextJson_Reflection_

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.523136554112554 > 2.2811432688132594.
IsChangePoint: Marked as a change because one of 12/3/2022 2:55:34 AM, 1/15/2023 4:08:29 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -30.515269836987443 (T) = (0 -2503734.258843636) / Math.Sqrt((909067361.7575506 / (46)) + (894140897.2584221 / (9))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (46) + (9) - 2, .025) and -0.15342448435714337 = (2170696.2985436213 - 2503734.258843636) / 2170696.2985436213 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### MicroBenchmarks.Serializers.Json_ToStream&lt;CollectionsOfPrimitives&gt;.SystemTextJson_SourceGen_

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.6201835186170213 > 2.413639889179864.
IsChangePoint: Marked as a change because one of 12/3/2022 2:55:34 AM, 1/15/2023 4:08:29 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -30.330328773247228 (T) = (0 -2647235.154946631) / Math.Sqrt((678985290.0401511 / (45)) + (992600892.4841977 / (9))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (45) + (9) - 2, .025) and -0.1471711573938167 = (2307620.0424710046 - 2647235.154946631) / 2307620.0424710046 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS ubuntu 18.04
Baseline 2eca2d319bf8d1247edf38facfdd501e0e2f0030
Compare 51a8dd5323721b363e61069575511f783e7ea6d3
Diff Diff

Regressions in System.Text.Json.Serialization.Tests.WriteJson<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
SerializeToWriter - Duration of single invocation 761.49 ns 904.63 ns 1.19 0.11 False
SerializeToUtf8Bytes - Duration of single invocation 1.52 μs 1.71 μs 1.13 0.04 False
SerializeToStream - Duration of single invocation 3.58 μs 4.05 μs 1.13 0.05 False
SerializeToWriter - Duration of single invocation 746.43 ns 901.64 ns 1.21 0.07 False
SerializeToUtf8Bytes - Duration of single invocation 1.53 μs 1.73 μs 1.13 0.05 False
SerializeToString - Duration of single invocation 1.97 μs 2.16 μs 1.10 0.09 False
SerializeToString - Duration of single invocation 1.95 μs 2.14 μs 1.10 0.16 False
SerializeToStream - Duration of single invocation 3.71 μs 4.12 μs 1.11 0.06 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.WriteJson&lt;Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Serialization.Tests.WriteJson<Int32>.SerializeToWriter(Mode: Reflection)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 904.6283684899622 > 799.1313875921929.
IsChangePoint: Marked as a change because one of 1/15/2023 4:08:29 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -26.71345796111925 (T) = (0 -922.3618495811716) / Math.Sqrt((296.078777259918 / (45)) + (283.11852946053534 / (9))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (45) + (9) - 2, .025) and -0.2174649006190831 = (757.6085759122492 - 922.3618495811716) / 757.6085759122492 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Serialization.Tests.WriteJson&lt;Int32&gt;.SerializeToUtf8Bytes(Mode: SourceGen)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.7077130188748806 > 1.6000431347516704.
IsChangePoint: Marked as a change because one of 12/2/2022 5:21:30 AM, 1/15/2023 4:08:29 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -17.383016966852708 (T) = (0 -1735.759778517023) / Math.Sqrt((433.3908823195409 / (45)) + (1031.2585608739446 / (9))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (45) + (9) - 2, .025) and -0.12563820330219513 = (1542.0228039746366 - 1735.759778517023) / 1542.0228039746366 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Serialization.Tests.WriteJson&lt;Int32&gt;.SerializeToStream(Mode: Reflection)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 4.047583846070811 > 3.8027521880381108.
IsChangePoint: Marked as a change because one of 12/2/2022 5:21:30 AM, 12/9/2022 1:52:32 PM, 1/3/2023 9:28:36 AM, 1/15/2023 4:08:29 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -18.289820978090763 (T) = (0 -4066.564054636928) / Math.Sqrt((4581.172773041748 / (45)) + (2772.91955395299 / (9))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (45) + (9) - 2, .025) and -0.10018170023325208 = (3696.265856607837 - 4066.564054636928) / 3696.265856607837 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Serialization.Tests.WriteJson&lt;Int32&gt;.SerializeToWriter(Mode: SourceGen)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 901.644088226202 > 790.6399957692029.
IsChangePoint: Marked as a change because one of 12/5/2022 3:49:45 AM, 1/15/2023 4:08:29 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -29.523458884005198 (T) = (0 -909.9484798025373) / Math.Sqrt((277.98081775741554 / (45)) + (205.08906179291955 / (9))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (45) + (9) - 2, .025) and -0.2115592654293387 = (751.0556897768266 - 909.9484798025373) / 751.0556897768266 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Serialization.Tests.WriteJson&lt;Int32&gt;.SerializeToUtf8Bytes(Mode: Reflection)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.727803892383734 > 1.6401029190619494.
IsChangePoint: Marked as a change because one of 12/2/2022 5:21:30 AM, 1/15/2023 4:08:29 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -18.85265802053377 (T) = (0 -1741.110210098418) / Math.Sqrt((471.6123992707908 / (46)) + (854.4298494554378 / (9))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (46) + (9) - 2, .025) and -0.12492690716860036 = (1547.7540798457105 - 1741.110210098418) / 1547.7540798457105 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Serialization.Tests.WriteJson&lt;Int32&gt;.SerializeToString(Mode: SourceGen)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.163096228198349 > 2.0637061939233643.
IsChangePoint: Marked as a change because one of 12/6/2022 11:39:16 PM, 12/21/2022 9:23:17 AM, 1/15/2023 4:08:29 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -14.7715971177912 (T) = (0 -2210.5679536493803) / Math.Sqrt((1684.5663837095658 / (45)) + (2181.395484177099 / (9))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (45) + (9) - 2, .025) and -0.12584482256140547 = (1963.4748140690697 - 2210.5679536493803) / 1963.4748140690697 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Serialization.Tests.WriteJson&lt;Int32&gt;.SerializeToString(Mode: Reflection)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.1411251669891285 > 2.0529732293576983.
IsChangePoint: Marked as a change because one of 12/7/2022 5:17:51 AM, 12/21/2022 9:23:17 AM, 1/15/2023 4:08:29 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -8.678687942680812 (T) = (0 -2255.1371662730594) / Math.Sqrt((2696.433355016894 / (45)) + (10190.376109433575 / (9))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (45) + (9) - 2, .025) and -0.15324018155057692 = (1955.4791814840673 - 2255.1371662730594) / 1955.4791814840673 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Serialization.Tests.WriteJson&lt;Int32&gt;.SerializeToStream(Mode: SourceGen)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 4.124724010527719 > 3.9339749129236328.
IsChangePoint: Marked as a change because one of 12/2/2022 5:21:30 AM, 1/15/2023 4:08:29 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -16.05950642622807 (T) = (0 -4090.1466313631736) / Math.Sqrt((5269.790543566284 / (45)) + (3837.0657959364285 / (9))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (45) + (9) - 2, .025) and -0.10075398191454123 = (3715.768190317315 - 4090.1466313631736) / 3715.768190317315 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 2eca2d319bf8d1247edf38facfdd501e0e2f0030
Compare 51a8dd5323721b363e61069575511f783e7ea6d3
Diff Diff

Regressions in System.Buffers.Text.Tests.Utf8FormatterTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
FormatterInt32 - Duration of single invocation 187.91 ns 300.14 ns 1.60 0.30 False
FormatterUInt64 - Duration of single invocation 395.37 ns 516.60 ns 1.31 0.09 False
FormatterInt64 - Duration of single invocation 391.81 ns 510.68 ns 1.30 0.07 False
FormatterUInt32 - Duration of single invocation 243.55 ns 369.81 ns 1.52 0.14 False
FormatterInt32 - Duration of single invocation 250.75 ns 384.42 ns 1.53 0.15 False
FormatterUInt64 - Duration of single invocation 176.50 ns 296.93 ns 1.68 0.28 False
FormatterDouble - Duration of single invocation 4.33 μs 5.38 μs 1.24 0.04 False
FormatterInt64 - Duration of single invocation 183.42 ns 315.39 ns 1.72 0.26 False
FormatterUInt32 - Duration of single invocation 196.17 ns 299.75 ns 1.53 0.28 False
FormatterDouble - Duration of single invocation 1.97 μs 2.74 μs 1.39 0.13 False
FormatterInt32 - Duration of single invocation 254.41 ns 375.27 ns 1.48 0.17 False
FormatterDouble - Duration of single invocation 4.49 μs 5.32 μs 1.19 0.03 False
FormatterInt64 - Duration of single invocation 394.32 ns 511.12 ns 1.30 0.12 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Buffers.Text.Tests.Utf8FormatterTests*'

Payloads

Baseline
Compare

Histogram

System.Buffers.Text.Tests.Utf8FormatterTests.FormatterInt32(value: 12345)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 300.13901881145637 > 194.7914285371582.
IsChangePoint: Marked as a change because one of 1/15/2023 4:08:29 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -40.48842766286809 (T) = (0 -307.4718197978181) / Math.Sqrt((39.903786195577105 / (46)) + (69.91039086074301 / (9))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (46) + (9) - 2, .025) and -0.6312104330795012 = (188.4930439154644 - 307.4718197978181) / 188.4930439154644 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Buffers.Text.Tests.Utf8FormatterTests.FormatterUInt64(value: 18446744073709551615)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 516.5956022962062 > 420.3175200870668.
IsChangePoint: Marked as a change because one of 1/15/2023 4:08:29 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -61.66807506610332 (T) = (0 -519.8806140422433) / Math.Sqrt((65.1059319510828 / (45)) + (24.520810424622468 / (9))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (45) + (9) - 2, .025) and -0.3197260166796533 = (393.93071552096086 - 519.8806140422433) / 393.93071552096086 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Buffers.Text.Tests.Utf8FormatterTests.FormatterInt64(value: -9223372036854775808)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 510.6847658628154 > 410.32092302939157.
IsChangePoint: Marked as a change because one of 1/15/2023 4:08:29 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -31.773602071511142 (T) = (0 -516.7538869053703) / Math.Sqrt((87.23188313461907 / (45)) + (124.82641808853224 / (9))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (45) + (9) - 2, .025) and -0.32357102830368173 = (390.4239937675681 - 516.7538869053703) / 390.4239937675681 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Buffers.Text.Tests.Utf8FormatterTests.FormatterUInt32(value: 4294967295)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 369.80869211214525 > 258.1820279257068.
IsChangePoint: Marked as a change because one of 1/15/2023 4:08:29 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -116.98620731925972 (T) = (0 -370.129247601111) / Math.Sqrt((25.954123882517194 / (45)) + (4.972245481149466 / (9))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (45) + (9) - 2, .025) and -0.5057314952151186 = (245.81357883347715 - 370.129247601111) / 245.81357883347715 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Buffers.Text.Tests.Utf8FormatterTests.FormatterInt32(value: -2147483648)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 384.42462798009274 > 266.16864113642805.
IsChangePoint: Marked as a change because one of 1/15/2023 4:08:29 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -53.82529540048504 (T) = (0 -379.80582932086736) / Math.Sqrt((47.587291618283004 / (46)) + (39.76225777796431 / (9))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (46) + (9) - 2, .025) and -0.4945913056815261 = (254.12019183911806 - 379.80582932086736) / 254.12019183911806 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Buffers.Text.Tests.Utf8FormatterTests.FormatterUInt64(value: 12345)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 296.927506557866 > 185.8136244761951.
IsChangePoint: Marked as a change because one of 1/15/2023 4:08:29 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -91.1376452271774 (T) = (0 -296.5013934423432) / Math.Sqrt((19.08915275883294 / (45)) + (11.081941569615156 / (9))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (45) + (9) - 2, .025) and -0.6542426110924704 = (179.23694593172894 - 296.5013934423432) / 179.23694593172894 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Buffers.Text.Tests.Utf8FormatterTests.FormatterDouble(value: -1.7976931348623157E+308)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 5.37655200204385 > 4.618404497391284.
IsChangePoint: Marked as a change because one of 1/13/2023 11:01:49 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -54.3996412180294 (T) = (0 -5340.073718937481) / Math.Sqrt((4805.341098799737 / (33)) + (3288.0413604983505 / (21))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (21) - 2, .025) and -0.2151964918319457 = (4394.411730803434 - 5340.073718937481) / 4394.411730803434 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Buffers.Text.Tests.Utf8FormatterTests.FormatterInt64(value: 12345)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 315.38768709622394 > 196.183557471691.
IsChangePoint: Marked as a change because one of 1/15/2023 4:08:29 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -38.86982677179527 (T) = (0 -308.80741495069327) / Math.Sqrt((42.0873248369263 / (45)) + (78.2747519676946 / (9))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (45) + (9) - 2, .025) and -0.6411071555662473 = (188.1701715231035 - 308.80741495069327) / 188.1701715231035 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Buffers.Text.Tests.Utf8FormatterTests.FormatterUInt32(value: 12345)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 299.75396409218683 > 185.04903017807413.
IsChangePoint: Marked as a change because one of 1/15/2023 4:08:29 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -55.194206203185416 (T) = (0 -299.67452871168723) / Math.Sqrt((48.14657078031565 / (45)) + (32.682639544346266 / (9))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (45) + (9) - 2, .025) and -0.6648637191625266 = (179.99943494620206 - 299.67452871168723) / 179.99943494620206 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Buffers.Text.Tests.Utf8FormatterTests.FormatterDouble(value: 12345)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.737689129840612 > 2.010693632158721.
IsChangePoint: Marked as a change because one of 1/13/2023 11:01:49 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -50.002018350868624 (T) = (0 -2812.4491114188136) / Math.Sqrt((2530.7506385221222 / (34)) + (5177.321215022543 / (21))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (34) + (21) - 2, .025) and -0.4673950793530791 = (1916.6270563335402 - 2812.4491114188136) / 1916.6270563335402 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Buffers.Text.Tests.Utf8FormatterTests.FormatterInt32(value: 2147483647)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 375.26682118615867 > 265.9169163345484.
IsChangePoint: Marked as a change because one of 1/15/2023 4:08:29 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -47.5829885468184 (T) = (0 -376.9539560931971) / Math.Sqrt((30.196427476896226 / (46)) + (54.58905293436107 / (9))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (46) + (9) - 2, .025) and -0.486485548932253 = (253.58736676852612 - 376.9539560931971) / 253.58736676852612 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Buffers.Text.Tests.Utf8FormatterTests.FormatterDouble(value: 1.7976931348623157E+308)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 5.319388384010933 > 4.659275727182896.
IsChangePoint: Marked as a change because one of 1/13/2023 11:01:49 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -49.901765447853755 (T) = (0 -5422.643626239114) / Math.Sqrt((4128.980930322688 / (33)) + (5170.574670329586 / (21))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (21) - 2, .025) and -0.21555882202161833 = (4461.029386649192 - 5422.643626239114) / 4461.029386649192 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Buffers.Text.Tests.Utf8FormatterTests.FormatterInt64(value: 9223372036854775807)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 511.1195649504225 > 412.53430027242376.
IsChangePoint: Marked as a change because one of 1/15/2023 4:08:29 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -53.25681305858769 (T) = (0 -511.63961729551716) / Math.Sqrt((58.68112916225676 / (46)) + (36.70191363781019 / (9))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (46) + (9) - 2, .025) and -0.31725314425977713 = (388.41404138991834 - 511.63961729551716) / 388.41404138991834 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 2eca2d319bf8d1247edf38facfdd501e0e2f0030
Compare 51a8dd5323721b363e61069575511f783e7ea6d3
Diff Diff

Regressions in System.Tests.Perf_UInt32

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
TryFormat - Duration of single invocation 156.13 ns 270.53 ns 1.73 0.18 False
TryFormat - Duration of single invocation 106.01 ns 224.48 ns 2.12 0.16 False
TryFormat - Duration of single invocation 191.06 ns 322.33 ns 1.69 0.14 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_UInt32*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_UInt32.TryFormat(value: 12345)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 270.53045834868584 > 159.02652491455487.
IsChangePoint: Marked as a change because one of 1/15/2023 4:08:29 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -45.79337044720575 (T) = (0 -276.83593573928505) / Math.Sqrt((28.397496778494396 / (46)) + (60.3705553666342 / (9))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (46) + (9) - 2, .025) and -0.8106172495149493 = (152.8958899587681 - 276.83593573928505) / 152.8958899587681 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_UInt32.TryFormat(value: 0)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 224.48334518154377 > 110.71290930979134.
IsChangePoint: Marked as a change because one of 1/15/2023 4:08:29 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -65.49575749793803 (T) = (0 -223.44149183179252) / Math.Sqrt((9.276896955258492 / (45)) + (27.547924828388993 / (9))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (45) + (9) - 2, .025) and -1.126832967729632 = (105.05831685988653 - 223.44149183179252) / 105.05831685988653 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_UInt32.TryFormat(value: 4294967295)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 322.3290129919583 > 203.17042631592605.
IsChangePoint: Marked as a change because one of 1/15/2023 4:08:29 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -68.38966542476774 (T) = (0 -323.0334086112086) / Math.Sqrt((21.214177385233114 / (46)) + (27.04150233293193 / (9))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (46) + (9) - 2, .025) and -0.6505304953215757 = (195.71489865037086 - 323.0334086112086) / 195.71489865037086 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
@performanceautofiler
Copy link
Author

performanceautofiler bot commented Jan 17, 2023 •
edited
Loading

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 2eca2d319bf8d1247edf38facfdd501e0e2f0030
Compare 51a8dd5323721b363e61069575511f783e7ea6d3
Diff Diff

Regressions in System.Text.Json.Tests.Perf_Basic

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
WriteBasicUtf16 - Duration of single invocation 29.85 ms 41.81 ms 1.40 0.09 False
WriteBasicUtf16 - Duration of single invocation 37.76 ms 49.99 ms 1.32 0.06 False
WriteBasicUtf8 - Duration of single invocation 35.63 ms 48.47 ms 1.36 0.08 False
WriteBasicUtf8 - Duration of single invocation 27.77 ms 42.05 ms 1.51 0.09 False
WriteBasicUtf16 - Duration of single invocation 21.89 μs 24.41 μs 1.12 0.01 False
WriteBasicUtf16 - Duration of single invocation 19.60 μs 22.09 μs 1.13 0.01 False
WriteBasicUtf8 - Duration of single invocation 22.74 μs 25.39 μs 1.12 0.01 False
WriteBasicUtf8 - Duration of single invocation 26.77 μs 29.62 μs 1.11 0.01 False
WriteBasicUtf8 - Duration of single invocation 24.33 μs 26.88 μs 1.10 0.01 False
WriteBasicUtf8 - Duration of single invocation 20.19 μs 22.99 μs 1.14 0.01 False
WriteBasicUtf16 - Duration of single invocation 23.99 μs 25.91 μs 1.08 0.01 False
WriteBasicUtf16 - Duration of single invocation 37.01 ms 49.62 ms 1.34 0.10 False
WriteBasicUtf8 - Duration of single invocation 37.81 ms 50.62 ms 1.34 0.05 False
WriteBasicUtf8 - Duration of single invocation 29.32 ms 42.66 ms 1.46 0.11 False
WriteBasicUtf16 - Duration of single invocation 27.95 ms 40.32 ms 1.44 0.08 False
WriteBasicUtf16 - Duration of single invocation 17.03 μs 19.53 μs 1.15 0.02 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Tests.Perf_Basic*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Tests.Perf_Basic.WriteBasicUtf16(Formatted: False, SkipValidation: False, DataSize: 100000)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 41.80860214444444 > 30.946623144230767.
IsChangePoint: Marked as a change because one of 12/19/2022 2:44:31 PM, 1/15/2023 4:08:29 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -53.96401654575514 (T) = (0 -42796780.87384345) / Math.Sqrt((307472340851.87494 / (46)) + (508153556622.35425 / (9))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (46) + (9) - 2, .025) and -0.4638250294326056 = (29236268.005631752 - 42796780.87384345) / 29236268.005631752 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Basic.WriteBasicUtf16(Formatted: True, SkipValidation: False, DataSize: 100000)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 49.986096145833336 > 39.05879781222527.
IsChangePoint: Marked as a change because one of 1/15/2023 4:08:29 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -37.58460667012675 (T) = (0 -50461912.33839031) / Math.Sqrt((528246361986.77985 / (45)) + (934902836761.801 / (9))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (45) + (9) - 2, .025) and -0.33914411371054154 = (37682211.960420676 - 50461912.33839031) / 37682211.960420676 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Basic.WriteBasicUtf8(Formatted: True, SkipValidation: True, DataSize: 100000)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 48.472400799999996 > 37.36268981991759.
IsChangePoint: Marked as a change because one of 1/15/2023 4:08:29 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -49.50394443847177 (T) = (0 -49183954.016707376) / Math.Sqrt((636835335321.7612 / (46)) + (527467013586.3407 / (9))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (46) + (9) - 2, .025) and -0.3715910844941685 = (35859050.53826302 - 49183954.016707376) / 35859050.53826302 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Basic.WriteBasicUtf8(Formatted: False, SkipValidation: True, DataSize: 100000)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 42.0518806875 > 29.13620468541667.
IsChangePoint: Marked as a change because one of 12/17/2022 3:28:28 PM, 1/15/2023 4:08:29 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -46.550256651077255 (T) = (0 -40599249.93615012) / Math.Sqrt((239635982408.91354 / (46)) + (640966497000.8523 / (9))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (46) + (9) - 2, .025) and -0.4640834557516257 = (27730147.33323889 - 40599249.93615012) / 27730147.33323889 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Basic.WriteBasicUtf16(Formatted: True, SkipValidation: True, DataSize: 10)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 24.407882848084885 > 22.85752268649787.
IsChangePoint: Marked as a change because one of 10/25/2022 8:17:52 AM, 12/17/2022 10:44:50 PM, 1/15/2023 4:08:29 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -26.528942496273743 (T) = (0 -24037.805291552766) / Math.Sqrt((49996.511468262186 / (45)) + (68549.24859802547 / (9))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (45) + (9) - 2, .025) and -0.11495571802175635 = (21559.425996040958 - 24037.805291552766) / 21559.425996040958 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Basic.WriteBasicUtf16(Formatted: False, SkipValidation: False, DataSize: 10)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 22.093921966818645 > 20.549367400664377.
IsChangePoint: Marked as a change because one of 10/25/2022 8:17:52 AM, 12/17/2022 10:44:50 PM, 1/15/2023 4:08:29 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -34.1438503253031 (T) = (0 -22071.305933956195) / Math.Sqrt((62189.58084730611 / (45)) + (36688.2760079179 / (9))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (45) + (9) - 2, .025) and -0.1290415881628519 = (19548.709423423516 - 22071.305933956195) / 19548.709423423516 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Basic.WriteBasicUtf8(Formatted: False, SkipValidation: False, DataSize: 10)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 25.39113368176539 > 23.971615888512503.
IsChangePoint: Marked as a change because one of 10/25/2022 8:17:52 AM, 12/5/2022 11:02:29 AM, 1/3/2023 9:28:36 AM, 1/15/2023 4:08:29 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -23.952056570194593 (T) = (0 -25336.455833020616) / Math.Sqrt((50425.56002610502 / (45)) + (78279.10659193364 / (9))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (45) + (9) - 2, .025) and -0.10335439261827506 = (22963.116839456143 - 25336.455833020616) / 22963.116839456143 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Basic.WriteBasicUtf8(Formatted: True, SkipValidation: False, DataSize: 10)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 29.621382638888896 > 28.24073517793433.
IsChangePoint: Marked as a change because one of 10/25/2022 8:17:52 AM, 12/14/2022 5:20:21 PM, 1/3/2023 9:28:36 AM, 1/15/2023 4:08:29 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -21.79506734271396 (T) = (0 -29754.891630305883) / Math.Sqrt((55067.94653385542 / (45)) + (138613.01875014228 / (9))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (45) + (9) - 2, .025) and -0.10429622531941794 = (26944.664799247388 - 29754.891630305883) / 26944.664799247388 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Basic.WriteBasicUtf8(Formatted: True, SkipValidation: True, DataSize: 10)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 26.877239770628073 > 25.84014419730134.
IsChangePoint: Marked as a change because one of 10/25/2022 8:17:52 AM, 12/14/2022 8:02:51 PM, 1/3/2023 9:28:36 AM, 1/15/2023 4:08:29 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -29.18544898451374 (T) = (0 -27082.168364727455) / Math.Sqrt((78074.1650742668 / (46)) + (41972.84418278366 / (9))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (46) + (9) - 2, .025) and -0.09403130179119758 = (24754.473039653712 - 27082.168364727455) / 24754.473039653712 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Basic.WriteBasicUtf8(Formatted: False, SkipValidation: True, DataSize: 10)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 22.989271109474714 > 21.5017314197467.
IsChangePoint: Marked as a change because one of 10/25/2022 8:17:52 AM, 12/14/2022 5:20:21 PM, 1/3/2023 5:57:54 PM, 1/15/2023 4:08:29 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -25.46289852184249 (T) = (0 -22924.252794117372) / Math.Sqrt((59404.52538064009 / (45)) + (65261.081580053535 / (9))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (45) + (9) - 2, .025) and -0.11462095092615864 = (20566.859769744322 - 22924.252794117372) / 20566.859769744322 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Basic.WriteBasicUtf16(Formatted: True, SkipValidation: False, DataSize: 10)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 25.91161033590572 > 25.148016916605282.
IsChangePoint: Marked as a change because one of 10/25/2022 8:17:52 AM, 12/17/2022 10:44:50 PM, 1/15/2023 4:08:29 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -32.28876889174867 (T) = (0 -26238.94433384536) / Math.Sqrt((69597.23564296568 / (45)) + (36649.37845696184 / (9))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (45) + (9) - 2, .025) and -0.10161431097059082 = (23818.630597424988 - 26238.94433384536) / 23818.630597424988 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Basic.WriteBasicUtf16(Formatted: True, SkipValidation: True, DataSize: 100000)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 49.619051980769235 > 38.660965095.
IsChangePoint: Marked as a change because one of 1/15/2023 4:08:29 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -63.49341412247219 (T) = (0 -49194294.649608254) / Math.Sqrt((632408878851.6664 / (45)) + (254381237999.3393 / (9))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (45) + (9) - 2, .025) and -0.36148462325394043 = (36132831.62319834 - 49194294.649608254) / 36132831.62319834 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Basic.WriteBasicUtf8(Formatted: True, SkipValidation: False, DataSize: 100000)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 50.61937751923077 > 39.56848768410257.
IsChangePoint: Marked as a change because one of 1/15/2023 4:08:29 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -44.977870160717984 (T) = (0 -50766811.87988401) / Math.Sqrt((685601094751.7382 / (45)) + (638803323081.2524 / (9))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (45) + (9) - 2, .025) and -0.3516071366744328 = (37560331.32881602 - 50766811.87988401) / 37560331.32881602 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Basic.WriteBasicUtf8(Formatted: False, SkipValidation: False, DataSize: 100000)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 42.66490376785714 > 30.796734079326924.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 1/15/2023 4:08:29 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -45.062712012160794 (T) = (0 -42610039.5163241) / Math.Sqrt((443814388447.62537 / (46)) + (677478941170.8932 / (9))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (46) + (9) - 2, .025) and -0.4454850458889092 = (29478021.67688343 - 42610039.5163241) / 29478021.67688343 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Basic.WriteBasicUtf16(Formatted: False, SkipValidation: True, DataSize: 100000)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 40.32185205555556 > 28.91438024820513.
IsChangePoint: Marked as a change because one of 12/19/2022 10:34:07 PM, 1/15/2023 4:08:29 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -42.36106572199991 (T) = (0 -41063880.2940171) / Math.Sqrt((528977940203.9959 / (45)) + (787197812672.3303 / (9))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (45) + (9) - 2, .025) and -0.4813606196786472 = (27720380.67471047 - 41063880.2940171) / 27720380.67471047 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Basic.WriteBasicUtf16(Formatted: False, SkipValidation: True, DataSize: 10)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 19.529157627935113 > 18.249395244913604.
IsChangePoint: Marked as a change because one of 10/25/2022 8:17:52 AM, 12/17/2022 10:44:50 PM, 1/15/2023 4:08:29 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -30.599687961399646 (T) = (0 -19443.16199283701) / Math.Sqrt((52290.40767955562 / (45)) + (33266.603688684256 / (9))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (45) + (9) - 2, .025) and -0.12321229375658126 = (17310.318005698988 - 19443.16199283701) / 17310.318005698988 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS ubuntu 18.04
Baseline 2eca2d319bf8d1247edf38facfdd501e0e2f0030
Compare 51a8dd5323721b363e61069575511f783e7ea6d3
Diff Diff

Regressions in System.Tests.Perf_SByte

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ToString - Duration of single invocation 212.71 ns 342.53 ns 1.61 0.17 False
ToString - Duration of single invocation 148.59 ns 277.08 ns 1.86 0.13 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_SByte*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_SByte.ToString(value: -128)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 342.52870732231844 > 221.86902123824206.
IsChangePoint: Marked as a change because one of 12/8/2022 9:11:55 AM, 1/15/2023 4:08:29 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -47.43792526649656 (T) = (0 -334.68133286664903) / Math.Sqrt((34.08488454140149 / (46)) + (55.67472489399774 / (9))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (46) + (9) - 2, .025) and -0.5950268284018988 = (209.8280272827608 - 334.68133286664903) / 209.8280272827608 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_SByte.ToString(value: 127)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 277.0791910929543 > 155.65080835071242.
IsChangePoint: Marked as a change because one of 12/1/2022 11:48:13 PM, 12/8/2022 9:11:55 AM, 1/15/2023 4:08:29 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -93.58239074886355 (T) = (0 -271.6679596193516) / Math.Sqrt((3.2866584603361813 / (45)) + (14.970728033260107 / (9))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (45) + (9) - 2, .025) and -0.8312604127736486 = (148.3502606862342 - 271.6679596193516) / 148.3502606862342 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 2eca2d319bf8d1247edf38facfdd501e0e2f0030
Compare 51a8dd5323721b363e61069575511f783e7ea6d3
Diff Diff

Regressions in System.Tests.Perf_Version

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
TryFormat3 - Duration of single invocation 489.85 ns 836.05 ns 1.71 0.11 False
ToStringL - Duration of single invocation 1.15 μs 1.68 μs 1.46 0.05 False
TryFormat4 - Duration of single invocation 635.39 ns 1.10 μs 1.74 0.14 False
ToString3 - Duration of single invocation 634.48 ns 1.01 μs 1.59 0.05 False
ToString2 - Duration of single invocation 484.25 ns 751.92 ns 1.55 0.07 False
ToString4 - Duration of single invocation 777.82 ns 1.28 μs 1.64 0.07 False
TryFormatL - Duration of single invocation 996.62 ns 1.55 μs 1.55 0.10 False
TryFormat2 - Duration of single invocation 369.59 ns 566.82 ns 1.53 0.16 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Version*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Version.TryFormat3

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 836.0526230738802 > 508.723535091425.
IsChangePoint: Marked as a change because one of 1/15/2023 4:08:29 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -55.32641489676398 (T) = (0 -846.8522332539115) / Math.Sqrt((197.2569415891052 / (45)) + (348.208888711257 / (9))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (45) + (9) - 2, .025) and -0.7506231532735189 = (483.7433068735374 - 846.8522332539115) / 483.7433068735374 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Version.ToStringL

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.6817820983168248 > 1.2490776068521272.
IsChangePoint: Marked as a change because one of 12/8/2022 9:11:55 AM, 1/15/2023 4:08:29 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -64.1870720481898 (T) = (0 -1713.3337109905224) / Math.Sqrt((594.4004888801196 / (45)) + (486.2902497323866 / (9))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (45) + (9) - 2, .025) and -0.4434209014326776 = (1186.995220375388 - 1713.3337109905224) / 1186.995220375388 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Version.TryFormat4

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.1043926511826045 > 664.1889377287941.
IsChangePoint: Marked as a change because one of 1/15/2023 4:08:29 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -87.27155191918386 (T) = (0 -1109.229179902115) / Math.Sqrt((430.2215186256098 / (46)) + (178.42880304718975 / (9))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (46) + (9) - 2, .025) and -0.7391029721864476 = (637.8168502050007 - 1109.229179902115) / 637.8168502050007 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Version.ToString3

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.0070280461263514 > 666.9641267647421.
IsChangePoint: Marked as a change because one of 12/8/2022 9:11:55 AM, 1/15/2023 4:08:29 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -136.73543659458315 (T) = (0 -1004.9742680258945) / Math.Sqrt((146.70434005759887 / (45)) + (39.29143597103704 / (9))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (45) + (9) - 2, .025) and -0.6018561155068093 = (627.3811101366816 - 1004.9742680258945) / 627.3811101366816 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Version.ToString2

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 751.9240469214229 > 507.2789842281721.
IsChangePoint: Marked as a change because one of 12/8/2022 9:11:55 AM, 1/15/2023 4:08:29 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -66.78481753770906 (T) = (0 -735.3217793255749) / Math.Sqrt((63.8843934663227 / (46)) + (119.11409481214343 / (9))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (46) + (9) - 2, .025) and -0.5321425433869561 = (479.9303971418166 - 735.3217793255749) / 479.9303971418166 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Version.ToString4

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.2776312198666557 > 829.2567121255537.
IsChangePoint: Marked as a change because one of 12/8/2022 9:11:55 AM, 1/15/2023 4:08:29 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -74.07263456967182 (T) = (0 -1285.213881691032) / Math.Sqrt((360.9115437950891 / (45)) + (332.27871417244734 / (9))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (45) + (9) - 2, .025) and -0.6296372425361002 = (788.6502886316808 - 1285.213881691032) / 788.6502886316808 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Version.TryFormatL

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.545043499053502 > 1042.6087092373468.
IsChangePoint: Marked as a change because one of 1/15/2023 4:08:29 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -63.348980108754084 (T) = (0 -1507.0465736547594) / Math.Sqrt((270.2527287548947 / (46)) + (513.8615517224698 / (9))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (46) + (9) - 2, .025) and -0.5005249587797185 = (1004.3462221916952 - 1507.0465736547594) / 1004.3462221916952 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Version.TryFormat2

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 566.8232246100873 > 347.3147805601256.
IsChangePoint: Marked as a change because one of 1/15/2023 4:08:29 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -75.7635560841846 (T) = (0 -571.7242948217595) / Math.Sqrt((92.24220779578224 / (45)) + (73.37170367852745 / (9))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (45) + (9) - 2, .025) and -0.733925184959136 = (329.72835263087404 - 571.7242948217595) / 329.72835263087404 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented Jan 17, 2023 •
edited
Loading

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 2eca2d319bf8d1247edf38facfdd501e0e2f0030
Compare 51a8dd5323721b363e61069575511f783e7ea6d3
Diff Diff

Regressions in System.Tests.Perf_Int16

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ToString - Duration of single invocation 239.76 ns 381.61 ns 1.59 0.09 False
ToString - Duration of single invocation 169.42 ns 290.71 ns 1.72 0.11 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Int16*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Int16.ToString(value: -32768)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 381.61237146066463 > 245.55799977053977.
IsChangePoint: Marked as a change because one of 12/8/2022 9:11:55 AM, 1/15/2023 4:08:29 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -31.262661253828398 (T) = (0 -354.15292703635197) / Math.Sqrt((24.968591859718035 / (46)) + (132.79005589498152 / (9))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (46) + (9) - 2, .025) and -0.5273154230814793 = (231.87936275914802 - 354.15292703635197) / 231.87936275914802 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Int16.ToString(value: 32767)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 290.7139922709111 > 179.11918678260594.
IsChangePoint: Marked as a change because one of 12/1/2022 11:48:13 PM, 12/8/2022 9:11:55 AM, 1/15/2023 4:08:29 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -121.77368562943496 (T) = (0 -292.08901755017155) / Math.Sqrt((9.692573038804555 / (46)) + (7.2287953712405715 / (9))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (46) + (9) - 2, .025) and -0.7235291816111316 = (169.47146626036857 - 292.08901755017155) / 169.47146626036857 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS ubuntu 18.04
Baseline 2eca2d319bf8d1247edf38facfdd501e0e2f0030
Compare 51a8dd5323721b363e61069575511f783e7ea6d3
Diff Diff

Regressions in System.Tests.Perf_Uri

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ParseAbsoluteUri - Duration of single invocation 11.29 μs 12.14 μs 1.08 0.01 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Uri*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Uri.ParseAbsoluteUri

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 12.138506816473663 > 11.815326323329762.
IsChangePoint: Marked as a change because one of 12/8/2022 5:23:10 PM, 1/3/2023 9:28:36 AM, 1/15/2023 4:08:29 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -38.909118640448156 (T) = (0 -12160.907210321902) / Math.Sqrt((8894.584783420829 / (45)) + (3446.0092411584988 / (9))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (45) + (9) - 2, .025) and -0.08353056330295663 = (11223.40949317707 - 12160.907210321902) / 11223.40949317707 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 2eca2d319bf8d1247edf38facfdd501e0e2f0030
Compare 51a8dd5323721b363e61069575511f783e7ea6d3
Diff Diff

Regressions in Microsoft.Extensions.Logging.Formatting

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
TwoArguments - Duration of single invocation 1.53 μs 1.70 μs 1.11 0.04 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Microsoft.Extensions.Logging.Formatting*'

Payloads

Baseline
Compare

Histogram

Microsoft.Extensions.Logging.Formatting.TwoArguments

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.7020375292381378 > 1.570655695583137.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 12/8/2022 9:11:55 AM, 1/15/2023 4:08:29 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -26.92460174575973 (T) = (0 -1700.5462609709393) / Math.Sqrt((549.4440164254357 / (45)) + (380.40488147820867 / (9))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (45) + (9) - 2, .025) and -0.13232393932448003 = (1501.8195782255107 - 1700.5462609709393) / 1501.8195782255107 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 2eca2d319bf8d1247edf38facfdd501e0e2f0030
Compare 51a8dd5323721b363e61069575511f783e7ea6d3
Diff Diff

Regressions in MicroBenchmarks.Serializers.Json_ToString<CollectionsOfPrimitives>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
SystemTextJson_SourceGen_ - Duration of single invocation 2.30 ms 2.65 ms 1.15 0.02 False
SystemTextJson_Reflection_ - Duration of single invocation 2.34 ms 2.71 ms 1.16 0.02 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Json_ToString&lt;CollectionsOfPrimitives&gt;*'

Payloads

Baseline
Compare

Histogram

MicroBenchmarks.Serializers.Json_ToString<CollectionsOfPrimitives>.SystemTextJson_SourceGen_

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.6505375442546577 > 2.4299331909613366.
IsChangePoint: Marked as a change because one of 12/21/2022 9:23:17 AM, 1/15/2023 4:08:29 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -49.7441695462595 (T) = (0 -2640667.8900939007) / Math.Sqrt((711856844.1615887 / (46)) + (254729103.15998143 / (9))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (46) + (9) - 2, .025) and -0.14238743763814374 = (2311534.4261430367 - 2640667.8900939007) / 2311534.4261430367 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### MicroBenchmarks.Serializers.Json_ToString&lt;CollectionsOfPrimitives&gt;.SystemTextJson_Reflection_

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.707961588405797 > 2.460156980873507.
IsChangePoint: Marked as a change because one of 12/21/2022 9:23:17 AM, 1/15/2023 4:08:29 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -25.02864551306796 (T) = (0 -2685404.879683627) / Math.Sqrt((572767916.7989439 / (46)) + (1500155472.357887 / (9))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (46) + (9) - 2, .025) and -0.14252236271794186 = (2350417.7837668997 - 2685404.879683627) / 2350417.7837668997 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 2eca2d319bf8d1247edf38facfdd501e0e2f0030
Compare 51a8dd5323721b363e61069575511f783e7ea6d3
Diff Diff

Regressions in System.Tests.Perf_Byte

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ToString - Duration of single invocation 140.77 ns 254.79 ns 1.81 0.16 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Byte*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Byte.ToString(value: 255)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 254.78779569421312 > 148.63273826366833.
IsChangePoint: Marked as a change because one of 12/1/2022 11:48:13 PM, 12/8/2022 9:11:55 AM, 1/15/2023 4:08:29 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -85.03231911627641 (T) = (0 -257.63515965274723) / Math.Sqrt((6.539732904512917 / (46)) + (15.686726907285717 / (9))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (46) + (9) - 2, .025) and -0.828684525885749 = (140.8855141528351 - 257.63515965274723) / 140.8855141528351 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented Jan 17, 2023 •
edited
Loading

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 2eca2d319bf8d1247edf38facfdd501e0e2f0030
Compare 51a8dd5323721b363e61069575511f783e7ea6d3
Diff Diff

Regressions in System.Tests.Perf_Int64

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
TryFormat - Duration of single invocation 425.16 ns 558.47 ns 1.31 0.09 False
ToString - Duration of single invocation 423.44 ns 565.07 ns 1.33 0.09 False
TryFormat - Duration of single invocation 337.97 ns 472.05 ns 1.40 0.11 False
TryFormat - Duration of single invocation 169.97 ns 286.54 ns 1.69 0.22 False
ToString - Duration of single invocation 175.20 ns 293.48 ns 1.68 0.07 False
ToString - Duration of single invocation 355.06 ns 475.04 ns 1.34 0.07 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Int64*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Int64.TryFormat(value: -9223372036854775808)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 558.4693839880755 > 450.1845924226269.
IsChangePoint: Marked as a change because one of 12/17/2022 3:28:28 PM, 1/15/2023 4:08:29 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -31.16493269503481 (T) = (0 -554.8981023111664) / Math.Sqrt((74.02601048783164 / (45)) + (138.4931635137288 / (9))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (45) + (9) - 2, .025) and -0.3017326068246602 = (426.27656355996135 - 554.8981023111664) / 426.27656355996135 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Int64.ToString(value: -9223372036854775808)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 565.0707006012922 > 455.0809412485792.
IsChangePoint: Marked as a change because one of 12/8/2022 9:11:55 AM, 1/15/2023 4:08:29 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -49.27183044638389 (T) = (0 -555.1523642197321) / Math.Sqrt((79.887690480479 / (45)) + (42.75764596460334 / (9))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (45) + (9) - 2, .025) and -0.29321409618136124 = (429.28109572807904 - 555.1523642197321) / 429.28109572807904 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Int64.TryFormat(value: 9223372036854775807)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 472.0483067975999 > 355.53797225878776.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 1/15/2023 4:08:29 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -48.26410766232176 (T) = (0 -471.3464628065736) / Math.Sqrt((58.22428739432844 / (45)) + (53.7725819775875 / (9))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (45) + (9) - 2, .025) and -0.3813372957582958 = (341.2247423232168 - 471.3464628065736) / 341.2247423232168 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Int64.TryFormat(value: 12345)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 286.5430049046879 > 179.97527251660685.
IsChangePoint: Marked as a change because one of 1/15/2023 4:08:29 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -56.482331308872794 (T) = (0 -292.87536741573786) / Math.Sqrt((26.819414779070655 / (45)) + (36.265785669485666 / (9))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (45) + (9) - 2, .025) and -0.7087372991459483 = (171.3987091884289 - 292.87536741573786) / 171.3987091884289 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Int64.ToString(value: 12345)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 293.47763042830167 > 187.69384431233684.
IsChangePoint: Marked as a change because one of 12/8/2022 9:11:55 AM, 1/15/2023 4:08:29 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -63.72579091399848 (T) = (0 -300.6789927911666) / Math.Sqrt((11.78053679285669 / (45)) + (30.896992193406938 / (9))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (45) + (9) - 2, .025) and -0.6874413352721701 = (178.18633839656985 - 300.6789927911666) / 178.18633839656985 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Int64.ToString(value: 9223372036854775807)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 475.03514047527386 > 370.30297213586886.
IsChangePoint: Marked as a change because one of 12/8/2022 9:11:55 AM, 1/15/2023 4:08:29 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -31.430348862478162 (T) = (0 -483.69532692268797) / Math.Sqrt((136.17002052768032 / (46)) + (124.54014821924943 / (9))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (46) + (9) - 2, .025) and -0.36299485677924814 = (354.87685409588283 - 483.69532692268797) / 354.87685409588283 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS ubuntu 18.04
Baseline 2eca2d319bf8d1247edf38facfdd501e0e2f0030
Compare 51a8dd5323721b363e61069575511f783e7ea6d3
Diff Diff

Regressions in Microsoft.Extensions.Logging.FormattingOverhead

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
FourArguments_DefineMessage - Duration of single invocation 2.56 μs 2.94 μs 1.15 0.01 False
TwoArguments - Duration of single invocation 1.84 μs 2.12 μs 1.16 0.04 False
TwoArguments_DefineMessage - Duration of single invocation 1.47 μs 1.70 μs 1.15 0.03 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Microsoft.Extensions.Logging.FormattingOverhead*'

Payloads

Baseline
Compare

Histogram

Microsoft.Extensions.Logging.FormattingOverhead.FourArguments_DefineMessage

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.9431430601661375 > 2.6868416632388996.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 12/3/2022 2:55:34 AM, 1/15/2023 4:08:29 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -36.864754464115485 (T) = (0 -2932.3621814645194) / Math.Sqrt((874.1340240380152 / (45)) + (654.9736510171642 / (9))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (45) + (9) - 2, .025) and -0.1372869912410907 = (2578.383648145409 - 2932.3621814645194) / 2578.383648145409 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### Microsoft.Extensions.Logging.FormattingOverhead.TwoArguments

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.1219441584688257 > 1.9348487579043185.
IsChangePoint: Marked as a change because one of 12/8/2022 9:11:55 AM, 1/15/2023 4:08:29 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -25.53758292123072 (T) = (0 -2082.6541305079586) / Math.Sqrt((1201.5204995662014 / (45)) + (490.83471546924886 / (9))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (45) + (9) - 2, .025) and -0.1242525095876285 = (1852.4789695794125 - 2082.6541305079586) / 1852.4789695794125 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### Microsoft.Extensions.Logging.FormattingOverhead.TwoArguments_DefineMessage

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.696493623094154 > 1.5512948719515327.
IsChangePoint: Marked as a change because one of 11/15/2022 5:29:43 PM, 12/8/2022 9:11:55 AM, 1/15/2023 4:08:29 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -20.601179916097767 (T) = (0 -1708.1589803165004) / Math.Sqrt((744.6680013479969 / (45)) + (817.177293438555 / (9))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (45) + (9) - 2, .025) and -0.1427992731134585 = (1494.7147941937064 - 1708.1589803165004) / 1494.7147941937064 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 2eca2d319bf8d1247edf38facfdd501e0e2f0030
Compare 51a8dd5323721b363e61069575511f783e7ea6d3
Diff Diff

Regressions in System.Tests.Perf_Int32

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
TryFormat - Duration of single invocation 194.15 ns 321.15 ns 1.65 0.17 False
TryFormat - Duration of single invocation 156.73 ns 297.14 ns 1.90 0.19 False
ToString - Duration of single invocation 291.57 ns 416.28 ns 1.43 0.09 False
TryFormat - Duration of single invocation 106.44 ns 219.61 ns 2.06 0.22 False
TryFormat - Duration of single invocation 296.22 ns 408.48 ns 1.38 0.13 False
ToString - Duration of single invocation 169.23 ns 295.83 ns 1.75 0.11 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Int32*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Int32.TryFormat(value: 2147483647)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 321.1467215520923 > 204.64887279460024.
IsChangePoint: Marked as a change because one of 1/15/2023 4:08:29 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -51.86125072888809 (T) = (0 -320.2517055441675) / Math.Sqrt((22.49792333769131 / (46)) + (45.23559757649962 / (9))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (46) + (9) - 2, .025) and -0.6137030016224222 = (198.45765002741237 - 320.2517055441675) / 198.45765002741237 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Int32.TryFormat(value: 12345)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 297.1400323169597 > 164.29648032219774.
IsChangePoint: Marked as a change because one of 1/15/2023 4:08:29 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -44.98333737902407 (T) = (0 -280.8706593672865) / Math.Sqrt((21.90695835256325 / (45)) + (66.16338274709958 / (9))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (45) + (9) - 2, .025) and -0.8128769076486964 = (154.93090467547304 - 280.8706593672865) / 154.93090467547304 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Int32.ToString(value: -2147483648)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 416.2752036587273 > 306.3284530005489.
IsChangePoint: Marked as a change because one of 12/8/2022 9:11:55 AM, 1/15/2023 4:08:29 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -51.31539165101203 (T) = (0 -413.5896433404533) / Math.Sqrt((42.0563158810269 / (46)) + (41.77274188457168 / (9))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (46) + (9) - 2, .025) and -0.4133211098999865 = (292.6367125229743 - 413.5896433404533) / 292.6367125229743 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Int32.TryFormat(value: 4)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 219.60621872348355 > 112.7542600152662.
IsChangePoint: Marked as a change because one of 1/15/2023 4:08:29 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -97.85235447257642 (T) = (0 -222.62392135980218) / Math.Sqrt((11.125777548892106 / (45)) + (10.174812324188498 / (9))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (45) + (9) - 2, .025) and -1.065805238907069 = (107.76617135388 - 222.62392135980218) / 107.76617135388 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Int32.TryFormat(value: -2147483648)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 408.48361860093667 > 311.3897775642138.
IsChangePoint: Marked as a change because one of 1/15/2023 4:08:29 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -77.50124384574308 (T) = (0 -412.11029296885107) / Math.Sqrt((54.83427165603804 / (46)) + (9.750568129645083 / (9))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (46) + (9) - 2, .025) and -0.3960237979986354 = (295.20291384692706 - 412.11029296885107) / 295.20291384692706 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Int32.ToString(value: 12345)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 295.8335469439574 > 178.29728154066183.
IsChangePoint: Marked as a change because one of 12/1/2022 11:48:13 PM, 12/8/2022 9:11:55 AM, 1/15/2023 4:08:29 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -36.593794062145626 (T) = (0 -292.929246958228) / Math.Sqrt((12.259867127283512 / (45)) + (100.05538016170946 / (9))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (45) + (9) - 2, .025) and -0.7289083705397881 = (169.43017452496434 - 292.929246958228) / 169.43017452496434 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 2eca2d319bf8d1247edf38facfdd501e0e2f0030
Compare 51a8dd5323721b363e61069575511f783e7ea6d3
Diff Diff

Regressions in System.Tests.Perf_UInt64

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ToString - Duration of single invocation 167.33 ns 291.00 ns 1.74 0.11 False
ToString - Duration of single invocation 341.19 ns 470.72 ns 1.38 0.07 False
TryFormat - Duration of single invocation 338.66 ns 466.16 ns 1.38 0.13 False
TryFormat - Duration of single invocation 165.80 ns 288.95 ns 1.74 0.24 False
TryFormat - Duration of single invocation 127.20 ns 244.26 ns 1.92 0.23 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_UInt64*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_UInt64.ToString(value: 12345)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 290.99873285262714 > 176.03067714841816.
IsChangePoint: Marked as a change because one of 12/8/2022 9:11:55 AM, 1/15/2023 4:08:29 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -68.39124786653672 (T) = (0 -290.62452573209913) / Math.Sqrt((5.502948306089623 / (46)) + (27.941778580941005 / (9))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (46) + (9) - 2, .025) and -0.7317690407667656 = (167.81944872014859 - 290.62452573209913) / 167.81944872014859 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_UInt64.ToString(value: 18446744073709551615)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 470.71930236904944 > 363.1670299915617.
IsChangePoint: Marked as a change because one of 12/8/2022 9:11:55 AM, 1/15/2023 4:08:29 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -31.50803295493609 (T) = (0 -475.1675789608689) / Math.Sqrt((82.8620018863198 / (45)) + (126.90980595633089 / (9))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (45) + (9) - 2, .025) and -0.36009997510422825 = (349.3622437015746 - 475.1675789608689) / 349.3622437015746 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_UInt64.TryFormat(value: 18446744073709551615)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 466.1556715809864 > 356.1369160292734.
IsChangePoint: Marked as a change because one of 12/19/2022 10:34:07 PM, 1/15/2023 4:08:29 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -66.48361468685736 (T) = (0 -466.8901896540011) / Math.Sqrt((39.01452359217315 / (45)) + (23.139019167823385 / (9))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (45) + (9) - 2, .025) and -0.3587499153888009 = (343.61745628547277 - 466.8901896540011) / 343.61745628547277 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_UInt64.TryFormat(value: 12345)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 288.94912732476433 > 176.08501435248962.
IsChangePoint: Marked as a change because one of 1/15/2023 4:08:29 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -76.47086841155858 (T) = (0 -289.54760408690777) / Math.Sqrt((63.94273220340748 / (46)) + (9.280227235533957 / (9))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (46) + (9) - 2, .025) and -0.6976542478262736 = (170.55746448822134 - 289.54760408690777) / 170.55746448822134 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_UInt64.TryFormat(value: 0)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 244.25893594270653 > 123.00845369173437.
IsChangePoint: Marked as a change because one of 1/15/2023 4:08:29 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -55.05802285070104 (T) = (0 -242.6464784854951) / Math.Sqrt((19.308374515766243 / (45)) + (42.05446435015634 / (9))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (45) + (9) - 2, .025) and -1.051352448376665 = (118.28609885030389 - 242.6464784854951) / 118.28609885030389 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 2eca2d319bf8d1247edf38facfdd501e0e2f0030
Compare 51a8dd5323721b363e61069575511f783e7ea6d3
Diff Diff

Regressions in System.Text.Json.Tests.Perf_Deep

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
WriteDeepUtf8 - Duration of single invocation 79.76 ms 96.65 ms 1.21 0.05 False
WriteDeepUtf8 - Duration of single invocation 112.81 ms 127.34 ms 1.13 0.03 False
WriteDeepUtf16 - Duration of single invocation 110.28 ms 125.02 ms 1.13 0.03 False
WriteDeepUtf16 - Duration of single invocation 81.92 ms 97.31 ms 1.19 0.05 False
WriteDeepUtf16 - Duration of single invocation 115.17 ms 131.50 ms 1.14 0.02 False
WriteDeepUtf16 - Duration of single invocation 84.24 ms 101.15 ms 1.20 0.06 False
WriteDeepUtf8 - Duration of single invocation 109.34 ms 127.96 ms 1.17 0.03 False
WriteDeepUtf8 - Duration of single invocation 84.84 ms 99.85 ms 1.18 0.03 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Tests.Perf_Deep*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Tests.Perf_Deep.WriteDeepUtf8(Formatted: False, SkipValidation: True)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 96.64878227083332 > 84.02411966189904.
IsChangePoint: Marked as a change because one of 10/25/2022 8:17:52 AM, 1/15/2023 4:08:29 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -24.39908128265583 (T) = (0 -96678428.41003765) / Math.Sqrt((2768212794292.3145 / (46)) + (3191656456428.1772 / (9))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (46) + (9) - 2, .025) and -0.19409045325645136 = (80964074.49400678 - 96678428.41003765) / 80964074.49400678 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Deep.WriteDeepUtf8(Formatted: True, SkipValidation: False)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 127.33971557142857 > 118.69543337625.
IsChangePoint: Marked as a change because one of 10/25/2022 8:17:52 AM, 1/4/2023 7:18:24 AM, 1/15/2023 4:08:29 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -29.34153057391292 (T) = (0 -130148696.67260888) / Math.Sqrt((2944054151235.713 / (45)) + (2363692566996.931 / (9))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (45) + (9) - 2, .025) and -0.14827298745205883 = (113342992.55911276 - 130148696.67260888) / 113342992.55911276 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Deep.WriteDeepUtf16(Formatted: True, SkipValidation: True)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 125.0244543 > 116.82143436250001.
IsChangePoint: Marked as a change because one of 10/25/2022 8:17:52 AM, 1/15/2023 4:08:29 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -29.93076418804875 (T) = (0 -125642396.73174603) / Math.Sqrt((3244324622229.825 / (45)) + (1657019029947.148 / (9))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (45) + (9) - 2, .025) and -0.13711445516287835 = (110492304.58841474 - 125642396.73174603) / 110492304.58841474 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Deep.WriteDeepUtf16(Formatted: False, SkipValidation: True)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 97.3066935 > 85.29824329625.
IsChangePoint: Marked as a change because one of 10/25/2022 8:17:52 AM, 1/15/2023 4:08:29 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -36.79472663123063 (T) = (0 -97205721.41276455) / Math.Sqrt((1692986558948.1477 / (45)) + (1338906151614.814 / (9))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (45) + (9) - 2, .025) and -0.1953424088072525 = (81320398.82175623 - 97205721.41276455) / 81320398.82175623 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Deep.WriteDeepUtf16(Formatted: True, SkipValidation: False)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 131.5009038076923 > 119.77417098500001.
IsChangePoint: Marked as a change because one of 10/25/2022 12:27:43 PM, 1/5/2023 6:34:45 PM, 1/15/2023 4:08:29 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -32.857780215970195 (T) = (0 -130679883.13302809) / Math.Sqrt((2495830524011.9683 / (45)) + (1943327610698.8381 / (9))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (45) + (9) - 2, .025) and -0.15072947760336095 = (113562644.97994503 - 130679883.13302809) / 113562644.97994503 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Deep.WriteDeepUtf16(Formatted: False, SkipValidation: False)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 101.14921817857143 > 88.48308927187499.
IsChangePoint: Marked as a change because one of 10/25/2022 8:17:52 AM, 11/9/2022 6:57:30 AM, 12/14/2022 5:20:21 PM, 1/15/2023 4:08:29 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -18.00319827381978 (T) = (0 -100107283.18042327) / Math.Sqrt((2151082780473.1758 / (46)) + (6623506074396.919 / (9))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (46) + (9) - 2, .025) and -0.18920911083730937 = (84179714.28922102 - 100107283.18042327) / 84179714.28922102 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Deep.WriteDeepUtf8(Formatted: True, SkipValidation: True)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 127.95661026666667 > 115.03077091153847.
IsChangePoint: Marked as a change because one of 10/25/2022 8:17:52 AM, 1/15/2023 4:08:29 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -18.341321359931197 (T) = (0 -128304432.7324074) / Math.Sqrt((2826404517739.809 / (46)) + (7860306522241.69 / (9))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (46) + (9) - 2, .025) and -0.1603803790781838 = (110571011.92484275 - 128304432.7324074) / 110571011.92484275 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Deep.WriteDeepUtf8(Formatted: False, SkipValidation: False)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 99.85390079999999 > 87.84428057312502.
IsChangePoint: Marked as a change because one of 10/25/2022 8:17:52 AM, 11/15/2022 8:48:03 PM, 12/14/2022 5:20:21 PM, 1/15/2023 4:08:29 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -43.46615809377788 (T) = (0 -99507631.9291514) / Math.Sqrt((2119879408795.142 / (46)) + (687002980616.5857 / (9))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (46) + (9) - 2, .025) and -0.18040498708873684 = (84299569.23053132 - 99507631.9291514) / 84299569.23053132 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@kotlarmilos
Copy link
Member

Could be dotnet/runtime#80668. Similar to #11805 and #11831.

@kotlarmilos kotlarmilos mentioned this issue Jan 18, 2023
Closed
@kotlarmilos
Copy link
Member

Mitigated by dotnet/runtime#82093.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
arch-x64 branch-refs/heads/main kind-micro_mono mono-interpreter os-linux perf-regression runtime-mono
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@kotlarmilos