[Perf] Windows/arm64: 4 Regressions on 1/15/2023 10:11:50 PM

[performanceautofiler]

Run Information

Architecture	arm64
OS	Windows 10.0.25094
Baseline	2eca2d319bf8d1247edf38facfdd501e0e2f0030
Compare	51a8dd5323721b363e61069575511f783e7ea6d3
Diff	Diff

Regressions in System.Buffers.Text.Tests.Utf8FormatterTests

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
FormatterInt64 - Duration of single invocation	59.60 ns	79.60 ns	1.34	0.68	False
FormatterInt32 - Duration of single invocation	12.22 ns	24.44 ns	2.00	0.77	False
FormatterInt32 - Duration of single invocation	26.99 ns	50.08 ns	1.86	0.75	False
FormatterInt64 - Duration of single invocation	11.45 ns	26.89 ns	2.35	0.77	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Buffers.Text.Tests.Utf8FormatterTests*'

Payloads

Baseline
Compare

Histogram

System.Buffers.Text.Tests.Utf8FormatterTests.FormatterInt64(value: 9223372036854775807)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 79.60429317764353 > 69.9026592349715.
IsChangePoint: Marked as a change because one of 10/10/2022 3:36:43 PM, 1/15/2023 4:47:52 PM, 1/19/2023 1:09:52 AM falls between 1/10/2023 12:53:48 PM and 1/19/2023 1:09:52 AM.
IsRegressionStdDev: Marked as regression because -11.678490499089765 (T) = (0 -83.8515226526501) / Math.Sqrt((26.96106148469378 / (31)) + (30.394509003644178 / (12))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (31) + (12) - 2, .025) and -0.3457313624985111 = (62.309257991111785 - 83.8515226526501) / 62.309257991111785 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Buffers.Text.Tests.Utf8FormatterTests.FormatterInt32(value: 12345)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 24.439602161734864 > 13.972052515482275.
IsChangePoint: Marked as a change because one of 1/15/2023 4:47:52 PM, 1/19/2023 1:09:52 AM falls between 1/10/2023 12:53:48 PM and 1/19/2023 1:09:52 AM.
IsRegressionStdDev: Marked as regression because -17.32839580318711 (T) = (0 -25.051515021449507) / Math.Sqrt((3.5695783311140645 / (31)) + (4.486682970576205 / (12))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (31) + (12) - 2, .025) and -0.9369401091209877 = (12.933551689844586 - 25.051515021449507) / 12.933551689844586 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Buffers.Text.Tests.Utf8FormatterTests.FormatterInt32(value: 2147483647)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 50.07927170174519 > 28.74615098114294.
IsChangePoint: Marked as a change because one of 1/15/2023 4:47:52 PM, 1/19/2023 1:09:52 AM falls between 1/10/2023 12:53:48 PM and 1/19/2023 1:09:52 AM.
IsRegressionStdDev: Marked as regression because -13.591532442596002 (T) = (0 -45.471446437980894) / Math.Sqrt((8.512837571559484 / (31)) + (16.378848831242923 / (12))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (31) + (12) - 2, .025) and -0.620023697618298 = (28.06838350872979 - 45.471446437980894) / 28.06838350872979 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Buffers.Text.Tests.Utf8FormatterTests.FormatterInt64(value: 12345)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 26.89463643311037 > 14.672022791550491.
IsChangePoint: Marked as a change because one of 1/15/2023 10:11:50 PM, 1/19/2023 1:09:52 AM falls between 1/10/2023 12:53:48 PM and 1/19/2023 1:09:52 AM.
IsRegressionStdDev: Marked as regression because -17.205960225848052 (T) = (0 -26.862266774259105) / Math.Sqrt((3.141338725456139 / (32)) + (5.9522703432239314 / (11))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (11) - 2, .025) and -1.0497381872090106 = (13.105218481993367 - 26.862266774259105) / 13.105218481993367 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

[dotnet-issue-labeler]

I couldn't figure out the best area label to add to this issue. If you have write-permissions please help me learn by adding exactly one area label.

[cincuranet]

Probably caused by #80668.

cc @EgorBo

[ghost]

Tagging subscribers to this area: @dotnet/area-system-buffers
See info in area-owners.md if you want to be subscribed.

Issue Details

---

Run Information

Architecture	arm64
OS	Windows 10.0.25094
Baseline	2eca2d319bf8d1247edf38facfdd501e0e2f0030
Compare	51a8dd5323721b363e61069575511f783e7ea6d3
Diff	Diff

Regressions in System.Buffers.Text.Tests.Utf8FormatterTests

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
FormatterInt64 - Duration of single invocation	59.60 ns	79.60 ns	1.34	0.68	False
FormatterInt32 - Duration of single invocation	12.22 ns	24.44 ns	2.00	0.77	False
FormatterInt32 - Duration of single invocation	26.99 ns	50.08 ns	1.86	0.75	False
FormatterInt64 - Duration of single invocation	11.45 ns	26.89 ns	2.35	0.77	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Buffers.Text.Tests.Utf8FormatterTests*'

Payloads

Baseline
Compare

Histogram

System.Buffers.Text.Tests.Utf8FormatterTests.FormatterInt64(value: 9223372036854775807)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 79.60429317764353 > 69.9026592349715.
IsChangePoint: Marked as a change because one of 10/10/2022 3:36:43 PM, 1/15/2023 4:47:52 PM, 1/19/2023 1:09:52 AM falls between 1/10/2023 12:53:48 PM and 1/19/2023 1:09:52 AM.
IsRegressionStdDev: Marked as regression because -11.678490499089765 (T) = (0 -83.8515226526501) / Math.Sqrt((26.96106148469378 / (31)) + (30.394509003644178 / (12))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (31) + (12) - 2, .025) and -0.3457313624985111 = (62.309257991111785 - 83.8515226526501) / 62.309257991111785 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Buffers.Text.Tests.Utf8FormatterTests.FormatterInt32(value: 12345)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 24.439602161734864 > 13.972052515482275.
IsChangePoint: Marked as a change because one of 1/15/2023 4:47:52 PM, 1/19/2023 1:09:52 AM falls between 1/10/2023 12:53:48 PM and 1/19/2023 1:09:52 AM.
IsRegressionStdDev: Marked as regression because -17.32839580318711 (T) = (0 -25.051515021449507) / Math.Sqrt((3.5695783311140645 / (31)) + (4.486682970576205 / (12))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (31) + (12) - 2, .025) and -0.9369401091209877 = (12.933551689844586 - 25.051515021449507) / 12.933551689844586 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Buffers.Text.Tests.Utf8FormatterTests.FormatterInt32(value: 2147483647)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 50.07927170174519 > 28.74615098114294.
IsChangePoint: Marked as a change because one of 1/15/2023 4:47:52 PM, 1/19/2023 1:09:52 AM falls between 1/10/2023 12:53:48 PM and 1/19/2023 1:09:52 AM.
IsRegressionStdDev: Marked as regression because -13.591532442596002 (T) = (0 -45.471446437980894) / Math.Sqrt((8.512837571559484 / (31)) + (16.378848831242923 / (12))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (31) + (12) - 2, .025) and -0.620023697618298 = (28.06838350872979 - 45.471446437980894) / 28.06838350872979 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Buffers.Text.Tests.Utf8FormatterTests.FormatterInt64(value: 12345)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 26.89463643311037 > 14.672022791550491.
IsChangePoint: Marked as a change because one of 1/15/2023 10:11:50 PM, 1/19/2023 1:09:52 AM falls between 1/10/2023 12:53:48 PM and 1/19/2023 1:09:52 AM.
IsRegressionStdDev: Marked as regression because -17.205960225848052 (T) = (0 -26.862266774259105) / Math.Sqrt((3.141338725456139 / (32)) + (5.9522703432239314 / (11))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (11) - 2, .025) and -1.0497381872090106 = (13.105218481993367 - 26.862266774259105) / 13.105218481993367 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Author:	performanceautofiler[bot]
Assignees:	-
Labels:	area-System.Buffers, tenet-performance, tenet-performance-benchmarks, untriaged
Milestone:	-

[EgorBo]

#80745