Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Perf] Regressions in System.Tests.Perf_String #70161

Closed
performanceautofiler bot opened this issue Jun 2, 2022 · 6 comments
Closed

[Perf] Regressions in System.Tests.Perf_String #70161

performanceautofiler bot opened this issue Jun 2, 2022 · 6 comments
Assignees
@tannergooding
Labels
arch-arm64 area-CodeGen-coreclr CLR JIT compiler in src/coreclr/src/jit and related components such as SuperPMI runtime-coreclr specific to the CoreCLR runtime tenet-performance Performance related issue tenet-performance-benchmarks Issue from performance benchmark
Milestone
7.0.0

Comments

@performanceautofiler
Copy link

Run Information

Architecture arm64
OS Windows 10.0.19041
Baseline e86511ec00d9fad4667c71e73017931b74a049d6
Compare 0dba0ee1b234100ade81d2b4f799cc118150785c
Diff Diff

Regressions in System.Tests.Perf_String

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ToUpperInvariant - Duration of single invocation 18.76 ns 20.22 ns 1.08 0.29 False
ToUpperInvariant - Duration of single invocation 5.65 ns 6.90 ns 1.22 0.36 False
Replace_String - Duration of single invocation 14.42 ns 16.56 ns 1.15 0.21 False
Insert - Duration of single invocation 18.24 ns 20.03 ns 1.10 0.21 False
Remove_IntInt - Duration of single invocation 14.72 ns 16.60 ns 1.13 0.38 False
TrimStart - Duration of single invocation 2.26 ns 3.43 ns 1.52 0.37 False
TrimStart_CharArr - Duration of single invocation 4.26 ns 5.83 ns 1.37 0.39 False
Remove_IntInt - Duration of single invocation 14.14 ns 16.28 ns 1.15 0.29 False
Replace_Char - Duration of single invocation 5.06 ns 6.89 ns 1.36 0.33 False
ToLower - Duration of single invocation 15.56 ns 17.37 ns 1.12 0.18 False
Remove_Int - Duration of single invocation 13.44 ns 15.25 ns 1.13 0.26 False
Replace_Char - Duration of single invocation 12.49 ns 15.20 ns 1.22 0.10 False
Remove_IntInt - Duration of single invocation 13.58 ns 15.11 ns 1.11 0.26 False
Trim_CharArr - Duration of single invocation 7.73 ns 9.31 ns 1.20 0.18 False
ToLowerInvariant - Duration of single invocation 5.19 ns 7.19 ns 1.39 0.33 False
Trim - Duration of single invocation 16.52 ns 18.76 ns 1.14 0.21 False
TrimEnd - Duration of single invocation 2.72 ns 4.09 ns 1.50 0.43 False
TrimEnd_CharArr - Duration of single invocation 16.52 ns 17.88 ns 1.08 0.17 False
Remove_Int - Duration of single invocation 2.89 ns 4.25 ns 1.47 0.37 False
Trim_CharArr - Duration of single invocation 17.79 ns 19.98 ns 1.12 0.09 False
IndexOfAny - Duration of single invocation 18.37 ns 19.55 ns 1.06 0.06 False
ToLowerInvariant - Duration of single invocation 18.39 ns 20.18 ns 1.10 0.29 False
Trim - Duration of single invocation 3.44 ns 5.06 ns 1.47 0.43 False
Trim - Duration of single invocation 16.69 ns 18.85 ns 1.13 0.22 False
ToLowerInvariant - Duration of single invocation 18.66 ns 20.45 ns 1.10 0.33 False
ToUpperInvariant - Duration of single invocation 18.79 ns 20.22 ns 1.08 0.30 False
Remove_Int - Duration of single invocation 12.35 ns 14.74 ns 1.19 0.29 False
Substring_Int - Duration of single invocation 10.28 ns 12.56 ns 1.22 0.26 False
Substring_IntInt - Duration of single invocation 10.94 ns 12.65 ns 1.16 0.33 False

graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_String*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_String.ToUpperInvariant(s: "test")

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 20.22159224976476 > 19.54862607317232.
IsChangePoint: Marked as a change because one of 2/3/2022 8:57:20 PM, 2/4/2022 3:19:13 PM, 5/24/2022 3:42:52 PM, 6/2/2022 5:13:37 AM falls between 5/24/2022 3:42:52 PM and 6/2/2022 5:13:37 AM.
IsRegressionStdDev: Marked as regression because -11.434891142397158 (T) = (0 -20.46928031622022) / Math.Sqrt((0.08374089370265282 / (6)) + (0.47594668455995925 / (32))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (6) + (32) - 2, .025) and -0.10479341288407733 = (18.52769945720884 - 20.46928031622022) / 18.52769945720884 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.ToUpperInvariant(s: "TEST")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 6.8960155828797 > 5.561910605738338.
IsChangePoint: Marked as a change because one of 3/15/2022 7:21:49 PM, 3/16/2022 12:48:00 PM, 5/24/2022 3:42:52 PM, 6/2/2022 5:13:37 AM falls between 5/24/2022 3:42:52 PM and 6/2/2022 5:13:37 AM.
IsRegressionStdDev: Marked as regression because -19.96816313113974 (T) = (0 -7.035606127092163) / Math.Sqrt((0.021565857610617654 / (6)) + (0.1129460930133774 / (32))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (6) + (32) - 2, .025) and -0.3150094828225154 = (5.3502322371022375 - 7.035606127092163) / 5.3502322371022375 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.Replace_String(text: "This is a very nice sentence", oldValue: "bad", newValue: "nice")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 16.559502738456917 > 15.170373902449262.
IsChangePoint: Marked as a change because one of 2/23/2022 2:01:04 AM, 5/24/2022 10:45:34 PM, 6/2/2022 5:13:37 AM falls between 5/24/2022 3:42:52 PM and 6/2/2022 5:13:37 AM.
IsRegressionStdDev: Marked as regression because -12.703711538327958 (T) = (0 -16.330748829527142) / Math.Sqrt((0.09692622613801072 / (7)) + (0.1257371159682088 / (31))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (7) + (31) - 2, .025) and -0.1161757973088164 = (14.630982743848955 - 16.330748829527142) / 14.630982743848955 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.Insert(s1: "dzsdzsDDZSDZSDZSddsz", i: 7, s2: "Test")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 20.02677533400781 > 19.158389685802124.
IsChangePoint: Marked as a change because one of 3/12/2022 4:35:06 AM, 3/13/2022 1:54:51 AM, 5/24/2022 10:45:34 PM, 6/2/2022 5:13:37 AM falls between 5/24/2022 3:42:52 PM and 6/2/2022 5:13:37 AM.
IsRegressionStdDev: Marked as regression because -6.5788991988663135 (T) = (0 -20.22068259203967) / Math.Sqrt((0.39785713785073296 / (7)) + (0.3658785949447587 / (31))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (7) + (31) - 2, .025) and -0.0931829828923221 = (18.497070397620153 - 20.22068259203967) / 18.497070397620153 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.Remove_IntInt(s: "dzsdzsDDZSDZSDZSddsz", i1: 10, i2: 1)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 16.60043289209919 > 15.855240962737735.
IsChangePoint: Marked as a change because one of 4/8/2022 8:38:01 PM, 5/24/2022 10:45:34 PM, 6/2/2022 5:13:37 AM falls between 5/24/2022 3:42:52 PM and 6/2/2022 5:13:37 AM.
IsRegressionStdDev: Marked as regression because -4.981293023059961 (T) = (0 -16.943602677231233) / Math.Sqrt((0.5201858752237558 / (7)) + (0.6917348991790302 / (31))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (7) + (31) - 2, .025) and -0.10057846662556892 = (15.395179163536794 - 16.943602677231233) / 15.395179163536794 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.TrimStart(s: "Test")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.4344275252180414 > 2.3717537187299724.
IsChangePoint: Marked as a change because one of 3/15/2022 7:21:49 PM, 3/16/2022 12:48:00 PM, 5/24/2022 3:42:52 PM, 6/2/2022 5:13:37 AM falls between 5/24/2022 3:42:52 PM and 6/2/2022 5:13:37 AM.
IsRegressionStdDev: Marked as regression because -16.434774092238705 (T) = (0 -3.458672180972311) / Math.Sqrt((0.01622995235973337 / (6)) + (0.06712069541561386 / (32))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (6) + (32) - 2, .025) and -0.4909743780524818 = (2.3197395152356983 - 3.458672180972311) / 2.3197395152356983 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.TrimStart_CharArr(s: "Test", c: [' ', ' '])

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 5.834109041212096 > 4.459995061895248.
IsChangePoint: Marked as a change because one of 2/26/2022 10:27:55 AM, 4/16/2022 8:46:33 PM, 5/24/2022 3:42:52 PM, 6/2/2022 5:13:37 AM falls between 5/24/2022 3:42:52 PM and 6/2/2022 5:13:37 AM.
IsRegressionStdDev: Marked as regression because -12.181972728401577 (T) = (0 -5.620540644267087) / Math.Sqrt((0.025684065324774925 / (6)) + (0.21559611110714164 / (32))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (6) + (32) - 2, .025) and -0.29450770212585925 = (4.3418363869422745 - 5.620540644267087) / 4.3418363869422745 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.Remove_IntInt(s: "dzsdzsDDZSDZSDZSddsz", i1: 7, i2: 4)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 16.28249522263666 > 15.171857401825909.
IsChangePoint: Marked as a change because one of 1/22/2022 7:10:06 AM, 2/19/2022 11:51:48 PM, 2/21/2022 8:41:29 AM, 5/24/2022 3:42:52 PM, 6/2/2022 5:13:37 AM falls between 5/24/2022 3:42:52 PM and 6/2/2022 5:13:37 AM.
IsRegressionStdDev: Marked as regression because -6.612205732393991 (T) = (0 -16.339961270980808) / Math.Sqrt((0.3399480338110617 / (6)) + (0.535848650749657 / (32))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (6) + (32) - 2, .025) and -0.12313611547007394 = (14.548513796248045 - 16.339961270980808) / 14.548513796248045 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.Replace_Char(text: "Hello", oldChar: 'a', newChar: 'b')

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 6.890475456170126 > 5.312595955773796.
IsChangePoint: Marked as a change because one of 3/13/2022 11:50:20 AM, 3/14/2022 10:40:48 AM, 5/24/2022 3:42:52 PM, 6/2/2022 5:13:37 AM falls between 5/24/2022 3:42:52 PM and 6/2/2022 5:13:37 AM.
IsRegressionStdDev: Marked as regression because -22.029724862201 (T) = (0 -6.91536316838754) / Math.Sqrt((0.017905354119611187 / (6)) + (0.11858324750762354 / (32))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (6) + (32) - 2, .025) and -0.3523724015795005 = (5.113505096902862 - 6.91536316838754) / 5.113505096902862 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.ToLower(s: "test")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 17.36764255095269 > 16.315454378098288.
IsChangePoint: Marked as a change because one of 1/27/2022 1:01:32 PM, 3/11/2022 5:51:22 PM, 5/24/2022 10:45:34 PM, 6/2/2022 5:13:37 AM falls between 5/24/2022 3:42:52 PM and 6/2/2022 5:13:37 AM.
IsRegressionStdDev: Marked as regression because -7.610228808719198 (T) = (0 -17.480136179904928) / Math.Sqrt((0.4099987314980002 / (7)) + (0.1595855795009979 / (31))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (7) + (31) - 2, .025) and -0.12346616311931936 = (15.5591122845846 - 17.480136179904928) / 15.5591122845846 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.Remove_Int(s: "dzsdzsDDZSDZSDZSddsz", i: 10)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 15.250529754971732 > 13.748606391864017.
IsChangePoint: Marked as a change because one of 2/26/2022 3:32:05 AM, 2/27/2022 3:52:50 PM, 3/4/2022 4:19:52 AM, 3/5/2022 3:49:08 AM, 5/24/2022 3:42:52 PM, 6/2/2022 5:13:37 AM falls between 5/24/2022 3:42:52 PM and 6/2/2022 5:13:37 AM.
IsRegressionStdDev: Marked as regression because -14.410310426162884 (T) = (0 -15.039772718074143) / Math.Sqrt((0.04996101918062833 / (6)) + (0.2736065638324052 / (32))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (6) + (32) - 2, .025) and -0.14217112536454465 = (13.167705244933352 - 15.039772718074143) / 13.167705244933352 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.Replace_Char(text: "This is a very nice sentence", oldChar: 'z', newChar: 'y')

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 15.195637446664497 > 13.090368123163138.
IsChangePoint: Marked as a change because one of 2/19/2022 11:51:48 PM, 2/21/2022 8:41:29 AM, 5/24/2022 3:42:52 PM, 6/2/2022 5:13:37 AM falls between 5/24/2022 3:42:52 PM and 6/2/2022 5:13:37 AM.
IsRegressionStdDev: Marked as regression because -20.981514709721022 (T) = (0 -14.439333822575286) / Math.Sqrt((0.0001882412559320843 / (6)) + (0.28012641237124725 / (32))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (6) + (32) - 2, .025) and -0.15767179968088177 = (12.472735214380762 - 14.439333822575286) / 12.472735214380762 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.Remove_IntInt(s: "dzsdzsDDZSDZSDZSddsz", i1: 0, i2: 8)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 15.110752633459938 > 14.38870614377471.
IsChangePoint: Marked as a change because one of 3/13/2022 11:50:20 AM, 3/14/2022 10:40:48 AM, 5/24/2022 3:42:52 PM, 6/2/2022 5:13:37 AM falls between 5/24/2022 3:42:52 PM and 6/2/2022 5:13:37 AM.
IsRegressionStdDev: Marked as regression because -5.576061613994758 (T) = (0 -15.562200242121042) / Math.Sqrt((0.4277870787616548 / (6)) + (0.5658603196569941 / (32))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (6) + (32) - 2, .025) and -0.11967305803240645 = (13.898878900835916 - 15.562200242121042) / 13.898878900835916 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.Trim_CharArr(s: "Test", c: [' ', ' '])

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 9.314619161586394 > 7.94684310758377.
IsChangePoint: Marked as a change because one of 2/2/2022 8:22:24 PM, 2/3/2022 2:32:50 PM, 5/24/2022 10:45:34 PM, 6/2/2022 5:13:37 AM falls between 5/24/2022 3:42:52 PM and 6/2/2022 5:13:37 AM.
IsRegressionStdDev: Marked as regression because -12.382320233210294 (T) = (0 -8.93610041947187) / Math.Sqrt((0.04270568050227077 / (7)) + (0.16159848475180946 / (31))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (7) + (31) - 2, .025) and -0.1728634224761735 = (7.619046044257899 - 8.93610041947187) / 7.619046044257899 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.ToLowerInvariant(s: "test")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 7.194082951034547 > 5.4313586862375915.
IsChangePoint: Marked as a change because one of 5/24/2022 3:42:52 PM, 6/2/2022 5:13:37 AM falls between 5/24/2022 3:42:52 PM and 6/2/2022 5:13:37 AM.
IsRegressionStdDev: Marked as regression because -18.862552245110795 (T) = (0 -6.826151160113803) / Math.Sqrt((0.01335050421873693 / (6)) + (0.16427638025268373 / (32))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (6) + (32) - 2, .025) and -0.310688529431201 = (5.208065079409939 - 6.826151160113803) / 5.208065079409939 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.Trim(s: " Test")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 18.75619882911518 > 17.484118594496522.
IsChangePoint: Marked as a change because one of 4/17/2022 3:50:42 AM, 5/24/2022 3:42:52 PM, 6/2/2022 5:13:37 AM falls between 5/24/2022 3:42:52 PM and 6/2/2022 5:13:37 AM.
IsRegressionStdDev: Marked as regression because -11.222656744087747 (T) = (0 -18.571596418594776) / Math.Sqrt((0.12372458218182872 / (6)) + (0.3111502393134056 / (32))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (6) + (32) - 2, .025) and -0.11764946758215311 = (16.616655720126012 - 18.571596418594776) / 16.616655720126012 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.TrimEnd(s: "Test")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 4.085928323382124 > 2.5211927810851744.
IsChangePoint: Marked as a change because one of 2/3/2022 2:32:50 PM, 2/4/2022 8:03:09 AM, 5/24/2022 3:42:52 PM, 6/2/2022 5:13:37 AM falls between 5/24/2022 3:42:52 PM and 6/2/2022 5:13:37 AM.
IsRegressionStdDev: Marked as regression because -18.61422017230779 (T) = (0 -4.121374421745569) / Math.Sqrt((0.02637083732563198 / (6)) + (0.10811556951103422 / (32))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (6) + (32) - 2, .025) and -0.6617242451165817 = (2.480179508638285 - 4.121374421745569) / 2.480179508638285 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.TrimEnd_CharArr(s: "Test ", c: [' ', ' '])

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 17.880319648407813 > 17.378609917035234.
IsChangePoint: Marked as a change because one of 2/23/2022 1:54:47 PM, 2/26/2022 10:27:55 AM, 5/24/2022 3:42:52 PM, 6/2/2022 5:13:37 AM falls between 5/24/2022 3:42:52 PM and 6/2/2022 5:13:37 AM.
IsRegressionStdDev: Marked as regression because -8.676581618448592 (T) = (0 -17.552345342761885) / Math.Sqrt((0.0467793708120652 / (6)) + (0.17080383805884472 / (32))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (6) + (32) - 2, .025) and -0.0600541556130419 = (16.557970411059948 - 17.552345342761885) / 16.557970411059948 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.Remove_Int(s: "dzsdzsDDZSDZSDZSddsz", i: 0)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 4.252892761827513 > 3.066033058616817.
IsChangePoint: Marked as a change because one of 1/27/2022 5:13:24 AM, 5/24/2022 3:42:52 PM, 6/2/2022 5:13:37 AM falls between 5/24/2022 3:42:52 PM and 6/2/2022 5:13:37 AM.
IsRegressionStdDev: Marked as regression because -11.849667862170177 (T) = (0 -4.203668463245272) / Math.Sqrt((0.03029080283658165 / (6)) + (0.16226677943440643 / (32))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (6) + (32) - 2, .025) and -0.39580056800917984 = (3.011654071212289 - 4.203668463245272) / 3.011654071212289 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.Trim_CharArr(s: "Test ", c: [' ', ' '])

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 19.98023124526524 > 18.641785538723262.
IsChangePoint: Marked as a change because one of 3/15/2022 4:36:24 AM, 3/16/2022 12:48:00 PM, 5/24/2022 3:42:52 PM, 6/2/2022 5:13:37 AM falls between 5/24/2022 3:42:52 PM and 6/2/2022 5:13:37 AM.
IsRegressionStdDev: Marked as regression because -23.767092898320307 (T) = (0 -19.738503064147302) / Math.Sqrt((0.010847395549482278 / (6)) + (0.16046699771331163 / (32))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (6) + (32) - 2, .025) and -0.11044059219777048 = (17.775379613132746 - 19.738503064147302) / 17.775379613132746 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.IndexOfAny

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 19.551848643916593 > 19.273682115064265.
IsChangePoint: Marked as a change because one of 1/27/2022 1:01:32 PM, 1/28/2022 4:34:00 PM, 2/26/2022 3:25:25 PM, 5/31/2022 5:02:26 PM, 6/2/2022 5:13:37 AM falls between 5/24/2022 3:42:52 PM and 6/2/2022 5:13:37 AM.
IsRegressionStdDev: Marked as regression because -18.021367529284696 (T) = (0 -19.853362348495228) / Math.Sqrt((0.007181468006083057 / (33)) + (0.035901305938680524 / (5))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (5) - 2, .025) and -0.08468595188081778 = (18.303327625908683 - 19.853362348495228) / 18.303327625908683 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.ToLowerInvariant(s: "TeSt")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 20.17888952916716 > 19.520307155666195.
IsChangePoint: Marked as a change because one of 5/24/2022 3:42:52 PM, 6/2/2022 5:13:37 AM falls between 5/24/2022 3:42:52 PM and 6/2/2022 5:13:37 AM.
IsRegressionStdDev: Marked as regression because -6.761479562211135 (T) = (0 -20.498823638322616) / Math.Sqrt((0.31506985798527676 / (6)) + (0.440339178917229 / (32))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (6) + (32) - 2, .025) and -0.09279330867806515 = (18.7581891978454 - 20.498823638322616) / 18.7581891978454 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.Trim(s: "Test")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 5.057817925574822 > 3.5981354631503706.
IsChangePoint: Marked as a change because one of 2/14/2022 9:07:48 PM, 2/15/2022 10:33:05 PM, 5/24/2022 3:42:52 PM, 6/2/2022 5:13:37 AM falls between 5/24/2022 3:42:52 PM and 6/2/2022 5:13:37 AM.
IsRegressionStdDev: Marked as regression because -23.01202121045961 (T) = (0 -5.089226264887139) / Math.Sqrt((0.012439456050047207 / (6)) + (0.0948877906944843 / (32))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (6) + (32) - 2, .025) and -0.472670706262856 = (3.455780198006306 - 5.089226264887139) / 3.455780198006306 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.Trim(s: "Test ")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 18.845340114122486 > 17.5511031437366.
IsChangePoint: Marked as a change because one of 4/16/2022 8:46:33 PM, 5/24/2022 3:42:52 PM, 6/2/2022 5:13:37 AM falls between 5/24/2022 3:42:52 PM and 6/2/2022 5:13:37 AM.
IsRegressionStdDev: Marked as regression because -12.497606096374176 (T) = (0 -18.639027056469462) / Math.Sqrt((0.12297646619032847 / (6)) + (0.18886785504451822 / (32))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (6) + (32) - 2, .025) and -0.1222599457811303 = (16.608475715932354 - 18.639027056469462) / 16.608475715932354 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.ToLowerInvariant(s: "TEST")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 20.449937973379868 > 19.445639505686984.
IsChangePoint: Marked as a change because one of 2/13/2022 4:40:10 PM, 2/14/2022 9:07:48 PM, 2/15/2022 4:45:35 PM, 2/16/2022 10:21:50 AM, 5/24/2022 3:42:52 PM, 6/2/2022 5:13:37 AM falls between 5/24/2022 3:42:52 PM and 6/2/2022 5:13:37 AM.
IsRegressionStdDev: Marked as regression because -5.420982851669959 (T) = (0 -20.29145178723937) / Math.Sqrt((0.4547879409439268 / (6)) + (0.48019225863310533 / (32))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (6) + (32) - 2, .025) and -0.08755229791066428 = (18.65790898168484 - 20.29145178723937) / 18.65790898168484 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.ToUpperInvariant(s: "TeSt")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 20.21807933295926 > 19.362349165746938.
IsChangePoint: Marked as a change because one of 2/6/2022 10:04:17 PM, 2/7/2022 11:21:03 PM, 3/7/2022 12:49:56 AM, 3/7/2022 10:45:01 PM, 3/12/2022 3:27:04 PM, 3/13/2022 11:50:20 AM, 5/24/2022 3:42:52 PM, 6/2/2022 5:13:37 AM falls between 5/24/2022 3:42:52 PM and 6/2/2022 5:13:37 AM.
IsRegressionStdDev: Marked as regression because -5.652868403177702 (T) = (0 -20.325290339015005) / Math.Sqrt((0.4969146689924385 / (6)) + (0.38698036706513494 / (32))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (6) + (32) - 2, .025) and -0.09371224015358146 = (18.583764168316232 - 20.325290339015005) / 18.583764168316232 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.Remove_Int(s: "dzsdzsDDZSDZSDZSddsz", i: 7)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 14.744298289023352 > 13.042573946062095.
IsChangePoint: Marked as a change because one of 2/12/2022 4:06:00 AM, 2/12/2022 8:07:31 PM, 5/24/2022 3:42:52 PM, 6/2/2022 5:13:37 AM falls between 5/24/2022 3:42:52 PM and 6/2/2022 5:13:37 AM.
IsRegressionStdDev: Marked as regression because -10.623326354624487 (T) = (0 -14.284173371832146) / Math.Sqrt((0.1542627486815108 / (6)) + (0.21360203683227535 / (32))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (6) + (32) - 2, .025) and -0.15451902283200866 = (12.372401917461175 - 14.284173371832146) / 12.372401917461175 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.Substring_Int(s: "dzsdzsDDZSDZSDZSddsz", i: 10)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 12.555304832597384 > 10.872120107657818.
IsChangePoint: Marked as a change because one of 2/4/2022 3:19:13 PM, 2/6/2022 6:42:03 AM, 5/24/2022 10:45:34 PM, 6/2/2022 5:13:37 AM falls between 5/24/2022 3:42:52 PM and 6/2/2022 5:13:37 AM.
IsRegressionStdDev: Marked as regression because -8.67280047335682 (T) = (0 -12.423749379685697) / Math.Sqrt((0.24268527246797397 / (7)) + (0.19064200421739796 / (31))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (7) + (31) - 2, .025) and -0.1641967651391572 = (10.671520271919565 - 12.423749379685697) / 10.671520271919565 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.Substring_IntInt(s: "dzsdzsDDZSDZSDZSddsz", i1: 0, i2: 8)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 12.653527363554327 > 11.40488219056948.
IsChangePoint: Marked as a change because one of 3/6/2022 4:27:35 PM, 3/7/2022 3:58:30 PM, 5/24/2022 3:42:52 PM, 6/2/2022 5:13:37 AM falls between 5/24/2022 3:42:52 PM and 6/2/2022 5:13:37 AM.
IsRegressionStdDev: Marked as regression because -8.297962636483726 (T) = (0 -12.670852254134145) / Math.Sqrt((0.18025464340157066 / (6)) + (0.3214056006587698 / (32))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (6) + (32) - 2, .025) and -0.1509049346713526 = (11.009469046852578 - 12.670852254134145) / 11.009469046852578 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
@performanceautofiler
Copy link
Author

Run Information

Architecture arm64
OS Windows 10.0.19041
Baseline e86511ec00d9fad4667c71e73017931b74a049d6
Compare 0dba0ee1b234100ade81d2b4f799cc118150785c
Diff Diff

Regressions in System.Net.Primitives.Tests.IPAddressPerformanceTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GetAddressBytes - Duration of single invocation 22.88 ns 24.58 ns 1.07 0.10 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Net.Primitives.Tests.IPAddressPerformanceTests*'

Payloads

Baseline
Compare

Histogram

System.Net.Primitives.Tests.IPAddressPerformanceTests.GetAddressBytes(address: 1020:3040:5060:7080:9010:1112:1314:1516)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 24.575909234297303 > 24.072725233354987.
IsChangePoint: Marked as a change because one of 3/6/2022 1:47:20 PM, 3/7/2022 12:49:56 AM, 3/22/2022 12:26:23 PM, 5/24/2022 3:42:52 PM, 6/2/2022 5:13:37 AM falls between 5/24/2022 3:42:52 PM and 6/2/2022 5:13:37 AM.
IsRegressionStdDev: Marked as regression because -18.659912460466078 (T) = (0 -24.665950981127935) / Math.Sqrt((0.014013525239643925 / (6)) + (0.20629620377471308 / (32))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (6) + (32) - 2, .025) and -0.07630487119202181 = (22.91725294693693 - 24.665950981127935) / 22.91725294693693 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture arm64
OS Windows 10.0.19041
Baseline e86511ec00d9fad4667c71e73017931b74a049d6
Compare 0dba0ee1b234100ade81d2b4f799cc118150785c
Diff Diff

Regressions in System.Text.RegularExpressions.Tests.Perf_Regex_Common

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
CtorInvoke - Duration of single invocation 328.69 μs 381.30 μs 1.16 0.01 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Common*'

Payloads

Baseline
Compare

Histogram

System.Text.RegularExpressions.Tests.Perf_Regex_Common.CtorInvoke(Options: Compiled)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 381.3020949074074 > 345.0833413461539.
IsChangePoint: Marked as a change because one of 2/26/2022 10:27:55 AM, 3/14/2022 7:02:48 PM, 5/24/2022 3:42:52 PM, 6/2/2022 5:13:37 AM falls between 5/24/2022 3:42:52 PM and 6/2/2022 5:13:37 AM.
IsRegressionStdDev: Marked as regression because -35.74669789348127 (T) = (0 -381741.88996898464) / Math.Sqrt((608110.094102625 / (6)) + (67317731.24433766 / (32))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (6) + (32) - 2, .025) and -0.1615086934138739 = (328660.3812210648 - 381741.88996898464) / 328660.3812210648 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture arm64
OS Windows 10.0.19041
Baseline e86511ec00d9fad4667c71e73017931b74a049d6
Compare 0dba0ee1b234100ade81d2b4f799cc118150785c
Diff Diff

Regressions in Microsoft.Extensions.Primitives.StringSegmentBenchmark

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
SubString - Duration of single invocation 10.05 ns 11.75 ns 1.17 0.28 False
GetValue - Duration of single invocation 1.94 ns 3.20 ns 1.65 0.43 False

graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'Microsoft.Extensions.Primitives.StringSegmentBenchmark*'

Payloads

Baseline
Compare

Histogram

Microsoft.Extensions.Primitives.StringSegmentBenchmark.SubString

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 11.748892137265518 > 10.539520375845216.
IsChangePoint: Marked as a change because one of 5/24/2022 3:42:52 PM, 6/2/2022 5:13:37 AM falls between 5/24/2022 3:42:52 PM and 6/2/2022 5:13:37 AM.
IsRegressionStdDev: Marked as regression because -12.450576621364137 (T) = (0 -11.801846472709784) / Math.Sqrt((0.0637415883013489 / (6)) + (0.2051778860129851 / (32))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (6) + (32) - 2, .025) and -0.15968145176709034 = (10.176800236587779 - 11.801846472709784) / 10.176800236587779 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### Microsoft.Extensions.Primitives.StringSegmentBenchmark.GetValue

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.1959015575231295 > 2.1931390051977284.
IsChangePoint: Marked as a change because one of 5/24/2022 10:45:34 PM, 6/2/2022 5:13:37 AM falls between 5/24/2022 3:42:52 PM and 6/2/2022 5:13:37 AM.
IsRegressionStdDev: Marked as regression because -8.644499362632347 (T) = (0 -3.1402975331719807) / Math.Sqrt((0.05792996829923169 / (7)) + (0.13673408120517802 / (31))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (7) + (31) - 2, .025) and -0.44939238029400475 = (2.1666303589473688 - 3.1402975331719807) / 2.1666303589473688 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture arm64
OS Windows 10.0.19041
Baseline e86511ec00d9fad4667c71e73017931b74a049d6
Compare 0dba0ee1b234100ade81d2b4f799cc118150785c
Diff Diff

Regressions in System.IO.Tests.Perf_Path

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Combine - Duration of single invocation 3.18 ns 4.92 ns 1.55 0.41 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.IO.Tests.Perf_Path*'

Payloads

Baseline
Compare

Histogram

System.IO.Tests.Perf_Path.Combine

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 4.922189142978524 > 3.2980251997764225.
IsChangePoint: Marked as a change because one of 2/11/2022 12:34:36 AM, 5/24/2022 3:42:52 PM, 6/2/2022 5:13:37 AM falls between 5/24/2022 3:42:52 PM and 6/2/2022 5:13:37 AM.
IsRegressionStdDev: Marked as regression because -21.862177285852944 (T) = (0 -4.88217204079947) / Math.Sqrt((0.004918164851055981 / (6)) + (0.1777151259348738 / (32))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (6) + (32) - 2, .025) and -0.5563926936069156 = (3.1368510407775774 - 4.88217204079947) / 3.1368510407775774 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture arm64
OS Windows 10.0.19041
Baseline e86511ec00d9fad4667c71e73017931b74a049d6
Compare 0dba0ee1b234100ade81d2b4f799cc118150785c
Diff Diff

Regressions in System.Tests.Perf_UInt16

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ToString - Duration of single invocation 1.61 ns 2.71 ns 1.68 0.45 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_UInt16*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_UInt16.ToString(value: 0)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.7126712049657775 > 1.8615168234138078.
IsChangePoint: Marked as a change because one of 3/11/2022 5:51:22 PM, 5/24/2022 3:42:52 PM, 6/2/2022 5:13:37 AM falls between 5/24/2022 3:42:52 PM and 6/2/2022 5:13:37 AM.
IsRegressionStdDev: Marked as regression because -9.939751223852417 (T) = (0 -2.6969437758732053) / Math.Sqrt((0.030816866806062763 / (6)) + (0.11186516870250203 / (32))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (6) + (32) - 2, .025) and -0.5207247427680791 = (1.773459522309165 - 2.6969437758732053) / 1.773459522309165 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture arm64
OS Windows 10.0.19041
Baseline e86511ec00d9fad4667c71e73017931b74a049d6
Compare 0dba0ee1b234100ade81d2b4f799cc118150785c
Diff Diff

Regressions in System.Tests.Perf_UInt64

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ToString - Duration of single invocation 14.52 ns 16.20 ns 1.12 0.23 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_UInt64*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_UInt64.ToString(value: 12345)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 16.20468851480538 > 15.273783178387445.
IsChangePoint: Marked as a change because one of 5/5/2022 4:01:04 PM, 5/24/2022 3:42:52 PM, 6/2/2022 5:13:37 AM falls between 5/24/2022 3:42:52 PM and 6/2/2022 5:13:37 AM.
IsRegressionStdDev: Marked as regression because -24.25372667440816 (T) = (0 -16.318383498057365) / Math.Sqrt((0.011400717798991097 / (6)) + (0.10017911119725446 / (32))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (6) + (32) - 2, .025) and -0.11784096858357596 = (14.598126170607705 - 16.318383498057365) / 14.598126170607705 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture arm64
OS Windows 10.0.19041
Baseline e86511ec00d9fad4667c71e73017931b74a049d6
Compare 0dba0ee1b234100ade81d2b4f799cc118150785c
Diff Diff

Regressions in System.IO.Tests.Perf_RandomAccess

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ReadScatterAsync - Duration of single invocation 744.25 μs 1.45 ms 1.95 0.41 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.IO.Tests.Perf_RandomAccess*'

Payloads

Baseline
Compare

Histogram

System.IO.Tests.Perf_RandomAccess.ReadScatterAsync(fileSize: 1048576, buffersSize: 16384, options: None)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.4534361647727272 > 789.7818333333332.
IsChangePoint: Marked as a change because one of 5/23/2022 7:50:08 PM, 5/24/2022 3:42:52 PM, 6/2/2022 5:13:37 AM falls between 5/24/2022 3:42:52 PM and 6/2/2022 5:13:37 AM.
IsRegressionStdDev: Marked as regression because -3.764001984035213 (T) = (0 -1403297.5101942136) / Math.Sqrt((94168365163.9964 / (6)) + (4571499845.99702 / (32))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (6) + (32) - 2, .025) and -0.5095593711086853 = (929607.3656007127 - 1403297.5101942136) / 929607.3656007127 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture arm64
OS Windows 10.0.19041
Baseline e86511ec00d9fad4667c71e73017931b74a049d6
Compare 0dba0ee1b234100ade81d2b4f799cc118150785c
Diff Diff

Regressions in XmlDocumentTests.XmlDocumentTests.Perf_XmlDocument

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GetDocumentElement - Duration of single invocation 4.89 ns 7.37 ns 1.51 0.49 True

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'XmlDocumentTests.XmlDocumentTests.Perf_XmlDocument*'

Payloads

Baseline
Compare

Histogram

XmlDocumentTests.XmlDocumentTests.Perf_XmlDocument.GetDocumentElement

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 7.365736313523484 > 5.151178518700287.
IsChangePoint: Marked as a change because one of 5/24/2022 3:42:52 PM, 6/2/2022 5:13:37 AM falls between 5/24/2022 3:42:52 PM and 6/2/2022 5:13:37 AM.
IsRegressionStdDev: Marked as regression because -16.871991670057845 (T) = (0 -6.758921479685904) / Math.Sqrt((0.03312672807608957 / (6)) + (0.17796033382097645 / (32))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (6) + (32) - 2, .025) and -0.3564618855306792 = (4.982758123750494 - 6.758921479685904) / 4.982758123750494 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture arm64
OS Windows 10.0.19041
Baseline e86511ec00d9fad4667c71e73017931b74a049d6
Compare 0dba0ee1b234100ade81d2b4f799cc118150785c
Diff Diff

Regressions in System.Tests.Perf_Byte

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ToString - Duration of single invocation 1.62 ns 2.71 ns 1.68 0.42 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_Byte*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Byte.ToString(value: 0)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.712339870658082 > 1.6975165295479184.
IsChangePoint: Marked as a change because one of 2/26/2022 3:32:05 AM, 2/27/2022 3:52:50 PM, 3/11/2022 5:51:22 PM, 5/24/2022 3:42:52 PM, 6/2/2022 5:13:37 AM falls between 5/24/2022 3:42:52 PM and 6/2/2022 5:13:37 AM.
IsRegressionStdDev: Marked as regression because -11.159855161851535 (T) = (0 -2.688630940967903) / Math.Sqrt((0.026913524078425143 / (6)) + (0.0970088366010443 / (32))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (6) + (32) - 2, .025) and -0.5621973946372818 = (1.7210571149314724 - 2.688630940967903) / 1.7210571149314724 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture arm64
OS Windows 10.0.19041
Baseline e86511ec00d9fad4667c71e73017931b74a049d6
Compare 0dba0ee1b234100ade81d2b4f799cc118150785c
Diff Diff

Regressions in System.Tests.Perf_Int16

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ToString - Duration of single invocation 1.93 ns 3.09 ns 1.60 0.45 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_Int16*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Int16.ToString(value: 0)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.0946884607474163 > 1.6966199250855036.
IsChangePoint: Marked as a change because one of 3/15/2022 12:53:39 AM, 3/15/2022 11:00:34 PM, 5/24/2022 3:42:52 PM, 6/2/2022 5:13:37 AM falls between 5/24/2022 3:42:52 PM and 6/2/2022 5:13:37 AM.
IsRegressionStdDev: Marked as regression because -15.065771068423869 (T) = (0 -2.8337055891007963) / Math.Sqrt((0.01612774343676352 / (6)) + (0.10306431316626731 / (32))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (6) + (32) - 2, .025) and -0.691131296845589 = (1.675627193693602 - 2.8337055891007963) / 1.675627193693602 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture arm64
OS Windows 10.0.19041
Baseline e86511ec00d9fad4667c71e73017931b74a049d6
Compare 0dba0ee1b234100ade81d2b4f799cc118150785c
Diff Diff

Regressions in System.Threading.Tests.Perf_Interlocked

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
CompareExchange_object_Match - Duration of single invocation 3.55 ns 7.37 ns 2.08 0.41 True

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Threading.Tests.Perf_Interlocked*'

Payloads

Baseline
Compare

Histogram

System.Threading.Tests.Perf_Interlocked.CompareExchange_object_Match

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 7.3715814840574945 > 3.744560142968401.
IsChangePoint: Marked as a change because one of 5/24/2022 3:42:52 PM, 6/2/2022 5:13:37 AM falls between 5/24/2022 3:42:52 PM and 6/2/2022 5:13:37 AM.
IsRegressionStdDev: Marked as regression because -18.799138751492798 (T) = (0 -7.00711222912566) / Math.Sqrt((0.008866530493438707 / (6)) + (0.9940645157899638 / (32))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (6) + (32) - 2, .025) and -0.9378901098817293 = (3.6158460138657236 - 7.00711222912566) / 3.6158460138657236 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler performanceautofiler bot added arm64 untriaged New issue has not been triaged by the area owner labels Jun 2, 2022
@ghost ghost added the untriaged New issue has not been triaged by the area owner label Jun 2, 2022
@dotnet-issue-labeler dotnet-issue-labeler bot added the area-CodeGen-coreclr CLR JIT compiler in src/coreclr/src/jit and related components such as SuperPMI label Jun 2, 2022
@DrewScoggins DrewScoggins transferred this issue from dotnet/perf-autofiling-issues Jun 2, 2022
@ghost
Copy link

ghost commented Jun 2, 2022

Tagging subscribers to this area: @JulieLeeMSFT
See info in area-owners.md if you want to be subscribed.

Issue Details

Run Information

Architecture arm64
OS Windows 10.0.19041
Baseline e86511ec00d9fad4667c71e73017931b74a049d6
Compare 0dba0ee1b234100ade81d2b4f799cc118150785c
Diff Diff

Regressions in System.Tests.Perf_String

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ToUpperInvariant - Duration of single invocation 18.76 ns 20.22 ns 1.08 0.29 False
ToUpperInvariant - Duration of single invocation 5.65 ns 6.90 ns 1.22 0.36 False
Replace_String - Duration of single invocation 14.42 ns 16.56 ns 1.15 0.21 False
Insert - Duration of single invocation 18.24 ns 20.03 ns 1.10 0.21 False
Remove_IntInt - Duration of single invocation 14.72 ns 16.60 ns 1.13 0.38 False
TrimStart - Duration of single invocation 2.26 ns 3.43 ns 1.52 0.37 False
TrimStart_CharArr - Duration of single invocation 4.26 ns 5.83 ns 1.37 0.39 False
Remove_IntInt - Duration of single invocation 14.14 ns 16.28 ns 1.15 0.29 False
Replace_Char - Duration of single invocation 5.06 ns 6.89 ns 1.36 0.33 False
ToLower - Duration of single invocation 15.56 ns 17.37 ns 1.12 0.18 False
Remove_Int - Duration of single invocation 13.44 ns 15.25 ns 1.13 0.26 False
Replace_Char - Duration of single invocation 12.49 ns 15.20 ns 1.22 0.10 False
Remove_IntInt - Duration of single invocation 13.58 ns 15.11 ns 1.11 0.26 False
Trim_CharArr - Duration of single invocation 7.73 ns 9.31 ns 1.20 0.18 False
ToLowerInvariant - Duration of single invocation 5.19 ns 7.19 ns 1.39 0.33 False
Trim - Duration of single invocation 16.52 ns 18.76 ns 1.14 0.21 False
TrimEnd - Duration of single invocation 2.72 ns 4.09 ns 1.50 0.43 False
TrimEnd_CharArr - Duration of single invocation 16.52 ns 17.88 ns 1.08 0.17 False
Remove_Int - Duration of single invocation 2.89 ns 4.25 ns 1.47 0.37 False
Trim_CharArr - Duration of single invocation 17.79 ns 19.98 ns 1.12 0.09 False
IndexOfAny - Duration of single invocation 18.37 ns 19.55 ns 1.06 0.06 False
ToLowerInvariant - Duration of single invocation 18.39 ns 20.18 ns 1.10 0.29 False
Trim - Duration of single invocation 3.44 ns 5.06 ns 1.47 0.43 False
Trim - Duration of single invocation 16.69 ns 18.85 ns 1.13 0.22 False
ToLowerInvariant - Duration of single invocation 18.66 ns 20.45 ns 1.10 0.33 False
ToUpperInvariant - Duration of single invocation 18.79 ns 20.22 ns 1.08 0.30 False
Remove_Int - Duration of single invocation 12.35 ns 14.74 ns 1.19 0.29 False
Substring_Int - Duration of single invocation 10.28 ns 12.56 ns 1.22 0.26 False
Substring_IntInt - Duration of single invocation 10.94 ns 12.65 ns 1.16 0.33 False

graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_String*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_String.ToUpperInvariant(s: "test")

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 20.22159224976476 > 19.54862607317232.
IsChangePoint: Marked as a change because one of 2/3/2022 8:57:20 PM, 2/4/2022 3:19:13 PM, 5/24/2022 3:42:52 PM, 6/2/2022 5:13:37 AM falls between 5/24/2022 3:42:52 PM and 6/2/2022 5:13:37 AM.
IsRegressionStdDev: Marked as regression because -11.434891142397158 (T) = (0 -20.46928031622022) / Math.Sqrt((0.08374089370265282 / (6)) + (0.47594668455995925 / (32))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (6) + (32) - 2, .025) and -0.10479341288407733 = (18.52769945720884 - 20.46928031622022) / 18.52769945720884 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.ToUpperInvariant(s: "TEST")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 6.8960155828797 > 5.561910605738338.
IsChangePoint: Marked as a change because one of 3/15/2022 7:21:49 PM, 3/16/2022 12:48:00 PM, 5/24/2022 3:42:52 PM, 6/2/2022 5:13:37 AM falls between 5/24/2022 3:42:52 PM and 6/2/2022 5:13:37 AM.
IsRegressionStdDev: Marked as regression because -19.96816313113974 (T) = (0 -7.035606127092163) / Math.Sqrt((0.021565857610617654 / (6)) + (0.1129460930133774 / (32))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (6) + (32) - 2, .025) and -0.3150094828225154 = (5.3502322371022375 - 7.035606127092163) / 5.3502322371022375 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.Replace_String(text: "This is a very nice sentence", oldValue: "bad", newValue: "nice")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 16.559502738456917 > 15.170373902449262.
IsChangePoint: Marked as a change because one of 2/23/2022 2:01:04 AM, 5/24/2022 10:45:34 PM, 6/2/2022 5:13:37 AM falls between 5/24/2022 3:42:52 PM and 6/2/2022 5:13:37 AM.
IsRegressionStdDev: Marked as regression because -12.703711538327958 (T) = (0 -16.330748829527142) / Math.Sqrt((0.09692622613801072 / (7)) + (0.1257371159682088 / (31))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (7) + (31) - 2, .025) and -0.1161757973088164 = (14.630982743848955 - 16.330748829527142) / 14.630982743848955 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.Insert(s1: "dzsdzsDDZSDZSDZSddsz", i: 7, s2: "Test")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 20.02677533400781 > 19.158389685802124.
IsChangePoint: Marked as a change because one of 3/12/2022 4:35:06 AM, 3/13/2022 1:54:51 AM, 5/24/2022 10:45:34 PM, 6/2/2022 5:13:37 AM falls between 5/24/2022 3:42:52 PM and 6/2/2022 5:13:37 AM.
IsRegressionStdDev: Marked as regression because -6.5788991988663135 (T) = (0 -20.22068259203967) / Math.Sqrt((0.39785713785073296 / (7)) + (0.3658785949447587 / (31))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (7) + (31) - 2, .025) and -0.0931829828923221 = (18.497070397620153 - 20.22068259203967) / 18.497070397620153 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.Remove_IntInt(s: "dzsdzsDDZSDZSDZSddsz", i1: 10, i2: 1)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 16.60043289209919 > 15.855240962737735.
IsChangePoint: Marked as a change because one of 4/8/2022 8:38:01 PM, 5/24/2022 10:45:34 PM, 6/2/2022 5:13:37 AM falls between 5/24/2022 3:42:52 PM and 6/2/2022 5:13:37 AM.
IsRegressionStdDev: Marked as regression because -4.981293023059961 (T) = (0 -16.943602677231233) / Math.Sqrt((0.5201858752237558 / (7)) + (0.6917348991790302 / (31))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (7) + (31) - 2, .025) and -0.10057846662556892 = (15.395179163536794 - 16.943602677231233) / 15.395179163536794 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.TrimStart(s: "Test")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.4344275252180414 > 2.3717537187299724.
IsChangePoint: Marked as a change because one of 3/15/2022 7:21:49 PM, 3/16/2022 12:48:00 PM, 5/24/2022 3:42:52 PM, 6/2/2022 5:13:37 AM falls between 5/24/2022 3:42:52 PM and 6/2/2022 5:13:37 AM.
IsRegressionStdDev: Marked as regression because -16.434774092238705 (T) = (0 -3.458672180972311) / Math.Sqrt((0.01622995235973337 / (6)) + (0.06712069541561386 / (32))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (6) + (32) - 2, .025) and -0.4909743780524818 = (2.3197395152356983 - 3.458672180972311) / 2.3197395152356983 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.TrimStart_CharArr(s: "Test", c: [' ', ' '])

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 5.834109041212096 > 4.459995061895248.
IsChangePoint: Marked as a change because one of 2/26/2022 10:27:55 AM, 4/16/2022 8:46:33 PM, 5/24/2022 3:42:52 PM, 6/2/2022 5:13:37 AM falls between 5/24/2022 3:42:52 PM and 6/2/2022 5:13:37 AM.
IsRegressionStdDev: Marked as regression because -12.181972728401577 (T) = (0 -5.620540644267087) / Math.Sqrt((0.025684065324774925 / (6)) + (0.21559611110714164 / (32))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (6) + (32) - 2, .025) and -0.29450770212585925 = (4.3418363869422745 - 5.620540644267087) / 4.3418363869422745 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.Remove_IntInt(s: "dzsdzsDDZSDZSDZSddsz", i1: 7, i2: 4)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 16.28249522263666 > 15.171857401825909.
IsChangePoint: Marked as a change because one of 1/22/2022 7:10:06 AM, 2/19/2022 11:51:48 PM, 2/21/2022 8:41:29 AM, 5/24/2022 3:42:52 PM, 6/2/2022 5:13:37 AM falls between 5/24/2022 3:42:52 PM and 6/2/2022 5:13:37 AM.
IsRegressionStdDev: Marked as regression because -6.612205732393991 (T) = (0 -16.339961270980808) / Math.Sqrt((0.3399480338110617 / (6)) + (0.535848650749657 / (32))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (6) + (32) - 2, .025) and -0.12313611547007394 = (14.548513796248045 - 16.339961270980808) / 14.548513796248045 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.Replace_Char(text: "Hello", oldChar: 'a', newChar: 'b')

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 6.890475456170126 > 5.312595955773796.
IsChangePoint: Marked as a change because one of 3/13/2022 11:50:20 AM, 3/14/2022 10:40:48 AM, 5/24/2022 3:42:52 PM, 6/2/2022 5:13:37 AM falls between 5/24/2022 3:42:52 PM and 6/2/2022 5:13:37 AM.
IsRegressionStdDev: Marked as regression because -22.029724862201 (T) = (0 -6.91536316838754) / Math.Sqrt((0.017905354119611187 / (6)) + (0.11858324750762354 / (32))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (6) + (32) - 2, .025) and -0.3523724015795005 = (5.113505096902862 - 6.91536316838754) / 5.113505096902862 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.ToLower(s: "test")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 17.36764255095269 > 16.315454378098288.
IsChangePoint: Marked as a change because one of 1/27/2022 1:01:32 PM, 3/11/2022 5:51:22 PM, 5/24/2022 10:45:34 PM, 6/2/2022 5:13:37 AM falls between 5/24/2022 3:42:52 PM and 6/2/2022 5:13:37 AM.
IsRegressionStdDev: Marked as regression because -7.610228808719198 (T) = (0 -17.480136179904928) / Math.Sqrt((0.4099987314980002 / (7)) + (0.1595855795009979 / (31))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (7) + (31) - 2, .025) and -0.12346616311931936 = (15.5591122845846 - 17.480136179904928) / 15.5591122845846 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.Remove_Int(s: "dzsdzsDDZSDZSDZSddsz", i: 10)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 15.250529754971732 > 13.748606391864017.
IsChangePoint: Marked as a change because one of 2/26/2022 3:32:05 AM, 2/27/2022 3:52:50 PM, 3/4/2022 4:19:52 AM, 3/5/2022 3:49:08 AM, 5/24/2022 3:42:52 PM, 6/2/2022 5:13:37 AM falls between 5/24/2022 3:42:52 PM and 6/2/2022 5:13:37 AM.
IsRegressionStdDev: Marked as regression because -14.410310426162884 (T) = (0 -15.039772718074143) / Math.Sqrt((0.04996101918062833 / (6)) + (0.2736065638324052 / (32))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (6) + (32) - 2, .025) and -0.14217112536454465 = (13.167705244933352 - 15.039772718074143) / 13.167705244933352 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.Replace_Char(text: "This is a very nice sentence", oldChar: 'z', newChar: 'y')

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 15.195637446664497 > 13.090368123163138.
IsChangePoint: Marked as a change because one of 2/19/2022 11:51:48 PM, 2/21/2022 8:41:29 AM, 5/24/2022 3:42:52 PM, 6/2/2022 5:13:37 AM falls between 5/24/2022 3:42:52 PM and 6/2/2022 5:13:37 AM.
IsRegressionStdDev: Marked as regression because -20.981514709721022 (T) = (0 -14.439333822575286) / Math.Sqrt((0.0001882412559320843 / (6)) + (0.28012641237124725 / (32))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (6) + (32) - 2, .025) and -0.15767179968088177 = (12.472735214380762 - 14.439333822575286) / 12.472735214380762 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.Remove_IntInt(s: "dzsdzsDDZSDZSDZSddsz", i1: 0, i2: 8)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 15.110752633459938 > 14.38870614377471.
IsChangePoint: Marked as a change because one of 3/13/2022 11:50:20 AM, 3/14/2022 10:40:48 AM, 5/24/2022 3:42:52 PM, 6/2/2022 5:13:37 AM falls between 5/24/2022 3:42:52 PM and 6/2/2022 5:13:37 AM.
IsRegressionStdDev: Marked as regression because -5.576061613994758 (T) = (0 -15.562200242121042) / Math.Sqrt((0.4277870787616548 / (6)) + (0.5658603196569941 / (32))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (6) + (32) - 2, .025) and -0.11967305803240645 = (13.898878900835916 - 15.562200242121042) / 13.898878900835916 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.Trim_CharArr(s: "Test", c: [' ', ' '])

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 9.314619161586394 > 7.94684310758377.
IsChangePoint: Marked as a change because one of 2/2/2022 8:22:24 PM, 2/3/2022 2:32:50 PM, 5/24/2022 10:45:34 PM, 6/2/2022 5:13:37 AM falls between 5/24/2022 3:42:52 PM and 6/2/2022 5:13:37 AM.
IsRegressionStdDev: Marked as regression because -12.382320233210294 (T) = (0 -8.93610041947187) / Math.Sqrt((0.04270568050227077 / (7)) + (0.16159848475180946 / (31))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (7) + (31) - 2, .025) and -0.1728634224761735 = (7.619046044257899 - 8.93610041947187) / 7.619046044257899 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.ToLowerInvariant(s: "test")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 7.194082951034547 > 5.4313586862375915.
IsChangePoint: Marked as a change because one of 5/24/2022 3:42:52 PM, 6/2/2022 5:13:37 AM falls between 5/24/2022 3:42:52 PM and 6/2/2022 5:13:37 AM.
IsRegressionStdDev: Marked as regression because -18.862552245110795 (T) = (0 -6.826151160113803) / Math.Sqrt((0.01335050421873693 / (6)) + (0.16427638025268373 / (32))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (6) + (32) - 2, .025) and -0.310688529431201 = (5.208065079409939 - 6.826151160113803) / 5.208065079409939 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.Trim(s: " Test")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 18.75619882911518 > 17.484118594496522.
IsChangePoint: Marked as a change because one of 4/17/2022 3:50:42 AM, 5/24/2022 3:42:52 PM, 6/2/2022 5:13:37 AM falls between 5/24/2022 3:42:52 PM and 6/2/2022 5:13:37 AM.
IsRegressionStdDev: Marked as regression because -11.222656744087747 (T) = (0 -18.571596418594776) / Math.Sqrt((0.12372458218182872 / (6)) + (0.3111502393134056 / (32))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (6) + (32) - 2, .025) and -0.11764946758215311 = (16.616655720126012 - 18.571596418594776) / 16.616655720126012 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.TrimEnd(s: "Test")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 4.085928323382124 > 2.5211927810851744.
IsChangePoint: Marked as a change because one of 2/3/2022 2:32:50 PM, 2/4/2022 8:03:09 AM, 5/24/2022 3:42:52 PM, 6/2/2022 5:13:37 AM falls between 5/24/2022 3:42:52 PM and 6/2/2022 5:13:37 AM.
IsRegressionStdDev: Marked as regression because -18.61422017230779 (T) = (0 -4.121374421745569) / Math.Sqrt((0.02637083732563198 / (6)) + (0.10811556951103422 / (32))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (6) + (32) - 2, .025) and -0.6617242451165817 = (2.480179508638285 - 4.121374421745569) / 2.480179508638285 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.TrimEnd_CharArr(s: "Test ", c: [' ', ' '])

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 17.880319648407813 > 17.378609917035234.
IsChangePoint: Marked as a change because one of 2/23/2022 1:54:47 PM, 2/26/2022 10:27:55 AM, 5/24/2022 3:42:52 PM, 6/2/2022 5:13:37 AM falls between 5/24/2022 3:42:52 PM and 6/2/2022 5:13:37 AM.
IsRegressionStdDev: Marked as regression because -8.676581618448592 (T) = (0 -17.552345342761885) / Math.Sqrt((0.0467793708120652 / (6)) + (0.17080383805884472 / (32))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (6) + (32) - 2, .025) and -0.0600541556130419 = (16.557970411059948 - 17.552345342761885) / 16.557970411059948 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.Remove_Int(s: "dzsdzsDDZSDZSDZSddsz", i: 0)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 4.252892761827513 > 3.066033058616817.
IsChangePoint: Marked as a change because one of 1/27/2022 5:13:24 AM, 5/24/2022 3:42:52 PM, 6/2/2022 5:13:37 AM falls between 5/24/2022 3:42:52 PM and 6/2/2022 5:13:37 AM.
IsRegressionStdDev: Marked as regression because -11.849667862170177 (T) = (0 -4.203668463245272) / Math.Sqrt((0.03029080283658165 / (6)) + (0.16226677943440643 / (32))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (6) + (32) - 2, .025) and -0.39580056800917984 = (3.011654071212289 - 4.203668463245272) / 3.011654071212289 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.Trim_CharArr(s: "Test ", c: [' ', ' '])

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 19.98023124526524 > 18.641785538723262.
IsChangePoint: Marked as a change because one of 3/15/2022 4:36:24 AM, 3/16/2022 12:48:00 PM, 5/24/2022 3:42:52 PM, 6/2/2022 5:13:37 AM falls between 5/24/2022 3:42:52 PM and 6/2/2022 5:13:37 AM.
IsRegressionStdDev: Marked as regression because -23.767092898320307 (T) = (0 -19.738503064147302) / Math.Sqrt((0.010847395549482278 / (6)) + (0.16046699771331163 / (32))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (6) + (32) - 2, .025) and -0.11044059219777048 = (17.775379613132746 - 19.738503064147302) / 17.775379613132746 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.IndexOfAny

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 19.551848643916593 > 19.273682115064265.
IsChangePoint: Marked as a change because one of 1/27/2022 1:01:32 PM, 1/28/2022 4:34:00 PM, 2/26/2022 3:25:25 PM, 5/31/2022 5:02:26 PM, 6/2/2022 5:13:37 AM falls between 5/24/2022 3:42:52 PM and 6/2/2022 5:13:37 AM.
IsRegressionStdDev: Marked as regression because -18.021367529284696 (T) = (0 -19.853362348495228) / Math.Sqrt((0.007181468006083057 / (33)) + (0.035901305938680524 / (5))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (5) - 2, .025) and -0.08468595188081778 = (18.303327625908683 - 19.853362348495228) / 18.303327625908683 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.ToLowerInvariant(s: "TeSt")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 20.17888952916716 > 19.520307155666195.
IsChangePoint: Marked as a change because one of 5/24/2022 3:42:52 PM, 6/2/2022 5:13:37 AM falls between 5/24/2022 3:42:52 PM and 6/2/2022 5:13:37 AM.
IsRegressionStdDev: Marked as regression because -6.761479562211135 (T) = (0 -20.498823638322616) / Math.Sqrt((0.31506985798527676 / (6)) + (0.440339178917229 / (32))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (6) + (32) - 2, .025) and -0.09279330867806515 = (18.7581891978454 - 20.498823638322616) / 18.7581891978454 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.Trim(s: "Test")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 5.057817925574822 > 3.5981354631503706.
IsChangePoint: Marked as a change because one of 2/14/2022 9:07:48 PM, 2/15/2022 10:33:05 PM, 5/24/2022 3:42:52 PM, 6/2/2022 5:13:37 AM falls between 5/24/2022 3:42:52 PM and 6/2/2022 5:13:37 AM.
IsRegressionStdDev: Marked as regression because -23.01202121045961 (T) = (0 -5.089226264887139) / Math.Sqrt((0.012439456050047207 / (6)) + (0.0948877906944843 / (32))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (6) + (32) - 2, .025) and -0.472670706262856 = (3.455780198006306 - 5.089226264887139) / 3.455780198006306 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.Trim(s: "Test ")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 18.845340114122486 > 17.5511031437366.
IsChangePoint: Marked as a change because one of 4/16/2022 8:46:33 PM, 5/24/2022 3:42:52 PM, 6/2/2022 5:13:37 AM falls between 5/24/2022 3:42:52 PM and 6/2/2022 5:13:37 AM.
IsRegressionStdDev: Marked as regression because -12.497606096374176 (T) = (0 -18.639027056469462) / Math.Sqrt((0.12297646619032847 / (6)) + (0.18886785504451822 / (32))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (6) + (32) - 2, .025) and -0.1222599457811303 = (16.608475715932354 - 18.639027056469462) / 16.608475715932354 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.ToLowerInvariant(s: "TEST")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 20.449937973379868 > 19.445639505686984.
IsChangePoint: Marked as a change because one of 2/13/2022 4:40:10 PM, 2/14/2022 9:07:48 PM, 2/15/2022 4:45:35 PM, 2/16/2022 10:21:50 AM, 5/24/2022 3:42:52 PM, 6/2/2022 5:13:37 AM falls between 5/24/2022 3:42:52 PM and 6/2/2022 5:13:37 AM.
IsRegressionStdDev: Marked as regression because -5.420982851669959 (T) = (0 -20.29145178723937) / Math.Sqrt((0.4547879409439268 / (6)) + (0.48019225863310533 / (32))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (6) + (32) - 2, .025) and -0.08755229791066428 = (18.65790898168484 - 20.29145178723937) / 18.65790898168484 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.ToUpperInvariant(s: "TeSt")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 20.21807933295926 > 19.362349165746938.
IsChangePoint: Marked as a change because one of 2/6/2022 10:04:17 PM, 2/7/2022 11:21:03 PM, 3/7/2022 12:49:56 AM, 3/7/2022 10:45:01 PM, 3/12/2022 3:27:04 PM, 3/13/2022 11:50:20 AM, 5/24/2022 3:42:52 PM, 6/2/2022 5:13:37 AM falls between 5/24/2022 3:42:52 PM and 6/2/2022 5:13:37 AM.
IsRegressionStdDev: Marked as regression because -5.652868403177702 (T) = (0 -20.325290339015005) / Math.Sqrt((0.4969146689924385 / (6)) + (0.38698036706513494 / (32))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (6) + (32) - 2, .025) and -0.09371224015358146 = (18.583764168316232 - 20.325290339015005) / 18.583764168316232 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.Remove_Int(s: "dzsdzsDDZSDZSDZSddsz", i: 7)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 14.744298289023352 > 13.042573946062095.
IsChangePoint: Marked as a change because one of 2/12/2022 4:06:00 AM, 2/12/2022 8:07:31 PM, 5/24/2022 3:42:52 PM, 6/2/2022 5:13:37 AM falls between 5/24/2022 3:42:52 PM and 6/2/2022 5:13:37 AM.
IsRegressionStdDev: Marked as regression because -10.623326354624487 (T) = (0 -14.284173371832146) / Math.Sqrt((0.1542627486815108 / (6)) + (0.21360203683227535 / (32))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (6) + (32) - 2, .025) and -0.15451902283200866 = (12.372401917461175 - 14.284173371832146) / 12.372401917461175 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.Substring_Int(s: "dzsdzsDDZSDZSDZSddsz", i: 10)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 12.555304832597384 > 10.872120107657818.
IsChangePoint: Marked as a change because one of 2/4/2022 3:19:13 PM, 2/6/2022 6:42:03 AM, 5/24/2022 10:45:34 PM, 6/2/2022 5:13:37 AM falls between 5/24/2022 3:42:52 PM and 6/2/2022 5:13:37 AM.
IsRegressionStdDev: Marked as regression because -8.67280047335682 (T) = (0 -12.423749379685697) / Math.Sqrt((0.24268527246797397 / (7)) + (0.19064200421739796 / (31))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (7) + (31) - 2, .025) and -0.1641967651391572 = (10.671520271919565 - 12.423749379685697) / 10.671520271919565 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.Substring_IntInt(s: "dzsdzsDDZSDZSDZSddsz", i1: 0, i2: 8)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 12.653527363554327 > 11.40488219056948.
IsChangePoint: Marked as a change because one of 3/6/2022 4:27:35 PM, 3/7/2022 3:58:30 PM, 5/24/2022 3:42:52 PM, 6/2/2022 5:13:37 AM falls between 5/24/2022 3:42:52 PM and 6/2/2022 5:13:37 AM.
IsRegressionStdDev: Marked as regression because -8.297962636483726 (T) = (0 -12.670852254134145) / Math.Sqrt((0.18025464340157066 / (6)) + (0.3214056006587698 / (32))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (6) + (32) - 2, .025) and -0.1509049346713526 = (11.009469046852578 - 12.670852254134145) / 11.009469046852578 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Author: performanceautofiler[bot]
Assignees: tannergooding
Labels:

area-CodeGen-coreclr, untriaged, refs/heads/main, RunKind=micro, Windows 10.0.19041, Regression, CoreClr, arm64
Milestone: -

@DrewScoggins DrewScoggins added tenet-performance Performance related issue tenet-performance-benchmarks Issue from performance benchmark labels Jun 2, 2022
@DrewScoggins
Copy link
Member

Seems like it might be #69672

@DrewScoggins DrewScoggins changed the title [Perf] Changes at 5/24/2022 10:45:34 PM [Perf] Regressions in System.Tests.Perf_String Jun 2, 2022
@DrewScoggins
Copy link
Member

DrewScoggins commented Jun 2, 2022 •
edited
Loading

Also seeing regressions here:

dotnet/perf-autofiling-issues#5774
dotnet/perf-autofiling-issues#5773

@JulieLeeMSFT JulieLeeMSFT removed the untriaged New issue has not been triaged by the area owner label Jun 8, 2022
@JulieLeeMSFT JulieLeeMSFT added this to the 7.0.0 milestone Jun 8, 2022
@JulieLeeMSFT
Copy link
Member

cc @janvorli.

@tannergooding
Copy link
Member

Most of these were resolved in 2d5c8dc

A few were mostly fixed but still have a small regression related to #70182

@ghost ghost locked as resolved and limited conversation to collaborators Jul 17, 2022
@jeffhandley jeffhandley added arch-arm64 runtime-coreclr specific to the CoreCLR runtime and removed arm64 labels Dec 28, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees

@tannergooding tannergooding

Labels
arch-arm64 area-CodeGen-coreclr CLR JIT compiler in src/coreclr/src/jit and related components such as SuperPMI runtime-coreclr specific to the CoreCLR runtime tenet-performance Performance related issue tenet-performance-benchmarks Issue from performance benchmark
Projects
None yet
Milestone
7.0.0
Development

No branches or pull requests
4 participants
@jeffhandley @DrewScoggins @tannergooding @JulieLeeMSFT