Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Perf] Linux/x64: 103 Improvements on 12/8/2022 5:23:10 PM #10670

Closed
performanceautofiler bot opened this issue Dec 13, 2022 · 5 comments
Closed

[Perf] Linux/x64: 103 Improvements on 12/8/2022 5:23:10 PM #10670

performanceautofiler bot opened this issue Dec 13, 2022 · 5 comments
Labels
arch-x64 branch-refs/heads/main kind-micro_mono mono-interpreter os-linux perf-improvement runtime-mono untriaged

Comments

@performanceautofiler
Copy link

No description provided.

@performanceautofiler
Copy link
Author

performanceautofiler bot commented Dec 13, 2022 •
edited
Loading

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 8c795cbde56c3047e8a7f73f4963ec513607f44d
Compare a30de6d40f69ef612b514344a5ec83fffd10b957
Diff Diff

Improvements in System.Tests.Perf_Int64

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ToString - Duration of single invocation 273.41 ns 183.32 ns 0.67 0.07 False
ToString - Duration of single invocation 456.21 ns 366.58 ns 0.80 0.03 False
ToString - Duration of single invocation 516.61 ns 448.79 ns 0.87 0.09 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Int64*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Int64.ToString(value: 12345)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 183.321060848792 < 254.9307616791147.
IsChangePoint: Marked as a change because one of 12/8/2022 9:11:55 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 60.930741195247975 (T) = (0 -185.24591142256145) / Math.Sqrt((59.94181384486181 / (299)) + (41.82988603715731 / (21))) is greater than 1.967451947860886 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (21) - 2, .975) and 0.3275132356411672 = (275.4640258223966 - 185.24591142256145) / 275.4640258223966 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Int64.ToString(value: 9223372036854775807)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 366.5787620438197 < 425.3818044821641.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 12/8/2022 9:11:55 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 40.221472020572726 (T) = (0 -354.2851618142584) / Math.Sqrt((1132.6744432344813 / (299)) + (89.2966082724712 / (21))) is greater than 1.967451947860886 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (21) - 2, .975) and 0.24352314059487581 = (468.3357559580342 - 354.2851618142584) / 468.3357559580342 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Int64.ToString(value: -9223372036854775808)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 448.7935910859916 < 491.1491833524364.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 12/8/2022 9:11:55 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 36.92178806133207 (T) = (0 -433.85283310821313) / Math.Sqrt((2043.3478818898666 / (299)) + (58.4521552723881 / (21))) is greater than 1.967451947860886 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (21) - 2, .975) and 0.2088092646303883 = (548.3542889383493 - 433.85283310821313) / 548.3542889383493 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS ubuntu 18.04
Baseline 8c795cbde56c3047e8a7f73f4963ec513607f44d
Compare a30de6d40f69ef612b514344a5ec83fffd10b957
Diff Diff

Improvements in System.IO.Tests.Perf_Path

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GetFileNameWithoutExtension - Duration of single invocation 437.94 ns 355.21 ns 0.81 0.07 False
GetDirectoryName - Duration of single invocation 806.45 ns 705.88 ns 0.88 0.07 False
GetFileName - Duration of single invocation 362.05 ns 264.89 ns 0.73 0.11 False
ChangeExtension - Duration of single invocation 370.13 ns 280.33 ns 0.76 0.06 False
GetExtension - Duration of single invocation 277.34 ns 190.47 ns 0.69 0.12 False
GetTempPath - Duration of single invocation 1.27 μs 1.13 μs 0.90 0.05 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.IO.Tests.Perf_Path*'

Payloads

Baseline
Compare

Histogram

System.IO.Tests.Perf_Path.GetFileNameWithoutExtension

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 355.2082717960013 < 406.72307727344037.
IsChangePoint: Marked as a change because one of 9/21/2022 9:54:19 PM, 10/10/2022 3:36:43 PM, 12/8/2022 9:11:55 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 48.38060454321816 (T) = (0 -342.62211944236753) / Math.Sqrt((272.78668901040555 / (299)) + (70.73860684454957 / (21))) is greater than 1.967451947860886 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (21) - 2, .975) and 0.2261017848324675 = (442.7224572009087 - 342.62211944236753) / 442.7224572009087 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.Perf_Path.GetDirectoryName

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 705.876874652842 < 761.2402615995135.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 12/8/2022 9:11:55 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 31.041272739751914 (T) = (0 -682.7786141470056) / Math.Sqrt((579.6409458651257 / (299)) + (335.7874700406036 / (21))) is greater than 1.967451947860886 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (21) - 2, .975) and 0.16142575963067984 = (814.2136751616649 - 682.7786141470056) / 814.2136751616649 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.Perf_Path.GetFileName

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 264.89347083147493 < 351.93782232140273.
IsChangePoint: Marked as a change because one of 9/21/2022 9:54:19 PM, 12/8/2022 9:11:55 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 69.64125565977788 (T) = (0 -271.6045589409834) / Math.Sqrt((184.19128891711347 / (299)) + (46.06582797791194 / (21))) is greater than 1.967451947860886 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (21) - 2, .975) and 0.30059572041391275 = (388.3369988838516 - 271.6045589409834) / 388.3369988838516 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.Perf_Path.ChangeExtension

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 280.32713046612207 < 351.7399567718445.
IsChangePoint: Marked as a change because one of 12/8/2022 9:11:55 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 101.79185162396321 (T) = (0 -278.00685090185175) / Math.Sqrt((116.32146716017382 / (299)) + (16.18562700300743 / (21))) is greater than 1.967451947860886 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (21) - 2, .975) and 0.2828028209566127 = (387.62959340228184 - 278.00685090185175) / 387.62959340228184 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.Perf_Path.GetExtension

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 190.4743302514658 < 265.9650172408761.
IsChangePoint: Marked as a change because one of 12/8/2022 9:11:55 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 85.88926004692843 (T) = (0 -191.40663254737717) / Math.Sqrt((68.14748732504505 / (299)) + (21.53615415292971 / (21))) is greater than 1.967451947860886 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (21) - 2, .975) and 0.33439092172464907 = (287.56613873616004 - 191.40663254737717) / 287.56613873616004 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.Perf_Path.GetTempPath

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.1342310299274188 < 1.1966567332421154.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 12/3/2022 2:55:34 AM, 12/8/2022 9:11:55 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 32.01088956521407 (T) = (0 -1066.8477049311837) / Math.Sqrt((2063.6748545442447 / (299)) + (1182.7804739771207 / (21))) is greater than 1.967451947860886 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (21) - 2, .975) and 0.19262560937492113 = (1321.3791734281012 - 1066.8477049311837) / 1321.3791734281012 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 8c795cbde56c3047e8a7f73f4963ec513607f44d
Compare a30de6d40f69ef612b514344a5ec83fffd10b957
Diff Diff

Improvements in System.Tests.Perf_DateTime

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ToString - Duration of single invocation 684.64 ns 624.60 ns 0.91 0.05 False
ToString - Duration of single invocation 507.68 ns 422.04 ns 0.83 0.04 False
op_Subtraction - Duration of single invocation 26.03 ns 22.91 ns 0.88 0.27 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_DateTime*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_DateTime.ToString(format: "o")

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 624.6031191235386 < 650.1705199474117.
IsChangePoint: Marked as a change because one of 10/22/2022 11:54:51 PM, 12/3/2022 2:55:34 AM, 12/8/2022 9:11:55 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 46.39643300350949 (T) = (0 -586.2771328311346) / Math.Sqrt((1277.283373121189 / (299)) + (189.23856139229204 / (21))) is greater than 1.967451947860886 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (21) - 2, .975) and 0.22385869462768754 = (755.3742195822182 - 586.2771328311346) / 755.3742195822182 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_DateTime.ToString(format: "r")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 422.03940397891597 < 480.9571302551608.
IsChangePoint: Marked as a change because one of 10/22/2022 11:54:51 PM, 12/8/2022 9:11:55 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 48.629045441149174 (T) = (0 -420.8762892919975) / Math.Sqrt((758.5146741243686 / (299)) + (59.925847286780176 / (21))) is greater than 1.967451947860886 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (21) - 2, .975) and 0.21151732951384208 = (533.7800119722303 - 420.8762892919975) / 533.7800119722303 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_DateTime.op_Subtraction

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 22.9093815770987 < 24.46056830722816.
IsChangePoint: Marked as a change because one of 9/28/2022 6:26:25 AM, 11/15/2022 5:29:43 PM, 12/2/2022 5:21:30 AM, 12/8/2022 9:11:55 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 11.53372494816317 (T) = (0 -22.66096000789906) / Math.Sqrt((2.3227763275441426 / (299)) + (0.4444495545010124 / (21))) is greater than 1.967451947860886 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (21) - 2, .975) and 0.07967588971824069 = (24.62280380871621 - 22.66096000789906) / 24.62280380871621 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 8c795cbde56c3047e8a7f73f4963ec513607f44d
Compare a30de6d40f69ef612b514344a5ec83fffd10b957
Diff Diff

Improvements in System.Text.Json.Tests.Perf_Get

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GetString - Duration of single invocation 45.53 μs 36.54 μs 0.80 0.04 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Tests.Perf_Get*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Tests.Perf_Get.GetString

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 36.54376565158545 < 43.113072202993244.
IsChangePoint: Marked as a change because one of 12/8/2022 9:11:55 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 34.24641321053238 (T) = (0 -35279.80239413248) / Math.Sqrt((1127961.5446259244 / (299)) + (1640636.217406646 / (21))) is greater than 1.967451947860886 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (21) - 2, .975) and 0.21740229159411678 = (45080.380398756686 - 35279.80239413248) / 45080.380398756686 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 8c795cbde56c3047e8a7f73f4963ec513607f44d
Compare a30de6d40f69ef612b514344a5ec83fffd10b957
Diff Diff

Improvements in System.IO.Tests.StringReaderReadLineTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ReadLine - Duration of single invocation 290.15 μs 208.99 μs 0.72 0.04 False
ReadLineAsync - Duration of single invocation 3.26 ms 2.61 ms 0.80 0.07 False
ReadLine - Duration of single invocation 958.34 μs 680.85 μs 0.71 0.04 False
ReadLineAsync - Duration of single invocation 383.61 μs 308.56 μs 0.80 0.03 False
ReadLineAsync - Duration of single invocation 1.37 ms 1.12 ms 0.82 0.04 False
ReadLine - Duration of single invocation 2.23 ms 1.55 ms 0.69 0.13 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.IO.Tests.StringReaderReadLineTests*'

Payloads

Baseline
Compare

Histogram

System.IO.Tests.StringReaderReadLineTests.ReadLine(LineLengthRange: [ 9, 32])

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 208.99057616666664 < 275.5412892304455.
IsChangePoint: Marked as a change because one of 9/21/2022 9:54:19 PM, 12/8/2022 9:11:55 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 124.61280755730523 (T) = (0 -209171.5704801195) / Math.Sqrt((90860403.09322478 / (299)) + (6963792.362523444 / (21))) is greater than 1.967451947860886 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (21) - 2, .975) and 0.32199408984917943 = (308509.95153359335 - 209171.5704801195) / 308509.95153359335 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.StringReaderReadLineTests.ReadLineAsync(LineLengthRange: [   1,    1])

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.608735251302083 < 3.107539364040178.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 12/8/2022 9:11:55 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 61.95302689680598 (T) = (0 -2544770.35680775) / Math.Sqrt((12207361278.60696 / (299)) + (2809943837.132434 / (21))) is greater than 1.967451947860886 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (21) - 2, .975) and 0.2434102942623969 = (3363474.7307681656 - 2544770.35680775) / 3363474.7307681656 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.StringReaderReadLineTests.ReadLine(LineLengthRange: [   1,    8])

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 680.850216226708 < 925.4239721284412.
IsChangePoint: Marked as a change because one of 9/21/2022 9:54:19 PM, 12/8/2022 9:11:55 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 119.24612835785088 (T) = (0 -662732.4172987326) / Math.Sqrt((835062817.6582379 / (299)) + (103432305.03864214 / (21))) is greater than 1.967451947860886 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (21) - 2, .975) and 0.3332791405073986 = (994017.8229958124 - 662732.4172987326) / 994017.8229958124 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.StringReaderReadLineTests.ReadLineAsync(LineLengthRange: [   9,   32])

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 308.56242000000003 < 365.7305767949544.
IsChangePoint: Marked as a change because one of 9/21/2022 9:54:19 PM, 12/8/2022 9:11:55 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 64.9074488555637 (T) = (0 -305556.83542321454) / Math.Sqrt((140609540.79199153 / (299)) + (42318861.944798335 / (21))) is greater than 1.967451947860886 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (21) - 2, .975) and 0.2508758440031671 = (407885.439252217 - 305556.83542321454) / 407885.439252217 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.StringReaderReadLineTests.ReadLineAsync(LineLengthRange: [   1,    8])

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.1151669040178573 < 1.3281070819756153.
IsChangePoint: Marked as a change because one of 9/21/2022 9:54:19 PM, 12/8/2022 9:11:55 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 62.8764814302038 (T) = (0 -1072983.929728244) / Math.Sqrt((1425014624.928044 / (299)) + (534877825.73467827 / (21))) is greater than 1.967451947860886 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (21) - 2, .975) and 0.2436991357463161 = (1418726.303832883 - 1072983.929728244) / 1418726.303832883 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.StringReaderReadLineTests.ReadLine(LineLengthRange: [   1,    1])

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.5465044068749998 < 2.114851351628728.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/8/2022 9:11:55 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 78.98143731152426 (T) = (0 -1508915.2438801078) / Math.Sqrt((7569540368.104515 / (299)) + (1639898529.811191 / (21))) is greater than 1.967451947860886 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (21) - 2, .975) and 0.3473747136650939 = (2312069.8438671613 - 1508915.2438801078) / 2312069.8438671613 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 8c795cbde56c3047e8a7f73f4963ec513607f44d
Compare a30de6d40f69ef612b514344a5ec83fffd10b957
Diff Diff

Improvements in System.Tests.Perf_Int32

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ToString - Duration of single invocation 374.71 ns 297.91 ns 0.80 0.08 False
ToStringHex - Duration of single invocation 420.54 ns 319.77 ns 0.76 0.07 False
ToStringHex - Duration of single invocation 306.51 ns 224.03 ns 0.73 0.09 False
ToStringHex - Duration of single invocation 349.83 ns 258.42 ns 0.74 0.08 False
ToStringHex - Duration of single invocation 411.78 ns 321.66 ns 0.78 0.06 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Int32*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Int32.ToString(value: -2147483648)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 297.9123808595696 < 355.7296889269981.
IsChangePoint: Marked as a change because one of 10/3/2022 8:45:37 PM, 10/10/2022 9:53:50 PM, 12/8/2022 9:11:55 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 46.107170716560574 (T) = (0 -303.195029717638) / Math.Sqrt((512.8585184305898 / (299)) + (49.827063218011844 / (21))) is greater than 1.967451947860886 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (21) - 2, .975) and 0.2351629092024765 = (396.417790619288 - 303.195029717638) / 396.417790619288 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Int32.ToStringHex(value: -2147483648)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 319.77111273311607 < 392.5937255190613.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 12/8/2022 9:11:55 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 76.07792536943077 (T) = (0 -321.9533317848146) / Math.Sqrt((90.90346232350986 / (299)) + (26.52075571231781 / (21))) is greater than 1.967451947860886 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (21) - 2, .975) and 0.22827220190295755 = (417.1850911405552 - 321.9533317848146) / 417.1850911405552 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Int32.ToStringHex(value: 4)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 224.02628198219006 < 290.1685397152197.
IsChangePoint: Marked as a change because one of 12/8/2022 9:11:55 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 52.4514784483042 (T) = (0 -223.63367362441537) / Math.Sqrt((65.87678939772859 / (299)) + (55.24393830436692 / (21))) is greater than 1.967451947860886 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (21) - 2, .975) and 0.28367838429509484 = (312.1972989804948 - 223.63367362441537) / 312.1972989804948 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Int32.ToStringHex(value: 12345)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 258.4198447157947 < 332.41811560222266.
IsChangePoint: Marked as a change because one of 9/30/2022 6:30:54 PM, 12/8/2022 9:11:55 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 44.04546606157513 (T) = (0 -266.81063758751526) / Math.Sqrt((65.41577250410444 / (299)) + (79.71678551881817 / (21))) is greater than 1.967451947860886 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (21) - 2, .975) and 0.2485574786219425 = (355.0645990836609 - 266.81063758751526) / 355.0645990836609 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Int32.ToStringHex(value: 2147483647)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 321.6582519998261 < 391.60211918455735.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 12/8/2022 9:11:55 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 70.01462739843404 (T) = (0 -323.6415964284112) / Math.Sqrt((116.66947549832793 / (299)) + (28.653370133433004 / (21))) is greater than 1.967451947860886 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (21) - 2, .975) and 0.22273489566777913 = (416.38508486298826 - 323.6415964284112) / 416.38508486298826 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented Dec 13, 2022 •
edited
Loading

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 8c795cbde56c3047e8a7f73f4963ec513607f44d
Compare a30de6d40f69ef612b514344a5ec83fffd10b957
Diff Diff

Improvements in System.Tests.Perf_DateTimeOffset

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ToString - Duration of single invocation 833.11 ns 733.51 ns 0.88 0.03 False
ToString - Duration of single invocation 629.62 ns 541.10 ns 0.86 0.05 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_DateTimeOffset*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_DateTimeOffset.ToString(format: "o")

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 733.5111123143067 < 785.7282946546197.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/8/2022 9:11:55 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 45.64918441436715 (T) = (0 -731.5725255440913) / Math.Sqrt((1492.8024475621837 / (299)) + (114.47173060477394 / (21))) is greater than 1.967451947860886 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (21) - 2, .975) and 0.16781230736618166 = (879.0955838684819 - 731.5725255440913) / 879.0955838684819 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_DateTimeOffset.ToString(format: "r")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 541.0980456558032 < 593.1731095563775.
IsChangePoint: Marked as a change because one of 10/22/2022 11:54:51 PM, 12/8/2022 9:11:55 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 49.77605507918915 (T) = (0 -533.5815680311696) / Math.Sqrt((1089.794031292837 / (299)) + (47.828367794086546 / (21))) is greater than 1.967451947860886 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (21) - 2, .975) and 0.18501809154271956 = (654.7158439887488 - 533.5815680311696) / 654.7158439887488 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS ubuntu 18.04
Baseline 8c795cbde56c3047e8a7f73f4963ec513607f44d
Compare a30de6d40f69ef612b514344a5ec83fffd10b957
Diff Diff

Improvements in System.Text.Tests.Perf_StringBuilder

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ToString_SingleSegment - Duration of single invocation 284.97 ns 146.10 ns 0.51 0.07 False
ToString_MultipleSegments - Duration of single invocation 403.07 ns 273.63 ns 0.68 0.05 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Tests.Perf_StringBuilder*'

Payloads

Baseline
Compare

Histogram

System.Text.Tests.Perf_StringBuilder.ToString_SingleSegment(length: 100)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 146.09559223618479 < 251.99161460413896.
IsChangePoint: Marked as a change because one of 12/8/2022 9:11:55 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 171.27334237197044 (T) = (0 -145.87755696056092) / Math.Sqrt((93.27532204886947 / (299)) + (4.963213051198318 / (21))) is greater than 1.967451947860886 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (21) - 2, .975) and 0.46506396817784884 = (272.7009367151033 - 145.87755696056092) / 272.7009367151033 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Tests.Perf_StringBuilder.ToString_MultipleSegments(length: 100)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 273.63206080555295 < 393.0080515921571.
IsChangePoint: Marked as a change because one of 12/3/2022 2:55:34 AM, 12/8/2022 9:11:55 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 96.95129439372775 (T) = (0 -275.1461899625406) / Math.Sqrt((841.3110793457905 / (299)) + (37.15547355631074 / (21))) is greater than 1.967451947860886 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (21) - 2, .975) and 0.4299854229002739 = (482.70026946065764 - 275.1461899625406) / 482.70026946065764 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 8c795cbde56c3047e8a7f73f4963ec513607f44d
Compare a30de6d40f69ef612b514344a5ec83fffd10b957
Diff Diff

Improvements in System.Numerics.Tests.Perf_BigInteger

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ToStringX - Duration of single invocation 716.96 ns 642.29 ns 0.90 0.08 False
ToStringD - Duration of single invocation 696.38 ns 603.83 ns 0.87 0.03 False
ToStringX - Duration of single invocation 614.56 ns 534.76 ns 0.87 0.12 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_BigInteger*'

Payloads

Baseline
Compare

Histogram

System.Numerics.Tests.Perf_BigInteger.ToStringX(numberString: -2147483648)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 642.2866437875872 < 677.6253888188628.
IsChangePoint: Marked as a change because one of 9/29/2022 8:01:59 PM, 12/8/2022 9:11:55 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 33.104614851156526 (T) = (0 -605.192842265805) / Math.Sqrt((801.7372495144768 / (294)) + (269.5739132112624 / (21))) is greater than 1.9675720194858106 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (294) + (21) - 2, .975) and 0.17749691758263297 = (735.7940112359468 - 605.192842265805) / 735.7940112359468 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_BigInteger.ToStringD(numberString: -2147483648)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 603.8289600714521 < 677.6695734275814.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 12/8/2022 9:11:55 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 36.19367567352122 (T) = (0 -606.0323998971207) / Math.Sqrt((244.21843357496343 / (299)) + (100.74879219007484 / (21))) is greater than 1.967451947860886 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (21) - 2, .975) and 0.12396719834314832 = (691.7919040827286 - 606.0323998971207) / 691.7919040827286 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_BigInteger.ToStringX(numberString: 123)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 534.7629489404458 < 593.512539558116.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 12/8/2022 9:11:55 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 39.967132314959024 (T) = (0 -515.3940267795593) / Math.Sqrt((643.6045969284977 / (299)) + (144.40113985621173 / (21))) is greater than 1.967451947860886 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (21) - 2, .975) and 0.188977736277415 = (635.4869031756358 - 515.3940267795593) / 635.4869031756358 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 8c795cbde56c3047e8a7f73f4963ec513607f44d
Compare a30de6d40f69ef612b514344a5ec83fffd10b957
Diff Diff

Improvements in System.Tests.Perf_Int16

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ToString - Duration of single invocation 321.46 ns 240.86 ns 0.75 0.10 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Int16*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Int16.ToString(value: -32768)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 240.86283393761278 < 305.04501104916903.
IsChangePoint: Marked as a change because one of 10/3/2022 8:45:37 PM, 10/10/2022 9:53:50 PM, 12/8/2022 9:11:55 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 56.96925810490962 (T) = (0 -241.13563472087364) / Math.Sqrt((200.5551174624962 / (299)) + (37.483559014598676 / (21))) is greater than 1.967451947860886 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (21) - 2, .975) and 0.2701927120097891 = (330.41001191551277 - 241.13563472087364) / 330.41001191551277 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 8c795cbde56c3047e8a7f73f4963ec513607f44d
Compare a30de6d40f69ef612b514344a5ec83fffd10b957
Diff Diff

Improvements in System.Tests.Perf_Single

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ToString - Duration of single invocation 2.40 μs 2.07 μs 0.86 0.24 False
ToStringWithCultureInfo - Duration of single invocation 2.25 μs 2.12 μs 0.94 0.21 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Single*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Single.ToString(value: 3.4028235E+38)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.0739984927879194 < 2.234767021128135.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/11/2022 6:23:40 AM, 12/8/2022 9:11:55 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 21.878336988821296 (T) = (0 -2080.9013281297243) / Math.Sqrt((21224.571657182325 / (299)) + (3551.6525721066646 / (21))) is greater than 1.967451947860886 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (21) - 2, .975) and 0.1400942070319326 = (2419.9177923284424 - 2080.9013281297243) / 2419.9177923284424 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Single.ToStringWithCultureInfo(value: 3.4028235E+38, culture: zh)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.1180287508544087 < 2.139645489454285.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/11/2022 6:23:40 AM, 12/8/2022 9:11:55 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 24.45794453892466 (T) = (0 -2088.8421333585698) / Math.Sqrt((19149.281028682584 / (299)) + (2346.977146224276 / (21))) is greater than 1.967451947860886 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (21) - 2, .975) and 0.13438612924791662 = (2413.133850944061 - 2088.8421333585698) / 2413.133850944061 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 8c795cbde56c3047e8a7f73f4963ec513607f44d
Compare a30de6d40f69ef612b514344a5ec83fffd10b957
Diff Diff

Improvements in System.Net.NetworkInformation.Tests.PhysicalAddressTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
PAMedium - Duration of single invocation 487.27 ns 353.75 ns 0.73 0.03 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Net.NetworkInformation.Tests.PhysicalAddressTests*'

Payloads

Baseline
Compare

Histogram

System.Net.NetworkInformation.Tests.PhysicalAddressTests.PAMedium

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 353.752584470827 < 455.8521425127347.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/8/2022 9:11:55 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 90.40500671837444 (T) = (0 -353.5956460623291) / Math.Sqrt((379.3709246797074 / (299)) + (25.323136186968902 / (21))) is greater than 1.967451947860886 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (21) - 2, .975) and 0.28683568700689205 = (495.8123108800962 - 353.5956460623291) / 495.8123108800962 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 8c795cbde56c3047e8a7f73f4963ec513607f44d
Compare a30de6d40f69ef612b514344a5ec83fffd10b957
Diff Diff

Improvements in System.Diagnostics.Perf_Activity

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ActivityAllocations - Duration of single invocation 2.46 μs 2.24 μs 0.91 0.02 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Diagnostics.Perf_Activity*'

Payloads

Baseline
Compare

Histogram

System.Diagnostics.Perf_Activity.ActivityAllocations(idFormat: W3C)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.2357976277459413 < 2.3232385670587683.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/8/2022 9:11:55 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 40.06531271311641 (T) = (0 -2176.826048077425) / Math.Sqrt((8712.656145025345 / (299)) + (1079.0690218300938 / (21))) is greater than 1.967451947860886 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (21) - 2, .975) and 0.14174916861991713 = (2536.3518082202722 - 2176.826048077425) / 2536.3518082202722 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 8c795cbde56c3047e8a7f73f4963ec513607f44d
Compare a30de6d40f69ef612b514344a5ec83fffd10b957
Diff Diff

Improvements in System.Tests.Perf_Guid

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GuidToString - Duration of single invocation 646.70 ns 553.30 ns 0.86 0.05 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Guid*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Guid.GuidToString

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 553.3002184786194 < 608.8913123247852.
IsChangePoint: Marked as a change because one of 11/16/2022 11:42:33 AM, 12/8/2022 9:11:55 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 45.057551814419014 (T) = (0 -541.4955528446258) / Math.Sqrt((293.1448978086229 / (299)) + (87.1904355228575 / (21))) is greater than 1.967451947860886 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (21) - 2, .975) and 0.15860917749589115 = (643.5719743567579 - 541.4955528446258) / 643.5719743567579 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 8c795cbde56c3047e8a7f73f4963ec513607f44d
Compare a30de6d40f69ef612b514344a5ec83fffd10b957
Diff Diff

Improvements in System.Xml.Linq.Perf_XName

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
NonEmptyNameSpaceToString - Duration of single invocation 453.10 ns 345.91 ns 0.76 0.07 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Xml.Linq.Perf_XName*'

Payloads

Baseline
Compare

Histogram

System.Xml.Linq.Perf_XName.NonEmptyNameSpaceToString

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 345.9062365715606 < 430.5976377679428.
IsChangePoint: Marked as a change because one of 12/8/2022 9:11:55 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 78.81808705049727 (T) = (0 -349.15902135832346) / Math.Sqrt((236.4975288313838 / (299)) + (35.00667140204116 / (21))) is greater than 1.967451947860886 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (21) - 2, .975) and 0.2613966703616176 = (472.72873997098077 - 349.15902135832346) / 472.72873997098077 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 8c795cbde56c3047e8a7f73f4963ec513607f44d
Compare a30de6d40f69ef612b514344a5ec83fffd10b957
Diff Diff

Improvements in Microsoft.Extensions.Logging.FormattingOverhead

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
TwoArguments - Duration of single invocation 1.97 μs 1.85 μs 0.94 0.07 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Microsoft.Extensions.Logging.FormattingOverhead*'

Payloads

Baseline
Compare

Histogram

Microsoft.Extensions.Logging.FormattingOverhead.TwoArguments

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.8514998797089546 < 1.8631889700610884.
IsChangePoint: Marked as a change because one of 9/21/2022 9:54:19 PM, 10/11/2022 6:23:40 AM, 12/8/2022 9:11:55 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 21.588658491688314 (T) = (0 -1850.1741555643114) / Math.Sqrt((5955.441026204053 / (299)) + (924.846699983124 / (21))) is greater than 1.967451947860886 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (21) - 2, .975) and 0.08535224957697675 = (2022.8269896346528 - 1850.1741555643114) / 2022.8269896346528 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 8c795cbde56c3047e8a7f73f4963ec513607f44d
Compare a30de6d40f69ef612b514344a5ec83fffd10b957
Diff Diff

Improvements in System.Security.Cryptography.X509Certificates.Tests.X509Certificate2Tests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
CertProp - Duration of single invocation 948.00 ns 812.33 ns 0.86 0.04 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Security.Cryptography.X509Certificates.Tests.X509Certificate2Tests*'

Payloads

Baseline
Compare

Histogram

System.Security.Cryptography.X509Certificates.Tests.X509Certificate2Tests.CertProp

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 812.3286921363548 < 884.3004224797127.
IsChangePoint: Marked as a change because one of 10/22/2022 11:54:51 PM, 12/8/2022 9:11:55 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 43.00961897190931 (T) = (0 -794.9327067572818) / Math.Sqrt((3047.7373122625827 / (299)) + (131.9556968482506 / (21))) is greater than 1.967451947860886 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (21) - 2, .975) and 0.1800719845673165 = (969.5152401126196 - 794.9327067572818) / 969.5152401126196 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 8c795cbde56c3047e8a7f73f4963ec513607f44d
Compare a30de6d40f69ef612b514344a5ec83fffd10b957
Diff Diff

Improvements in System.Tests.Perf_String

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
TrimEnd - Duration of single invocation 280.33 ns 202.10 ns 0.72 0.07 False
Substring_Int - Duration of single invocation 198.63 ns 95.24 ns 0.48 0.08 False
Trim_CharArr - Duration of single invocation 311.94 ns 224.02 ns 0.72 0.09 False
ToLower - Duration of single invocation 451.63 ns 377.96 ns 0.84 0.14 False
ToUpperInvariant - Duration of single invocation 378.31 ns 321.59 ns 0.85 0.15 False
Trim - Duration of single invocation 408.22 ns 349.93 ns 0.86 0.08 False
ToLowerInvariant - Duration of single invocation 392.29 ns 321.09 ns 0.82 0.10 False
Remove_IntInt - Duration of single invocation 247.49 ns 156.96 ns 0.63 0.07 False
CtorCharCount - Duration of single invocation 502.36 ns 414.57 ns 0.83 0.05 False
Remove_Int - Duration of single invocation 204.38 ns 111.41 ns 0.55 0.08 False
Split - Duration of single invocation 7.39 μs 5.25 μs 0.71 0.04 False
TrimStart_CharArr - Duration of single invocation 252.90 ns 173.98 ns 0.69 0.09 False
Split - Duration of single invocation 6.44 μs 4.52 μs 0.70 0.04 False
Join_List - Duration of single invocation 987.31 ns 925.06 ns 0.94 0.10 False
Join_Enumerable - Duration of single invocation 4.31 μs 3.94 μs 0.91 0.03 False
ToUpper - Duration of single invocation 427.52 ns 362.68 ns 0.85 0.10 False
Substring_Int - Duration of single invocation 191.97 ns 94.88 ns 0.49 0.09 False
Remove_IntInt - Duration of single invocation 270.45 ns 177.35 ns 0.66 0.08 False
PadLeft - Duration of single invocation 284.59 ns 195.66 ns 0.69 0.12 False
Substring_IntInt - Duration of single invocation 188.52 ns 97.97 ns 0.52 0.06 False
ToLower - Duration of single invocation 461.32 ns 374.89 ns 0.81 0.11 False
Remove_IntInt - Duration of single invocation 247.88 ns 157.99 ns 0.64 0.07 False
Trim - Duration of single invocation 355.24 ns 256.46 ns 0.72 0.06 False
Concat_str_str - Duration of single invocation 373.79 ns 254.75 ns 0.68 0.03 False
TrimStart - Duration of single invocation 283.56 ns 208.15 ns 0.73 0.11 False
ToUpperInvariant - Duration of single invocation 374.35 ns 324.16 ns 0.87 0.10 False
Join_Array - Duration of single invocation 945.95 ns 849.04 ns 0.90 0.07 False
Concat_str_str_str - Duration of single invocation 481.50 ns 365.11 ns 0.76 0.03 False
Trim_CharArr - Duration of single invocation 274.88 ns 202.73 ns 0.74 0.10 False
Insert - Duration of single invocation 265.78 ns 188.91 ns 0.71 0.10 False
TrimEnd_CharArr - Duration of single invocation 253.93 ns 176.47 ns 0.69 0.12 False
ToUpper - Duration of single invocation 429.09 ns 361.38 ns 0.84 0.12 False
Insert - Duration of single invocation 273.85 ns 174.97 ns 0.64 0.09 False
ToLowerInvariant - Duration of single invocation 399.85 ns 326.98 ns 0.82 0.11 False
CtorCharCount - Duration of single invocation 165.22 ns 83.10 ns 0.50 0.05 False
Substring_IntInt - Duration of single invocation 184.57 ns 94.03 ns 0.51 0.08 False
CtorCharCount - Duration of single invocation 201.67 ns 110.50 ns 0.55 0.08 False
Trim - Duration of single invocation 332.28 ns 252.25 ns 0.76 0.08 False
Substring_IntInt - Duration of single invocation 194.77 ns 102.68 ns 0.53 0.06 False
Trim_CharArr - Duration of single invocation 295.51 ns 197.66 ns 0.67 0.11 False
Remove_Int - Duration of single invocation 201.69 ns 106.13 ns 0.53 0.10 False
Concat_str_str_str_str - Duration of single invocation 618.75 ns 491.41 ns 0.79 0.05 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_String*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_String.TrimEnd(s: "Test ")

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 202.09886794915204 < 265.9766210423043.
IsChangePoint: Marked as a change because one of 12/8/2022 9:11:55 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 90.76285883408214 (T) = (0 -196.78973121648033) / Math.Sqrt((81.22241560960478 / (299)) + (16.302970121006354 / (21))) is greater than 1.967451947860886 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (21) - 2, .975) and 0.3207223900318258 = (289.7044276576412 - 196.78973121648033) / 289.7044276576412 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.Substring_Int(s: "dzsdzsDDZSDZSDZSddsz", i: 10)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 95.23542682530979 < 184.68226752580046.
IsChangePoint: Marked as a change because one of 12/8/2022 9:11:55 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 175.65410216524512 (T) = (0 -95.08983965992535) / Math.Sqrt((54.16639093005168 / (299)) + (4.590113198531304 / (21))) is greater than 1.967451947860886 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (21) - 2, .975) and 0.5387268478410229 = (206.14648655543854 - 95.08983965992535) / 206.14648655543854 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.Trim_CharArr(s: " Te st  ", c: [' ', ' '])

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 224.02052580865933 < 298.4013223842638.
IsChangePoint: Marked as a change because one of 12/8/2022 9:11:55 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 87.54936792792148 (T) = (0 -220.43692965580505) / Math.Sqrt((60.0633292563787 / (299)) + (17.585438205300427 / (21))) is greater than 1.967451947860886 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (21) - 2, .975) and 0.288101023562006 = (309.6463640933539 - 220.43692965580505) / 309.6463640933539 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.ToLower(s: "TEST")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 377.9586463076851 < 435.420133475981.
IsChangePoint: Marked as a change because one of 12/8/2022 9:11:55 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 45.62071681837219 (T) = (0 -368.667913956953) / Math.Sqrt((187.2615438309284 / (299)) + (73.67084142242936 / (21))) is greater than 1.967451947860886 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (21) - 2, .975) and 0.20103151070166975 = (461.4298547377311 - 368.667913956953) / 461.4298547377311 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.ToUpperInvariant(s: "TeSt")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 321.58868305924517 < 364.2324585535894.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/8/2022 9:11:55 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 41.078140939000214 (T) = (0 -302.29262094206496) / Math.Sqrt((132.4856797869297 / (299)) + (87.47247382248104 / (21))) is greater than 1.967451947860886 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (21) - 2, .975) and 0.22583622506891657 = (390.47631874660533 - 302.29262094206496) / 390.47631874660533 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.Trim(s: " Te st  ")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 349.9273193615588 < 394.8601849684624.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/8/2022 9:11:55 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 47.571648224006076 (T) = (0 -330.06559236356173) / Math.Sqrt((228.14728120876768 / (299)) + (80.46181681877968 / (21))) is greater than 1.967451947860886 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (21) - 2, .975) and 0.23602107723237045 = (432.03494563416626 - 330.06559236356173) / 432.03494563416626 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.ToLowerInvariant(s: "TeSt")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 321.09415016979244 < 381.2530674254907.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/8/2022 9:11:55 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 59.74134048941857 (T) = (0 -319.49083392344596) / Math.Sqrt((146.9057155333089 / (299)) + (36.115153076275185 / (21))) is greater than 1.967451947860886 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (21) - 2, .975) and 0.2175568743830992 = (408.32467365797373 - 319.49083392344596) / 408.32467365797373 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.Remove_IntInt(s: "dzsdzsDDZSDZSDZSddsz", i1: 7, i2: 4)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 156.95503374194104 < 242.8556613443186.
IsChangePoint: Marked as a change because one of 11/18/2022 6:07:58 AM, 12/8/2022 9:11:55 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 72.20686473936239 (T) = (0 -152.59553296747404) / Math.Sqrt((89.41961663890933 / (299)) + (21.16346790055659 / (21))) is greater than 1.967451947860886 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (21) - 2, .975) and 0.35104538813565933 = (235.14053244662517 - 152.59553296747404) / 235.14053244662517 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.CtorCharCount(size: 100)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 414.5743106942465 < 478.71534263605116.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 12/8/2022 9:11:55 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 53.34207703972094 (T) = (0 -413.41050955217855) / Math.Sqrt((739.4002876382365 / (299)) + (35.17065638944159 / (21))) is greater than 1.967451947860886 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (21) - 2, .975) and 0.20809641062551135 = (522.046515635476 - 413.41050955217855) / 522.046515635476 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.Remove_Int(s: "dzsdzsDDZSDZSDZSddsz", i: 10)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 111.40662962717536 < 194.0508515986584.
IsChangePoint: Marked as a change because one of 12/8/2022 9:11:55 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 139.85989459747418 (T) = (0 -110.45603428028645) / Math.Sqrt((60.894626005673544 / (299)) + (9.137736895021217 / (21))) is greater than 1.967451947860886 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (21) - 2, .975) and 0.5029836780415409 = (222.23824329358428 - 110.45603428028645) / 222.23824329358428 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.Split(s: "A B C D E F G H I J K L M N O P Q R S T U V W X Y Z", arr: [' '], options: RemoveEmptyEntries)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 5.24838790865154 < 7.055361437418132.
IsChangePoint: Marked as a change because one of 12/8/2022 9:11:55 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 140.86026516549862 (T) = (0 -5268.215457028756) / Math.Sqrt((43878.5143773642 / (299)) + (3273.366640372476 / (21))) is greater than 1.967451947860886 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (21) - 2, .975) and 0.3174683917449599 = (7718.639537438379 - 5268.215457028756) / 7718.639537438379 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.TrimStart_CharArr(s: " Test", c: [' ', ' '])

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 173.97591608924043 < 242.62900209789515.
IsChangePoint: Marked as a change because one of 12/8/2022 9:11:55 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 89.19310251281468 (T) = (0 -170.82547829557356) / Math.Sqrt((46.809569277791 / (299)) + (16.79167656010456 / (21))) is greater than 1.967451947860886 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (21) - 2, .975) and 0.33799206949039623 = (258.04143790856807 - 170.82547829557356) / 258.04143790856807 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.Split(s: "A B C D E F G H I J K L M N O P Q R S T U V W X Y Z", arr: [' '], options: None)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 4.516269645340752 < 6.143318757469659.
IsChangePoint: Marked as a change because one of 12/8/2022 9:11:55 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 92.40649774974364 (T) = (0 -4377.957506220407) / Math.Sqrt((25141.148819938564 / (296)) + (11406.582227476541 / (21))) is greater than 1.9675235316906419 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (296) + (21) - 2, .975) and 0.34597363541367154 = (6693.854779064547 - 4377.957506220407) / 6693.854779064547 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.Join_List

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 925.0571912819911 < 927.2697350553115.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 11/15/2022 8:48:03 PM, 12/2/2022 5:21:30 AM, 12/8/2022 9:11:55 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 21.377612423573936 (T) = (0 -901.9924656648105) / Math.Sqrt((1477.8599065468527 / (299)) + (506.339835223322 / (21))) is greater than 1.967451947860886 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (21) - 2, .975) and 0.11327843567830499 = (1017.2217547847696 - 901.9924656648105) / 1017.2217547847696 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.Join_Enumerable

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 3.9357715138523064 < 4.090559478578941.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 11/15/2022 5:29:43 PM, 12/8/2022 9:11:55 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 23.69545422917365 (T) = (0 -3724.771148142759) / Math.Sqrt((13701.597420939692 / (299)) + (14962.467837652413 / (21))) is greater than 1.967451947860886 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (21) - 2, .975) and 0.14906889053232592 = (4377.288721378283 - 3724.771148142759) / 4377.288721378283 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.ToUpper(s: "TeSt")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 362.6801102489766 < 415.4288781974695.
IsChangePoint: Marked as a change because one of 12/8/2022 9:11:55 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 47.34304487136528 (T) = (0 -351.81781683319826) / Math.Sqrt((154.99659161215433 / (299)) + (65.12142095112206 / (21))) is greater than 1.967451947860886 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (21) - 2, .975) and 0.20382796359464597 = (441.88667869022936 - 351.81781683319826) / 441.88667869022936 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.Substring_Int(s: "dzsdzsDDZSDZSDZSddsz", i: 7)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 94.87863728458782 < 182.39146647423328.
IsChangePoint: Marked as a change because one of 12/8/2022 9:11:55 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 151.8008399766105 (T) = (0 -95.97717142300988) / Math.Sqrt((51.25418355798372 / (295)) + (7.093053156781312 / (21))) is greater than 1.9675476980854363 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (295) + (21) - 2, .975) and 0.5307766549601917 = (204.54474918520367 - 95.97717142300988) / 204.54474918520367 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.Remove_IntInt(s: "dzsdzsDDZSDZSDZSddsz", i1: 10, i2: 1)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 177.3545899310663 < 256.7873167455347.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 12/8/2022 9:11:55 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 90.05004425456711 (T) = (0 -167.03260895692225) / Math.Sqrt((117.14767826453837 / (299)) + (11.683279992580035 / (21))) is greater than 1.967451947860886 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (21) - 2, .975) and 0.34424187982694465 = (254.71679849399675 - 167.03260895692225) / 254.71679849399675 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.PadLeft(n: 18)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 195.65587850916148 < 270.5050276554075.
IsChangePoint: Marked as a change because one of 12/8/2022 9:11:55 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 54.782568326633296 (T) = (0 -196.3434975172138) / Math.Sqrt((77.08370449019127 / (299)) + (48.31982090521904 / (21))) is greater than 1.967451947860886 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (21) - 2, .975) and 0.30858681029273893 = (283.9741856824341 - 196.3434975172138) / 283.9741856824341 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.Substring_IntInt(s: "dzsdzsDDZSDZSDZSddsz", i1: 0, i2: 8)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 97.97000720318115 < 176.55764839048288.
IsChangePoint: Marked as a change because one of 12/8/2022 9:11:55 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 148.33908998905403 (T) = (0 -95.1995924691886) / Math.Sqrt((21.39609877472335 / (298)) + (6.945945924937751 / (21))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (298) + (21) - 2, .975) and 0.49714190368769123 = (189.31701242822038 - 95.1995924691886) / 189.31701242822038 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.ToLower(s: "TeSt")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 374.88557135070795 < 435.18413529676195.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/8/2022 9:11:55 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 28.1774122118155 (T) = (0 -367.6092578389123) / Math.Sqrt((162.17998596335084 / (299)) + (209.72107713305405 / (21))) is greater than 1.967451947860886 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (21) - 2, .975) and 0.19918000303774633 = (459.04105695832095 - 367.6092578389123) / 459.04105695832095 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.Remove_IntInt(s: "dzsdzsDDZSDZSDZSddsz", i1: 0, i2: 8)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 157.9918208663995 < 236.83040675999283.
IsChangePoint: Marked as a change because one of 11/15/2022 5:29:43 PM, 12/8/2022 9:11:55 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 78.59675998078576 (T) = (0 -151.45914499919132) / Math.Sqrt((77.93376540645271 / (299)) + (17.30727220706914 / (21))) is greater than 1.967451947860886 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (21) - 2, .975) and 0.35085437099059763 = (233.32074996841357 - 151.45914499919132) / 233.32074996841357 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.Trim(s: "Test ")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 256.46041308282264 < 313.4949151763562.
IsChangePoint: Marked as a change because one of 12/8/2022 9:11:55 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 68.91623708502938 (T) = (0 -247.28019138747817) / Math.Sqrt((98.1879676717946 / (299)) + (34.97687335033122 / (21))) is greater than 1.967451947860886 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (21) - 2, .975) and 0.2824034770872938 = (344.59502449060665 - 247.28019138747817) / 344.59502449060665 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.Concat_str_str(size: 100)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 254.748074230306 < 350.4535142813739.
IsChangePoint: Marked as a change because one of 12/3/2022 2:55:34 AM, 12/8/2022 9:11:55 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 89.98690284274943 (T) = (0 -254.44359063681108) / Math.Sqrt((351.2732855688185 / (299)) + (35.87809025414584 / (21))) is greater than 1.967451947860886 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (21) - 2, .975) and 0.3752061877005757 = (407.2440949765237 - 254.44359063681108) / 407.2440949765237 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.TrimStart(s: " Test")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 208.14634183483344 < 269.1444016529457.
IsChangePoint: Marked as a change because one of 12/8/2022 9:11:55 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 82.10865930469606 (T) = (0 -199.95326431422578) / Math.Sqrt((79.33856959083292 / (299)) + (20.077643150742045 / (21))) is greater than 1.967451947860886 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (21) - 2, .975) and 0.3121619652714632 = (290.69817924962473 - 199.95326431422578) / 290.69817924962473 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.ToUpperInvariant(s: "test")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 324.1573807876195 < 360.2431082726973.
IsChangePoint: Marked as a change because one of 12/8/2022 9:11:55 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 40.797009006465075 (T) = (0 -303.1349634665058) / Math.Sqrt((111.02846692937464 / (299)) + (87.53130922332423 / (21))) is greater than 1.967451947860886 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (21) - 2, .975) and 0.22284523477065915 = (390.0573952950667 - 303.1349634665058) / 390.0573952950667 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.Join_Array

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 849.0406737110311 < 898.7765175531637.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 11/15/2022 8:48:03 PM, 12/2/2022 5:21:30 AM, 12/8/2022 9:11:55 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 26.389587070189496 (T) = (0 -861.1202358562166) / Math.Sqrt((1445.5757446738276 / (299)) + (302.95258519964904 / (21))) is greater than 1.967451947860886 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (21) - 2, .975) and 0.11855108489949984 = (976.9372009018064 - 861.1202358562166) / 976.9372009018064 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.Concat_str_str_str(size: 100)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 365.11349925291944 < 460.90761287821533.
IsChangePoint: Marked as a change because one of 12/3/2022 2:55:34 AM, 12/8/2022 9:11:55 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 107.01897151162335 (T) = (0 -363.2052447244539) / Math.Sqrt((688.0738101153036 / (299)) + (14.44103558113603 / (21))) is greater than 1.967451947860886 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (21) - 2, .975) and 0.33748857483913214 = (548.2248772332675 - 363.2052447244539) / 548.2248772332675 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.Trim_CharArr(s: "Test ", c: [' ', ' '])

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 202.7348033665724 < 261.96351709971947.
IsChangePoint: Marked as a change because one of 12/8/2022 9:11:55 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 78.5744252479096 (T) = (0 -191.81523135019188) / Math.Sqrt((52.10031617229498 / (299)) + (23.301757882630255 / (21))) is greater than 1.967451947860886 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (21) - 2, .975) and 0.3170088407772961 = (280.8458480904676 - 191.81523135019188) / 280.8458480904676 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.Insert(s1: "Test", i: 2, s2: " Test")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 188.90515809962378 < 251.79678833492173.
IsChangePoint: Marked as a change because one of 12/8/2022 9:11:55 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 90.61793039960723 (T) = (0 -177.34735888103054) / Math.Sqrt((59.64243802397525 / (299)) + (18.549284799504086 / (21))) is greater than 1.967451947860886 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (21) - 2, .975) and 0.34712611212041045 = (271.6410660212206 - 177.34735888103054) / 271.6410660212206 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.TrimEnd_CharArr(s: "Test ", c: [' ', ' '])

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 176.47377281623858 < 242.27239966288025.
IsChangePoint: Marked as a change because one of 12/8/2022 9:11:55 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 85.78030055436001 (T) = (0 -168.94045702232384) / Math.Sqrt((53.03367635106104 / (299)) + (18.786464588963945 / (21))) is greater than 1.967451947860886 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (21) - 2, .975) and 0.3445662520686718 = (257.7536746551907 - 168.94045702232384) / 257.7536746551907 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.ToUpper(s: "test")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 361.3758031670881 < 419.190897328346.
IsChangePoint: Marked as a change because one of 12/8/2022 9:11:55 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 31.037478823465527 (T) = (0 -353.41172931428434) / Math.Sqrt((178.58797630854582 / (299)) + (159.87514345467292 / (21))) is greater than 1.967451947860886 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (21) - 2, .975) and 0.2010510339928321 = (442.3458122494318 - 353.41172931428434) / 442.3458122494318 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.Insert(s1: "dzsdzsDDZSDZSDZSddsz", i: 7, s2: "Test")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 174.96831019522676 < 261.03786348122.
IsChangePoint: Marked as a change because one of 12/8/2022 9:11:55 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 85.08306288386535 (T) = (0 -172.76450063467942) / Math.Sqrt((94.3360126933847 / (299)) + (29.556863579545194 / (21))) is greater than 1.967451947860886 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (21) - 2, .975) and 0.39262885794959446 = (284.4463437157206 - 172.76450063467942) / 284.4463437157206 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.ToLowerInvariant(s: "TEST")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 326.97713193617096 < 380.82886309652963.
IsChangePoint: Marked as a change because one of 10/10/2022 3:36:43 PM, 12/8/2022 9:11:55 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 52.21563447185237 (T) = (0 -319.7869897914702) / Math.Sqrt((128.14117065521717 / (299)) + (50.57412016038596 / (21))) is greater than 1.967451947860886 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (21) - 2, .975) and 0.21569632975602993 = (407.7336393083503 - 319.7869897914702) / 407.7336393083503 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.CtorCharCount(size: 1)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 83.09998710696003 < 160.5026347288666.
IsChangePoint: Marked as a change because one of 12/8/2022 9:11:55 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 236.78324832389544 (T) = (0 -78.86704116446246) / Math.Sqrt((17.482582578939702 / (299)) + (1.9598192389591305 / (21))) is greater than 1.967451947860886 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (21) - 2, .975) and 0.5391122406161104 = (171.11984329957295 - 78.86704116446246) / 171.11984329957295 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.Substring_IntInt(s: "dzsdzsDDZSDZSDZSddsz", i1: 7, i2: 4)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 94.02943795314731 < 175.7430922960183.
IsChangePoint: Marked as a change because one of 12/8/2022 9:11:55 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 213.8957013348835 (T) = (0 -94.89980481131394) / Math.Sqrt((23.670082336222784 / (295)) + (2.3725528416937585 / (21))) is greater than 1.9675476980854363 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (295) + (21) - 2, .975) and 0.49767359354229807 = (188.92059742693402 - 94.89980481131394) / 188.92059742693402 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.CtorCharCount(size: 10)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 110.49892664297394 < 190.89561778674118.
IsChangePoint: Marked as a change because one of 12/8/2022 9:11:55 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 236.845032764448 (T) = (0 -111.22354188172173) / Math.Sqrt((30.32758715162468 / (296)) + (1.2168312801706804 / (21))) is greater than 1.9675235316906419 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (296) + (21) - 2, .975) and 0.4602911049029982 = (206.08061659189727 - 111.22354188172173) / 206.08061659189727 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.Trim(s: " Test")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 252.25173832657964 < 310.9204300940648.
IsChangePoint: Marked as a change because one of 12/8/2022 9:11:55 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 59.86605867177386 (T) = (0 -249.26837173300555) / Math.Sqrt((102.36836875624758 / (299)) + (46.78687591430764 / (21))) is greater than 1.967451947860886 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (21) - 2, .975) and 0.2779995922456171 = (345.246857281283 - 249.26837173300555) / 345.246857281283 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.Substring_IntInt(s: "dzsdzsDDZSDZSDZSddsz", i1: 10, i2: 1)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 102.6810672357705 < 185.44622124054993.
IsChangePoint: Marked as a change because one of 12/8/2022 9:11:55 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 170.37012221544643 (T) = (0 -103.6708160543879) / Math.Sqrt((22.150002537212252 / (299)) + (4.657611450288014 / (21))) is greater than 1.967451947860886 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (21) - 2, .975) and 0.4719886722927945 = (196.34203020711664 - 103.6708160543879) / 196.34203020711664 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.Trim_CharArr(s: " Test", c: [' ', ' '])

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 197.65940834201777 < 264.96482281414546.
IsChangePoint: Marked as a change because one of 12/8/2022 9:11:55 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 102.22186522781391 (T) = (0 -191.90932710416996) / Math.Sqrt((55.66010371472732 / (299)) + (12.193876197945784 / (21))) is greater than 1.967451947860886 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (21) - 2, .975) and 0.3180750423629481 = (281.4229409774908 - 191.90932710416996) / 281.4229409774908 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.Remove_Int(s: "dzsdzsDDZSDZSDZSddsz", i: 7)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 106.13300822780289 < 190.25971092516195.
IsChangePoint: Marked as a change because one of 12/8/2022 9:11:55 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 151.28659681200267 (T) = (0 -108.97485142394996) / Math.Sqrt((25.37725490733608 / (299)) + (6.168872911718408 / (21))) is greater than 1.967451947860886 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (21) - 2, .975) and 0.4606963764030447 = (202.0658617072292 - 108.97485142394996) / 202.0658617072292 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.Concat_str_str_str_str(size: 100)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 491.40720456988373 < 597.9959973254515.
IsChangePoint: Marked as a change because one of 11/15/2022 10:02:59 PM, 12/3/2022 2:55:34 AM, 12/8/2022 9:11:55 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 94.37261950106941 (T) = (0 -491.75172464239336) / Math.Sqrt((1325.2196441206231 / (299)) + (22.157223095647666 / (21))) is greater than 1.967451947860886 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (21) - 2, .975) and 0.3101310361358388 = (712.8190285412257 - 491.75172464239336) / 712.8190285412257 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented Dec 13, 2022 •
edited
Loading

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 8c795cbde56c3047e8a7f73f4963ec513607f44d
Compare a30de6d40f69ef612b514344a5ec83fffd10b957
Diff Diff

Improvements in System.IO.Tests.StreamReaderReadLineTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ReadLineAsync - Duration of single invocation 6.31 ms 5.78 ms 0.92 0.03 False
ReadLine - Duration of single invocation 1.21 ms 900.38 μs 0.74 0.03 False
ReadLineAsync - Duration of single invocation 730.38 μs 665.11 μs 0.91 0.02 False
ReadLine - Duration of single invocation 2.93 ms 2.08 ms 0.71 0.08 False
ReadLine - Duration of single invocation 361.85 μs 290.62 μs 0.80 0.02 False
ReadLineAsync - Duration of single invocation 307.99 μs 288.36 μs 0.94 0.01 False
ReadLineAsync - Duration of single invocation 2.61 ms 2.41 ms 0.92 0.01 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.IO.Tests.StreamReaderReadLineTests*'

Payloads

Baseline
Compare

Histogram

System.IO.Tests.StreamReaderReadLineTests.ReadLineAsync(LineLengthRange: [ 1, 1])

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 5.781684168055555 < 5.950409685750532.
IsChangePoint: Marked as a change because one of 11/8/2022 2:29:58 AM, 12/8/2022 9:11:55 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 11.745360132591372 (T) = (0 -5466113.484429872) / Math.Sqrt((134986778100.60608 / (299)) + (12306311181.999872 / (21))) is greater than 1.967451947860886 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (21) - 2, .975) and 0.06473112067097818 = (5844429.9871833185 - 5466113.484429872) / 5844429.9871833185 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.StreamReaderReadLineTests.ReadLine(LineLengthRange: [   1,    8])

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 900.3844960407242 < 1.1644287170596033.
IsChangePoint: Marked as a change because one of 11/7/2022 4:10:19 PM, 12/8/2022 9:11:55 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 24.296740886869877 (T) = (0 -910657.2228271158) / Math.Sqrt((15411408664.283915 / (299)) + (289536456.0534099 / (21))) is greater than 1.967451947860886 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (21) - 2, .975) and 0.17739548919341286 = (1107041.3678308066 - 910657.2228271158) / 1107041.3678308066 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.StreamReaderReadLineTests.ReadLineAsync(LineLengthRange: [   9,   32])

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 665.1119619791667 < 696.5620527088797.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 11/15/2022 5:29:43 PM, 12/8/2022 9:11:55 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 23.489978388317272 (T) = (0 -652242.0135008564) / Math.Sqrt((956257614.585796 / (299)) + (101086248.65326986 / (21))) is greater than 1.967451947860886 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (21) - 2, .975) and 0.0925085544246865 = (718730.7568363477 - 652242.0135008564) / 718730.7568363477 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.StreamReaderReadLineTests.ReadLine(LineLengthRange: [   1,    1])

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.07538456640625 < 2.682618003325892.
IsChangePoint: Marked as a change because one of 11/7/2022 4:10:19 PM, 12/8/2022 9:11:55 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 16.612060452708555 (T) = (0 -2093331.8546644116) / Math.Sqrt((132917196552.22185 / (299)) + (2667700168.0320163 / (21))) is greater than 1.967451947860886 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (21) - 2, .975) and 0.15946854719503437 = (2490486.0462730858 - 2093331.8546644116) / 2490486.0462730858 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.StreamReaderReadLineTests.ReadLine(LineLengthRange: [   9,   32])

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 290.62372434413584 < 353.1460677638197.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 12/8/2022 9:11:55 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 68.60132360235187 (T) = (0 -294690.69802976743) / Math.Sqrt((122574882.50863497 / (299)) + (29714249.879063167 / (21))) is greater than 1.967451947860886 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (21) - 2, .975) and 0.23924046592691564 = (387363.7921459913 - 294690.69802976743) / 387363.7921459913 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.StreamReaderReadLineTests.ReadLineAsync(LineLengthRange: [  33,  128])

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 288.35695181818187 < 292.44852837928926.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 12/8/2022 9:11:55 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 30.289447833659352 (T) = (0 -283773.0495655833) / Math.Sqrt((35668433.7613569 / (299)) + (17577819.481900714 / (21))) is greater than 1.967451947860886 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (21) - 2, .975) and 0.09451597511808232 = (313393.7670546861 - 283773.0495655833) / 313393.7670546861 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.StreamReaderReadLineTests.ReadLineAsync(LineLengthRange: [   1,    8])

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.4102836741071427 < 2.5016034418105155.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 12/8/2022 9:11:55 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 12.596021747804704 (T) = (0 -2306408.023406313) / Math.Sqrt((21735342585.065056 / (299)) + (1976141244.162625 / (21))) is greater than 1.967451947860886 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (21) - 2, .975) and 0.06588544138924173 = (2469084.7628330174 - 2306408.023406313) / 2469084.7628330174 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS ubuntu 18.04
Baseline 8c795cbde56c3047e8a7f73f4963ec513607f44d
Compare a30de6d40f69ef612b514344a5ec83fffd10b957
Diff Diff

Improvements in System.Text.Tests.Perf_Encoding

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GetString - Duration of single invocation 1.34 μs 1.27 μs 0.95 0.14 False
GetString - Duration of single invocation 314.20 ns 230.05 ns 0.73 0.11 False
GetString - Duration of single invocation 305.67 ns 203.75 ns 0.67 0.14 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Tests.Perf_Encoding*'

Payloads

Baseline
Compare

Histogram

System.Text.Tests.Perf_Encoding.GetString(size: 512, encName: "utf-8")

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.2688463448827325 < 1.283100018729689.
IsChangePoint: Marked as a change because one of 12/8/2022 9:11:55 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 16.00987099221307 (T) = (0 -1192.791276777296) / Math.Sqrt((2330.72254025726 / (299)) + (1595.20886405346 / (21))) is greater than 1.967451947860886 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (21) - 2, .975) and 0.10940006798927664 = (1339.312112998156 - 1192.791276777296) / 1339.312112998156 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Tests.Perf_Encoding.GetString(size: 16, encName: "ascii")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 230.04547785188262 < 301.77396138787515.
IsChangePoint: Marked as a change because one of 12/8/2022 9:11:55 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 74.71255939095616 (T) = (0 -225.19734397294394) / Math.Sqrt((89.46493916770011 / (298)) + (26.820185632819488 / (21))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (298) + (21) - 2, .975) and 0.29412162770576356 = (319.0313697259353 - 225.19734397294394) / 319.0313697259353 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Tests.Perf_Encoding.GetString(size: 16, encName: "utf-8")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 203.74584371776746 < 287.9163493978321.
IsChangePoint: Marked as a change because one of 12/8/2022 9:11:55 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 94.31573386789624 (T) = (0 -202.19037289159124) / Math.Sqrt((101.63040306756668 / (296)) + (24.215024395304177 / (21))) is greater than 1.9675235316906419 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (296) + (21) - 2, .975) and 0.3633123159306255 = (317.5660185529209 - 202.19037289159124) / 317.5660185529209 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 8c795cbde56c3047e8a7f73f4963ec513607f44d
Compare a30de6d40f69ef612b514344a5ec83fffd10b957
Diff Diff

Improvements in System.Tests.Perf_Uri

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GetComponents - Duration of single invocation 531.69 ns 431.34 ns 0.81 0.11 False
Unescape - Duration of single invocation 1.10 μs 1.04 μs 0.94 0.04 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Uri*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Uri.GetComponents

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 431.3357164821648 < 502.0961805384608.
IsChangePoint: Marked as a change because one of 12/8/2022 9:11:55 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 32.200141913443474 (T) = (0 -429.54319230418974) / Math.Sqrt((232.80972234621174 / (299)) + (192.22291857859025 / (21))) is greater than 1.967451947860886 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (21) - 2, .975) and 0.19110200700567426 = (531.0226951041562 - 429.54319230418974) / 531.0226951041562 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Uri.Unescape

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.0417804999658495 < 1.0422442124921463.
IsChangePoint: Marked as a change because one of 10/3/2022 2:18:04 AM, 12/8/2022 9:11:55 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 23.585916205483734 (T) = (0 -986.4532785033615) / Math.Sqrt((345.00425031671296 / (299)) + (542.7919001112131 / (21))) is greater than 1.967451947860886 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (21) - 2, .975) and 0.1105114485280572 = (1109.0117763415387 - 986.4532785033615) / 1109.0117763415387 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 8c795cbde56c3047e8a7f73f4963ec513607f44d
Compare a30de6d40f69ef612b514344a5ec83fffd10b957
Diff Diff

Improvements in System.Tests.Perf_Double

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ToStringWithCultureInfo - Duration of single invocation 1.63 μs 1.51 μs 0.93 0.04 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Double*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Double.ToStringWithCultureInfo(value: 12345, culture: zh)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.509256890617864 < 1.55645474114073.
IsChangePoint: Marked as a change because one of 9/28/2022 6:26:25 AM, 11/15/2022 8:48:03 PM, 12/3/2022 2:55:34 AM, 12/8/2022 9:11:55 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 30.44060108981933 (T) = (0 -1499.442836620047) / Math.Sqrt((2119.785518568005 / (299)) + (578.964398938009 / (21))) is greater than 1.967451947860886 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (21) - 2, .975) and 0.10675844269317171 = (1678.6532426244794 - 1499.442836620047) / 1678.6532426244794 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 8c795cbde56c3047e8a7f73f4963ec513607f44d
Compare a30de6d40f69ef612b514344a5ec83fffd10b957
Diff Diff

Improvements in Microsoft.Extensions.Primitives.StringSegmentBenchmark

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
SubString - Duration of single invocation 207.30 ns 118.67 ns 0.57 0.07 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Microsoft.Extensions.Primitives.StringSegmentBenchmark*'

Payloads

Baseline
Compare

Histogram

Microsoft.Extensions.Primitives.StringSegmentBenchmark.SubString

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 118.6664901181421 < 197.52466962378517.
IsChangePoint: Marked as a change because one of 12/8/2022 9:11:55 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 190.80980071180937 (T) = (0 -116.53686472768281) / Math.Sqrt((22.074631230722883 / (299)) + (3.143305534799363 / (21))) is greater than 1.967451947860886 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (21) - 2, .975) and 0.43632722803904694 = (206.7455987314491 - 116.53686472768281) / 206.7455987314491 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 8c795cbde56c3047e8a7f73f4963ec513607f44d
Compare a30de6d40f69ef612b514344a5ec83fffd10b957
Diff Diff

Improvements in System.Tests.Perf_Version

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ToString2 - Duration of single invocation 553.43 ns 509.99 ns 0.92 0.08 False
ToString3 - Duration of single invocation 704.18 ns 644.60 ns 0.92 0.06 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Version*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Version.ToString2

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 509.9883045667402 < 531.7519419073268.
IsChangePoint: Marked as a change because one of 10/3/2022 8:45:37 PM, 10/10/2022 9:53:50 PM, 12/8/2022 9:11:55 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 27.129455768846917 (T) = (0 -493.47765269061176) / Math.Sqrt((836.0782122503247 / (295)) + (159.0740993861239 / (21))) is greater than 1.9675476980854363 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (295) + (21) - 2, .975) and 0.1506494595133117 = (581.0058735087671 - 493.47765269061176) / 581.0058735087671 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Version.ToString3

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 644.6022137469638 < 682.4007166573844.
IsChangePoint: Marked as a change because one of 10/3/2022 8:45:37 PM, 10/10/2022 9:53:50 PM, 12/8/2022 9:11:55 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 20.356264713332294 (T) = (0 -652.7929875250368) / Math.Sqrt((1571.1767540496157 / (299)) + (188.77349091923298 / (21))) is greater than 1.967451947860886 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (21) - 2, .975) and 0.10529727506897334 = (729.6199836379856 - 652.7929875250368) / 729.6199836379856 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 8c795cbde56c3047e8a7f73f4963ec513607f44d
Compare a30de6d40f69ef612b514344a5ec83fffd10b957
Diff Diff

Improvements in System.Diagnostics.Tracing.Perf_EventListener

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
WriteEvent_MixedParams - Duration of single invocation 892.12 ns 812.29 ns 0.91 0.03 False
WriteEvent_StringParams - Duration of single invocation 1.16 μs 900.12 ns 0.77 0.03 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Diagnostics.Tracing.Perf_EventListener*'

Payloads

Baseline
Compare

Histogram

System.Diagnostics.Tracing.Perf_EventListener.WriteEvent_MixedParams

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 812.2866783216783 < 867.5684535995839.
IsChangePoint: Marked as a change because one of 12/3/2022 2:55:34 AM, 12/8/2022 9:11:55 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 45.91781965206618 (T) = (0 -812.3529006663221) / Math.Sqrt((814.9656508425143 / (299)) + (164.14996536927956 / (21))) is greater than 1.967451947860886 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (21) - 2, .975) and 0.15506915700743307 = (961.4430664989567 - 812.3529006663221) / 961.4430664989567 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Diagnostics.Tracing.Perf_EventListener.WriteEvent_StringParams

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 900.1242529424375 < 1.0956545121178807.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 12/8/2022 9:11:55 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 72.80354704674357 (T) = (0 -883.5695522084583) / Math.Sqrt((798.8504929893911 / (299)) + (157.90468601455007 / (21))) is greater than 1.967451947860886 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (21) - 2, .975) and 0.2082589243095945 = (1115.9829637965638 - 883.5695522084583) / 1115.9829637965638 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 8c795cbde56c3047e8a7f73f4963ec513607f44d
Compare a30de6d40f69ef612b514344a5ec83fffd10b957
Diff Diff

Improvements in System.Tests.Perf_UInt64

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ToString - Duration of single invocation 257.67 ns 174.90 ns 0.68 0.08 False
ToString - Duration of single invocation 459.19 ns 350.23 ns 0.76 0.05 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_UInt64*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_UInt64.ToString(value: 12345)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 174.89570160355964 < 247.9860468713307.
IsChangePoint: Marked as a change because one of 12/8/2022 9:11:55 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 88.63826274039715 (T) = (0 -175.52999313947714) / Math.Sqrt((39.45065940886523 / (295)) + (20.398566097605677 / (21))) is greater than 1.9675476980854363 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (295) + (21) - 2, .975) and 0.34676665699829656 = (268.70948187196166 - 175.52999313947714) / 268.70948187196166 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_UInt64.ToString(value: 18446744073709551615)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 350.2269511503755 < 432.2167285007788.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 12/8/2022 9:11:55 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 45.22046816489317 (T) = (0 -345.7060066364836) / Math.Sqrt((3015.5643842246463 / (299)) + (18.77002975113522 / (21))) is greater than 1.967451947860886 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (21) - 2, .975) and 0.3023710257065848 = (495.5442210332901 - 345.7060066364836) / 495.5442210332901 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 8c795cbde56c3047e8a7f73f4963ec513607f44d
Compare a30de6d40f69ef612b514344a5ec83fffd10b957
Diff Diff

Improvements in System.Tests.Perf_SByte

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ToString - Duration of single invocation 293.33 ns 217.86 ns 0.74 0.11 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_SByte*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_SByte.ToString(value: -128)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 217.8579823505816 < 279.1322904172701.
IsChangePoint: Marked as a change because one of 10/3/2022 8:45:37 PM, 12/8/2022 9:11:55 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 64.09527295014615 (T) = (0 -215.03690897424843) / Math.Sqrt((286.9974202364058 / (295)) + (16.490462138017214 / (21))) is greater than 1.9675476980854363 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (295) + (21) - 2, .975) and 0.2832671111333436 = (300.0237777762347 - 215.03690897424843) / 300.0237777762347 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@lewing lewing mentioned this issue Dec 13, 2022
Merged
@lewing
Copy link
Member

lewing commented Dec 13, 2022

this is dotnet/runtime#79392

@kotlarmilos kotlarmilos mentioned this issue Apr 4, 2023
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
arch-x64 branch-refs/heads/main kind-micro_mono mono-interpreter os-linux perf-improvement runtime-mono untriaged
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@lewing @kotlarmilos