Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Perf] Linux/x64: 160 Regressions on 8/17/2022 6:09:21 PM #7981

Closed
performanceautofiler bot opened this issue Aug 23, 2022 · 12 comments
Closed

[Perf] Linux/x64: 160 Regressions on 8/17/2022 6:09:21 PM #7981

performanceautofiler bot opened this issue Aug 23, 2022 · 12 comments
Labels
arch-wasm arch-x64 branch-refs/heads/main kind-micro os-linux perf-regression PGO Applied if there were any profile guided optimization updates in the observed interval. wasm-aot

Comments

@performanceautofiler
Copy link

performanceautofiler bot commented Aug 23, 2022 •
edited
Loading

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 543bcc5ee7d6a2b9471b016770227421c43a756e
Compare f142128e89b63577a9bbba7e2b760ec82102a7a9
Diff Diff

Regressions in System.Tests.Perf_String

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Format_OneArg - Duration of single invocation 2.24 μs 2.55 μs 1.14 0.08 False
Replace_Char - Duration of single invocation 95.02 ns 125.90 ns 1.33 0.19 False
Replace_Char - Duration of single invocation 69.37 ns 98.36 ns 1.42 0.26 False
Format_OneArg - Duration of single invocation 1.10 μs 1.43 μs 1.30 0.09 False
Replace_String - Duration of single invocation 136.69 ns 363.61 ns 2.66 0.12 False
Replace_String - Duration of single invocation 162.05 ns 764.76 ns 4.72 0.07 False
Replace_Char - Duration of single invocation 24.69 ns 289.36 ns 11.72 0.55 False
IndexOfAny - Duration of single invocation 79.07 ns 802.44 ns 10.15 0.31 False
Replace_String - Duration of single invocation 31.44 ns 280.15 ns 8.91 0.52 False
Replace_Char - Duration of single invocation 15.32 ns 76.85 ns 5.02 0.68 False
Replace_String - Duration of single invocation 146.76 ns 233.28 ns 1.59 0.07 False
Replace_Char - Duration of single invocation 170.36 ns 364.96 ns 2.14 0.02 False
Format_MultipleArgs - Duration of single invocation 529.02 ns 854.38 ns 1.62 0.13 False

graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_String*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_String.Format_OneArg(s: "Testing {0}, {0:C}, {0:E} - {0:F4}{0:G}{0:N} , !!", o: 3.14159)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.5516874625643573 > 2.350611819642669.
IsChangePoint: Marked as a change because one of 6/24/2022 5:01:29 PM, 6/28/2022 9:06:37 PM, 8/5/2022 10:47:59 PM, 8/23/2022 11:50:53 AM falls between 8/5/2022 10:47:59 PM and 8/23/2022 11:50:53 AM.
IsRegressionStdDev: Marked as regression because -38.79578953518803 (T) = (0 -2563.7163581969467) / Math.Sqrt((454.87117869714154 / (14)) + (930.0274088828977 / (28))) is less than -2.0210753903043583 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (28) - 2, .025) and -0.1398142456624475 = (2249.2404950658806 - 2563.7163581969467) / 2249.2404950658806 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.Replace_Char(text: "This is a very nice sentence", oldChar: 'i', newChar: 'I')

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 125.90268824435714 > 101.73812095373543.
IsChangePoint: Marked as a change because one of 6/24/2022 12:39:19 PM, 6/29/2022 4:06:39 AM, 8/5/2022 10:47:59 PM, 8/23/2022 11:50:53 AM falls between 8/5/2022 10:47:59 PM and 8/23/2022 11:50:53 AM.
IsRegressionStdDev: Marked as regression because -28.19418272941041 (T) = (0 -128.3050576893062) / Math.Sqrt((11.774123315218525 / (14)) + (18.26513705560435 / (29))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (29) - 2, .025) and -0.36332811572868035 = (94.11164943277716 - 128.3050576893062) / 94.11164943277716 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.Replace_Char(text: "Hello", oldChar: 'l', newChar: '!')

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 98.36287430188123 > 68.20174507294821.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/23/2022 11:50:53 AM falls between 8/5/2022 10:47:59 PM and 8/23/2022 11:50:53 AM.
IsRegressionStdDev: Marked as regression because -32.646923659413666 (T) = (0 -100.2224874025874) / Math.Sqrt((6.4138230698879735 / (14)) + (21.48488045099303 / (29))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (29) - 2, .025) and -0.5544478253206164 = (64.47465509620162 - 100.2224874025874) / 64.47465509620162 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.Format_OneArg(s: "Testing {0}, {0:C}, {0:D5}, {0:E} - {0:F4}{0:G}{0:N}  {0:X} !!", o: 8)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.4344054878048782 > 1.1605645880607696.
IsChangePoint: Marked as a change because one of 6/24/2022 10:40:04 AM, 6/29/2022 1:11:29 AM, 8/5/2022 10:47:59 PM, 8/23/2022 11:50:53 AM falls between 8/5/2022 10:47:59 PM and 8/23/2022 11:50:53 AM.
IsRegressionStdDev: Marked as regression because -44.261218747176585 (T) = (0 -1471.663986324827) / Math.Sqrt((314.28462297584457 / (14)) + (1327.0239262407026 / (29))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (29) - 2, .025) and -0.33047659631188225 = (1106.117905722145 - 1471.663986324827) / 1106.117905722145 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.Replace_String(text: "This is a very nice sentence", oldValue: "nice", newValue: "bad")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 363.608794745425 > 143.77835024641746.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/23/2022 11:50:53 AM falls between 8/5/2022 10:47:59 PM and 8/23/2022 11:50:53 AM.
IsRegressionStdDev: Marked as regression because -124.96765950555645 (T) = (0 -347.3291751657902) / Math.Sqrt((10.319928796325312 / (14)) + (57.48832107905517 / (28))) is less than -2.0210753903043583 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (28) - 2, .025) and -1.5063241969077996 = (138.5811043895721 - 347.3291751657902) / 138.5811043895721 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.Replace_String(text: "This is a very nice sentence. This is another very nice sentence.", oldValue: "a", newValue: "")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 764.759308027231 > 167.68205416757223.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/23/2022 11:50:53 AM falls between 8/5/2022 10:47:59 PM and 8/23/2022 11:50:53 AM.
IsRegressionStdDev: Marked as regression because -134.87212820251094 (T) = (0 -782.548426892428) / Math.Sqrt((13.322034129219215 / (14)) + (565.8316900027708 / (28))) is less than -2.0210753903043583 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (28) - 2, .025) and -3.826408896964876 = (162.13885802020204 - 782.548426892428) / 162.13885802020204 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.Replace_Char(text: "This is a very nice sentence", oldChar: 'z', newChar: 'y')

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 289.35599893092285 > 25.287906320219577.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/23/2022 11:50:53 AM falls between 8/5/2022 10:47:59 PM and 8/23/2022 11:50:53 AM.
IsRegressionStdDev: Marked as regression because -99.98218219727052 (T) = (0 -288.377274892454) / Math.Sqrt((3.3652378870109554 / (14)) + (188.46364743346518 / (28))) is less than -2.0210753903043583 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (28) - 2, .025) and -10.821810471096704 = (24.393664202070514 - 288.377274892454) / 24.393664202070514 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.IndexOfAny

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 802.4409741362872 > 73.09304220658181.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/23/2022 11:50:53 AM falls between 8/5/2022 10:47:59 PM and 8/23/2022 11:50:53 AM.
IsRegressionStdDev: Marked as regression because -182.10788141307557 (T) = (0 -813.7766362361342) / Math.Sqrt((22.196872375281252 / (14)) + (404.2380861527335 / (27))) is less than -2.0226909200346674 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (27) - 2, .025) and -10.18313001926548 = (72.76823526456539 - 813.7766362361342) / 72.76823526456539 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.Replace_String(text: "This is a very nice sentence", oldValue: "bad", newValue: "nice")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 280.1536335030515 > 35.636487595757515.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/23/2022 11:50:53 AM falls between 8/5/2022 10:47:59 PM and 8/23/2022 11:50:53 AM.
IsRegressionStdDev: Marked as regression because -273.36779454336 (T) = (0 -281.2164450470928) / Math.Sqrt((2.8023748373775828 / (14)) + (15.494713214793327 / (25))) is less than -2.026192463026769 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (25) - 2, .025) and -7.350228598819173 = (33.677694175565485 - 281.2164450470928) / 33.677694175565485 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.Replace_Char(text: "Hello", oldChar: 'a', newChar: 'b')

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 76.84936149358121 > 18.282816393967657.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/23/2022 11:50:53 AM falls between 8/5/2022 10:47:59 PM and 8/23/2022 11:50:53 AM.
IsRegressionStdDev: Marked as regression because -46.79705112899162 (T) = (0 -65.5071128647858) / Math.Sqrt((7.556530895537607 / (14)) + (15.565143026298111 / (28))) is less than -2.0210753903043583 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (28) - 2, .025) and -2.964579874608719 = (16.523090702328446 - 65.5071128647858) / 16.523090702328446 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.Replace_String(text: "This is a very nice sentence. This is another very nice sentence.", oldValue: "a", newValue: "b")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 233.28303305149277 > 152.85767414876875.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/23/2022 11:50:53 AM falls between 8/5/2022 10:47:59 PM and 8/23/2022 11:50:53 AM.
IsRegressionStdDev: Marked as regression because -83.0389948012548 (T) = (0 -237.70080891605662) / Math.Sqrt((9.63630059919198 / (14)) + (13.794542493264334 / (27))) is less than -2.0226909200346674 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (27) - 2, .025) and -0.6195919590567344 = (146.76586135590338 - 237.70080891605662) / 146.76586135590338 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.Replace_Char(text: "yfesgj0sg1ijslnjsb3uofdz3tbzf6ysgblu3at20nfab2wei1kxfbvsbpzwhanjczcqa2psra3aacxb67qnwbnfp2tok6v0a58l", oldChar: 'b', newChar: '+')

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 364.95583104041975 > 181.89205555824867.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/23/2022 11:50:53 AM falls between 8/5/2022 10:47:59 PM and 8/23/2022 11:50:53 AM.
IsRegressionStdDev: Marked as regression because -116.16669505847699 (T) = (0 -353.66583335705974) / Math.Sqrt((12.87366151610382 / (14)) + (40.051677336329575 / (28))) is less than -2.0210753903043583 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (28) - 2, .025) and -1.014191277467959 = (175.58701465615187 - 353.66583335705974) / 175.58701465615187 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.Format_MultipleArgs

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 854.3849195442017 > 547.6817332098481.
IsChangePoint: Marked as a change because one of 6/24/2022 10:40:04 AM, 6/27/2022 8:54:23 AM, 8/5/2022 10:47:59 PM, 8/23/2022 11:50:53 AM falls between 8/5/2022 10:47:59 PM and 8/23/2022 11:50:53 AM.
IsRegressionStdDev: Marked as regression because -97.8970278454593 (T) = (0 -852.4908035609617) / Math.Sqrt((88.87602574889881 / (14)) + (136.77708491956312 / (28))) is less than -2.0210753903043583 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (28) - 2, .025) and -0.6257128366247446 = (524.3796963127123 - 852.4908035609617) / 524.3796963127123 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
@performanceautofiler performanceautofiler bot added AOT=true PGO Applied if there were any profile guided optimization updates in the observed interval. labels Aug 23, 2022
@performanceautofiler
Copy link
Author

performanceautofiler bot commented Aug 23, 2022 •
edited
Loading

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 543bcc5ee7d6a2b9471b016770227421c43a756e
Compare f142128e89b63577a9bbba7e2b760ec82102a7a9
Diff Diff

Regressions in System.Collections.Concurrent.IsEmpty<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Dictionary - Duration of single invocation 41.46 ns 49.66 ns 1.20 0.63 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.Concurrent.IsEmpty&lt;String&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.Concurrent.IsEmpty<String>.Dictionary(Size: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 49.658626368610264 > 36.18699342442162.
IsChangePoint: Marked as a change because one of 6/26/2022 6:49:05 AM, 8/5/2022 10:47:59 PM, 8/23/2022 11:50:53 AM falls between 8/5/2022 10:47:59 PM and 8/23/2022 11:50:53 AM.
IsRegressionStdDev: Marked as regression because -10.72522395181331 (T) = (0 -51.81463297191358) / Math.Sqrt((15.636176589741593 / (14)) + (11.830566589709452 / (29))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (29) - 2, .025) and -0.34336636417027694 = (38.5707386710673 - 51.81463297191358) / 38.5707386710673 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS ubuntu 18.04
Baseline 543bcc5ee7d6a2b9471b016770227421c43a756e
Compare f142128e89b63577a9bbba7e2b760ec82102a7a9
Diff Diff

Regressions in System.Tests.Perf_Uri

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Unescape - Duration of single invocation 172.19 ns 188.20 ns 1.09 0.16 False
BuilderToString - Duration of single invocation 441.53 ns 641.20 ns 1.45 0.15 False

graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_Uri*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Uri.Unescape

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 188.19808512769364 > 172.96289854910964.
IsChangePoint: Marked as a change because one of 6/28/2022 4:38:49 AM, 6/28/2022 4:04:15 PM, 8/5/2022 10:47:59 PM, 8/23/2022 11:50:53 AM falls between 8/5/2022 10:47:59 PM and 8/23/2022 11:50:53 AM.
IsRegressionStdDev: Marked as regression because -9.501141689992092 (T) = (0 -184.39353084358436) / Math.Sqrt((38.47472329519107 / (14)) + (36.05622696387691 / (28))) is less than -2.0210753903043583 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (28) - 2, .025) and -0.11546701986899406 = (165.3061252005823 - 184.39353084358436) / 165.3061252005823 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Uri.BuilderToString

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 641.2047211863875 > 462.0218121432754.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/23/2022 11:50:53 AM falls between 8/5/2022 10:47:59 PM and 8/23/2022 11:50:53 AM.
IsRegressionStdDev: Marked as regression because -72.5076031055035 (T) = (0 -636.9472204593927) / Math.Sqrt((36.363894411937835 / (14)) + (140.6022416797446 / (29))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (29) - 2, .025) and -0.4505872348826976 = (439.09611579540734 - 636.9472204593927) / 439.09611579540734 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 543bcc5ee7d6a2b9471b016770227421c43a756e
Compare f142128e89b63577a9bbba7e2b760ec82102a7a9
Diff Diff

Regressions in System.Text.Json.Document.Tests.Perf_EnumerateArray

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Parse - Duration of single invocation 18.34 μs 23.25 μs 1.27 0.06 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Json.Document.Tests.Perf_EnumerateArray*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Document.Tests.Perf_EnumerateArray.Parse(TestCase: ArrayOfNumbers)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 23.248668515779087 > 19.37420696056155.
IsChangePoint: Marked as a change because one of 6/24/2022 10:40:04 AM, 6/28/2022 9:06:37 PM, 8/5/2022 10:47:59 PM, 8/23/2022 11:50:53 AM falls between 8/5/2022 10:47:59 PM and 8/23/2022 11:50:53 AM.
IsRegressionStdDev: Marked as regression because -68.65658633148475 (T) = (0 -23448.63360165593) / Math.Sqrt((34770.20889890047 / (14)) + (84812.94881458329 / (28))) is less than -2.0210753903043583 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (28) - 2, .025) and -0.2777790320564871 = (18351.08654421818 - 23448.63360165593) / 18351.08654421818 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 543bcc5ee7d6a2b9471b016770227421c43a756e
Compare f142128e89b63577a9bbba7e2b760ec82102a7a9
Diff Diff

Regressions in System.Text.Json.Document.Tests.Perf_EnumerateObject

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Parse - Duration of single invocation 29.06 μs 33.40 μs 1.15 0.04 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Json.Document.Tests.Perf_EnumerateObject*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Document.Tests.Perf_EnumerateObject.Parse(TestCase: NumericProperties)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 33.4046993006993 > 30.95888046708099.
IsChangePoint: Marked as a change because one of 6/24/2022 3:13:27 AM, 6/24/2022 12:39:19 PM, 7/19/2022 10:33:10 AM, 8/5/2022 10:47:59 PM, 8/23/2022 11:50:53 AM falls between 8/5/2022 10:47:59 PM and 8/23/2022 11:50:53 AM.
IsRegressionStdDev: Marked as regression because -29.5925264824497 (T) = (0 -33583.31936569467) / Math.Sqrt((65353.7388232744 / (14)) + (385678.3029517717 / (28))) is less than -2.0210753903043583 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (28) - 2, .025) and -0.13593099542234224 = (29564.5769866578 - 33583.31936569467) / 29564.5769866578 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 543bcc5ee7d6a2b9471b016770227421c43a756e
Compare f142128e89b63577a9bbba7e2b760ec82102a7a9
Diff Diff

Regressions in System.Xml.Linq.Perf_XName

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
CreateElementWithNamespaceImplicitOperator - Duration of single invocation 279.17 ns 327.45 ns 1.17 0.16 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Xml.Linq.Perf_XName*'

Payloads

Baseline
Compare

Histogram

System.Xml.Linq.Perf_XName.CreateElementWithNamespaceImplicitOperator

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 327.45271524508297 > 293.6253706633243.
IsChangePoint: Marked as a change because one of 6/25/2022 6:06:34 PM, 6/27/2022 8:54:23 AM, 7/15/2022 2:19:08 PM, 8/5/2022 10:47:59 PM, 8/23/2022 11:50:53 AM falls between 8/5/2022 10:47:59 PM and 8/23/2022 11:50:53 AM.
IsRegressionStdDev: Marked as regression because -27.527528150566102 (T) = (0 -329.38963692422175) / Math.Sqrt((34.58288337650626 / (14)) + (24.662017104467278 / (28))) is less than -2.0210753903043583 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (28) - 2, .025) and -0.1806142531492598 = (278.99852644128504 - 329.38963692422175) / 278.99852644128504 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 543bcc5ee7d6a2b9471b016770227421c43a756e
Compare f142128e89b63577a9bbba7e2b760ec82102a7a9
Diff Diff

Regressions in System.Tests.Perf_Array

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
IndexOfShort - Duration of single invocation 57.69 ns 648.94 ns 11.25 0.31 False
IndexOfChar - Duration of single invocation 52.81 ns 659.71 ns 12.49 0.36 False

graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_Array*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Array.IndexOfShort

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 648.9401649595046 > 60.52226892704296.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/23/2022 11:50:53 AM falls between 8/5/2022 10:47:59 PM and 8/23/2022 11:50:53 AM.
IsRegressionStdDev: Marked as regression because -101.79112244749088 (T) = (0 -667.1346828797216) / Math.Sqrt((7.6586987949255745 / (14)) + (1032.2073229815899 / (29))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (29) - 2, .025) and -11.086188984673008 = (55.19810121501016 - 667.1346828797216) / 55.19810121501016 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Array.IndexOfChar

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 659.7112065653679 > 54.61373471012041.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/23/2022 11:50:53 AM falls between 8/5/2022 10:47:59 PM and 8/23/2022 11:50:53 AM.
IsRegressionStdDev: Marked as regression because -255.9527641880929 (T) = (0 -657.7499326160003) / Math.Sqrt((2.3135687507552047 / (14)) + (152.07818844406543 / (28))) is less than -2.0210753903043583 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (28) - 2, .025) and -11.59140085700741 = (52.23802657747545 - 657.7499326160003) / 52.23802657747545 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented Aug 23, 2022 •
edited
Loading

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 543bcc5ee7d6a2b9471b016770227421c43a756e
Compare f142128e89b63577a9bbba7e2b760ec82102a7a9
Diff Diff

Regressions in System.Collections.ContainsFalse<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
List - Duration of single invocation 186.60 μs 2.47 ms 13.22 0.04 False
Array - Duration of single invocation 189.84 μs 2.46 ms 12.95 0.06 False
Queue - Duration of single invocation 186.46 μs 2.46 ms 13.20 0.02 False
ICollection - Duration of single invocation 186.65 μs 3.05 ms 16.31 0.03 False
Stack - Duration of single invocation 148.17 μs 2.47 ms 16.68 0.03 False

graph
graph
graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.ContainsFalse&lt;Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.ContainsFalse<Int32>.List(Size: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.4661447716836737 > 193.79015418410634.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/23/2022 11:50:53 AM falls between 8/5/2022 10:47:59 PM and 8/23/2022 11:50:53 AM.
IsRegressionStdDev: Marked as regression because -118.34933107580751 (T) = (0 -2478031.6665363153) / Math.Sqrt((1050801.4543754798 / (14)) + (10887650922.36119 / (29))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (29) - 2, .025) and -12.420529103606032 = (184644.85620544423 - 2478031.6665363153) / 184644.85620544423 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.ContainsFalse&lt;Int32&gt;.Array(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.4589952375 > 198.77967614834336.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/23/2022 11:50:53 AM falls between 8/5/2022 10:47:59 PM and 8/23/2022 11:50:53 AM.
IsRegressionStdDev: Marked as regression because -167.09345182499396 (T) = (0 -2458525.755269949) / Math.Sqrt((191255.97387730263 / (14)) + (4794575338.031776 / (26))) is less than -2.0243941639098457 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (26) - 2, .025) and -11.982517096253591 = (189372.04064837427 - 2458525.755269949) / 189372.04064837427 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.ContainsFalse&lt;Int32&gt;.Queue(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.460594778846154 > 195.68580234160373.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/23/2022 11:50:53 AM falls between 8/5/2022 10:47:59 PM and 8/23/2022 11:50:53 AM.
IsRegressionStdDev: Marked as regression because -187.54313445534967 (T) = (0 -2449870.6307675955) / Math.Sqrt((714329.7028732462 / (14)) + (4220842154.0656714 / (29))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (29) - 2, .025) and -12.107527489536787 = (186905.62600179392 - 2449870.6307675955) / 186905.62600179392 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.ContainsFalse&lt;Int32&gt;.ICollection(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.0450156183035713 > 196.43006513020836.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/23/2022 11:50:53 AM falls between 8/5/2022 10:47:59 PM and 8/23/2022 11:50:53 AM.
IsRegressionStdDev: Marked as regression because -90.76572091669816 (T) = (0 -2469922.037953116) / Math.Sqrt((1083311.6941798208 / (14)) + (17705444368.297913 / (28))) is less than -2.0210753903043583 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (28) - 2, .025) and -12.182850152757988 = (187358.7281454749 - 2469922.037953116) / 187358.7281454749 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.ContainsFalse&lt;Int32&gt;.Stack(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.471943003434066 > 155.34581797022406.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/23/2022 11:50:53 AM falls between 8/5/2022 10:47:59 PM and 8/23/2022 11:50:53 AM.
IsRegressionStdDev: Marked as regression because -118.30306713181547 (T) = (0 -2509967.9688262152) / Math.Sqrt((874624.8610895702 / (14)) + (11157762239.10775 / (28))) is less than -2.0210753903043583 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (28) - 2, .025) and -15.937818044612174 = (148187.20818792962 - 2509967.9688262152) / 148187.20818792962 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS ubuntu 18.04
Baseline 543bcc5ee7d6a2b9471b016770227421c43a756e
Compare f142128e89b63577a9bbba7e2b760ec82102a7a9
Diff Diff

Regressions in System.Memory.SequenceReader

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
TryReadTo - Duration of single invocation 534.36 ns 5.85 μs 10.95 0.03 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Memory.SequenceReader*'

Payloads

Baseline
Compare

Histogram

System.Memory.SequenceReader.TryReadTo

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 5.8493668943363275 > 561.0029986377739.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/23/2022 11:50:53 AM falls between 8/5/2022 10:47:59 PM and 8/23/2022 11:50:53 AM.
IsRegressionStdDev: Marked as regression because -167.24979779781037 (T) = (0 -5939.934692971001) / Math.Sqrt((48.84019802293152 / (14)) + (29097.491319256995 / (28))) is less than -2.0210753903043583 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (28) - 2, .025) and -10.013471188950046 = (539.3335662357398 - 5939.934692971001) / 539.3335662357398 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 543bcc5ee7d6a2b9471b016770227421c43a756e
Compare f142128e89b63577a9bbba7e2b760ec82102a7a9
Diff Diff

Regressions in System.Text.Json.Document.Tests.Perf_DocumentParse

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Parse - Duration of single invocation 2.19 μs 2.52 μs 1.15 0.02 False
Parse - Duration of single invocation 2.00 μs 2.17 μs 1.09 0.08 False
Parse - Duration of single invocation 1.43 ms 1.57 ms 1.10 0.01 False

graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Json.Document.Tests.Perf_DocumentParse*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Document.Tests.Perf_DocumentParse.Parse(IsDataIndented: True, TestRandomAccess: False, TestCase: Json400B)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.5178344108705537 > 2.311987551767818.
IsChangePoint: Marked as a change because one of 6/28/2022 4:38:49 AM, 6/28/2022 4:04:15 PM, 8/5/2022 10:47:59 PM, 8/23/2022 11:50:53 AM falls between 8/5/2022 10:47:59 PM and 8/23/2022 11:50:53 AM.
IsRegressionStdDev: Marked as regression because -28.605301000677606 (T) = (0 -2481.0323250318106) / Math.Sqrt((548.3508649960379 / (14)) + (1408.6571736135502 / (28))) is less than -2.0210753903043583 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (28) - 2, .025) and -0.12241148301720506 = (2210.4480955259255 - 2481.0323250318106) / 2210.4480955259255 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Document.Tests.Perf_DocumentParse.Parse(IsDataIndented: False, TestRandomAccess: False, TestCase: Json400B)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.1653742992131892 > 2.038905780005349.
IsChangePoint: Marked as a change because one of 6/24/2022 10:40:04 AM, 6/28/2022 7:07:47 PM, 8/5/2022 10:47:59 PM, 8/23/2022 11:50:53 AM falls between 8/5/2022 10:47:59 PM and 8/23/2022 11:50:53 AM.
IsRegressionStdDev: Marked as regression because -17.98448025285158 (T) = (0 -2177.9914915281875) / Math.Sqrt((1157.7605502438864 / (14)) + (1876.104171013748 / (29))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (29) - 2, .025) and -0.11141755015292519 = (1959.6518799154353 - 2177.9914915281875) / 1959.6518799154353 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Document.Tests.Perf_DocumentParse.Parse(IsDataIndented: True, TestRandomAccess: False, TestCase: Json400KB)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.5685566683333334 > 1.4992818779829549.
IsChangePoint: Marked as a change because one of 6/17/2022 9:56:33 AM, 7/19/2022 5:42:31 PM, 8/5/2022 10:47:59 PM, 8/23/2022 11:50:53 AM falls between 8/5/2022 10:47:59 PM and 8/23/2022 11:50:53 AM.
IsRegressionStdDev: Marked as regression because -12.01635366096734 (T) = (0 -1515867.1436410353) / Math.Sqrt((111630366.65240838 / (14)) + (1110311605.6652582 / (28))) is less than -2.0210753903043583 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (28) - 2, .025) and -0.057872770624726434 = (1432938.9939263116 - 1515867.1436410353) / 1432938.9939263116 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 543bcc5ee7d6a2b9471b016770227421c43a756e
Compare f142128e89b63577a9bbba7e2b760ec82102a7a9
Diff Diff

Regressions in MicroBenchmarks.Serializers.Xml_FromStream<XmlElement>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
DataContractSerializer_ - Duration of single invocation 5.34 μs 6.62 μs 1.24 0.11 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'MicroBenchmarks.Serializers.Xml_FromStream&lt;XmlElement&gt;*'

Payloads

Baseline
Compare

Histogram

MicroBenchmarks.Serializers.Xml_FromStream<XmlElement>.DataContractSerializer_

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 6.6229771198254435 > 5.6177409673230905.
IsChangePoint: Marked as a change because one of 6/24/2022 5:01:29 PM, 6/25/2022 2:40:01 AM, 8/5/2022 10:47:59 PM, 8/23/2022 11:50:53 AM falls between 8/5/2022 10:47:59 PM and 8/23/2022 11:50:53 AM.
IsRegressionStdDev: Marked as regression because -27.552109635727273 (T) = (0 -6838.503505916858) / Math.Sqrt((17037.856091286594 / (14)) + (38012.77532060141 / (26))) is less than -2.0243941639098457 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (26) - 2, .025) and -0.26348187163239706 = (5412.4270869685115 - 6838.503505916858) / 5412.4270869685115 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented Aug 23, 2022 •
edited
Loading

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 543bcc5ee7d6a2b9471b016770227421c43a756e
Compare f142128e89b63577a9bbba7e2b760ec82102a7a9
Diff Diff

Regressions in System.Text.RegularExpressions.Tests.Perf_Regex_Common

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
MatchWord - Duration of single invocation 2.90 μs 4.04 μs 1.39 0.17 False
Uri_IsMatch - Duration of single invocation 454.64 ns 499.44 ns 1.10 0.06 False
Backtracking - Duration of single invocation 2.45 μs 3.24 μs 1.32 0.10 False
MatchWord - Duration of single invocation 521.60 ns 1.86 μs 3.56 0.04 False
Uri_IsNotMatch - Duration of single invocation 447.94 ns 599.67 ns 1.34 0.04 False
SplitWords - Duration of single invocation 12.36 μs 49.96 μs 4.04 0.08 False
Uri_IsNotMatch - Duration of single invocation 446.25 ns 593.19 ns 1.33 0.05 False
ReplaceWords - Duration of single invocation 13.28 μs 68.75 μs 5.18 0.02 False
ReplaceWords - Duration of single invocation 110.43 μs 158.40 μs 1.43 0.11 False
MatchesWord - Duration of single invocation 4.94 μs 36.17 μs 7.32 0.04 False
MatchesWords - Duration of single invocation 11.68 μs 68.21 μs 5.84 0.04 False
SplitWords - Duration of single invocation 13.58 μs 70.03 μs 5.16 0.06 False
Uri_IsNotMatch - Duration of single invocation 444.95 ns 584.16 ns 1.31 0.06 False
MatchesWord - Duration of single invocation 4.92 μs 35.73 μs 7.26 0.03 False
MatchesWords - Duration of single invocation 108.04 μs 153.13 μs 1.42 0.11 False
MatchWord - Duration of single invocation 442.03 ns 1.36 μs 3.09 0.06 False
Backtracking - Duration of single invocation 2.54 μs 3.00 μs 1.18 0.08 False
ReplaceWords - Duration of single invocation 12.34 μs 50.75 μs 4.11 0.07 False
MatchesWords - Duration of single invocation 10.10 μs 47.92 μs 4.75 0.04 False
OneNodeBacktracking - Duration of single invocation 1.63 μs 2.07 μs 1.27 0.10 False
MatchesWord - Duration of single invocation 6.57 μs 44.84 μs 6.83 0.03 False
Uri_IsMatch - Duration of single invocation 455.72 ns 507.12 ns 1.11 0.05 False
Backtracking - Duration of single invocation 2.45 μs 2.92 μs 1.19 0.10 False
Uri_IsMatch - Duration of single invocation 453.61 ns 536.24 ns 1.18 0.03 False
OneNodeBacktracking - Duration of single invocation 1.62 μs 2.10 μs 1.30 0.11 False

graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Common*'

Payloads

Baseline
Compare

Histogram

System.Text.RegularExpressions.Tests.Perf_Regex_Common.MatchWord(Options: None)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 4.035325988188244 > 3.058069162543387.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/23/2022 11:50:53 AM falls between 8/5/2022 10:47:59 PM and 8/23/2022 11:50:53 AM.
IsRegressionStdDev: Marked as regression because -76.30398674786828 (T) = (0 -4064.880369396592) / Math.Sqrt((929.372238411742 / (14)) + (4331.177257341925 / (28))) is less than -2.0210753903043583 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (28) - 2, .025) and -0.3871588871075008 = (2930.3639310365284 - 4064.880369396592) / 2930.3639310365284 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.Uri_IsMatch(Options: Compiled)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 499.4395652795868 > 480.1444340379784.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/23/2022 11:50:53 AM falls between 8/5/2022 10:47:59 PM and 8/23/2022 11:50:53 AM.
IsRegressionStdDev: Marked as regression because -24.140732619914985 (T) = (0 -514.032947116126) / Math.Sqrt((29.752279392756684 / (14)) + (89.71558146691532 / (28))) is less than -2.0210753903043583 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (28) - 2, .025) and -0.12159956714938126 = (458.3034464096437 - 514.032947116126) / 458.3034464096437 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.Backtracking(Options: None)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.2391768820555438 > 2.5780127447204717.
IsChangePoint: Marked as a change because one of 6/24/2022 5:01:29 PM, 6/27/2022 8:24:30 PM, 8/5/2022 10:47:59 PM, 8/23/2022 11:50:53 AM falls between 8/5/2022 10:47:59 PM and 8/23/2022 11:50:53 AM.
IsRegressionStdDev: Marked as regression because -34.102895100657605 (T) = (0 -2948.173012249717) / Math.Sqrt((407.1945204507192 / (14)) + (4637.439281487996 / (27))) is less than -2.0226909200346674 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (27) - 2, .025) and -0.19607613468796728 = (2464.870694053968 - 2948.173012249717) / 2464.870694053968 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.MatchWord(Options: IgnoreCase, Compiled)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.8558130660304955 > 544.0263343584903.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/23/2022 11:50:53 AM falls between 8/5/2022 10:47:59 PM and 8/23/2022 11:50:53 AM.
IsRegressionStdDev: Marked as regression because -313.9090792931773 (T) = (0 -1874.5914525127068) / Math.Sqrt((38.76438659560182 / (14)) + (446.12672136431644 / (28))) is less than -2.0210753903043583 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (28) - 2, .025) and -2.625433113659538 = (517.0668975935074 - 1874.5914525127068) / 517.0668975935074 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.Uri_IsNotMatch(Options: Compiled)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 599.6663978022406 > 471.43001759852376.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/23/2022 11:50:53 AM falls between 8/5/2022 10:47:59 PM and 8/23/2022 11:50:53 AM.
IsRegressionStdDev: Marked as regression because -70.39812448052683 (T) = (0 -594.1614175177172) / Math.Sqrt((34.7262092388468 / (14)) + (45.861781335396856 / (29))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (29) - 2, .025) and -0.3137019879683943 = (452.2802149645615 - 594.1614175177172) / 452.2802149645615 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.SplitWords(Options: Compiled)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 49.95591707933972 > 12.554794607944187.
IsChangePoint: Marked as a change because one of 7/5/2022 3:17:18 PM, 8/5/2022 10:47:59 PM, 8/23/2022 11:50:53 AM falls between 8/5/2022 10:47:59 PM and 8/23/2022 11:50:53 AM.
IsRegressionStdDev: Marked as regression because -114.07830512895214 (T) = (0 -50771.79952086185) / Math.Sqrt((47102.17948320081 / (14)) + (2813289.612322355 / (25))) is less than -2.026192463026769 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (25) - 2, .025) and -3.2538314829713695 = (11935.545572058472 - 50771.79952086185) / 11935.545572058472 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.Uri_IsNotMatch(Options: IgnoreCase, Compiled)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 593.1908261891024 > 471.5091607783785.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/23/2022 11:50:53 AM falls between 8/5/2022 10:47:59 PM and 8/23/2022 11:50:53 AM.
IsRegressionStdDev: Marked as regression because -85.52055490717883 (T) = (0 -592.7175035215353) / Math.Sqrt((24.613985181656737 / (14)) + (29.981861154316327 / (29))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (29) - 2, .025) and -0.3176804560427088 = (449.81884705310074 - 592.7175035215353) / 449.81884705310074 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.ReplaceWords(Options: IgnoreCase, Compiled)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 68.74777898311305 > 13.843925153380757.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/23/2022 11:50:53 AM falls between 8/5/2022 10:47:59 PM and 8/23/2022 11:50:53 AM.
IsRegressionStdDev: Marked as regression because -115.47555962180213 (T) = (0 -69747.71172103864) / Math.Sqrt((8310.22713809762 / (14)) + (6696693.378337342 / (28))) is less than -2.0210753903043583 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (28) - 2, .025) and -4.282067962326111 = (13204.622170427965 - 69747.71172103864) / 13204.622170427965 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.ReplaceWords(Options: None)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 158.39701474964986 > 114.79327169646707.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/23/2022 11:50:53 AM falls between 8/5/2022 10:47:59 PM and 8/23/2022 11:50:53 AM.
IsRegressionStdDev: Marked as regression because -40.79409137549634 (T) = (0 -156678.38103898245) / Math.Sqrt((14535834.615208637 / (14)) + (9350322.924823465 / (28))) is less than -2.0210753903043583 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (28) - 2, .025) and -0.4388478645766285 = (108891.55476147858 - 156678.38103898245) / 108891.55476147858 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.MatchesWord(Options: None)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 36.170727860983106 > 5.12959258366233.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/23/2022 11:50:53 AM falls between 8/5/2022 10:47:59 PM and 8/23/2022 11:50:53 AM.
IsRegressionStdDev: Marked as regression because -237.54845244072874 (T) = (0 -36103.4703608266) / Math.Sqrt((2069.2173477453784 / (14)) + (480294.6408247448 / (28))) is less than -2.0210753903043583 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (28) - 2, .025) and -6.432087995590884 = (4857.782951741843 - 36103.4703608266) / 4857.782951741843 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.MatchesWords(Options: IgnoreCase, Compiled)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 68.20964454295704 > 12.304877370638177.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/23/2022 11:50:53 AM falls between 8/5/2022 10:47:59 PM and 8/23/2022 11:50:53 AM.
IsRegressionStdDev: Marked as regression because -144.455821292185 (T) = (0 -68499.59483560102) / Math.Sqrt((21436.907931271788 / (14)) + (4419109.736099011 / (29))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (29) - 2, .025) and -4.791889920186304 = (11826.812280541 - 68499.59483560102) / 11826.812280541 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.SplitWords(Options: IgnoreCase, Compiled)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 70.02945562130178 > 14.355911416751553.
IsChangePoint: Marked as a change because one of 7/5/2022 3:17:18 PM, 8/5/2022 10:47:59 PM, 8/23/2022 11:50:53 AM falls between 8/5/2022 10:47:59 PM and 8/23/2022 11:50:53 AM.
IsRegressionStdDev: Marked as regression because -100.81102357702365 (T) = (0 -70957.48081414145) / Math.Sqrt((19197.52841175587 / (14)) + (9007881.874340966 / (28))) is less than -2.0210753903043583 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (28) - 2, .025) and -4.195971002350081 = (13656.250348981577 - 70957.48081414145) / 13656.250348981577 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.Uri_IsNotMatch(Options: None)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 584.1606492076749 > 465.23264235782176.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/23/2022 11:50:53 AM falls between 8/5/2022 10:47:59 PM and 8/23/2022 11:50:53 AM.
IsRegressionStdDev: Marked as regression because -44.56345108726677 (T) = (0 -594.2192920145534) / Math.Sqrt((78.92038015570309 / (14)) + (148.73709475887532 / (29))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (29) - 2, .025) and -0.32638438183196494 = (447.99931313563445 - 594.2192920145534) / 447.99931313563445 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.MatchesWord(Options: Compiled)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 35.730436846191566 > 5.1369339384973.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/23/2022 11:50:53 AM falls between 8/5/2022 10:47:59 PM and 8/23/2022 11:50:53 AM.
IsRegressionStdDev: Marked as regression because -157.13486616565802 (T) = (0 -36518.4808007439) / Math.Sqrt((1729.4264890357667 / (14)) + (1132228.9654739003 / (28))) is less than -2.0210753903043583 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (28) - 2, .025) and -6.495314719261042 = (4872.174440774414 - 36518.4808007439) / 4872.174440774414 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.MatchesWords(Options: None)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 153.1279324667367 > 113.04373821591582.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/23/2022 11:50:53 AM falls between 8/5/2022 10:47:59 PM and 8/23/2022 11:50:53 AM.
IsRegressionStdDev: Marked as regression because -79.80902039920016 (T) = (0 -154858.3697745503) / Math.Sqrt((2549890.0089574326 / (14)) + (4769296.992631573 / (28))) is less than -2.0210753903043583 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (28) - 2, .025) and -0.44085654682834186 = (107476.60488160972 - 154858.3697745503) / 107476.60488160972 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.MatchWord(Options: Compiled)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.3643355481699477 > 450.6320864391529.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/23/2022 11:50:53 AM falls between 8/5/2022 10:47:59 PM and 8/23/2022 11:50:53 AM.
IsRegressionStdDev: Marked as regression because -247.33765652844005 (T) = (0 -1370.4783163965649) / Math.Sqrt((107.74354885830323 / (14)) + (189.4290444621899 / (29))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (29) - 2, .025) and -2.1323784776749295 = (437.52002708619995 - 1370.4783163965649) / 437.52002708619995 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.Backtracking(Options: IgnoreCase, Compiled)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.999530572032734 > 2.6589912583939683.
IsChangePoint: Marked as a change because one of 6/24/2022 12:39:19 PM, 6/28/2022 4:04:15 PM, 6/29/2022 4:06:39 AM, 7/12/2022 7:47:24 PM, 8/5/2022 10:47:59 PM, 8/23/2022 11:50:53 AM falls between 8/5/2022 10:47:59 PM and 8/23/2022 11:50:53 AM.
IsRegressionStdDev: Marked as regression because -41.49702945331888 (T) = (0 -3020.215935206733) / Math.Sqrt((357.5798793659266 / (14)) + (2912.3258307917017 / (27))) is less than -2.0226909200346674 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (27) - 2, .025) and -0.18863079676334427 = (2540.9201439427757 - 3020.215935206733) / 2540.9201439427757 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.ReplaceWords(Options: Compiled)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 50.75184541666666 > 12.143364455559539.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/23/2022 11:50:53 AM falls between 8/5/2022 10:47:59 PM and 8/23/2022 11:50:53 AM.
IsRegressionStdDev: Marked as regression because -108.74744631660074 (T) = (0 -50327.73885868017) / Math.Sqrt((79105.55756114118 / (14)) + (3517556.341344595 / (29))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (29) - 2, .025) and -3.3454640752341045 = (11581.671827759581 - 50327.73885868017) / 11581.671827759581 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.MatchesWords(Options: Compiled)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 47.91625816191985 > 10.696920042305413.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/23/2022 11:50:53 AM falls between 8/5/2022 10:47:59 PM and 8/23/2022 11:50:53 AM.
IsRegressionStdDev: Marked as regression because -132.370213986018 (T) = (0 -48379.15991578456) / Math.Sqrt((21361.35768695698 / (14)) + (2291568.7015327956 / (28))) is less than -2.0210753903043583 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (28) - 2, .025) and -3.762044094349303 = (10159.326322322768 - 48379.15991578456) / 10159.326322322768 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.OneNodeBacktracking(Options: None)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.074751734537456 > 1.6836555163822886.
IsChangePoint: Marked as a change because one of 6/28/2022 4:38:49 AM, 6/29/2022 1:11:29 AM, 8/5/2022 10:47:59 PM, 8/23/2022 11:50:53 AM falls between 8/5/2022 10:47:59 PM and 8/23/2022 11:50:53 AM.
IsRegressionStdDev: Marked as regression because -89.3012439249026 (T) = (0 -2089.7383825307156) / Math.Sqrt((202.6096661145524 / (14)) + (424.13739274656774 / (29))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (29) - 2, .025) and -0.2995656354124217 = (1608.0283485393409 - 2089.7383825307156) / 1608.0283485393409 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.MatchesWord(Options: IgnoreCase, Compiled)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 44.84400567949243 > 6.909777436889604.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/23/2022 11:50:53 AM falls between 8/5/2022 10:47:59 PM and 8/23/2022 11:50:53 AM.
IsRegressionStdDev: Marked as regression because -152.8255925801414 (T) = (0 -45134.60533440922) / Math.Sqrt((8725.489220011705 / (14)) + (1825465.9177857372 / (29))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (29) - 2, .025) and -5.836046917707663 = (6602.442300022166 - 45134.60533440922) / 6602.442300022166 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.Uri_IsMatch(Options: IgnoreCase, Compiled)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 507.1150262341568 > 482.7014638466213.
IsChangePoint: Marked as a change because one of 6/24/2022 10:40:04 AM, 6/25/2022 6:06:34 PM, 8/5/2022 10:47:59 PM, 8/23/2022 11:50:53 AM falls between 8/5/2022 10:47:59 PM and 8/23/2022 11:50:53 AM.
IsRegressionStdDev: Marked as regression because -32.16929589058821 (T) = (0 -514.1264164535977) / Math.Sqrt((16.440001607480827 / (14)) + (48.83558299533619 / (28))) is less than -2.0210753903043583 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (28) - 2, .025) and -0.11968537243167464 = (459.17043225905917 - 514.1264164535977) / 459.17043225905917 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.Backtracking(Options: Compiled)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.9173284510207904 > 2.591927353654952.
IsChangePoint: Marked as a change because one of 6/24/2022 6:48:53 PM, 6/29/2022 4:06:39 AM, 8/5/2022 10:47:59 PM, 8/23/2022 11:50:53 AM falls between 8/5/2022 10:47:59 PM and 8/23/2022 11:50:53 AM.
IsRegressionStdDev: Marked as regression because -62.781127431459055 (T) = (0 -2937.3927459558986) / Math.Sqrt((556.432351718883 / (14)) + (461.84190249664397 / (29))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (29) - 2, .025) and -0.18972638145776738 = (2468.964958444245 - 2937.3927459558986) / 2468.964958444245 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.Uri_IsMatch(Options: None)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 536.2418932927883 > 485.6196538514088.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/23/2022 11:50:53 AM falls between 8/5/2022 10:47:59 PM and 8/23/2022 11:50:53 AM.
IsRegressionStdDev: Marked as regression because -21.731864515652468 (T) = (0 -512.4978600947402) / Math.Sqrt((45.26367399520388 / (14)) + (69.74883981253647 / (29))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (29) - 2, .025) and -0.1119609884135343 = (460.8955398929374 - 512.4978600947402) / 460.8955398929374 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.OneNodeBacktracking(Options: Compiled)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.1045444331215433 > 1.6961022481054546.
IsChangePoint: Marked as a change because one of 6/24/2022 10:40:04 AM, 6/28/2022 9:06:37 PM, 8/5/2022 10:47:59 PM, 8/23/2022 11:50:53 AM falls between 8/5/2022 10:47:59 PM and 8/23/2022 11:50:53 AM.
IsRegressionStdDev: Marked as regression because -101.37685559790721 (T) = (0 -2102.3897553110824) / Math.Sqrt((150.92488272286548 / (14)) + (377.6563705954165 / (28))) is less than -2.0210753903043583 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (28) - 2, .025) and -0.3115503358055153 = (1602.9806084566758 - 2102.3897553110824) / 1602.9806084566758 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented Aug 23, 2022 •
edited
Loading

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 543bcc5ee7d6a2b9471b016770227421c43a756e
Compare f142128e89b63577a9bbba7e2b760ec82102a7a9
Diff Diff

Regressions in System.Text.Json.Tests.Perf_Reader

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ReadSingleSpanSequenceEmptyLoop - Duration of single invocation 11.87 μs 12.61 μs 1.06 0.05 False
ReadSingleSpanSequenceEmptyLoop - Duration of single invocation 117.84 μs 124.39 μs 1.06 0.04 False
ReadSpanEmptyLoop - Duration of single invocation 1.42 μs 1.67 μs 1.18 0.09 False
ReadMultiSpanSequenceEmptyLoop - Duration of single invocation 1.70 μs 1.96 μs 1.15 0.08 False
ReadSingleSpanSequenceEmptyLoop - Duration of single invocation 4.47 μs 4.86 μs 1.09 0.11 False
ReadReturnBytes - Duration of single invocation 11.74 μs 12.50 μs 1.06 0.04 False
ReadSingleSpanSequenceEmptyLoop - Duration of single invocation 3.71 μs 4.14 μs 1.12 0.16 False
ReadSpanEmptyLoop - Duration of single invocation 4.25 μs 4.91 μs 1.15 0.09 False
ReadReturnBytes - Duration of single invocation 1.44 μs 1.62 μs 1.13 0.07 False
ReadMultiSpanSequenceEmptyLoop - Duration of single invocation 2.02 μs 2.30 μs 1.13 0.10 False
ReadMultiSpanSequenceEmptyLoop - Duration of single invocation 5.28 μs 5.98 μs 1.13 0.11 False
ReadMultiSpanSequenceEmptyLoop - Duration of single invocation 112.87 μs 126.24 μs 1.12 0.09 False
ReadSpanEmptyLoop - Duration of single invocation 11.98 μs 13.68 μs 1.14 0.12 False
ReadReturnBytes - Duration of single invocation 1.66 μs 1.93 μs 1.17 0.04 False
ReadSpanEmptyLoop - Duration of single invocation 92.65 μs 99.80 μs 1.08 0.09 False
ReadSpanEmptyLoop - Duration of single invocation 9.36 μs 10.11 μs 1.08 0.07 False
ReadSpanEmptyLoop - Duration of single invocation 3.69 μs 4.14 μs 1.12 0.07 False
ReadSpanEmptyLoop - Duration of single invocation 1.14 μs 1.36 μs 1.20 0.07 False
ReadSingleSpanSequenceEmptyLoop - Duration of single invocation 1.20 μs 1.43 μs 1.19 0.12 False

graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Json.Tests.Perf_Reader*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Tests.Perf_Reader.ReadSingleSpanSequenceEmptyLoop(IsDataCompact: False, TestCase: Json4KB)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 12.613221351464313 > 12.461277161132662.
IsChangePoint: Marked as a change because one of 6/17/2022 12:17:51 PM, 7/19/2022 10:33:10 AM, 8/5/2022 10:47:59 PM, 8/23/2022 11:50:53 AM falls between 8/5/2022 10:47:59 PM and 8/23/2022 11:50:53 AM.
IsRegressionStdDev: Marked as regression because -13.937409746155648 (T) = (0 -12709.991858447944) / Math.Sqrt((23777.42642584325 / (14)) + (41037.65920232227 / (25))) is less than -2.026192463026769 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (25) - 2, .025) and -0.06766068433903008 = (11904.52364209372 - 12709.991858447944) / 11904.52364209372 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Reader.ReadSingleSpanSequenceEmptyLoop(IsDataCompact: False, TestCase: Json40KB)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 124.38895499338625 > 123.14741873122054.
IsChangePoint: Marked as a change because one of 6/10/2022 4:04:28 PM, 8/5/2022 10:47:59 PM, 8/23/2022 11:50:53 AM falls between 8/5/2022 10:47:59 PM and 8/23/2022 11:50:53 AM.
IsRegressionStdDev: Marked as regression because -14.0794844908739 (T) = (0 -125402.34429171348) / Math.Sqrt((1157256.9049888344 / (14)) + (6497647.846385304 / (28))) is less than -2.0210753903043583 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (28) - 2, .025) and -0.0672197948130517 = (117503.76529858183 - 125402.34429171348) / 117503.76529858183 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Reader.ReadSpanEmptyLoop(IsDataCompact: False, TestCase: Json400B)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.6705870675889043 > 1.4886446652625192.
IsChangePoint: Marked as a change because one of 6/24/2022 6:48:53 PM, 6/25/2022 6:06:34 PM, 6/28/2022 4:04:15 PM, 6/29/2022 4:06:39 AM, 8/5/2022 10:47:59 PM, 8/23/2022 11:50:53 AM falls between 8/5/2022 10:47:59 PM and 8/23/2022 11:50:53 AM.
IsRegressionStdDev: Marked as regression because -32.03032163680055 (T) = (0 -1671.6164600343445) / Math.Sqrt((561.9308404727673 / (14)) + (671.8895256702989 / (27))) is less than -2.0226909200346674 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (27) - 2, .025) and -0.18274640649037519 = (1413.3346344248205 - 1671.6164600343445) / 1413.3346344248205 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Reader.ReadMultiSpanSequenceEmptyLoop(IsDataCompact: True, TestCase: Json400B)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.9586051858513187 > 1.777974693072054.
IsChangePoint: Marked as a change because one of 6/22/2022 10:31:12 AM, 7/19/2022 8:53:09 AM, 8/5/2022 10:47:59 PM, 8/23/2022 11:50:53 AM falls between 8/5/2022 10:47:59 PM and 8/23/2022 11:50:53 AM.
IsRegressionStdDev: Marked as regression because -24.891043042714003 (T) = (0 -1938.0303145786695) / Math.Sqrt((611.4742780301284 / (14)) + (1541.716379140258 / (28))) is less than -2.0210753903043583 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (28) - 2, .025) and -0.14629178781111324 = (1690.6954539728583 - 1938.0303145786695) / 1690.6954539728583 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Reader.ReadSingleSpanSequenceEmptyLoop(IsDataCompact: False, TestCase: LotsOfNumbers)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 4.864362787982565 > 4.545769891424943.
IsChangePoint: Marked as a change because one of 6/22/2022 1:06:55 AM, 7/19/2022 10:33:10 AM, 8/5/2022 10:47:59 PM, 8/23/2022 11:50:53 AM falls between 8/5/2022 10:47:59 PM and 8/23/2022 11:50:53 AM.
IsRegressionStdDev: Marked as regression because -15.900135698565535 (T) = (0 -4883.973853298014) / Math.Sqrt((8538.298831566235 / (14)) + (14399.864292102231 / (29))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (29) - 2, .025) and -0.12144090289031392 = (4355.088030684847 - 4883.973853298014) / 4355.088030684847 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Reader.ReadReturnBytes(IsDataCompact: True, TestCase: Json4KB)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 12.503143216746157 > 12.324131044307697.
IsChangePoint: Marked as a change because one of 6/17/2022 9:56:33 AM, 7/19/2022 8:53:09 AM, 8/5/2022 10:47:59 PM, 8/23/2022 11:50:53 AM falls between 8/5/2022 10:47:59 PM and 8/23/2022 11:50:53 AM.
IsRegressionStdDev: Marked as regression because -13.063961832844592 (T) = (0 -12455.212807235222) / Math.Sqrt((29930.178573926638 / (14)) + (5847.948763613317 / (29))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (29) - 2, .025) and -0.05344399184006666 = (11823.32701473717 - 12455.212807235222) / 11823.32701473717 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Reader.ReadSingleSpanSequenceEmptyLoop(IsDataCompact: True, TestCase: LotsOfNumbers)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 4.135640140429493 > 3.875742245734397.
IsChangePoint: Marked as a change because one of 6/24/2022 10:40:04 AM, 6/29/2022 1:11:29 AM, 8/5/2022 10:47:59 PM, 8/23/2022 11:50:53 AM falls between 8/5/2022 10:47:59 PM and 8/23/2022 11:50:53 AM.
IsRegressionStdDev: Marked as regression because -12.931430690525042 (T) = (0 -4275.544933063481) / Math.Sqrt((9243.067113697243 / (14)) + (33645.48686794039 / (29))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (29) - 2, .025) and -0.1481642876451619 = (3723.809370375427 - 4275.544933063481) / 3723.809370375427 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Reader.ReadSpanEmptyLoop(IsDataCompact: False, TestCase: LotsOfNumbers)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 4.907127620607828 > 4.460535892543025.
IsChangePoint: Marked as a change because one of 6/24/2022 6:48:53 PM, 6/25/2022 5:15:47 AM, 7/19/2022 10:33:10 AM, 8/5/2022 10:47:59 PM, 8/23/2022 11:50:53 AM falls between 8/5/2022 10:47:59 PM and 8/23/2022 11:50:53 AM.
IsRegressionStdDev: Marked as regression because -18.527622473169757 (T) = (0 -4816.4828906731145) / Math.Sqrt((4758.814727171611 / (14)) + (16219.296078862555 / (27))) is less than -2.0226909200346674 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (27) - 2, .025) and -0.13375788950671846 = (4248.246415968665 - 4816.4828906731145) / 4248.246415968665 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Reader.ReadReturnBytes(IsDataCompact: True, TestCase: Json400B)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.6218510749584805 > 1.4971612348663204.
IsChangePoint: Marked as a change because one of 6/24/2022 10:40:04 AM, 6/28/2022 9:06:37 PM, 8/5/2022 10:47:59 PM, 8/23/2022 11:50:53 AM falls between 8/5/2022 10:47:59 PM and 8/23/2022 11:50:53 AM.
IsRegressionStdDev: Marked as regression because -30.87032489261679 (T) = (0 -1643.0296887041561) / Math.Sqrt((121.92819149030274 / (14)) + (1175.5798640328185 / (28))) is less than -2.0210753903043583 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (28) - 2, .025) and -0.15443437270090585 = (1423.2335137944103 - 1643.0296887041561) / 1423.2335137944103 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Reader.ReadMultiSpanSequenceEmptyLoop(IsDataCompact: False, TestCase: Json400B)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.2963312469791535 > 2.1769234911808915.
IsChangePoint: Marked as a change because one of 6/24/2022 12:39:19 PM, 6/25/2022 6:06:34 PM, 6/27/2022 5:19:23 PM, 7/19/2022 8:53:09 AM, 8/5/2022 10:47:59 PM, 8/23/2022 11:50:53 AM falls between 8/5/2022 10:47:59 PM and 8/23/2022 11:50:53 AM.
IsRegressionStdDev: Marked as regression because -12.324398748413092 (T) = (0 -2326.1609322462014) / Math.Sqrt((4623.901156282939 / (14)) + (1743.9479220287596 / (28))) is less than -2.0210753903043583 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (28) - 2, .025) and -0.11728557020280579 = (2081.9752749728655 - 2326.1609322462014) / 2081.9752749728655 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Reader.ReadMultiSpanSequenceEmptyLoop(IsDataCompact: True, TestCase: LotsOfNumbers)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 5.978300765689591 > 5.605796187848516.
IsChangePoint: Marked as a change because one of 6/24/2022 10:40:04 AM, 7/19/2022 10:33:10 AM, 8/5/2022 10:47:59 PM, 8/23/2022 11:50:53 AM falls between 8/5/2022 10:47:59 PM and 8/23/2022 11:50:53 AM.
IsRegressionStdDev: Marked as regression because -9.01856544266071 (T) = (0 -5929.127235510019) / Math.Sqrt((38163.513380608805 / (14)) + (17501.923033950163 / (29))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (29) - 2, .025) and -0.09621204717702486 = (5408.741174464157 - 5929.127235510019) / 5408.741174464157 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Reader.ReadMultiSpanSequenceEmptyLoop(IsDataCompact: True, TestCase: Json40KB)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 126.23628065718158 > 118.55053428507193.
IsChangePoint: Marked as a change because one of 6/17/2022 9:56:33 AM, 7/19/2022 10:33:10 AM, 8/5/2022 10:47:59 PM, 8/23/2022 11:50:53 AM falls between 8/5/2022 10:47:59 PM and 8/23/2022 11:50:53 AM.
IsRegressionStdDev: Marked as regression because -9.69756940859704 (T) = (0 -122457.73645034849) / Math.Sqrt((4192742.5510894423 / (14)) + (13413074.070764584 / (29))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (29) - 2, .025) and -0.07426165610874075 = (113992.4670623754 - 122457.73645034849) / 113992.4670623754 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Reader.ReadSpanEmptyLoop(IsDataCompact: True, TestCase: BroadTree)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 13.67699804372041 > 12.571512629772029.
IsChangePoint: Marked as a change because one of 7/10/2022 8:13:50 PM, 7/11/2022 5:09:40 AM, 8/5/2022 10:47:59 PM, 8/23/2022 11:50:53 AM falls between 8/5/2022 10:47:59 PM and 8/23/2022 11:50:53 AM.
IsRegressionStdDev: Marked as regression because -8.342648364326454 (T) = (0 -12905.550685671767) / Math.Sqrt((86440.76010806137 / (14)) + (90395.28240907758 / (27))) is less than -2.0226909200346674 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (27) - 2, .025) and -0.06732813619656963 = (12091.455521505108 - 12905.550685671767) / 12091.455521505108 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Reader.ReadReturnBytes(IsDataCompact: False, TestCase: Json400B)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.9332301034372446 > 1.7488712747408022.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/23/2022 11:50:53 AM falls between 8/5/2022 10:47:59 PM and 8/23/2022 11:50:53 AM.
IsRegressionStdDev: Marked as regression because -12.284443886414365 (T) = (0 -1941.6014072018897) / Math.Sqrt((3287.188062058943 / (14)) + (5954.635125600745 / (28))) is less than -2.0210753903043583 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (28) - 2, .025) and -0.15451653637500185 = (1681.7441292770125 - 1941.6014072018897) / 1681.7441292770125 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Reader.ReadSpanEmptyLoop(IsDataCompact: True, TestCase: Json40KB)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 99.79641682590234 > 97.49529907633277.
IsChangePoint: Marked as a change because one of 6/17/2022 9:56:33 AM, 7/19/2022 10:33:10 AM, 8/5/2022 10:47:59 PM, 8/23/2022 11:50:53 AM falls between 8/5/2022 10:47:59 PM and 8/23/2022 11:50:53 AM.
IsRegressionStdDev: Marked as regression because -19.016779220612186 (T) = (0 -100942.35858429682) / Math.Sqrt((265137.5448871564 / (14)) + (4270584.486459031 / (28))) is less than -2.0210753903043583 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (28) - 2, .025) and -0.08460913608760895 = (93067.95897775218 - 100942.35858429682) / 93067.95897775218 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Reader.ReadSpanEmptyLoop(IsDataCompact: True, TestCase: Json4KB)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 10.114262516244315 > 9.82705112745118.
IsChangePoint: Marked as a change because one of 6/17/2022 9:56:33 AM, 7/19/2022 10:33:10 AM, 8/5/2022 10:47:59 PM, 8/23/2022 11:50:53 AM falls between 8/5/2022 10:47:59 PM and 8/23/2022 11:50:53 AM.
IsRegressionStdDev: Marked as regression because -18.004786450918388 (T) = (0 -10175.614764752167) / Math.Sqrt((5846.434711971038 / (14)) + (45065.52973336598 / (28))) is less than -2.0210753903043583 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (28) - 2, .025) and -0.0865599646007763 = (9364.98223408304 - 10175.614764752167) / 9364.98223408304 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Reader.ReadSpanEmptyLoop(IsDataCompact: True, TestCase: LotsOfNumbers)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 4.142479098325685 > 3.8397191839892413.
IsChangePoint: Marked as a change because one of 6/17/2022 3:57:37 AM, 7/19/2022 10:33:10 AM, 8/5/2022 10:47:59 PM, 8/23/2022 11:50:53 AM falls between 8/5/2022 10:47:59 PM and 8/23/2022 11:50:53 AM.
IsRegressionStdDev: Marked as regression because -19.075482439885974 (T) = (0 -4128.337381330252) / Math.Sqrt((4581.0508649823905 / (14)) + (7857.676810907962 / (28))) is less than -2.0210753903043583 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (28) - 2, .025) and -0.1285657044389474 = (3658.0390181027205 - 4128.337381330252) / 3658.0390181027205 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Reader.ReadSpanEmptyLoop(IsDataCompact: True, TestCase: Json400B)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.3622938144780559 > 1.2032862286374135.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/23/2022 11:50:53 AM falls between 8/5/2022 10:47:59 PM and 8/23/2022 11:50:53 AM.
IsRegressionStdDev: Marked as regression because -42.08293671434285 (T) = (0 -1373.6756399389244) / Math.Sqrt((284.91052476164003 / (14)) + (233.9569426949648 / (29))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (29) - 2, .025) and -0.19518987155765113 = (1149.336747765992 - 1373.6756399389244) / 1149.336747765992 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Reader.ReadSingleSpanSequenceEmptyLoop(IsDataCompact: True, TestCase: Json400B)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.4289923415898615 > 1.2593976670257723.
IsChangePoint: Marked as a change because one of 6/24/2022 10:40:04 AM, 6/28/2022 9:06:37 PM, 8/5/2022 10:47:59 PM, 8/23/2022 11:50:53 AM falls between 8/5/2022 10:47:59 PM and 8/23/2022 11:50:53 AM.
IsRegressionStdDev: Marked as regression because -27.241544628754276 (T) = (0 -1436.2050929595987) / Math.Sqrt((670.4044322213813 / (14)) + (563.9676255633955 / (28))) is less than -2.0210753903043583 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (28) - 2, .025) and -0.18545742376347707 = (1211.5197595203983 - 1436.2050929595987) / 1211.5197595203983 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented Aug 23, 2022 •
edited
Loading

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 543bcc5ee7d6a2b9471b016770227421c43a756e
Compare f142128e89b63577a9bbba7e2b760ec82102a7a9
Diff Diff

Regressions in System.IO.Tests.StringReaderReadLineTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ReadLine - Duration of single invocation 86.41 μs 269.09 μs 3.11 0.05 False
ReadLine - Duration of single invocation 20.19 μs 209.92 μs 10.40 0.02 False
ReadLine - Duration of single invocation 551.53 μs 754.76 μs 1.37 0.12 False
ReadLine - Duration of single invocation 21.14 μs 215.05 μs 10.17 0.02 False
ReadLine - Duration of single invocation 21.77 μs 204.05 μs 9.37 0.02 False
ReadLine - Duration of single invocation 294.70 μs 470.47 μs 1.60 0.07 False
ReadLine - Duration of single invocation 40.46 μs 221.46 μs 5.47 0.02 False
ReadLine - Duration of single invocation 255.42 μs 503.29 μs 1.97 0.29 False

graph
graph
graph
graph
graph
graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.IO.Tests.StringReaderReadLineTests*'

Payloads

Baseline
Compare

Histogram

System.IO.Tests.StringReaderReadLineTests.ReadLine(LineLengthRange: [ 9, 32])

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 269.09145487288134 > 90.69123860748627.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/23/2022 11:50:53 AM falls between 8/5/2022 10:47:59 PM and 8/23/2022 11:50:53 AM.
IsRegressionStdDev: Marked as regression because -55.07614231894471 (T) = (0 -272921.31250492355) / Math.Sqrt((527342.3846924558 / (14)) + (318183466.2735709 / (28))) is less than -2.0210753903043583 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (28) - 2, .025) and -2.13876767122829 = (86951.74064862265 - 272921.31250492355) / 86951.74064862265 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.StringReaderReadLineTests.ReadLine(LineLengthRange: [1025, 2048])

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 209.92211519230773 > 21.21179322216496.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/23/2022 11:50:53 AM falls between 8/5/2022 10:47:59 PM and 8/23/2022 11:50:53 AM.
IsRegressionStdDev: Marked as regression because -218.05967826359245 (T) = (0 -212565.15126529807) / Math.Sqrt((77787.40189800599 / (14)) + (22445104.695158686 / (29))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (29) - 2, .025) and -9.607752330287159 = (20038.66084414358 - 212565.15126529807) / 20038.66084414358 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.StringReaderReadLineTests.ReadLine(LineLengthRange: [   1,    1])

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 754.7624985119047 > 570.4935576616381.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/23/2022 11:50:53 AM falls between 8/5/2022 10:47:59 PM and 8/23/2022 11:50:53 AM.
IsRegressionStdDev: Marked as regression because -49.805899336632535 (T) = (0 -765728.9384902457) / Math.Sqrt((57459175.48482486 / (14)) + (443235934.7301273 / (29))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (29) - 2, .025) and -0.4013475361132613 = (546423.2952619664 - 765728.9384902457) / 546423.2952619664 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.StringReaderReadLineTests.ReadLine(LineLengthRange: [ 129, 1024])

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 215.046326754386 > 22.264635787987395.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/23/2022 11:50:53 AM falls between 8/5/2022 10:47:59 PM and 8/23/2022 11:50:53 AM.
IsRegressionStdDev: Marked as regression because -60.59986408732086 (T) = (0 -212485.79652995308) / Math.Sqrt((17885.60353883081 / (14)) + (288719483.55487233 / (29))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (29) - 2, .025) and -8.992989405816612 = (21263.48662055737 - 212485.79652995308) / 21263.48662055737 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.StringReaderReadLineTests.ReadLine(LineLengthRange: [   0, 1024])

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 204.04883603238866 > 22.972197026976495.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/23/2022 11:50:53 AM falls between 8/5/2022 10:47:59 PM and 8/23/2022 11:50:53 AM.
IsRegressionStdDev: Marked as regression because -269.92091376332 (T) = (0 -204741.46434191658) / Math.Sqrt((31592.87978212796 / (14)) + (11870156.25705656 / (26))) is less than -2.0243941639098457 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (26) - 2, .025) and -8.344267974068527 = (21910.91532371491 - 204741.46434191658) / 21910.91532371491 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.StringReaderReadLineTests.ReadLine(LineLengthRange: [   1,    8])

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 470.47408137254905 > 311.4137087654689.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/23/2022 11:50:53 AM falls between 8/5/2022 10:47:59 PM and 8/23/2022 11:50:53 AM.
IsRegressionStdDev: Marked as regression because -142.34030273836603 (T) = (0 -472268.9445581814) / Math.Sqrt((11009528.421553712 / (14)) + (19461644.790319044 / (27))) is less than -2.0226909200346674 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (27) - 2, .025) and -0.5873482081696622 = (297520.696547574 - 472268.9445581814) / 297520.696547574 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.StringReaderReadLineTests.ReadLine(LineLengthRange: [  33,  128])

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 221.4611496185446 > 41.342590508321834.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/23/2022 11:50:53 AM falls between 8/5/2022 10:47:59 PM and 8/23/2022 11:50:53 AM.
IsRegressionStdDev: Marked as regression because -206.86308404145663 (T) = (0 -222566.1855464226) / Math.Sqrt((128240.44100925759 / (14)) + (21714750.345644146 / (28))) is less than -2.0210753903043583 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (28) - 2, .025) and -4.660152356068341 = (39321.589163197306 - 222566.1855464226) / 39321.589163197306 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.StringReaderReadLineTests.ReadLine(LineLengthRange: [   0,    0])

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 503.29475940860215 > 271.3051512730692.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/23/2022 11:50:53 AM falls between 8/5/2022 10:47:59 PM and 8/23/2022 11:50:53 AM.
IsRegressionStdDev: Marked as regression because -65.0966283778178 (T) = (0 -512148.8552916073) / Math.Sqrt((59532118.383911714 / (14)) + (311434367.67938715 / (28))) is less than -2.0210753903043583 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (28) - 2, .025) and -0.993579883027263 = (256899.08874577237 - 512148.8552916073) / 256899.08874577237 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS ubuntu 18.04
Baseline 543bcc5ee7d6a2b9471b016770227421c43a756e
Compare f142128e89b63577a9bbba7e2b760ec82102a7a9
Diff Diff

Regressions in System.Net.Tests.Perf_WebUtility

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
HtmlDecode_Entities - Duration of single invocation 380.57 ns 988.45 ns 2.60 0.03 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Net.Tests.Perf_WebUtility*'

Payloads

Baseline
Compare

Histogram

System.Net.Tests.Perf_WebUtility.HtmlDecode_Entities

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 988.4484305457953 > 397.6356320961072.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/23/2022 11:50:53 AM falls between 8/5/2022 10:47:59 PM and 8/23/2022 11:50:53 AM.
IsRegressionStdDev: Marked as regression because -134.76573681455102 (T) = (0 -944.4402595443016) / Math.Sqrt((13.079063220438643 / (14)) + (471.8432437705738 / (28))) is less than -2.0210753903043583 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (28) - 2, .025) and -1.5112053229743023 = (376.0904179773301 - 944.4402595443016) / 376.0904179773301 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 543bcc5ee7d6a2b9471b016770227421c43a756e
Compare f142128e89b63577a9bbba7e2b760ec82102a7a9
Diff Diff

Regressions in System.Text.Json.Tests.Utf8JsonReaderCommentsTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Utf8JsonReaderCommentParsing - Duration of single invocation 19.68 μs 134.36 μs 6.83 0.01 False
Utf8JsonReaderCommentParsing - Duration of single invocation 244.14 ns 312.82 ns 1.28 0.19 False
Utf8JsonReaderCommentParsing - Duration of single invocation 221.46 ns 332.70 ns 1.50 0.26 False
Utf8JsonReaderCommentParsing - Duration of single invocation 184.96 ns 275.73 ns 1.49 0.27 False
Utf8JsonReaderCommentParsing - Duration of single invocation 192.73 ns 274.70 ns 1.43 0.24 False
Utf8JsonReaderCommentParsing - Duration of single invocation 2.29 μs 32.05 μs 14.01 0.04 False
Utf8JsonReaderCommentParsing - Duration of single invocation 4.27 μs 31.98 μs 7.48 0.06 False
Utf8JsonReaderCommentParsing - Duration of single invocation 31.70 μs 231.80 μs 7.31 0.04 False
Utf8JsonReaderCommentParsing - Duration of single invocation 19.67 μs 135.44 μs 6.88 0.04 False
Utf8JsonReaderCommentParsing - Duration of single invocation 4.17 μs 31.31 μs 7.51 0.05 False
Utf8JsonReaderCommentParsing - Duration of single invocation 179.61 ns 263.60 ns 1.47 0.25 False
Utf8JsonReaderCommentParsing - Duration of single invocation 252.20 ns 317.23 ns 1.26 0.24 False
Utf8JsonReaderCommentParsing - Duration of single invocation 2.30 μs 30.65 μs 13.32 0.02 False
Utf8JsonReaderCommentParsing - Duration of single invocation 252.44 ns 345.29 ns 1.37 0.26 False
Utf8JsonReaderCommentParsing - Duration of single invocation 31.70 μs 212.09 μs 6.69 0.04 False

graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Json.Tests.Utf8JsonReaderCommentsTests*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Tests.Utf8JsonReaderCommentsTests.Utf8JsonReaderCommentParsing(CommentHandling: Skip, SegmentSize: 0, TestCase: LongMultiLine)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 134.35535477453584 > 20.677687727880983.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/23/2022 11:50:53 AM falls between 8/5/2022 10:47:59 PM and 8/23/2022 11:50:53 AM.
IsRegressionStdDev: Marked as regression because -225.98951081125537 (T) = (0 -135153.344405329) / Math.Sqrt((815782.2320625408 / (14)) + (5581831.257404132 / (28))) is less than -2.0210753903043583 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (28) - 2, .025) and -5.60922708294927 = (20449.19061020049 - 135153.344405329) / 20449.19061020049 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Utf8JsonReaderCommentsTests.Utf8JsonReaderCommentParsing(CommentHandling: Skip, SegmentSize: 100, TestCase: ShortMultiLine)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 312.8242826806474 > 248.6372148203218.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/23/2022 11:50:53 AM falls between 8/5/2022 10:47:59 PM and 8/23/2022 11:50:53 AM.
IsRegressionStdDev: Marked as regression because -25.613060544029366 (T) = (0 -324.3797025406497) / Math.Sqrt((97.17629469441496 / (14)) + (97.70135328874947 / (28))) is less than -2.0210753903043583 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (28) - 2, .025) and -0.3423031919114097 = (241.65904133680877 - 324.3797025406497) / 241.65904133680877 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Utf8JsonReaderCommentsTests.Utf8JsonReaderCommentParsing(CommentHandling: Skip, SegmentSize: 100, TestCase: ShortSingleLine)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 332.6957200569111 > 244.1354239237377.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/23/2022 11:50:53 AM falls between 8/5/2022 10:47:59 PM and 8/23/2022 11:50:53 AM.
IsRegressionStdDev: Marked as regression because -31.935221518077977 (T) = (0 -341.14616096220897) / Math.Sqrt((96.58794991652422 / (14)) + (126.97595477544284 / (28))) is less than -2.0210753903043583 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (28) - 2, .025) and -0.4631431706399717 = (233.15979448066815 - 341.14616096220897) / 233.15979448066815 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Utf8JsonReaderCommentsTests.Utf8JsonReaderCommentParsing(CommentHandling: Skip, SegmentSize: 0, TestCase: ShortSingleLine)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 275.7287334745965 > 190.90348356307757.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/23/2022 11:50:53 AM falls between 8/5/2022 10:47:59 PM and 8/23/2022 11:50:53 AM.
IsRegressionStdDev: Marked as regression because -40.501815823969615 (T) = (0 -282.7879787403771) / Math.Sqrt((51.7637553906346 / (14)) + (89.40072904309334 / (28))) is less than -2.0210753903043583 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (28) - 2, .025) and -0.6024409190321492 = (176.47326362033797 - 282.7879787403771) / 176.47326362033797 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Utf8JsonReaderCommentsTests.Utf8JsonReaderCommentParsing(CommentHandling: Allow, SegmentSize: 0, TestCase: ShortMultiLine)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 274.7042568953877 > 200.2085448287532.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/23/2022 11:50:53 AM falls between 8/5/2022 10:47:59 PM and 8/23/2022 11:50:53 AM.
IsRegressionStdDev: Marked as regression because -27.966525281985025 (T) = (0 -270.31277402154495) / Math.Sqrt((56.545250709514754 / (14)) + (124.86420033802521 / (29))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (29) - 2, .025) and -0.4262586665816302 = (189.52577141523292 - 270.31277402154495) / 189.52577141523292 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Utf8JsonReaderCommentsTests.Utf8JsonReaderCommentParsing(CommentHandling: Skip, SegmentSize: 0, TestCase: LongSingleLine)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 32.05093647634212 > 2.4048217775498495.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/23/2022 11:50:53 AM falls between 8/5/2022 10:47:59 PM and 8/23/2022 11:50:53 AM.
IsRegressionStdDev: Marked as regression because -91.64023453278679 (T) = (0 -30350.265691614764) / Math.Sqrt((114.12684439719571 / (14)) + (2717694.3852142007 / (29))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (29) - 2, .025) and -12.221540857328264 = (2295.5165376804484 - 30350.265691614764) / 2295.5165376804484 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Utf8JsonReaderCommentsTests.Utf8JsonReaderCommentParsing(CommentHandling: Allow, SegmentSize: 100, TestCase: LongSingleLine)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 31.97801565860215 > 4.494249781094996.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/23/2022 11:50:53 AM falls between 8/5/2022 10:47:59 PM and 8/23/2022 11:50:53 AM.
IsRegressionStdDev: Marked as regression because -128.51463497799747 (T) = (0 -32110.009654927744) / Math.Sqrt((2474.252707903194 / (14)) + (1259717.7495575892 / (27))) is less than -2.0226909200346674 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (27) - 2, .025) and -6.470468272697403 = (4298.259290154727 - 32110.009654927744) / 4298.259290154727 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Utf8JsonReaderCommentsTests.Utf8JsonReaderCommentParsing(CommentHandling: Skip, SegmentSize: 100, TestCase: LongMultiLine)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 231.796078247549 > 33.132160918540094.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/23/2022 11:50:53 AM falls between 8/5/2022 10:47:59 PM and 8/23/2022 11:50:53 AM.
IsRegressionStdDev: Marked as regression because -210.46211832861127 (T) = (0 -215110.5179045707) / Math.Sqrt((313797.1693706912 / (14)) + (21364323.731839955 / (29))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (29) - 2, .025) and -5.777105295058165 = (31740.766675327934 - 215110.5179045707) / 31740.766675327934 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Utf8JsonReaderCommentsTests.Utf8JsonReaderCommentParsing(CommentHandling: Allow, SegmentSize: 0, TestCase: LongMultiLine)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 135.44152288074713 > 20.67661923145365.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/23/2022 11:50:53 AM falls between 8/5/2022 10:47:59 PM and 8/23/2022 11:50:53 AM.
IsRegressionStdDev: Marked as regression because -139.22606131683963 (T) = (0 -137094.1381682944) / Math.Sqrt((823738.0662078352 / (14)) + (18003019.752045527 / (28))) is less than -2.0210753903043583 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (28) - 2, .025) and -5.700821427581814 = (20459.30333316893 - 137094.1381682944) / 20459.30333316893 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Utf8JsonReaderCommentsTests.Utf8JsonReaderCommentParsing(CommentHandling: Skip, SegmentSize: 100, TestCase: LongSingleLine)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 31.30854224281897 > 4.375426468770921.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/23/2022 11:50:53 AM falls between 8/5/2022 10:47:59 PM and 8/23/2022 11:50:53 AM.
IsRegressionStdDev: Marked as regression because -157.99482176579227 (T) = (0 -31735.341634660726) / Math.Sqrt((4795.2440867406995 / (14)) + (840848.8863684192 / (28))) is less than -2.0210753903043583 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (28) - 2, .025) and -6.555468936699371 = (4200.313958080331 - 31735.341634660726) / 4200.313958080331 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Utf8JsonReaderCommentsTests.Utf8JsonReaderCommentParsing(CommentHandling: Skip, SegmentSize: 0, TestCase: ShortMultiLine)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 263.59821951393536 > 189.41250127065697.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/23/2022 11:50:53 AM falls between 8/5/2022 10:47:59 PM and 8/23/2022 11:50:53 AM.
IsRegressionStdDev: Marked as regression because -35.860269666536546 (T) = (0 -265.9562138924223) / Math.Sqrt((41.43882557222722 / (14)) + (66.1733462983568 / (29))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (29) - 2, .025) and -0.4465576612652143 = (183.85455416952192 - 265.9562138924223) / 183.85455416952192 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Utf8JsonReaderCommentsTests.Utf8JsonReaderCommentParsing(CommentHandling: Allow, SegmentSize: 100, TestCase: ShortMultiLine)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 317.232764624233 > 254.7886215475907.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/23/2022 11:50:53 AM falls between 8/5/2022 10:47:59 PM and 8/23/2022 11:50:53 AM.
IsRegressionStdDev: Marked as regression because -25.124651790364858 (T) = (0 -332.1202384418749) / Math.Sqrt((27.259992344812556 / (14)) + (303.5543358757276 / (28))) is less than -2.0210753903043583 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (28) - 2, .025) and -0.3708546075207585 = (242.27240191614973 - 332.1202384418749) / 242.27240191614973 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Utf8JsonReaderCommentsTests.Utf8JsonReaderCommentParsing(CommentHandling: Allow, SegmentSize: 0, TestCase: LongSingleLine)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 30.6464546471173 > 2.4141794657632536.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/23/2022 11:50:53 AM falls between 8/5/2022 10:47:59 PM and 8/23/2022 11:50:53 AM.
IsRegressionStdDev: Marked as regression because -136.65006846279402 (T) = (0 -29901.031975125115) / Math.Sqrt((9.937798010956225 / (14)) + (1020136.5592765184 / (25))) is less than -2.026192463026769 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (25) - 2, .025) and -12.01752190131109 = (2296.983419871455 - 29901.031975125115) / 2296.983419871455 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Utf8JsonReaderCommentsTests.Utf8JsonReaderCommentParsing(CommentHandling: Allow, SegmentSize: 100, TestCase: ShortSingleLine)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 345.288944347182 > 243.8845547924991.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/23/2022 11:50:53 AM falls between 8/5/2022 10:47:59 PM and 8/23/2022 11:50:53 AM.
IsRegressionStdDev: Marked as regression because -35.84795575539659 (T) = (0 -349.97964637758344) / Math.Sqrt((55.98389454729155 / (14)) + (171.1668135251028 / (28))) is less than -2.0210753903043583 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (28) - 2, .025) and -0.4830545732514614 = (235.98568298824304 - 349.97964637758344) / 235.98568298824304 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Utf8JsonReaderCommentsTests.Utf8JsonReaderCommentParsing(CommentHandling: Allow, SegmentSize: 100, TestCase: LongMultiLine)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 212.08699330821202 > 33.4613419440184.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/23/2022 11:50:53 AM falls between 8/5/2022 10:47:59 PM and 8/23/2022 11:50:53 AM.
IsRegressionStdDev: Marked as regression because -253.7769437631809 (T) = (0 -215164.56232869453) / Math.Sqrt((433419.20815548743 / (14)) + (13205111.156873224 / (27))) is less than -2.0226909200346674 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (27) - 2, .025) and -5.691075580377781 = (32156.946927887824 - 215164.56232869453) / 32156.946927887824 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented Aug 23, 2022 •
edited
Loading

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 543bcc5ee7d6a2b9471b016770227421c43a756e
Compare f142128e89b63577a9bbba7e2b760ec82102a7a9
Diff Diff

Regressions in System.Tests.Perf_Version

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Parse2 - Duration of single invocation 102.55 ns 132.89 ns 1.30 0.31 False
TryParse3 - Duration of single invocation 117.91 ns 186.52 ns 1.58 0.34 False
TryParse4 - Duration of single invocation 145.95 ns 217.11 ns 1.49 0.28 False
Parse3 - Duration of single invocation 137.82 ns 183.79 ns 1.33 0.33 False
Parse4 - Duration of single invocation 163.88 ns 229.53 ns 1.40 0.26 False

graph
graph
graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_Version*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Version.Parse2

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 132.88547397848046 > 103.03926091301193.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/23/2022 11:50:53 AM falls between 8/5/2022 10:47:59 PM and 8/23/2022 11:50:53 AM.
IsRegressionStdDev: Marked as regression because -27.767654747790534 (T) = (0 -135.56011703851843) / Math.Sqrt((11.26884381051676 / (14)) + (30.126865209619243 / (28))) is less than -2.0210753903043583 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (28) - 2, .025) and -0.3906718439794591 = (97.4781488712737 - 135.56011703851843) / 97.4781488712737 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Version.TryParse3

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 186.51511386649256 > 126.80783320369301.
IsChangePoint: Marked as a change because one of 6/24/2022 10:40:04 AM, 6/29/2022 4:06:39 AM, 8/5/2022 10:47:59 PM, 8/23/2022 11:50:53 AM falls between 8/5/2022 10:47:59 PM and 8/23/2022 11:50:53 AM.
IsRegressionStdDev: Marked as regression because -26.87141790339949 (T) = (0 -179.84762786573936) / Math.Sqrt((30.301624730174066 / (14)) + (54.26385269779902 / (25))) is less than -2.026192463026769 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (25) - 2, .025) and -0.4515565851034233 = (123.89983946297569 - 179.84762786573936) / 123.89983946297569 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Version.TryParse4

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 217.11017434696893 > 156.02556493029974.
IsChangePoint: Marked as a change because one of 6/24/2022 10:57:01 PM, 6/26/2022 6:49:05 AM, 6/29/2022 1:11:29 AM, 8/5/2022 10:47:59 PM, 8/23/2022 11:50:53 AM falls between 8/5/2022 10:47:59 PM and 8/23/2022 11:50:53 AM.
IsRegressionStdDev: Marked as regression because -29.73884304746242 (T) = (0 -221.4409892141364) / Math.Sqrt((42.989758044043064 / (14)) + (70.79478779696304 / (28))) is less than -2.0210753903043583 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (28) - 2, .025) and -0.46579955038677373 = (151.07180866285967 - 221.4409892141364) / 151.07180866285967 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Version.Parse3

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 183.79019530321997 > 140.51136718461998.
IsChangePoint: Marked as a change because one of 6/24/2022 12:39:19 PM, 6/28/2022 4:04:15 PM, 6/29/2022 1:11:29 AM, 8/5/2022 10:47:59 PM, 8/23/2022 11:50:53 AM falls between 8/5/2022 10:47:59 PM and 8/23/2022 11:50:53 AM.
IsRegressionStdDev: Marked as regression because -25.093413458773373 (T) = (0 -194.06100712232058) / Math.Sqrt((39.886557967880584 / (14)) + (84.46279504846112 / (29))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (29) - 2, .025) and -0.4500700487090869 = (133.828712133653 - 194.06100712232058) / 133.828712133653 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Version.Parse4

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 229.52510641188726 > 173.47913011849766.
IsChangePoint: Marked as a change because one of 6/24/2022 10:40:04 AM, 6/29/2022 4:06:39 AM, 8/5/2022 10:47:59 PM, 8/23/2022 11:50:53 AM falls between 8/5/2022 10:47:59 PM and 8/23/2022 11:50:53 AM.
IsRegressionStdDev: Marked as regression because -27.83065582744794 (T) = (0 -235.62512711610853) / Math.Sqrt((41.33098213819585 / (14)) + (85.71089249814779 / (28))) is less than -2.0210753903043583 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (28) - 2, .025) and -0.4077373510734573 = (167.37861429650548 - 235.62512711610853) / 167.37861429650548 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS ubuntu 18.04
Baseline 543bcc5ee7d6a2b9471b016770227421c43a756e
Compare f142128e89b63577a9bbba7e2b760ec82102a7a9
Diff Diff

Regressions in System.Text.Json.Tests.Perf_Segment

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ReadMultiSegmentSequenceUsingSpan - Duration of single invocation 1.21 ms 1.31 ms 1.08 0.04 False
ReadSingleSegmentSequence - Duration of single invocation 1.18 ms 1.26 ms 1.06 0.05 False
ReadSingleSegmentSequenceByN - Duration of single invocation 11.91 μs 12.59 μs 1.06 0.04 False
ReadSingleSegmentSequence - Duration of single invocation 11.80 μs 12.76 μs 1.08 0.03 False
ReadSingleSegmentSequenceByN - Duration of single invocation 117.72 μs 125.64 μs 1.07 0.06 False
ReadSingleSegmentSequenceByN - Duration of single invocation 11.86 μs 12.84 μs 1.08 0.05 False

graph
graph
graph
graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Json.Tests.Perf_Segment*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Tests.Perf_Segment.ReadMultiSegmentSequenceUsingSpan(segmentSize: 8192, TestCase: Json400KB)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.3138642309027777 > 1.2668867514423077.
IsChangePoint: Marked as a change because one of 6/17/2022 9:56:33 AM, 7/19/2022 2:55:40 PM, 8/5/2022 10:47:59 PM, 8/23/2022 11:50:53 AM falls between 8/5/2022 10:47:59 PM and 8/23/2022 11:50:53 AM.
IsRegressionStdDev: Marked as regression because -8.832612902034107 (T) = (0 -1286957.4100793882) / Math.Sqrt((730020004.5746969 / (14)) + (332306096.67320937 / (29))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (29) - 2, .025) and -0.057904270223844805 = (1216515.942228948 - 1286957.4100793882) / 1216515.942228948 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Segment.ReadSingleSegmentSequence(TestCase: Json400KB)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.2572410714285716 > 1.240758706026786.
IsChangePoint: Marked as a change because one of 6/25/2022 6:06:34 PM, 6/30/2022 4:27:12 PM, 8/5/2022 10:47:59 PM, 8/23/2022 11:50:53 AM falls between 8/5/2022 10:47:59 PM and 8/23/2022 11:50:53 AM.
IsRegressionStdDev: Marked as regression because -20.51733771847067 (T) = (0 -1259583.4625203677) / Math.Sqrt((37595650.1515269 / (14)) + (352451476.28193635 / (28))) is less than -2.0210753903043583 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (28) - 2, .025) and -0.06798628111404846 = (1179400.414400884 - 1259583.4625203677) / 1179400.414400884 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Segment.ReadSingleSegmentSequenceByN(numberOfBytes: 4096, TestCase: Json4KB)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 12.588213749712214 > 12.515637436028303.
IsChangePoint: Marked as a change because one of 6/21/2022 11:38:18 AM, 7/19/2022 10:33:10 AM, 8/5/2022 10:47:59 PM, 8/23/2022 11:50:53 AM falls between 8/5/2022 10:47:59 PM and 8/23/2022 11:50:53 AM.
IsRegressionStdDev: Marked as regression because -15.317705637673985 (T) = (0 -12672.202444221704) / Math.Sqrt((20535.81036398369 / (14)) + (22127.45557737415 / (28))) is less than -2.0210753903043583 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (28) - 2, .025) and -0.06092605854773347 = (11944.472795369229 - 12672.202444221704) / 11944.472795369229 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Segment.ReadSingleSegmentSequence(TestCase: Json4KB)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 12.757053677066452 > 12.526546679443879.
IsChangePoint: Marked as a change because one of 6/17/2022 9:56:33 AM, 7/19/2022 10:33:10 AM, 8/5/2022 10:47:59 PM, 8/23/2022 11:50:53 AM falls between 8/5/2022 10:47:59 PM and 8/23/2022 11:50:53 AM.
IsRegressionStdDev: Marked as regression because -15.443974653024322 (T) = (0 -12709.413762106327) / Math.Sqrt((23697.426259320284 / (14)) + (27503.44560827027 / (27))) is less than -2.0226909200346674 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (27) - 2, .025) and -0.06754780438681304 = (11905.240879968336 - 12709.413762106327) / 11905.240879968336 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Segment.ReadSingleSegmentSequenceByN(numberOfBytes: 8192, TestCase: Json40KB)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 125.64186511243386 > 124.27589552531899.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/23/2022 11:50:53 AM falls between 8/5/2022 10:47:59 PM and 8/23/2022 11:50:53 AM.
IsRegressionStdDev: Marked as regression because -8.674742804104302 (T) = (0 -127097.64918947841) / Math.Sqrt((7023383.564901864 / (14)) + (10589473.726140928 / (29))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (29) - 2, .025) and -0.06785752177031634 = (119021.16771043884 - 127097.64918947841) / 119021.16771043884 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Segment.ReadSingleSegmentSequenceByN(numberOfBytes: 8192, TestCase: Json4KB)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 12.836572243589744 > 12.469607141755468.
IsChangePoint: Marked as a change because one of 6/17/2022 9:56:33 AM, 7/19/2022 8:53:09 AM, 8/5/2022 10:47:59 PM, 8/23/2022 11:50:53 AM falls between 8/5/2022 10:47:59 PM and 8/23/2022 11:50:53 AM.
IsRegressionStdDev: Marked as regression because -15.787585263709786 (T) = (0 -12719.566754294883) / Math.Sqrt((3972.5173419869543 / (14)) + (77359.70481630621 / (29))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (29) - 2, .025) and -0.07230537059622265 = (11861.888509634673 - 12719.566754294883) / 11861.888509634673 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 543bcc5ee7d6a2b9471b016770227421c43a756e
Compare f142128e89b63577a9bbba7e2b760ec82102a7a9
Diff Diff

Regressions in Microsoft.Extensions.Configuration.Xml.XmlConfigurationProviderBenchmarks

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Load - Duration of single invocation 277.65 μs 298.82 μs 1.08 0.10 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'Microsoft.Extensions.Configuration.Xml.XmlConfigurationProviderBenchmarks*'

Payloads

Baseline
Compare

Histogram

Microsoft.Extensions.Configuration.Xml.XmlConfigurationProviderBenchmarks.Load(FileName: "repeated.xml")

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 298.8236714975845 > 286.28722408634746.
IsChangePoint: Marked as a change because one of 7/1/2022 12:02:05 PM, 7/10/2022 5:59:30 PM, 7/18/2022 3:18:54 PM, 8/5/2022 10:47:59 PM, 8/23/2022 11:50:53 AM falls between 8/5/2022 10:47:59 PM and 8/23/2022 11:50:53 AM.
IsRegressionStdDev: Marked as regression because -6.88310600907928 (T) = (0 -301472.8099982475) / Math.Sqrt((42979330.934151694 / (14)) + (316904078.4245501 / (28))) is less than -2.0210753903043583 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (28) - 2, .025) and -0.09481488875156308 = (275364.18539395486 - 301472.8099982475) / 275364.18539395486 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 543bcc5ee7d6a2b9471b016770227421c43a756e
Compare f142128e89b63577a9bbba7e2b760ec82102a7a9
Diff Diff

Regressions in System.Collections.Concurrent.IsEmpty<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Dictionary - Duration of single invocation 76.39 ns 84.10 ns 1.10 0.43 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.Concurrent.IsEmpty&lt;Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.Concurrent.IsEmpty<Int32>.Dictionary(Size: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 84.0958860225178 > 77.7428330891405.
IsChangePoint: Marked as a change because one of 6/25/2022 6:06:34 PM, 8/5/2022 10:47:59 PM, 8/23/2022 11:50:53 AM falls between 8/5/2022 10:47:59 PM and 8/23/2022 11:50:53 AM.
IsRegressionStdDev: Marked as regression because -6.638448804558544 (T) = (0 -90.2199086364392) / Math.Sqrt((34.63026078641228 / (14)) + (23.459226759057398 / (28))) is less than -2.0210753903043583 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (28) - 2, .025) and -0.15459728625245242 = (78.13971997913794 - 90.2199086364392) / 78.13971997913794 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 543bcc5ee7d6a2b9471b016770227421c43a756e
Compare f142128e89b63577a9bbba7e2b760ec82102a7a9
Diff Diff

Regressions in System.Text.Json.Serialization.Tests.ReadJson<Location>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
DeserializeFromUtf8Bytes - Duration of single invocation 4.44 μs 4.72 μs 1.06 0.12 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Json.Serialization.Tests.ReadJson&lt;Location&gt;*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Serialization.Tests.ReadJson<Location>.DeserializeFromUtf8Bytes

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 4.7167483949541875 > 4.671149286056172.
IsChangePoint: Marked as a change because one of 6/13/2022 2:27:15 PM, 7/19/2022 10:33:10 AM, 8/5/2022 10:47:59 PM, 8/23/2022 11:50:53 AM falls between 8/5/2022 10:47:59 PM and 8/23/2022 11:50:53 AM.
IsRegressionStdDev: Marked as regression because -10.646466865726808 (T) = (0 -4719.71655251361) / Math.Sqrt((5639.382209450915 / (14)) + (8314.017184266457 / (28))) is less than -2.0210753903043583 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (28) - 2, .025) and -0.06345684775861557 = (4438.089389767976 - 4719.71655251361) / 4438.089389767976 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 543bcc5ee7d6a2b9471b016770227421c43a756e
Compare f142128e89b63577a9bbba7e2b760ec82102a7a9
Diff Diff

Regressions in System.Collections.ContainsTrue<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ICollection - Duration of single invocation 103.61 μs 1.32 ms 12.75 0.03 False
Array - Duration of single invocation 106.50 μs 1.23 ms 11.55 0.02 False
Queue - Duration of single invocation 96.57 μs 1.21 ms 12.55 0.04 False
List - Duration of single invocation 100.61 μs 1.26 ms 12.52 0.02 False
Stack - Duration of single invocation 83.72 μs 1.43 ms 17.05 0.07 False
ImmutableArray - Duration of single invocation 125.83 μs 1.25 ms 9.91 0.08 False

graph
graph
graph
graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.ContainsTrue&lt;Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.ContainsTrue<Int32>.ICollection(Size: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.3204940487351189 > 109.14635845064156.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/23/2022 11:50:53 AM falls between 8/5/2022 10:47:59 PM and 8/23/2022 11:50:53 AM.
IsRegressionStdDev: Marked as regression because -100.7628440772313 (T) = (0 -1252327.8761853708) / Math.Sqrt((2634485.7547261342 / (14)) + (3628098547.286273 / (28))) is less than -2.0210753903043583 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (28) - 2, .025) and -10.983786157142017 = (104501.85440258519 - 1252327.8761853708) / 104501.85440258519 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.ContainsTrue&lt;Int32&gt;.Array(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.2297522174556212 > 111.63461333778687.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/23/2022 11:50:53 AM falls between 8/5/2022 10:47:59 PM and 8/23/2022 11:50:53 AM.
IsRegressionStdDev: Marked as regression because -204.87108641837335 (T) = (0 -1238959.5870047088) / Math.Sqrt((3224255.2890859684 / (14)) + (878635111.1867132 / (29))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (29) - 2, .025) and -10.578891565738541 = (107001.57091640263 - 1238959.5870047088) / 107001.57091640263 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.ContainsTrue&lt;Int32&gt;.Queue(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.2121431981456041 > 101.51258619178186.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/23/2022 11:50:53 AM falls between 8/5/2022 10:47:59 PM and 8/23/2022 11:50:53 AM.
IsRegressionStdDev: Marked as regression because -139.12958104437044 (T) = (0 -1242698.0293413135) / Math.Sqrt((305147.74049519986 / (14)) + (1761971305.633365 / (26))) is less than -2.0243941639098457 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (26) - 2, .025) and -11.787631995561998 = (97179.68344511298 - 1242698.0293413135) / 97179.68344511298 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.ContainsTrue&lt;Int32&gt;.List(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.2599283663461538 > 105.77203708857249.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/23/2022 11:50:53 AM falls between 8/5/2022 10:47:59 PM and 8/23/2022 11:50:53 AM.
IsRegressionStdDev: Marked as regression because -208.26781418995307 (T) = (0 -1232406.4391492493) / Math.Sqrt((613011.761185817 / (14)) + (765914123.4490246 / (26))) is less than -2.0243941639098457 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (26) - 2, .025) and -11.179867311895812 = (101183.89696623253 - 1232406.4391492493) / 101183.89696623253 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.ContainsTrue&lt;Int32&gt;.Stack(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.4278253430555554 > 86.69658293775305.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/23/2022 11:50:53 AM falls between 8/5/2022 10:47:59 PM and 8/23/2022 11:50:53 AM.
IsRegressionStdDev: Marked as regression because -122.37384648290255 (T) = (0 -1278111.8789787954) / Math.Sqrt((389419.7797080269 / (14)) + (2765967339.211544 / (29))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (29) - 2, .025) and -14.433681115111046 = (82813.15840635076 - 1278111.8789787954) / 82813.15840635076 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.ContainsTrue&lt;Int32&gt;.ImmutableArray(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.2470414219674557 > 132.27599546370968.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/23/2022 11:50:53 AM falls between 8/5/2022 10:47:59 PM and 8/23/2022 11:50:53 AM.
IsRegressionStdDev: Marked as regression because -154.78887391032546 (T) = (0 -1268135.159394115) / Math.Sqrt((1331132.729081719 / (14)) + (1521212018.076903 / (28))) is less than -2.0210753903043583 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (28) - 2, .025) and -9.047327226834682 = (126216.16980953344 - 1268135.159394115) / 126216.16980953344 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 543bcc5ee7d6a2b9471b016770227421c43a756e
Compare f142128e89b63577a9bbba7e2b760ec82102a7a9
Diff Diff

Regressions in System.Tests.Perf_Enum

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Parse - Duration of single invocation 233.09 ns 281.62 ns 1.21 0.12 False
TryParseGeneric - Duration of single invocation 384.79 ns 694.95 ns 1.81 0.06 False
Parse - Duration of single invocation 389.96 ns 713.19 ns 1.83 0.09 False

graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_Enum*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Enum.Parse(text: "Red")

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 281.6150562224611 > 245.76129626849564.
IsChangePoint: Marked as a change because one of 7/13/2022 7:22:00 AM, 8/5/2022 10:47:59 PM, 8/23/2022 11:50:53 AM falls between 8/5/2022 10:47:59 PM and 8/23/2022 11:50:53 AM.
IsRegressionStdDev: Marked as regression because -21.583363895123185 (T) = (0 -279.8983654854116) / Math.Sqrt((27.540577260398518 / (14)) + (57.02608001375129 / (29))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (29) - 2, .025) and -0.18055039092830522 = (237.09141738991624 - 279.8983654854116) / 237.09141738991624 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Enum.TryParseGeneric(text: "Red, Orange, Yellow, Green, Blue")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 694.9515675835552 > 373.2237744190423.
IsChangePoint: Marked as a change because one of 7/18/2022 10:06:53 PM, 8/5/2022 10:47:59 PM, 8/23/2022 11:50:53 AM falls between 8/5/2022 10:47:59 PM and 8/23/2022 11:50:53 AM.
IsRegressionStdDev: Marked as regression because -99.72770168103483 (T) = (0 -692.8033621972049) / Math.Sqrt((84.79509119052696 / (14)) + (134.29224971737383 / (25))) is less than -2.026192463026769 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (25) - 2, .025) and -0.9479180119104775 = (355.66351250980927 - 692.8033621972049) / 355.66351250980927 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Enum.Parse(text: "Red, Orange, Yellow, Green, Blue")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 713.1869060263218 > 405.68836810658865.
IsChangePoint: Marked as a change because one of 7/13/2022 7:22:00 AM, 8/5/2022 10:47:59 PM, 8/23/2022 11:50:53 AM falls between 8/5/2022 10:47:59 PM and 8/23/2022 11:50:53 AM.
IsRegressionStdDev: Marked as regression because -83.92873580269011 (T) = (0 -726.4411198267983) / Math.Sqrt((45.1054669875723 / (14)) + (378.72473294371343 / (28))) is less than -2.0210753903043583 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (28) - 2, .025) and -0.8968541368594106 = (382.97152412022285 - 726.4411198267983) / 382.97152412022285 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented Aug 23, 2022 •
edited
Loading

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 543bcc5ee7d6a2b9471b016770227421c43a756e
Compare f142128e89b63577a9bbba7e2b760ec82102a7a9
Diff Diff

Regressions in System.Memory.Span<Byte>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
LastIndexOfAnyValues - Duration of single invocation 149.70 ns 3.01 μs 20.07 0.03 False
IndexOfValue - Duration of single invocation 138.67 ns 2.65 μs 19.11 0.02 False
IndexOfAnyThreeValues - Duration of single invocation 276.67 ns 3.73 μs 13.50 0.02 False
IndexOfAnyTwoValues - Duration of single invocation 200.08 ns 3.05 μs 15.22 0.02 False

graph
graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Memory.Span&lt;Byte&gt;*'

Payloads

Baseline
Compare

Histogram

System.Memory.Span<Byte>.LastIndexOfAnyValues(Size: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.005069486888802 > 157.0054445991761.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/23/2022 11:50:53 AM falls between 8/5/2022 10:47:59 PM and 8/23/2022 11:50:53 AM.
IsRegressionStdDev: Marked as regression because -142.6526103293986 (T) = (0 -3067.8936524974233) / Math.Sqrt((0.31544952674522203 / (14)) + (11716.73659992451 / (28))) is less than -2.0210753903043583 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (28) - 2, .025) and -19.495105972331594 = (149.6890846350871 - 3067.8936524974233) / 149.6890846350871 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Span&lt;Byte&gt;.IndexOfValue(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.6495521433202396 > 146.37516945438924.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/23/2022 11:50:53 AM falls between 8/5/2022 10:47:59 PM and 8/23/2022 11:50:53 AM.
IsRegressionStdDev: Marked as regression because -44.075812473203925 (T) = (0 -2497.057416503454) / Math.Sqrt((259.65556211673896 / (14)) + (81457.5742724803 / (29))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (29) - 2, .025) and -15.279241241750704 = (153.38905415931507 - 2497.057416503454) / 153.38905415931507 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Span&lt;Byte&gt;.IndexOfAnyThreeValues(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.734082833333333 > 290.8288738091121.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/23/2022 11:50:53 AM falls between 8/5/2022 10:47:59 PM and 8/23/2022 11:50:53 AM.
IsRegressionStdDev: Marked as regression because -69.71306579337316 (T) = (0 -3838.531627119351) / Math.Sqrt((1.2682850554058953 / (14)) + (75683.0523484087 / (29))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (29) - 2, .025) and -12.851460254023374 = (277.12108014058583 - 3838.531627119351) / 277.12108014058583 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Span&lt;Byte&gt;.IndexOfAnyTwoValues(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.0461933614330867 > 209.74119752252065.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/23/2022 11:50:53 AM falls between 8/5/2022 10:47:59 PM and 8/23/2022 11:50:53 AM.
IsRegressionStdDev: Marked as regression because -108.74125124029152 (T) = (0 -3104.8560263406293) / Math.Sqrt((0.1937329534100386 / (14)) + (20693.78650898296 / (29))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (29) - 2, .025) and -14.521636828768049 = (200.0340595900324 - 3104.8560263406293) / 200.0340595900324 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS ubuntu 18.04
Baseline 543bcc5ee7d6a2b9471b016770227421c43a756e
Compare f142128e89b63577a9bbba7e2b760ec82102a7a9
Diff Diff

Regressions in System.Xml.Linq.Perf_XElement

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GetElementWithNamespace - Duration of single invocation 298.26 ns 401.75 ns 1.35 0.24 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Xml.Linq.Perf_XElement*'

Payloads

Baseline
Compare

Histogram

System.Xml.Linq.Perf_XElement.GetElementWithNamespace

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 401.75145816509644 > 299.1777922280125.
IsChangePoint: Marked as a change because one of 6/24/2022 10:40:04 AM, 6/27/2022 5:19:23 PM, 8/5/2022 10:47:59 PM, 8/23/2022 11:50:53 AM falls between 8/5/2022 10:47:59 PM and 8/23/2022 11:50:53 AM.
IsRegressionStdDev: Marked as regression because -22.88794680213281 (T) = (0 -371.3257398974403) / Math.Sqrt((96.67452904726059 / (14)) + (151.5163300693113 / (29))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (29) - 2, .025) and -0.27335908429937344 = (291.6111758858271 - 371.3257398974403) / 291.6111758858271 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 543bcc5ee7d6a2b9471b016770227421c43a756e
Compare f142128e89b63577a9bbba7e2b760ec82102a7a9
Diff Diff

Regressions in System.Linq.Tests.Perf_Enumerable

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Contains_ElementNotFound - Duration of single invocation 124.32 ns 998.45 ns 8.03 0.33 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Linq.Tests.Perf_Enumerable*'

Payloads

Baseline
Compare

Histogram

System.Linq.Tests.Perf_Enumerable.Contains_ElementNotFound(input: ICollection)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 998.4536756151209 > 129.71356244641066.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/23/2022 11:50:53 AM falls between 8/5/2022 10:47:59 PM and 8/23/2022 11:50:53 AM.
IsRegressionStdDev: Marked as regression because -83.96695598922933 (T) = (0 -1010.4754241502075) / Math.Sqrt((21.300119274764548 / (14)) + (3081.3796957606455 / (28))) is less than -2.0210753903043583 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (28) - 2, .025) and -7.178190337064788 = (123.55733756532676 - 1010.4754241502075) / 123.55733756532676 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 543bcc5ee7d6a2b9471b016770227421c43a756e
Compare f142128e89b63577a9bbba7e2b760ec82102a7a9
Diff Diff

Regressions in System.IO.Tests.Perf_Path

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GetFileNameWithoutExtension - Duration of single invocation 109.03 ns 148.42 ns 1.36 0.24 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.IO.Tests.Perf_Path*'

Payloads

Baseline
Compare

Histogram

System.IO.Tests.Perf_Path.GetFileNameWithoutExtension

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 148.42365308801624 > 114.74546136153367.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/23/2022 11:50:53 AM falls between 8/5/2022 10:47:59 PM and 8/23/2022 11:50:53 AM.
IsRegressionStdDev: Marked as regression because -34.698195417822475 (T) = (0 -151.29899068786685) / Math.Sqrt((12.744288495830018 / (14)) + (15.412838246974943 / (29))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (29) - 2, .025) and -0.380019771114862 = (109.63537903927168 - 151.29899068786685) / 109.63537903927168 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 543bcc5ee7d6a2b9471b016770227421c43a756e
Compare f142128e89b63577a9bbba7e2b760ec82102a7a9
Diff Diff

Regressions in System.Tests.Perf_Environment

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
SetEnvironmentVariable - Duration of single invocation 194.79 ns 643.72 ns 3.30 0.16 False
GetEnvironmentVariable - Duration of single invocation 151.33 ns 494.56 ns 3.27 0.23 False
ExpandEnvironmentVariables - Duration of single invocation 382.06 ns 1.13 μs 2.95 0.11 False

graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_Environment*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Environment.SetEnvironmentVariable

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 643.7239185178689 > 213.09369271099766.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/23/2022 11:50:53 AM falls between 8/5/2022 10:47:59 PM and 8/23/2022 11:50:53 AM.
IsRegressionStdDev: Marked as regression because -132.25655567743905 (T) = (0 -651.5599982678131) / Math.Sqrt((49.2398659816711 / (14)) + (220.28555520436004 / (27))) is less than -2.0226909200346674 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (27) - 2, .025) and -2.2636753226781448 = (199.6399561379005 - 651.5599982678131) / 199.6399561379005 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Environment.GetEnvironmentVariable

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 494.55751708086154 > 158.47364365641357.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/23/2022 11:50:53 AM falls between 8/5/2022 10:47:59 PM and 8/23/2022 11:50:53 AM.
IsRegressionStdDev: Marked as regression because -116.4310419074876 (T) = (0 -490.7721960632389) / Math.Sqrt((90.23408819346187 / (14)) + (47.211757529656325 / (28))) is less than -2.0210753903043583 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (28) - 2, .025) and -2.0912555291507298 = (158.76144544998849 - 490.7721960632389) / 158.76144544998849 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Environment.ExpandEnvironmentVariables

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.1274812549493187 > 404.12774324999464.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/23/2022 11:50:53 AM falls between 8/5/2022 10:47:59 PM and 8/23/2022 11:50:53 AM.
IsRegressionStdDev: Marked as regression because -150.63612210848132 (T) = (0 -1062.689515754447) / Math.Sqrt((68.03089949492843 / (14)) + (430.36958960302144 / (28))) is less than -2.0210753903043583 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (28) - 2, .025) and -1.7590361184603165 = (385.16694603747425 - 1062.689515754447) / 385.16694603747425 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 543bcc5ee7d6a2b9471b016770227421c43a756e
Compare f142128e89b63577a9bbba7e2b760ec82102a7a9
Diff Diff

Regressions in System.Globalization.Tests.StringSearch

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
IndexOf_Word_NotFound - Duration of single invocation 139.08 ns 2.10 μs 15.12 0.08 False
IsPrefix_FirstHalf - Duration of single invocation 6.82 μs 8.03 μs 1.18 0.21 False
LastIndexOf_Word_NotFound - Duration of single invocation 144.78 ns 2.14 μs 14.80 0.05 False
IndexOf_Word_NotFound - Duration of single invocation 242.87 ns 2.70 μs 11.14 0.02 False
LastIndexOf_Word_NotFound - Duration of single invocation 635.72 ns 682.15 ns 1.07 0.10 False

graph
graph
graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Globalization.Tests.StringSearch*'

Payloads

Baseline
Compare

Histogram

System.Globalization.Tests.StringSearch.IndexOf_Word_NotFound(Options: (en-US, Ordinal, False))

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.102241201309685 > 146.25042125333198.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/23/2022 11:50:53 AM falls between 8/5/2022 10:47:59 PM and 8/23/2022 11:50:53 AM.
IsRegressionStdDev: Marked as regression because -216.07328967230188 (T) = (0 -2124.0521945534124) / Math.Sqrt((2.5191633776554614 / (14)) + (2442.504966196499 / (29))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (29) - 2, .025) and -14.286636145640532 = (138.94830584812166 - 2124.0521945534124) / 138.94830584812166 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Globalization.Tests.StringSearch.IsPrefix_FirstHalf(Options: (, IgnoreCase, True))

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 8.030053271509532 > 7.216435407390224.
IsChangePoint: Marked as a change because one of 7/10/2022 5:59:30 PM, 7/11/2022 1:18:03 PM, 8/22/2022 5:37:03 PM, 8/23/2022 11:50:53 AM falls between 8/5/2022 10:47:59 PM and 8/23/2022 11:50:53 AM.
IsRegressionStdDev: Marked as regression because -6.679233800516325 (T) = (0 -8020.001365867452) / Math.Sqrt((194181.81594506878 / (39)) + (57415.48025146133 / (4))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (39) + (4) - 2, .025) and -0.1309631318417717 = (7091.302218496629 - 8020.001365867452) / 7091.302218496629 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Globalization.Tests.StringSearch.LastIndexOf_Word_NotFound(Options: (en-US, Ordinal, False))

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.143099663809064 > 153.37867235480752.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/23/2022 11:50:53 AM falls between 8/5/2022 10:47:59 PM and 8/23/2022 11:50:53 AM.
IsRegressionStdDev: Marked as regression because -80.30936141902454 (T) = (0 -2199.4524462532004) / Math.Sqrt((2.377436332127871 / (14)) + (16995.015002842916 / (26))) is less than -2.0243941639098457 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (26) - 2, .025) and -14.070615046053588 = (145.94311111603585 - 2199.4524462532004) / 145.94311111603585 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Globalization.Tests.StringSearch.IndexOf_Word_NotFound(Options: (en-US, OrdinalIgnoreCase, False))

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.7047682972182274 > 257.52336573844406.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/23/2022 11:50:53 AM falls between 8/5/2022 10:47:59 PM and 8/23/2022 11:50:53 AM.
IsRegressionStdDev: Marked as regression because -226.60758741668346 (T) = (0 -2727.9813468807633) / Math.Sqrt((7.039920622659737 / (14)) + (3468.803675465384 / (29))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (29) - 2, .025) and -10.161400186272095 = (244.41210792137258 - 2727.9813468807633) / 244.41210792137258 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Globalization.Tests.StringSearch.LastIndexOf_Word_NotFound(Options: (en-US, IgnoreNonSpace, False))

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 682.1478327356152 > 668.712661395362.
IsChangePoint: Marked as a change because one of 7/26/2022 11:31:56 PM, 7/28/2022 6:38:38 AM, 8/22/2022 5:37:03 PM, 8/23/2022 11:50:53 AM falls between 8/5/2022 10:47:59 PM and 8/23/2022 11:50:53 AM.
IsRegressionStdDev: Marked as regression because -2.870215536045104 (T) = (0 -702.3979716555094) / Math.Sqrt((346.0450470505346 / (39)) + (1895.9417660213292 / (4))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (39) + (4) - 2, .025) and -0.09865085454277366 = (639.3277434329461 - 702.3979716555094) / 639.3277434329461 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 543bcc5ee7d6a2b9471b016770227421c43a756e
Compare f142128e89b63577a9bbba7e2b760ec82102a7a9
Diff Diff

Regressions in Microsoft.Extensions.Logging.FormattingOverhead

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
TwoArguments - Duration of single invocation 1.04 μs 1.14 μs 1.09 0.13 False
FourArguments_DefineMessage - Duration of single invocation 1.42 μs 1.62 μs 1.14 0.13 False

graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'Microsoft.Extensions.Logging.FormattingOverhead*'

Payloads

Baseline
Compare

Histogram

Microsoft.Extensions.Logging.FormattingOverhead.TwoArguments

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.141339026539498 > 1.075421527176927.
IsChangePoint: Marked as a change because one of 6/23/2022 10:44:43 PM, 8/5/2022 10:47:59 PM, 8/23/2022 11:50:53 AM falls between 8/5/2022 10:47:59 PM and 8/23/2022 11:50:53 AM.
IsRegressionStdDev: Marked as regression because -15.415914047835702 (T) = (0 -1158.0083120861318) / Math.Sqrt((807.2839543862087 / (14)) + (810.278058978474 / (29))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (29) - 2, .025) and -0.14047159098429918 = (1015.3767276979669 - 1158.0083120861318) / 1015.3767276979669 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### Microsoft.Extensions.Logging.FormattingOverhead.FourArguments_DefineMessage

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.6243565681627739 > 1.5501314010607476.
IsChangePoint: Marked as a change because one of 7/5/2022 3:17:18 PM, 8/5/2022 10:47:59 PM, 8/23/2022 11:50:53 AM falls between 8/5/2022 10:47:59 PM and 8/23/2022 11:50:53 AM.
IsRegressionStdDev: Marked as regression because -14.271240221783382 (T) = (0 -1640.2479841161314) / Math.Sqrt((1064.3387475109448 / (14)) + (2210.1069630772977 / (28))) is less than -2.0210753903043583 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (28) - 2, .025) and -0.12146242670996156 = (1462.5973595282483 - 1640.2479841161314) / 1462.5973595282483 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 543bcc5ee7d6a2b9471b016770227421c43a756e
Compare f142128e89b63577a9bbba7e2b760ec82102a7a9
Diff Diff

Regressions in System.IO.Tests.Perf_StreamWriter

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
WriteFormat - Duration of single invocation 173.47 ns 426.99 ns 2.46 0.21 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.IO.Tests.Perf_StreamWriter*'

Payloads

Baseline
Compare

Histogram

System.IO.Tests.Perf_StreamWriter.WriteFormat

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 426.9929217651048 > 181.18342021582447.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/23/2022 11:50:53 AM falls between 8/5/2022 10:47:59 PM and 8/23/2022 11:50:53 AM.
IsRegressionStdDev: Marked as regression because -80.47307427579044 (T) = (0 -433.877681872193) / Math.Sqrt((16.881308028509636 / (14)) + (275.08503803058505 / (29))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (29) - 2, .025) and -1.541041487861923 = (170.7479724139669 - 433.877681872193) / 170.7479724139669 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented Aug 23, 2022 •
edited
Loading

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 543bcc5ee7d6a2b9471b016770227421c43a756e
Compare f142128e89b63577a9bbba7e2b760ec82102a7a9
Diff Diff

Regressions in System.Memory.ReadOnlySpan

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
IndexOfString - Duration of single invocation 34.13 ns 487.61 ns 14.29 0.10 False
IndexOfString - Duration of single invocation 23.31 ns 101.98 ns 4.37 0.31 False
IndexOfString - Duration of single invocation 32.55 ns 453.78 ns 13.94 0.17 False
IndexOfString - Duration of single invocation 234.36 ns 4.61 μs 19.69 0.02 False
IndexOfString - Duration of single invocation 234.19 ns 4.60 μs 19.65 0.01 False
IndexOfString - Duration of single invocation 78.74 ns 873.11 ns 11.09 0.04 False
IndexOfString - Duration of single invocation 24.67 ns 36.17 ns 1.47 0.20 False
IndexOfString - Duration of single invocation 93.45 ns 1.73 μs 18.56 0.01 False

graph
graph
graph
graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Memory.ReadOnlySpan*'

Payloads

Baseline
Compare

Histogram

System.Memory.ReadOnlySpan.IndexOfString(input: "だだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだすだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだ", value: "す", comparisonType: Ordinal)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 487.61459682895224 > 33.784328185896165.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/23/2022 11:50:53 AM falls between 8/5/2022 10:47:59 PM and 8/23/2022 11:50:53 AM.
IsRegressionStdDev: Marked as regression because -304.2230570152591 (T) = (0 -490.3302481916936) / Math.Sqrt((0.5028662940680104 / (14)) + (64.71450814850557 / (29))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (29) - 2, .025) and -14.21388221323135 = (32.229133979048335 - 490.3302481916936) / 32.229133979048335 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.ReadOnlySpan.IndexOfString(input: "Hello WorldbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbareallyreallylongHello WorldbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbareallyreallylongHello Worldbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbareallyreallylong!xyz", value: "w", comparisonType: OrdinalIgnoreCase)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 101.98013908331971 > 24.663213899276023.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/23/2022 11:50:53 AM falls between 8/5/2022 10:47:59 PM and 8/23/2022 11:50:53 AM.
IsRegressionStdDev: Marked as regression because -121.40656375209959 (T) = (0 -103.50581043172582) / Math.Sqrt((1.033988285663497 / (14)) + (9.596951436578491 / (27))) is less than -2.0226909200346674 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (27) - 2, .025) and -3.3200995462258622 = (23.959126247947427 - 103.50581043172582) / 23.959126247947427 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.ReadOnlySpan.IndexOfString(input: "Hello Worldbbbbbbbbbbbbbbcbbbbbbbbbbbbbbbbbbba!", value: "y", comparisonType: Ordinal)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 453.7848295476578 > 34.03516543601816.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/23/2022 11:50:53 AM falls between 8/5/2022 10:47:59 PM and 8/23/2022 11:50:53 AM.
IsRegressionStdDev: Marked as regression because -137.5390467262803 (T) = (0 -464.62135574801874) / Math.Sqrt((0.7465493746276208 / (14)) + (274.5015625746037 / (28))) is less than -2.0210753903043583 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (28) - 2, .025) and -13.16237377669304 = (32.806742928409655 - 464.62135574801874) / 32.806742928409655 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.ReadOnlySpan.IndexOfString(input: "AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAXAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA", value: "X", comparisonType: Ordinal)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 4.614070490539966 > 245.6993616631503.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/23/2022 11:50:53 AM falls between 8/5/2022 10:47:59 PM and 8/23/2022 11:50:53 AM.
IsRegressionStdDev: Marked as regression because -101.24192606537102 (T) = (0 -4718.334087518352) / Math.Sqrt((0.05060724263704915 / (14)) + (56893.1014870835 / (29))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (29) - 2, .025) and -19.15816743090678 = (234.06562643606867 - 4718.334087518352) / 234.06562643606867 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.ReadOnlySpan.IndexOfString(input: "だだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだxだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだ", value: "x", comparisonType: Ordinal)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 4.60164357679835 > 245.60467572935048.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/23/2022 11:50:53 AM falls between 8/5/2022 10:47:59 PM and 8/23/2022 11:50:53 AM.
IsRegressionStdDev: Marked as regression because -173.76302404045188 (T) = (0 -4672.242852086905) / Math.Sqrt((0.1567034407743528 / (14)) + (18264.945734808905 / (28))) is less than -2.0210753903043583 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (28) - 2, .025) and -18.94960565531547 = (234.20226609050843 - 4672.242852086905) / 234.20226609050843 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.ReadOnlySpan.IndexOfString(input: "AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAXAAAAAAAAAAAAAAAAAAAAAAAAAAAAA", value: "x", comparisonType: OrdinalIgnoreCase)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 873.10607839328 > 82.72251524773952.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/23/2022 11:50:53 AM falls between 8/5/2022 10:47:59 PM and 8/23/2022 11:50:53 AM.
IsRegressionStdDev: Marked as regression because -196.35099467445784 (T) = (0 -880.8949865329986) / Math.Sqrt((0.7610023579611145 / (14)) + (481.96388306839737 / (29))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (29) - 2, .025) and -10.133116370497365 = (79.12384611979449 - 880.8949865329986) / 79.12384611979449 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.ReadOnlySpan.IndexOfString(input: "StrIng", value: "string", comparisonType: OrdinalIgnoreCase)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 36.170697722320114 > 26.19735640294146.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/23/2022 11:50:53 AM falls between 8/5/2022 10:47:59 PM and 8/23/2022 11:50:53 AM.
IsRegressionStdDev: Marked as regression because -108.15649360972516 (T) = (0 -36.37797819442204) / Math.Sqrt((0.037995025208353364 / (14)) + (0.2483548816882856 / (29))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (29) - 2, .025) and -0.4614303015997112 = (24.892037721266604 - 36.37797819442204) / 24.892037721266604 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.ReadOnlySpan.IndexOfString(input: "Hello WorldbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbareallyreallylongHello WorldbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbareallyreallylongHello Worldbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbareallyreallylong!xyz", value: "~", comparisonType: Ordinal)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.7340257789372409 > 98.51782727816838.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/23/2022 11:50:53 AM falls between 8/5/2022 10:47:59 PM and 8/23/2022 11:50:53 AM.
IsRegressionStdDev: Marked as regression because -214.82544799861023 (T) = (0 -1759.3582825839767) / Math.Sqrt((0.7608732217210599 / (14)) + (1681.2130657035607 / (28))) is less than -2.0210753903043583 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (28) - 2, .025) and -17.721823517533725 = (93.97366025463644 - 1759.3582825839767) / 93.97366025463644 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS ubuntu 18.04
Baseline 543bcc5ee7d6a2b9471b016770227421c43a756e
Compare f142128e89b63577a9bbba7e2b760ec82102a7a9
Diff Diff

Regressions in MicroBenchmarks.Serializers.Json_ToStream<MyEventsListerViewModel>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
JsonNet_ - Duration of single invocation 2.59 ms 2.86 ms 1.10 0.09 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'MicroBenchmarks.Serializers.Json_ToStream&lt;MyEventsListerViewModel&gt;*'

Payloads

Baseline
Compare

Histogram

MicroBenchmarks.Serializers.Json_ToStream<MyEventsListerViewModel>.JsonNet_

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.858746665098039 > 2.8178917236652645.
IsChangePoint: Marked as a change because one of 7/5/2022 3:17:18 PM, 7/15/2022 2:19:08 PM, 8/5/2022 10:47:59 PM, 8/23/2022 11:50:53 AM falls between 8/5/2022 10:47:59 PM and 8/23/2022 11:50:53 AM.
IsRegressionStdDev: Marked as regression because -8.320213997090566 (T) = (0 -2827508.0106259184) / Math.Sqrt((2292345485.717487 / (14)) + (4766154291.923005 / (28))) is less than -2.0210753903043583 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (28) - 2, .025) and -0.05683062910139096 = (2675459.939148538 - 2827508.0106259184) / 2675459.939148538 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 543bcc5ee7d6a2b9471b016770227421c43a756e
Compare f142128e89b63577a9bbba7e2b760ec82102a7a9
Diff Diff

Regressions in Microsoft.Extensions.Primitives.StringSegmentBenchmark

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
IndexOfAny - Duration of single invocation 14.86 ns 50.92 ns 3.43 0.68 False
IndexOf - Duration of single invocation 18.35 ns 84.72 ns 4.62 0.61 False
LastIndexOf - Duration of single invocation 16.78 ns 65.08 ns 3.88 0.65 False

graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'Microsoft.Extensions.Primitives.StringSegmentBenchmark*'

Payloads

Baseline
Compare

Histogram

Microsoft.Extensions.Primitives.StringSegmentBenchmark.IndexOfAny

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 50.91802239054728 > 18.6576994961329.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/23/2022 11:50:53 AM falls between 8/5/2022 10:47:59 PM and 8/23/2022 11:50:53 AM.
IsRegressionStdDev: Marked as regression because -36.22759886723291 (T) = (0 -47.18507838790887) / Math.Sqrt((3.7206236675699813 / (14)) + (12.33523683258373 / (28))) is less than -2.0210753903043583 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (28) - 2, .025) and -1.8189146800730565 = (16.738739459360293 - 47.18507838790887) / 16.738739459360293 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### Microsoft.Extensions.Primitives.StringSegmentBenchmark.IndexOf

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 84.71913750255565 > 20.715502331823917.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/23/2022 11:50:53 AM falls between 8/5/2022 10:47:59 PM and 8/23/2022 11:50:53 AM.
IsRegressionStdDev: Marked as regression because -82.25137792950324 (T) = (0 -83.4934899552102) / Math.Sqrt((3.8017693886097863 / (14)) + (9.29909705199728 / (28))) is less than -2.0210753903043583 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (28) - 2, .025) and -3.2625845682962464 = (19.5875269141188 - 83.4934899552102) / 19.5875269141188 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### Microsoft.Extensions.Primitives.StringSegmentBenchmark.LastIndexOf

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 65.08190371520622 > 22.257768665279706.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/23/2022 11:50:53 AM falls between 8/5/2022 10:47:59 PM and 8/23/2022 11:50:53 AM.
IsRegressionStdDev: Marked as regression because -58.858950996418145 (T) = (0 -64.68431041811989) / Math.Sqrt((3.999640174588973 / (14)) + (8.924392534887998 / (29))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (29) - 2, .025) and -2.34410118116752 = (19.342808998242326 - 64.68431041811989) / 19.342808998242326 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 543bcc5ee7d6a2b9471b016770227421c43a756e
Compare f142128e89b63577a9bbba7e2b760ec82102a7a9
Diff Diff

Regressions in System.Memory.Span<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
IndexOfValue - Duration of single invocation 171.00 ns 2.36 μs 13.79 0.03 False
LastIndexOfValue - Duration of single invocation 135.04 ns 2.43 μs 18.00 0.02 False

graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Memory.Span&lt;Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Memory.Span<Int32>.IndexOfValue(Size: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.358176211048312 > 179.7444605194581.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/23/2022 11:50:53 AM falls between 8/5/2022 10:47:59 PM and 8/23/2022 11:50:53 AM.
IsRegressionStdDev: Marked as regression because -156.42227643933458 (T) = (0 -2406.3845005271896) / Math.Sqrt((2.2500015628255032 / (14)) + (5710.373732052404 / (28))) is less than -2.0210753903043583 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (28) - 2, .025) and -13.017862532623067 = (171.665579893292 - 2406.3845005271896) / 171.665579893292 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Span&lt;Int32&gt;.LastIndexOfValue(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.4310165814638838 > 142.03914147261904.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/23/2022 11:50:53 AM falls between 8/5/2022 10:47:59 PM and 8/23/2022 11:50:53 AM.
IsRegressionStdDev: Marked as regression because -202.03621957763792 (T) = (0 -2432.2987830384113) / Math.Sqrt((1.2799202029164007 / (14)) + (3744.9075123973444 / (29))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (29) - 2, .025) and -16.937601037227935 = (135.59777464056572 - 2432.2987830384113) / 135.59777464056572 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 543bcc5ee7d6a2b9471b016770227421c43a756e
Compare f142128e89b63577a9bbba7e2b760ec82102a7a9
Diff Diff

Regressions in System.Memory.Span<Char>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
LastIndexOfValue - Duration of single invocation 128.66 ns 2.41 μs 18.72 0.04 False
IndexOfAnyThreeValues - Duration of single invocation 195.46 ns 3.77 μs 19.30 0.03 False
IndexOfValue - Duration of single invocation 126.39 ns 2.40 μs 18.96 0.06 False
IndexOfAnyTwoValues - Duration of single invocation 236.91 ns 3.08 μs 12.98 0.02 False
LastIndexOfAnyValues - Duration of single invocation 95.95 ns 3.08 μs 32.08 0.09 False
IndexOfAnyFourValues - Duration of single invocation 279.58 ns 4.47 μs 15.98 0.03 False

graph
graph
graph
graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Memory.Span&lt;Char&gt;*'

Payloads

Baseline
Compare

Histogram

System.Memory.Span<Char>.LastIndexOfValue(Size: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.40842584367609 > 137.18103536974002.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/23/2022 11:50:53 AM falls between 8/5/2022 10:47:59 PM and 8/23/2022 11:50:53 AM.
IsRegressionStdDev: Marked as regression because -96.52800611586298 (T) = (0 -2473.9619278246505) / Math.Sqrt((0.9181860130930313 / (14)) + (16509.11084088377 / (28))) is less than -2.0210753903043583 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (28) - 2, .025) and -18.038191556690318 = (129.94731776166321 - 2473.9619278246505) / 129.94731776166321 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Span&lt;Char&gt;.IndexOfAnyThreeValues(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.772286871933099 > 206.48339309987557.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/23/2022 11:50:53 AM falls between 8/5/2022 10:47:59 PM and 8/23/2022 11:50:53 AM.
IsRegressionStdDev: Marked as regression because -112.12045581161223 (T) = (0 -3843.77923396972) / Math.Sqrt((3.370794594716354 / (14)) + (30665.190461492468 / (29))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (29) - 2, .025) and -18.468962449024534 = (197.4311288561917 - 3843.77923396972) / 197.4311288561917 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Span&lt;Char&gt;.IndexOfValue(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.3964984596995342 > 132.1323954061054.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/23/2022 11:50:53 AM falls between 8/5/2022 10:47:59 PM and 8/23/2022 11:50:53 AM.
IsRegressionStdDev: Marked as regression because -267.4182166173596 (T) = (0 -2396.127860023525) / Math.Sqrt((4.345582870190371 / (14)) + (2009.5129991916272 / (28))) is less than -2.0210753903043583 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (28) - 2, .025) and -18.05182642224808 = (125.76893190804047 - 2396.127860023525) / 125.76893190804047 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Span&lt;Char&gt;.IndexOfAnyTwoValues(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.0753977627923126 > 249.19577071103217.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/23/2022 11:50:53 AM falls between 8/5/2022 10:47:59 PM and 8/23/2022 11:50:53 AM.
IsRegressionStdDev: Marked as regression because -223.94947262972323 (T) = (0 -3091.5540573922735) / Math.Sqrt((1.1468207019903844 / (14)) + (4545.0201175809225 / (28))) is less than -2.0210753903043583 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (28) - 2, .025) and -12.012135047356978 = (237.59006851225612 - 3091.5540573922735) / 237.59006851225612 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Span&lt;Char&gt;.LastIndexOfAnyValues(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.077791397506381 > 101.56128111377362.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/23/2022 11:50:53 AM falls between 8/5/2022 10:47:59 PM and 8/23/2022 11:50:53 AM.
IsRegressionStdDev: Marked as regression because -156.1190606777848 (T) = (0 -3106.8022256131394) / Math.Sqrt((3.7970755950343804 / (14)) + (9657.426213825875 / (26))) is less than -2.0243941639098457 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (26) - 2, .025) and -31.075860596734632 = (96.85795385734455 - 3106.8022256131394) / 96.85795385734455 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Span&lt;Char&gt;.IndexOfAnyFourValues(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 4.467081662612098 > 293.4609871895166.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/23/2022 11:50:53 AM falls between 8/5/2022 10:47:59 PM and 8/23/2022 11:50:53 AM.
IsRegressionStdDev: Marked as regression because -85.5630623736574 (T) = (0 -4594.031599553125) / Math.Sqrt((0.2535476631661038 / (14)) + (73744.38721979049 / (29))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (29) - 2, .025) and -15.448123757917857 = (279.30429434795764 - 4594.031599553125) / 279.30429434795764 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@lewing
Copy link
Member

lewing commented Sep 15, 2022

looks like dotnet/runtime#73768

cc @adamsitnik @SamMonoRT @radekdoulik

@naricc naricc mentioned this issue Sep 15, 2022
Closed
@radekdoulik radekdoulik mentioned this issue Sep 15, 2022
Closed
@naricc naricc removed the untriaged label Sep 28, 2022
@adamsitnik
Copy link
Member

I looked at various regressions from this list and they seem to be solved:

image

image

image

image

image

image

image

@lewing I believe you can close this issue (I don't have the permissions to do that)

@lewing
Copy link
Member

lewing commented Sep 28, 2022

I don't have permission either

@EgorBo EgorBo mentioned this issue Sep 28, 2022
Closed
18 tasks
@naricc
Copy link

naricc commented Sep 28, 2022

Sorry, I didn't realize that since I transferred these, apparently only I can close them?

@naricc naricc closed this as completed Sep 28, 2022
@jeffhandley jeffhandley mentioned this issue Oct 6, 2022
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
arch-wasm arch-x64 branch-refs/heads/main kind-micro os-linux perf-regression PGO Applied if there were any profile guided optimization updates in the observed interval. wasm-aot
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@lewing @adamsitnik @naricc