[Perf] Windows/arm64: 6 Improvements on 10/6/2023 11:50:29 PM

[performanceautofiler]

Run Information

Name	Value
Architecture	arm64
OS	Windows 10.0.19041
Queue	SurfaceWindows
Baseline	f55549222e4dc1469190caa56e03361ee46208ca
Compare	f21a2666c577306e437f80fe934d76cdb15072a5
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Improvements in System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
Count - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	1.56 ms	1.47 ms	0.94	0.00	False
Count - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	320.86 μs	303.46 μs	0.95	0.00	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock*'

Payloads

Baseline
Compare

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "(?i)Sherlock|Holmes|Watson", Options: Compiled)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 8/27/2023 6:19:41 AM, 10/6/2023 9:14:38 AM, 10/11/2023 7:15:38 PM falls between 10/2/2023 7:07:56 PM and 10/11/2023 7:15:38 PM.
IsImprovementStdDev: Marked as improvement because 19.09759874396002 (T) = (0 -1461489.06503219) / Math.Sqrt((479213264.23872846 / (299)) + (314736819.999678 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.08765482028752052 = (1601903.6407829437 - 1461489.06503219) / 1601903.6407829437 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Sherlock|Holmes|Watson|Irene|Adler|John|Baker", Options: Compiled)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/15/2023 5:11:34 PM, 10/6/2023 9:14:38 AM, 10/11/2023 7:15:38 PM falls between 10/2/2023 7:07:56 PM and 10/11/2023 7:15:38 PM.
IsImprovementStdDev: Marked as improvement because 34.34964125501859 (T) = (0 -302916.3169496102) / Math.Sqrt((18585399.119986266 / (299)) + (1849959.3200064418 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.06456518894950103 = (323824.0798516289 - 302916.3169496102) / 323824.0798516289 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	arm64
OS	Windows 10.0.19041
Queue	SurfaceWindows
Baseline	f55549222e4dc1469190caa56e03361ee46208ca
Compare	f21a2666c577306e437f80fe934d76cdb15072a5
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Improvements in ByteMark

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
BenchNeural - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	834.98 ms	765.76 ms	0.92	0.00	True

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'ByteMark*'

Payloads

Baseline
Compare

ByteMark.BenchNeural

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 9/28/2023 11:34:37 PM, 10/6/2023 9:14:38 AM, 10/11/2023 7:15:38 PM falls between 10/2/2023 7:07:56 PM and 10/11/2023 7:15:38 PM.
IsImprovementStdDev: Marked as improvement because 196.09108890703018 (T) = (0 -766955150.3601953) / Math.Sqrt((40055164876125.29 / (299)) + (690788109546.3832 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.11316527525904561 = (864823093.8230615 - 766955150.3601953) / 864823093.8230615 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	arm64
OS	Windows 10.0.19041
Queue	SurfaceWindows
Baseline	f55549222e4dc1469190caa56e03361ee46208ca
Compare	f21a2666c577306e437f80fe934d76cdb15072a5
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Improvements in System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
Count - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	2.68 ms	2.53 ms	0.94	0.03	False
Count - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	42.89 ms	39.55 ms	0.92	0.01	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig*'

Payloads

Baseline
Compare

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig.Count(Pattern: "\p{Sm}", Options: Compiled)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/9/2023 7:27:03 AM, 5/9/2023 11:02:55 PM, 10/6/2023 9:14:38 AM, 10/11/2023 7:15:38 PM falls between 10/2/2023 7:07:56 PM and 10/11/2023 7:15:38 PM.
IsImprovementStdDev: Marked as improvement because 29.52349312640571 (T) = (0 -2537453.543792517) / Math.Sqrt((8078663426.1844225 / (299)) + (10915076.490233384 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.058807354506667145 = (2695998.0572972842 - 2537453.543792517) / 2695998.0572972842 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig.Count(Pattern: "(?i)Tom|Sawyer|Huckleberry|Finn", Options: Compiled)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 9/19/2023 8:08:41 AM, 10/6/2023 9:14:38 AM, 10/11/2023 7:15:38 PM falls between 10/2/2023 7:07:56 PM and 10/11/2023 7:15:38 PM.
IsImprovementStdDev: Marked as improvement because 90.86402005454875 (T) = (0 -39326837.91819292) / Math.Sqrt((244180679550.03146 / (299)) + (13051347210.77212 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.11219941669014354 = (44296927.32525186 - 39326837.91819292) / 44296927.32525186 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	arm64
OS	Windows 10.0.19041
Queue	SurfaceWindows
Baseline	f55549222e4dc1469190caa56e03361ee46208ca
Compare	f21a2666c577306e437f80fe934d76cdb15072a5
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Improvements in Benchstone.MDBenchF.MDLLoops

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
Test - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	961.26 ms	909.32 ms	0.95	0.02	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Benchstone.MDBenchF.MDLLoops*'

Payloads

Baseline
Compare

Benchstone.MDBenchF.MDLLoops.Test

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 9/28/2023 11:34:37 PM, 10/6/2023 9:14:38 AM, 10/11/2023 7:15:38 PM falls between 10/2/2023 7:07:56 PM and 10/11/2023 7:15:38 PM.
IsImprovementStdDev: Marked as improvement because 102.02913900055998 (T) = (0 -909138560.8058606) / Math.Sqrt((206502588186699.5 / (299)) + (60789564582.01882 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.0858790665109412 = (994549547.548177 - 909138560.8058606) / 994549547.548177 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

[EgorBo]

dotnet/runtime#92710