[Perf] Windows/arm64: 7 Improvements on 3/17/2023 1:20:12 AM #14430
Description
Run Information
| Name | Value |
|---|---|
| Architecture | arm64 |
| OS | Windows 10.0.25094 |
| Queue | AmpereWindows |
| Baseline | 004a2eb687eb57c58cf706e4edfa87858415add5 |
| Compare | 34c0472fe5a237f85c39bfc45e586ac0fd6e2f11 |
| Diff | Diff |
| Configs | CompilationMode:tiered, RunKind:micro |
Improvements in System.Numerics.Tests.Perf_VectorConvert
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| Narrow_long - Duration of single invocation | 1.22 μs | 983.32 ns | 0.81 | 0.60 | False | |||||
| Narrow_ulong - Duration of single invocation | 1.19 μs | 933.48 ns | 0.78 | 0.63 | False | |||||
| Narrow_uint - Duration of single invocation | 1.13 μs | 911.33 ns | 0.80 | 0.63 | False | |||||
| Narrow_short - Duration of single invocation | 1.03 μs | 851.80 ns | 0.83 | 0.63 | False | |||||
| Narrow_ushort - Duration of single invocation | 1.22 μs | 807.14 ns | 0.66 | 0.59 | False | |||||
| Narrow_int - Duration of single invocation | 1.12 μs | 788.70 ns | 0.71 | 0.65 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorConvert*'Payloads
Histogram
System.Numerics.Tests.Perf_VectorConvert.Narrow_long
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 983.317058635604 < 1.132262003409838.
IsChangePoint: Marked as a change because one of 3/16/2023 8:14:39 PM, 3/23/2023 3:52:33 AM falls between 3/14/2023 3:18:57 PM and 3/23/2023 3:52:33 AM.
IsImprovementStdDev: Marked as improvement because 13.683465916421985 (T) = (0 -909.8531794119201) / Math.Sqrt((7158.37935844616 / (299)) + (4876.025819094724 / (20))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (20) - 2, .975) and 0.19748587029502118 = (1133.7534701680597 - 909.8531794119201) / 1133.7534701680597 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Numerics.Tests.Perf_VectorConvert.Narrow_ulong
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 933.4792886558137 < 0.9651333623773708.
IsChangePoint: Marked as a change because one of 3/16/2023 8:14:39 PM, 3/23/2023 3:52:33 AM falls between 3/14/2023 3:18:57 PM and 3/23/2023 3:52:33 AM.
IsImprovementStdDev: Marked as improvement because 13.431922098250888 (T) = (0 -893.3397090337078) / Math.Sqrt((7245.496905793392 / (299)) + (5479.674840702769 / (20))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (20) - 2, .975) and 0.206128232497363 = (1125.2947208892151 - 893.3397090337078) / 1125.2947208892151 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Numerics.Tests.Perf_VectorConvert.Narrow_uint
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 911.3309609496695 < 1.1210340160098389.
IsChangePoint: Marked as a change because one of 3/16/2023 8:14:39 PM, 3/23/2023 3:52:33 AM falls between 3/14/2023 3:18:57 PM and 3/23/2023 3:52:33 AM.
IsImprovementStdDev: Marked as improvement because 19.62645584694399 (T) = (0 -914.428118388777) / Math.Sqrt((6706.997642013643 / (299)) + (1997.5843862207767 / (20))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (20) - 2, .975) and 0.19183379967795555 = (1131.4852291823 - 914.428118388777) / 1131.4852291823 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Numerics.Tests.Perf_VectorConvert.Narrow_short
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 851.7968685162132 < 1.0566461803285534.
IsChangePoint: Marked as a change because one of 3/16/2023 8:14:39 PM, 3/23/2023 3:52:33 AM falls between 3/14/2023 3:18:57 PM and 3/23/2023 3:52:33 AM.
IsImprovementStdDev: Marked as improvement because 12.679476771040795 (T) = (0 -892.3409127612779) / Math.Sqrt((6537.428053105647 / (299)) + (6634.412072902033 / (20))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (20) - 2, .975) and 0.21085128144587187 = (1130.76393812844 - 892.3409127612779) / 1130.76393812844 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Numerics.Tests.Perf_VectorConvert.Narrow_ushort
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 807.1436138832898 < 1.1161819326406037.
IsChangePoint: Marked as a change because one of 3/16/2023 8:14:39 PM, 3/23/2023 3:52:33 AM falls between 3/14/2023 3:18:57 PM and 3/23/2023 3:52:33 AM.
IsImprovementStdDev: Marked as improvement because 15.902027393092668 (T) = (0 -880.3709224935262) / Math.Sqrt((6483.878536235464 / (299)) + (4561.828835042211 / (20))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (20) - 2, .975) and 0.222075313075252 = (1131.6917142374841 - 880.3709224935262) / 1131.6917142374841 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Numerics.Tests.Perf_VectorConvert.Narrow_int
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 788.6991949747696 < 1.0727821421770178.
IsChangePoint: Marked as a change because one of 3/16/2023 8:14:39 PM, 3/23/2023 3:52:33 AM falls between 3/14/2023 3:18:57 PM and 3/23/2023 3:52:33 AM.
IsImprovementStdDev: Marked as improvement because 14.273527928663537 (T) = (0 -914.9513872855074) / Math.Sqrt((8167.387264409812 / (299)) + (4107.5280385313645 / (20))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (20) - 2, .975) and 0.192226661435622 = (1132.6833204369093 - 914.9513872855074) / 1132.6833204369093 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Name | Value |
|---|---|
| Architecture | arm64 |
| OS | Windows 10.0.25094 |
| Queue | AmpereWindows |
| Baseline | 004a2eb687eb57c58cf706e4edfa87858415add5 |
| Compare | 34c0472fe5a237f85c39bfc45e586ac0fd6e2f11 |
| Diff | Diff |
| Configs | CompilationMode:tiered, RunKind:micro |
Improvements in System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| Count - Duration of single invocation | 53.47 ms | 43.41 ms | 0.81 | 0.53 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig*'Payloads
Histogram
System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig.Count(Pattern: "\p{Sm}", Options: NonBacktracking)
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 43.409464 < 49.765238187499996.
IsChangePoint: Marked as a change because one of 1/3/2023 12:19:11 AM, 3/16/2023 8:14:39 PM, 3/23/2023 3:52:33 AM falls between 3/14/2023 3:18:57 PM and 3/23/2023 3:52:33 AM.
IsImprovementStdDev: Marked as improvement because 12.113590343089715 (T) = (0 -44409369.8864072) / Math.Sqrt((21523691713687.973 / (299)) + (5156058308963.793 / (19))) is greater than 1.9674995188240387 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (19) - 2, .975) and 0.13780822832686337 = (51507531.55557036 - 44409369.8864072) / 51507531.55557036 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository