[Perf] Linux/arm64: 7 Improvements on 3/13/2023 9:17:14 PM #14107
Description
Run Information
| Architecture | arm64 |
|---|---|
| OS | ubuntu 20.04 |
| Baseline | 46f9533d9b0663aa652b6dc7001417d4924343d2 |
| Compare | 2b185ac8c5c1302804a3336c62cb33782d9d4916 |
| Diff | Diff |
Improvements in StoreBlock.AnyLocation
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| InitBlockAllZeros128 - Duration of single invocation | 5.09 ns | 2.76 ns | 0.54 | 0.16 | False | |||||
| InitBlockAllOnes128 - Duration of single invocation | 5.13 ns | 2.76 ns | 0.54 | 0.20 | False | |||||
| CopyBlock128 - Duration of single invocation | 5.81 ns | 3.73 ns | 0.64 | 0.23 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'StoreBlock.AnyLocation*'Payloads
Histogram
StoreBlock.AnyLocation.InitBlockAllZeros128
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.759798330259609 < 4.872561365914348.
IsChangePoint: Marked as a change because one of 3/13/2023 2:14:21 PM, 3/16/2023 2:30:29 AM falls between 3/7/2023 7:00:18 AM and 3/16/2023 2:30:29 AM.
IsImprovementStdDev: Marked as improvement because 175.04554497851285 (T) = (0 -2.7525717125644666) / Math.Sqrt((0.04394078965075729 / (299)) + (0.00037756996309576456 / (8))) is greater than 1.967772354556772 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (8) - 2, .975) and 0.46980795895798166 = (5.191650382292936 - 2.7525717125644666) / 5.191650382292936 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### StoreBlock.AnyLocation.InitBlockAllOnes128
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.762790873833566 < 4.873551680223714.
IsChangePoint: Marked as a change because one of 3/13/2023 2:14:21 PM, 3/16/2023 2:30:29 AM falls between 3/7/2023 7:00:18 AM and 3/16/2023 2:30:29 AM.
IsImprovementStdDev: Marked as improvement because 163.49818519413944 (T) = (0 -2.7439742700227314) / Math.Sqrt((0.04723773093496164 / (299)) + (0.0005398802357431444 / (8))) is greater than 1.967772354556772 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (8) - 2, .975) and 0.4722122871755378 = (5.19901127545831 - 2.7439742700227314) / 5.19901127545831 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### StoreBlock.AnyLocation.CopyBlock128
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 3.7325124601450974 < 5.519691491429137.
IsChangePoint: Marked as a change because one of 3/13/2023 2:14:21 PM, 3/16/2023 2:30:29 AM falls between 3/7/2023 7:00:18 AM and 3/16/2023 2:30:29 AM.
IsImprovementStdDev: Marked as improvement because 142.1601870790738 (T) = (0 -3.7282338542670055) / Math.Sqrt((0.05388767193399177 / (299)) + (0.00042404527590819653 / (8))) is greater than 1.967772354556772 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (8) - 2, .975) and 0.3680204339378602 = (5.899294936856273 - 3.7282338542670055) / 5.899294936856273 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Architecture | arm64 |
|---|---|
| OS | ubuntu 20.04 |
| Baseline | 46f9533d9b0663aa652b6dc7001417d4924343d2 |
| Compare | 2b185ac8c5c1302804a3336c62cb33782d9d4916 |
| Diff | Diff |
Improvements in System.Tests.Perf_Single
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| Parse - Duration of single invocation | 124.90 ns | 115.30 ns | 0.92 | 0.02 | False | |||||
| TryParse - Duration of single invocation | 124.56 ns | 115.68 ns | 0.93 | 0.01 | False | |||||
| Parse - Duration of single invocation | 115.98 ns | 103.39 ns | 0.89 | 0.02 | False | |||||
| TryParse - Duration of single invocation | 116.23 ns | 102.85 ns | 0.88 | 0.02 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Single*'Payloads
Histogram
System.Tests.Perf_Single.Parse(value: "-3.4028235E+38")
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 115.29974191737732 < 118.61305084688989.
IsChangePoint: Marked as a change because one of 11/30/2022 2:05:04 AM, 1/27/2023 3:43:27 AM, 2/16/2023 5:39:51 AM, 3/13/2023 2:14:21 PM, 3/16/2023 2:30:29 AM falls between 3/7/2023 7:00:18 AM and 3/16/2023 2:30:29 AM.
IsImprovementStdDev: Marked as improvement because 39.98083210080784 (T) = (0 -115.3017110273394) / Math.Sqrt((5.9247265736415375 / (299)) + (0.23326832260947666 / (8))) is greater than 1.967772354556772 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (8) - 2, .975) and 0.07126697609783147 = (124.14946821088287 - 115.3017110273394) / 124.14946821088287 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Tests.Perf_Single.TryParse(value: "-3.4028235E+38")
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 115.68356857746753 < 118.7718991118427.
IsChangePoint: Marked as a change because one of 11/30/2022 2:05:04 AM, 1/27/2023 3:43:27 AM, 2/16/2023 5:39:51 AM, 3/13/2023 2:14:21 PM, 3/16/2023 2:30:29 AM falls between 3/7/2023 7:00:18 AM and 3/16/2023 2:30:29 AM.
IsImprovementStdDev: Marked as improvement because 42.803082447016166 (T) = (0 -115.7417088696214) / Math.Sqrt((6.200124306189473 / (299)) + (0.16139267404296856 / (8))) is greater than 1.967772354556772 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (8) - 2, .975) and 0.06959431557139342 = (124.39918500788426 - 115.7417088696214) / 124.39918500788426 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Tests.Perf_Single.Parse(value: "3.4028235E+38")
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 103.38661113765946 < 110.24675281510996.
IsChangePoint: Marked as a change because one of 11/30/2022 2:05:04 AM, 1/27/2023 3:43:27 AM, 2/16/2023 5:39:51 AM, 3/13/2023 2:14:21 PM, 3/16/2023 2:30:29 AM falls between 3/7/2023 7:00:18 AM and 3/16/2023 2:30:29 AM.
IsImprovementStdDev: Marked as improvement because 19.391766814566715 (T) = (0 -103.4362288880479) / Math.Sqrt((18.33644464378148 / (299)) + (0.5224025377895587 / (8))) is greater than 1.967772354556772 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (8) - 2, .975) and 0.062540187717135 = (110.33670727299136 - 103.4362288880479) / 110.33670727299136 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Tests.Perf_Single.TryParse(value: "3.4028235E+38")
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 102.84691549489936 < 110.53311950147523.
IsChangePoint: Marked as a change because one of 11/30/2022 2:05:04 AM, 1/27/2023 3:43:27 AM, 2/16/2023 5:39:51 AM, 3/13/2023 2:14:21 PM, 3/16/2023 2:30:29 AM falls between 3/7/2023 7:00:18 AM and 3/16/2023 2:30:29 AM.
IsImprovementStdDev: Marked as improvement because 27.433393565808142 (T) = (0 -102.71965961406045) / Math.Sqrt((22.472258106011 / (299)) + (0.1228368177293985 / (8))) is greater than 1.967772354556772 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (8) - 2, .975) and 0.07437321103102404 = (110.97308422596149 - 102.71965961406045) / 110.97308422596149 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository