Skip to content

[Perf] Linux/arm64: 10 Improvements on 3/17/2023 1:20:12 AM #14416

@performanceautofiler

Description

@performanceautofiler

Run Information

Name Value
Architecture arm64
OS ubuntu 20.04
Queue AmpereUbuntu
Baseline 004a2eb687eb57c58cf706e4edfa87858415add5
Compare 34c0472fe5a237f85c39bfc45e586ac0fd6e2f11
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Numerics.Tests.Perf_VectorConvert

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Narrow_uint - Duration of single invocation 840.77 ns 673.63 ns 0.80 0.00 False
Narrow_double - Duration of single invocation 840.90 ns 746.46 ns 0.89 0.00 False
Narrow_ushort - Duration of single invocation 840.76 ns 673.96 ns 0.80 0.00 False
Narrow_int - Duration of single invocation 840.76 ns 673.70 ns 0.80 0.00 False
Narrow_ulong - Duration of single invocation 840.77 ns 673.53 ns 0.80 0.00 False
Narrow_long - Duration of single invocation 839.75 ns 673.65 ns 0.80 0.00 False
Narrow_short - Duration of single invocation 839.77 ns 673.64 ns 0.80 0.00 False

graph
graph
graph
graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorConvert*'

Payloads

Baseline
Compare

Histogram

System.Numerics.Tests.Perf_VectorConvert.Narrow_uint


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 673.6337917850467 < 798.7323724293689.
IsChangePoint: Marked as a change because one of 3/16/2023 8:14:39 PM, 3/23/2023 3:52:33 AM falls between 3/14/2023 3:18:57 PM and 3/23/2023 3:52:33 AM.
IsImprovementStdDev: Marked as improvement because 1346.7339487664717 (T) = (0 -673.3858832512258) / Math.Sqrt((1.7339786160846244 / (299)) + (0.2003253151015099 / (20))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (20) - 2, .975) and 0.20096684399607964 = (842.7508648313486 - 673.3858832512258) / 842.7508648313486 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorConvert.Narrow_double

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 746.4597910485217 < 798.8569967306972.
IsChangePoint: Marked as a change because one of 3/16/2023 8:14:39 PM, 3/23/2023 3:52:33 AM falls between 3/14/2023 3:18:57 PM and 3/23/2023 3:52:33 AM.
IsImprovementStdDev: Marked as improvement because 171.98489698909847 (T) = (0 -739.2796083415932) / Math.Sqrt((3.7835595706968226 / (299)) + (7.020986383470197 / (20))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (20) - 2, .975) and 0.12303719378217433 = (842.9999574668007 - 739.2796083415932) / 842.9999574668007 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorConvert.Narrow_ushort

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 673.9619632101226 < 798.7172970299747.
IsChangePoint: Marked as a change because one of 3/16/2023 8:14:39 PM, 3/23/2023 3:52:33 AM falls between 3/14/2023 3:18:57 PM and 3/23/2023 3:52:33 AM.
IsImprovementStdDev: Marked as improvement because 1089.0778690847796 (T) = (0 -673.4747193987714) / Math.Sqrt((4.2953548489668965 / (299)) + (0.19697536931400342 / (20))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (20) - 2, .975) and 0.20104665095161836 = (842.9462373503216 - 673.4747193987714) / 842.9462373503216 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorConvert.Narrow_int

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 673.700867756276 < 798.6826440669115.
IsChangePoint: Marked as a change because one of 3/16/2023 8:14:39 PM, 3/23/2023 3:52:33 AM falls between 3/14/2023 3:18:57 PM and 3/23/2023 3:52:33 AM.
IsImprovementStdDev: Marked as improvement because 264.59592495810347 (T) = (0 -674.2333190223375) / Math.Sqrt((1.8038344480397255 / (299)) + (7.992441639149341 / (20))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (20) - 2, .975) and 0.19996733638831266 = (842.7572394089034 - 674.2333190223375) / 842.7572394089034 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorConvert.Narrow_ulong

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 673.5306051136773 < 798.7269982533123.
IsChangePoint: Marked as a change because one of 3/16/2023 8:14:39 PM, 3/23/2023 3:52:33 AM falls between 3/14/2023 3:18:57 PM and 3/23/2023 3:52:33 AM.
IsImprovementStdDev: Marked as improvement because 1380.8517607203319 (T) = (0 -673.6153851177048) / Math.Sqrt((4.395084392688365 / (299)) + (0.006830678145290009 / (20))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (20) - 2, .975) and 0.20089700855370718 = (842.9644142597082 - 673.6153851177048) / 842.9644142597082 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorConvert.Narrow_long

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 673.6525952098159 < 798.7033538170052.
IsChangePoint: Marked as a change because one of 3/16/2023 8:14:39 PM, 3/23/2023 3:52:33 AM falls between 3/14/2023 3:18:57 PM and 3/23/2023 3:52:33 AM.
IsImprovementStdDev: Marked as improvement because 1598.7942213246654 (T) = (0 -673.5802015606442) / Math.Sqrt((2.4293002617239625 / (299)) + (0.0618192531188115 / (20))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (20) - 2, .975) and 0.20087704101590104 = (842.8993235495906 - 673.5802015606442) / 842.8993235495906 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorConvert.Narrow_short

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 673.6443531413084 < 798.6961354772263.
IsChangePoint: Marked as a change because one of 3/16/2023 8:14:39 PM, 3/23/2023 3:52:33 AM falls between 3/14/2023 3:18:57 PM and 3/23/2023 3:52:33 AM.
IsImprovementStdDev: Marked as improvement because 1264.9433204688398 (T) = (0 -673.4947819707376) / Math.Sqrt((3.1302357436347785 / (299)) + (0.1492048966154511 / (20))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (20) - 2, .975) and 0.20095146674481762 = (842.8709320409349 - 673.4947819707376) / 842.8709320409349 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture arm64
OS ubuntu 20.04
Queue AmpereUbuntu
Baseline 004a2eb687eb57c58cf706e4edfa87858415add5
Compare 34c0472fe5a237f85c39bfc45e586ac0fd6e2f11
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Count - Duration of single invocation 37.45 ms 32.61 ms 0.87 0.01 False
Count - Duration of single invocation 37.45 ms 32.63 ms 0.87 0.01 False

graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig*'

Payloads

Baseline
Compare

Histogram

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig.Count(Pattern: "\p{Sm}", Options: None)


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 32.6131114 < 35.58684329394557.
IsChangePoint: Marked as a change because one of 1/3/2023 12:19:11 AM, 1/11/2023 5:39:19 AM, 1/21/2023 1:51:00 AM, 3/16/2023 8:14:39 PM, 3/23/2023 3:52:33 AM falls between 3/14/2023 3:18:57 PM and 3/23/2023 3:52:33 AM.
IsImprovementStdDev: Marked as improvement because 32.82555108958532 (T) = (0 -32600485.342899006) / Math.Sqrt((6618097081497 / (299)) + (926684427.58345 / (20))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (20) - 2, .975) and 0.13040390631266027 = (37489226.986592695 - 32600485.342899006) / 37489226.986592695 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig.Count(Pattern: "\\p{Sm}", Options: NonBacktracking)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 32.62532103296703 < 35.58350536619047.
IsChangePoint: Marked as a change because one of 1/3/2023 12:19:11 AM, 1/11/2023 5:39:19 AM, 1/21/2023 1:51:00 AM, 3/16/2023 8:14:39 PM, 3/23/2023 3:52:33 AM falls between 3/14/2023 3:18:57 PM and 3/23/2023 3:52:33 AM.
IsImprovementStdDev: Marked as improvement because 31.932774642924265 (T) = (0 -32710227.64225013) / Math.Sqrt((6767471252607.631 / (299)) + (3675897669.862565 / (20))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (20) - 2, .975) and 0.12851319626279648 = (37533818.64415917 - 32710227.64225013) / 37533818.64415917 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture arm64
OS ubuntu 20.04
Queue AmpereUbuntu
Baseline 004a2eb687eb57c58cf706e4edfa87858415add5
Compare 34c0472fe5a237f85c39bfc45e586ac0fd6e2f11
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Mariomkas

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Count - Duration of single invocation 20.21 ms 19.07 ms 0.94 0.01 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Mariomkas*'

Payloads

Baseline
Compare

Histogram

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Mariomkas.Count(Pattern: "(?:(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9])\.){3}(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9])", Options: NonBacktracking)


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 19.0672608 < 19.217149290000002.
IsChangePoint: Marked as a change because one of 1/3/2023 12:19:11 AM, 1/11/2023 5:39:19 AM, 1/21/2023 1:51:00 AM, 3/16/2023 8:14:39 PM, 3/23/2023 3:52:33 AM falls between 3/14/2023 3:18:57 PM and 3/23/2023 3:52:33 AM.
IsImprovementStdDev: Marked as improvement because 28.33007620099602 (T) = (0 -19078796.613333337) / Math.Sqrt((638868165953.4419 / (299)) + (249216732.50873303 / (20))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (20) - 2, .975) and 0.06440469421318604 = (20392146.57804264 - 19078796.613333337) / 20392146.57804264 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions