Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Perf] Windows/arm64: 9 Improvements on 9/28/2022 6:40:13 PM #8190

Open
performanceautofiler bot opened this issue Oct 6, 2022 · 1 comment
Open

[Perf] Windows/arm64: 9 Improvements on 9/28/2022 6:40:13 PM #8190

performanceautofiler bot opened this issue Oct 6, 2022 · 1 comment
Labels
Ampere arm64 CoreClr Improvement refs/heads/main RunKind=micro untriaged Windows 10.0.25094

Comments

@performanceautofiler
Copy link

Run Information

Architecture arm64
OS Windows 10.0.25094
Baseline cdb6e1d5f9075214c8a58ca75d5314b5dc64daed
Compare e3ed6c1c26d7d09e10253fd8f90e06e503ae1c61
Diff Diff

Improvements in System.IO.Tests.StreamReaderReadToEndTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ReadToEndAsync - Duration of single invocation 97.65 ms 86.92 ms 0.89 0.43 False
ReadToEnd - Duration of single invocation 106.62 ms 77.02 ms 0.72 0.32 False
ReadToEndAsync - Duration of single invocation 98.14 ms 88.85 ms 0.91 0.40 False
ReadToEnd - Duration of single invocation 109.73 ms 79.12 ms 0.72 0.36 False
ReadToEnd - Duration of single invocation 110.07 ms 82.67 ms 0.75 0.31 False
ReadToEnd - Duration of single invocation 104.51 ms 80.02 ms 0.77 0.36 False
ReadToEnd - Duration of single invocation 110.32 ms 83.18 ms 0.75 0.34 False
ReadToEndAsync - Duration of single invocation 102.71 ms 86.91 ms 0.85 0.36 False

graph
graph
graph
graph
graph
graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.IO.Tests.StreamReaderReadToEndTests*'
### Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.IO.Tests.StreamReaderReadToEndTests.ReadToEndAsync(LineLengthRange: [ 0, 0])

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 86.920655 < 98.930840175.
IsChangePoint: Marked as a change because one of 8/4/2022 3:21:43 PM, 9/4/2022 8:45:57 AM, 9/28/2022 11:19:50 AM, 10/6/2022 12:41:43 AM falls between 9/27/2022 9:53:04 AM and 10/6/2022 12:41:43 AM.
IsImprovementStdDev: Marked as improvement because 26.075612783378112 (T) = (0 -87279594.80857547) / Math.Sqrt((49681494998075.9 / (299)) + (11923236883367.08 / (34))) is greater than 1.9671567996108528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (34) - 2, .975) and 0.17680801673633076 = (106025807.5674429 - 87279594.80857547) / 106025807.5674429 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.StreamReaderReadToEndTests.ReadToEnd(LineLengthRange: [   9,   32])

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 77.0243425 < 100.80234064999999.
IsChangePoint: Marked as a change because one of 9/28/2022 11:19:50 AM, 10/6/2022 12:41:43 AM falls between 9/27/2022 9:53:04 AM and 10/6/2022 12:41:43 AM.
IsImprovementStdDev: Marked as improvement because 42.59362038981528 (T) = (0 -82372293.27221005) / Math.Sqrt((29902049541388.758 / (299)) + (5039375394690.11 / (34))) is greater than 1.9671567996108528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (34) - 2, .975) and 0.2048493191234501 = (103593312.88178656 - 82372293.27221005) / 103593312.88178656 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.StreamReaderReadToEndTests.ReadToEndAsync(LineLengthRange: [   1,    8])

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 88.846875 < 93.65018359374999.
IsChangePoint: Marked as a change because one of 8/5/2022 7:33:33 PM, 8/10/2022 11:28:06 PM, 9/28/2022 11:19:50 AM, 10/6/2022 12:41:43 AM falls between 9/27/2022 9:53:04 AM and 10/6/2022 12:41:43 AM.
IsImprovementStdDev: Marked as improvement because 26.102579995479548 (T) = (0 -86220324.53534827) / Math.Sqrt((54407352931192.69 / (299)) + (12525406589470.555 / (34))) is greater than 1.9671567996108528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (34) - 2, .975) and 0.18340232616741345 = (105584827.50836807 - 86220324.53534827) / 105584827.50836807 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.StreamReaderReadToEndTests.ReadToEnd(LineLengthRange: [ 129, 1024])

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 79.122033 < 102.31912205.
IsChangePoint: Marked as a change because one of 8/5/2022 7:33:33 PM, 8/10/2022 5:14:51 PM, 9/28/2022 11:19:50 AM, 10/6/2022 12:41:43 AM falls between 9/27/2022 9:53:04 AM and 10/6/2022 12:41:43 AM.
IsImprovementStdDev: Marked as improvement because 38.073296053179284 (T) = (0 -83263616.77699535) / Math.Sqrt((37407468347651.86 / (299)) + (4582380618977.189 / (34))) is greater than 1.9671567996108528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (34) - 2, .975) and 0.18904039183552215 = (102672951.819948 - 83263616.77699535) / 102672951.819948 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.StreamReaderReadToEndTests.ReadToEnd(LineLengthRange: [   0, 1024])

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 82.674542 < 101.875851.
IsChangePoint: Marked as a change because one of 8/5/2022 7:33:33 PM, 9/28/2022 11:19:50 AM, 10/6/2022 12:41:43 AM falls between 9/27/2022 9:53:04 AM and 10/6/2022 12:41:43 AM.
IsImprovementStdDev: Marked as improvement because 40.62577516980963 (T) = (0 -83001578.39077365) / Math.Sqrt((35077879201029.28 / (299)) + (4162517635987.756 / (34))) is greater than 1.9671567996108528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (34) - 2, .975) and 0.1933250156878402 = (102893457.71834974 - 83001578.39077365) / 102893457.71834974 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.StreamReaderReadToEndTests.ReadToEnd(LineLengthRange: [  33,  128])

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 80.021202 < 99.75765842499999.
IsChangePoint: Marked as a change because one of 8/5/2022 7:33:33 PM, 9/28/2022 11:19:50 AM, 10/6/2022 12:41:43 AM falls between 9/27/2022 9:53:04 AM and 10/6/2022 12:41:43 AM.
IsImprovementStdDev: Marked as improvement because 43.26896971096642 (T) = (0 -82338422.86996149) / Math.Sqrt((34178451751105.844 / (299)) + (3365366308184.812 / (34))) is greater than 1.9671567996108528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (34) - 2, .975) and 0.19529695692163565 = (102321500.55625318 - 82338422.86996149) / 102321500.55625318 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.StreamReaderReadToEndTests.ReadToEnd(LineLengthRange: [1025, 2048])

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 83.17845894736843 < 100.86543965.
IsChangePoint: Marked as a change because one of 9/28/2022 11:19:50 AM, 10/6/2022 12:41:43 AM falls between 9/27/2022 9:53:04 AM and 10/6/2022 12:41:43 AM.
IsImprovementStdDev: Marked as improvement because 39.89664944050176 (T) = (0 -82782333.36596678) / Math.Sqrt((36374957821713.27 / (299)) + (4254352198685.1025 / (34))) is greater than 1.9671567996108528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (34) - 2, .975) and 0.19316966848136274 = (102601910.3795363 - 82782333.36596678) / 102601910.3795363 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.StreamReaderReadToEndTests.ReadToEndAsync(LineLengthRange: [   0, 1024])

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 86.913844 < 97.1462661.
IsChangePoint: Marked as a change because one of 8/10/2022 1:54:59 AM, 8/10/2022 11:28:06 PM, 9/28/2022 11:19:50 AM, 10/6/2022 12:41:43 AM falls between 9/27/2022 9:53:04 AM and 10/6/2022 12:41:43 AM.
IsImprovementStdDev: Marked as improvement because 29.31897115940977 (T) = (0 -86529603.91684076) / Math.Sqrt((55237743053085.92 / (299)) + (8666434259139.278 / (34))) is greater than 1.9671567996108528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (34) - 2, .975) and 0.1834486435740401 = (105969579.54436606 - 86529603.91684076) / 105969579.54436606 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture arm64
OS Windows 10.0.25094
Baseline cdb6e1d5f9075214c8a58ca75d5314b5dc64daed
Compare e3ed6c1c26d7d09e10253fd8f90e06e503ae1c61
Diff Diff

Improvements in System.IO.Tests.Perf_File

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ReadAllBytes - Duration of single invocation 48.34 ms 42.52 ms 0.88 0.68 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.IO.Tests.Perf_File*'
### Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.IO.Tests.Perf_File.ReadAllBytes(size: 104857600)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 42.51623888888888 < 54.470719762499996.
IsChangePoint: Marked as a change because one of 9/28/2022 11:19:50 AM, 10/6/2022 12:41:43 AM falls between 9/27/2022 9:53:04 AM and 10/6/2022 12:41:43 AM.
IsImprovementStdDev: Marked as improvement because 22.834094429543715 (T) = (0 -38654549.1814803) / Math.Sqrt((55998588858362.07 / (299)) + (23383031030680.18 / (34))) is greater than 1.9671567996108528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (34) - 2, .975) and 0.35590954904507766 = (60014162.79991644 - 38654549.1814803) / 60014162.79991644 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
Ampere arm64 CoreClr Improvement refs/heads/main RunKind=micro untriaged Windows 10.0.25094
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@DrewScoggins