Skip to content

[Perf] Changes at 4/27/2022 1:24:17 PM #5094

@performanceautofiler

Description

@performanceautofiler

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 4ed596ef63e60ce54cfb41d55928f0fe45f65cf3
Compare 8f7761af3763d0b84acefbb117ed6becd83061d1
Diff Diff

Improvements in System.IO.Tests.Perf_Directory

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
MoveFiles - Duration of single invocation 5.70 μs 4.42 μs 0.77 0.10 True
MoveFolders - Duration of single invocation 6.14 μs 4.89 μs 0.80 0.14 True
CreateDirectory - Duration of single invocation 11.41 μs 8.24 μs 0.72 0.57 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.IO.Tests.Perf_Directory*'

Payloads

Baseline
Compare

Histogram

System.IO.Tests.Perf_Directory.MoveFiles


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 4.4185550903755875 < 5.467492187621759.
IsChangePoint: Marked as a change because one of 4/27/2022 8:46:50 AM, 5/3/2022 11:42:39 AM falls between 4/24/2022 10:52:31 PM and 5/3/2022 11:42:39 AM.
IsImprovementStdDev: Marked as improvement because 117.01873939237643 (T) = (0 -4417.738489937121) / Math.Sqrt((25438.311840417526 / (299)) + (1862.0187430100975 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.24450843993799198 = (5847.502107865413 - 4417.738489937121) / 5847.502107865413 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

```#### System.IO.Tests.Perf_Directory.MoveFolders

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 4.890694739393867 < 5.953107009629486.
IsChangePoint: Marked as a change because one of 4/27/2022 8:46:50 AM, 5/3/2022 11:42:39 AM falls between 4/24/2022 10:52:31 PM and 5/3/2022 11:42:39 AM.
IsImprovementStdDev: Marked as improvement because 86.45210562123398 (T) = (0 -4886.701382803054) / Math.Sqrt((37266.99475407966 / (299)) + (4385.781407227276 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.22710818331773497 = (6322.620161486289 - 4886.701382803054) / 6322.620161486289 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

```#### System.IO.Tests.Perf_Directory.CreateDirectory

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 8.242894444444442 < 10.207402403846153.
IsChangePoint: Marked as a change because one of 4/27/2022 8:46:50 AM, 5/3/2022 11:42:39 AM falls between 4/24/2022 10:52:31 PM and 5/3/2022 11:42:39 AM.
IsImprovementStdDev: Marked as improvement because 12.246814175630353 (T) = (0 -8650.472031227495) / Math.Sqrt((797076.3024468683 / (299)) + (839245.9759992415 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.2010792077788703 = (10827.69670717641 - 8650.472031227495) / 10827.69670717641 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS ubuntu 18.04
Baseline 4ed596ef63e60ce54cfb41d55928f0fe45f65cf3
Compare 8f7761af3763d0b84acefbb117ed6becd83061d1
Diff Diff

Improvements in System.IO.Tests.Perf_FileStream

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Flush - Duration of single invocation 1.23 ms 1.10 ms 0.89 0.04 True
Append - Duration of single invocation 538.64 μs 473.36 μs 0.88 0.03 True
CopyToFileAsync - Duration of single invocation 49.28 μs 45.53 μs 0.92 0.29 False
AppendAsync - Duration of single invocation 730.02 μs 687.91 μs 0.94 0.05 False
CopyToFile - Duration of single invocation 50.00 μs 42.24 μs 0.84 0.27 False
Flush - Duration of single invocation 1.27 ms 1.07 ms 0.84 0.06 True

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.IO.Tests.Perf_FileStream*'

Payloads

Baseline
Compare

Histogram

System.IO.Tests.Perf_FileStream.Flush(fileSize: 1024, options: Asynchronous)


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.0990216477777779 < 1.1973076193194827.
IsChangePoint: Marked as a change because one of 4/27/2022 8:46:50 AM, 5/3/2022 11:42:39 AM falls between 4/24/2022 10:52:31 PM and 5/3/2022 11:42:39 AM.
IsImprovementStdDev: Marked as improvement because 59.83160895000431 (T) = (0 -1075006.0894222439) / Math.Sqrt((697036027.2839366 / (299)) + (208224463.95673162 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.146502125168872 = (1259529.8958828025 - 1075006.0894222439) / 1259529.8958828025 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

```#### System.IO.Tests.Perf_FileStream.Append(fileSize: 1048576, userBufferSize: 4096, options: DeleteOnClose)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 473.36038027597397 < 510.70355554712296.
IsChangePoint: Marked as a change because one of 4/27/2022 8:46:50 AM, 5/3/2022 11:42:39 AM falls between 4/24/2022 10:52:31 PM and 5/3/2022 11:42:39 AM.
IsImprovementStdDev: Marked as improvement because 58.789893925475816 (T) = (0 -477465.5580478142) / Math.Sqrt((77474645.07914257 / (299)) + (25994097.834462553 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.11688421612541862 = (540660.2019420174 - 477465.5580478142) / 540660.2019420174 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

```#### System.IO.Tests.Perf_FileStream.CopyToFileAsync(fileSize: 1024, options: Asynchronous)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 45.53249890302359 < 46.74375025878481.
IsChangePoint: Marked as a change because one of 2/22/2022 11:58:49 AM, 4/27/2022 8:46:50 AM, 5/3/2022 11:42:39 AM falls between 4/24/2022 10:52:31 PM and 5/3/2022 11:42:39 AM.
IsImprovementStdDev: Marked as improvement because 15.836090995791952 (T) = (0 -45829.729166030884) / Math.Sqrt((3129490.9112991476 / (299)) + (2056848.8561467647 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.08973471072629302 = (50347.6620564078 - 45829.729166030884) / 50347.6620564078 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.Perf_FileStream.AppendAsync(fileSize: 1048576, userBufferSize: 4096, options: DeleteOnClose, Asynchronous)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 687.9061707932693 < 693.6354376693272.
IsChangePoint: Marked as a change because one of 4/27/2022 1:24:17 PM, 5/3/2022 11:42:39 AM falls between 4/24/2022 10:52:31 PM and 5/3/2022 11:42:39 AM.
IsImprovementStdDev: Marked as improvement because 29.691150553047652 (T) = (0 -678407.3841114759) / Math.Sqrt((196449261.6437951 / (299)) + (73014096.84728503 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.07328297071418956 = (732054.5135922468 - 678407.3841114759) / 732054.5135922468 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.Perf_FileStream.CopyToFile(fileSize: 1024, options: None)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 42.23988053857908 < 44.000917771832576.
IsChangePoint: Marked as a change because one of 4/27/2022 8:46:50 AM, 5/3/2022 11:42:39 AM falls between 4/24/2022 10:52:31 PM and 5/3/2022 11:42:39 AM.
IsImprovementStdDev: Marked as improvement because 22.939540412838845 (T) = (0 -42836.880922990036) / Math.Sqrt((2345126.401983269 / (299)) + (995387.7208753789 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.10035662820200734 = (47615.402131377785 - 42836.880922990036) / 47615.402131377785 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.Perf_FileStream.Flush(fileSize: 1024, options: None)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.073259187777778 < 1.191910836900359.
IsChangePoint: Marked as a change because one of 4/27/2022 8:46:50 AM, 5/3/2022 11:42:39 AM falls between 4/24/2022 10:52:31 PM and 5/3/2022 11:42:39 AM.
IsImprovementStdDev: Marked as improvement because 48.30122036664176 (T) = (0 -1070444.2084974018) / Math.Sqrt((719359077.4433931 / (299)) + (356896550.9793714 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.1475411320556531 = (1255713.617102387 - 1070444.2084974018) / 1255713.617102387 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions