[Perf] Changes at 12/7/2021 3:27:58 PM #2750
Description
Run Information
| Architecture | arm64 |
|---|---|
| OS | ubuntu 18.04 |
| Baseline | b72b7f03e7de338106f380b5468e2d7a1e952fb9 |
| Compare | 8369363c12ab747cb1c32523bcfbbfe5db8cd8b5 |
| Diff | Diff |
Improvements in System.Text.Json.Document.Tests.Perf_EnumerateObject
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| Parse - Duration of single invocation | 48.52 μs | 41.08 μs | 0.85 | 0.22 | False |
Repro
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Json.Document.Tests.Perf_EnumerateObject*'Payloads
Histogram
System.Text.Json.Document.Tests.Perf_EnumerateObject.Parse(TestCase: StringProperties)
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 41.08188478844705 < 46.13202134277488.
IsChangePoint: Marked as a change because one of 12/7/2021 2:09:29 PM, 1/5/2022 6:46:20 PM falls between 12/6/2021 2:46:54 PM and 1/5/2022 6:46:20 PM.
IsImprovementStdDev: Marked as improvement because 24.629329706258197 (T) = (0 -41620.81476552025) / Math.Sqrt((3263342.95993928 / (219)) + (1006144.1653241315 / (99))) is greater than 1.9674995188240387 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (219) + (99) - 2, .975) and 0.08565971688136824 = (45520.049301076375 - 41620.81476552025) / 45520.049301076375 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
| Architecture | arm64 |
|---|---|
| OS | ubuntu 18.04 |
| Baseline | b72b7f03e7de338106f380b5468e2d7a1e952fb9 |
| Compare | 8369363c12ab747cb1c32523bcfbbfe5db8cd8b5 |
| Diff | Diff |
Improvements in System.Text.Json.Tests.Perf_Strings
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| WriteStringsUtf8 - Duration of single invocation | 18.50 ms | 16.86 ms | 0.91 | 0.05 | True |
Repro
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Json.Tests.Perf_Strings*'Payloads
Histogram
System.Text.Json.Tests.Perf_Strings.WriteStringsUtf8(Formatted: False, SkipValidation: True, Escaped: OneEscaped)
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 16.861256558974365 < 17.80806674813874.
IsChangePoint: Marked as a change because one of 12/7/2021 2:09:29 PM, 1/5/2022 6:46:20 PM falls between 12/6/2021 2:46:54 PM and 1/5/2022 6:46:20 PM.
IsImprovementStdDev: Marked as improvement because 54.35938846498028 (T) = (0 -17114746.908161268) / Math.Sqrt((151137557984.48004 / (231)) + (17921774828.829033 / (99))) is greater than 1.9672228266464409 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (231) + (99) - 2, .975) and 0.08407840155374938 = (18685820.857586883 - 17114746.908161268) / 18685820.857586883 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository