Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Perf] Windows/x64: 1 Regression on 1/18/2023 1:01:18 AM #12006

Closed
performanceautofiler bot opened this issue Jan 24, 2023 · 2 comments
Closed

[Perf] Windows/x64: 1 Regression on 1/18/2023 1:01:18 AM #12006

performanceautofiler bot opened this issue Jan 24, 2023 · 2 comments

Comments

@performanceautofiler
Copy link

performanceautofiler bot commented Jan 24, 2023

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline 2e77a42d4563246f2362a498ca5906fdbd9274db
Compare 19fc478d67c2fc55e62b50676935f903a77f5a4d
Diff Diff

Regressions in System.Text.Perf_Utf8Encoding

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GetBytes - Duration of single invocation 127.23 μs 144.95 μs 1.14 0.02 False 1436061.9469026548 1488636.3636363635 1.0366101315107632 Trace Trace

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Perf_Utf8Encoding*'

Payloads

Baseline
Compare

Histogram

System.Text.Perf_Utf8Encoding.GetBytes(Input: Chinese)


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 144.952449845679 > 133.54047838618894.
IsChangePoint: Marked as a change because one of 12/21/2022 9:23:17 AM, 1/17/2023 8:36:05 PM, 1/24/2023 6:33:57 AM falls between 1/15/2023 4:08:29 PM and 1/24/2023 6:33:57 AM.
IsRegressionStdDev: Marked as regression because -35.687283578257905 (T) = (0 -147617.61720326805) / Math.Sqrt((5647909.848760539 / (30)) + (2989695.2155531375 / (38))) is less than -1.9965644189515832 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (38) - 2, .025) and -0.14273389516406024 = (129179.34597719695 - 147617.61720326805) / 129179.34597719695 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```### Baseline Jit Disasm

```assembly
; System.Text.Perf_Utf8Encoding.GetBytes()
       mov       rdx,rcx
       mov       rcx,[rdx+28]
       mov       rdx,[rdx+18]
       mov       rax,[rcx]
       mov       rax,[rax+58]
       jmp       qword ptr [rax+10]
; Total bytes of code 22

Compare Jit Disasm

; System.Text.Perf_Utf8Encoding.GetBytes()
       mov       rdx,rcx
       mov       rcx,[rdx+28]
       mov       rdx,[rdx+18]
       mov       rax,[rcx]
       mov       rax,[rax+58]
       jmp       qword ptr [rax+10]
; Total bytes of code 22

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@EgorBo
Copy link
Member

EgorBo commented Jan 24, 2023

ping @adamsitnik it seems that the recent regression is from dotnet/runtime#80709 but the previous improvement is dotnet/runtime#75012

@EgorBo EgorBo closed this as completed Jan 24, 2023
@EgorBo EgorBo removed the untriaged label Jan 24, 2023
@adamsitnik
Copy link
Member

@EgorBo in such case my only guess is code alignment (in dotnet/runtime#80709 I've added a new layer of abstraction which introduced regressions for small inputs and in dotnet/runtime#75012 I've removed it which solved these regressions)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants