Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Perf] Windows/x86: 4 Regressions on 11/24/2022 9:42:44 PM #10178

Closed
performanceautofiler bot opened this issue Nov 29, 2022 · 1 comment
Closed

[Perf] Windows/x86: 4 Regressions on 11/24/2022 9:42:44 PM #10178

performanceautofiler bot opened this issue Nov 29, 2022 · 1 comment
Assignees
@DrewScoggins
Labels
arch-x86 branch-refs/heads/main kind-micro os-windows perf-regression runtime-coreclr

Comments

@performanceautofiler
Copy link

performanceautofiler bot commented Nov 29, 2022 •
edited
Loading

Run Information

Architecture x86
OS Windows 10.0.18362
Baseline ac03fbd184b182a6632a50bbe70bc733e487264c
Compare 4832236b4820ca83bbc9ce96003c2029b85c6ce2
Diff Diff

Regressions in System.Tests.Perf_String

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ToUpperInvariant - Duration of single invocation 20.18 ns 22.89 ns 1.13 0.06 False Trace Trace
ToLowerInvariant - Duration of single invocation 20.27 ns 22.41 ns 1.11 0.07 False 234.50036882700073 251.33454622005507 1.0717874239484608
ToUpperInvariant - Duration of single invocation 20.19 ns 22.31 ns 1.11 0.03 False
ToLowerInvariant - Duration of single invocation 20.67 ns 22.39 ns 1.08 0.07 False 235.47489186033386 249.8169913257949 1.0609071283658036

graph
graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_String*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_String.ToUpperInvariant(s: "test")

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 22.88875740823189 > 21.213542069114958.
IsChangePoint: Marked as a change because one of 11/24/2022 8:21:24 PM, 11/29/2022 1:43:29 AM falls between 11/20/2022 12:28:28 PM and 11/29/2022 1:43:29 AM.
IsRegressionStdDev: Marked as regression because -12.540060711085625 (T) = (0 -22.758555955882965) / Math.Sqrt((0.2617366876105701 / (40)) + (0.45469232261831233 / (15))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (40) + (15) - 2, .025) and -0.11829513318528763 = (20.35111776893708 - 22.758555955882965) / 20.35111776893708 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```### Baseline Jit Disasm

```assembly
; System.Tests.Perf_String.ToUpperInvariant(System.String)
       cmp       [edx],dl
       mov       ecx,ds:[75C1388]
       cmp       [ecx],ecx
       call      dword ptr ds:[8FCBA68]; System.Globalization.TextInfo.ToUpper(System.String)
       ret
; Total bytes of code 17

; System.Globalization.TextInfo.ToUpper(System.String)
       push      ebp
       mov       ebp,esp
       test      edx,edx
       je        short M01_L00
       call      dword ptr ds:[0B92F1E0]; System.Globalization.TextInfo.ChangeCaseCommon[[System.Globalization.TextInfo+ToUpperConversion, System.Private.CoreLib]](System.String)
       pop       ebp
       ret
M01_L00:
       mov       ecx,15FB
       mov       edx,32A4000
       call      CORINFO_HELP_STRCNS
       mov       ecx,eax
       call      dword ptr ds:[8E673F0]
       int       3
; Total bytes of code 39

Compare Jit Disasm

; System.Tests.Perf_String.ToUpperInvariant(System.String)
       cmp       [edx],dl
       mov       ecx,ds:[706138C]
       cmp       [ecx],ecx
       call      dword ptr ds:[8B9BA98]; System.Globalization.TextInfo.ToUpper(System.String)
       ret
; Total bytes of code 17
; System.Globalization.TextInfo.ToUpper(System.String)
       push      ebp
       mov       ebp,esp
       test      edx,edx
       je        short M01_L00
       call      dword ptr ds:[0B44F1E0]; System.Globalization.TextInfo.ChangeCaseCommon[[System.Globalization.TextInfo+ToUpperConversion, System.Private.CoreLib]](System.String)
       pop       ebp
       ret
M01_L00:
       mov       ecx,15FB
       mov       edx,30A4000
       call      CORINFO_HELP_STRCNS
       mov       ecx,eax
       call      dword ptr ds:[8A47408]
       int       3
; Total bytes of code 39

System.Tests.Perf_String.ToLowerInvariant(s: "TeSt")

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 22.412301811691982 > 21.29055663319969.
IsChangePoint: Marked as a change because one of 11/24/2022 8:21:24 PM, 11/29/2022 1:43:29 AM falls between 11/20/2022 12:28:28 PM and 11/29/2022 1:43:29 AM.
IsRegressionStdDev: Marked as regression because -10.839011153381593 (T) = (0 -22.74661729136309) / Math.Sqrt((0.6719884150436979 / (40)) + (0.3544258668701032 / (15))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (40) + (15) - 2, .025) and -0.10596323438118446 = (20.56724544201546 - 22.74661729136309) / 20.56724544201546 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```### Baseline Jit Disasm

```assembly
; System.Tests.Perf_String.ToLowerInvariant(System.String)
       cmp       [edx],dl
       mov       ecx,ds:[7621388]
       cmp       [ecx],ecx
       call      dword ptr ds:[904B960]; System.Globalization.TextInfo.ToLower(System.String)
       ret
; Total bytes of code 17

; System.Globalization.TextInfo.ToLower(System.String)
       push      ebp
       mov       ebp,esp
       test      edx,edx
       je        short M01_L00
       call      dword ptr ds:[0B9633C0]; System.Globalization.TextInfo.ChangeCaseCommon[[System.Globalization.TextInfo+ToLowerConversion, System.Private.CoreLib]](System.String)
       pop       ebp
       ret
M01_L00:
       mov       ecx,15FB
       mov       edx,3984000
       call      CORINFO_HELP_STRCNS
       mov       ecx,eax
       call      dword ptr ds:[8EE73F0]
       int       3
; Total bytes of code 39

Compare Jit Disasm

; System.Tests.Perf_String.ToLowerInvariant(System.String)
       cmp       [edx],dl
       mov       ecx,ds:[72F138C]
       cmp       [ecx],ecx
       call      dword ptr ds:[8C9B960]; System.Globalization.TextInfo.ToLower(System.String)
       ret
; Total bytes of code 17
; System.Globalization.TextInfo.ToLower(System.String)
       push      ebp
       mov       ebp,esp
       test      edx,edx
       je        short M01_L00
       call      dword ptr ds:[0B5C32A0]; System.Globalization.TextInfo.ChangeCaseCommon[[System.Globalization.TextInfo+ToLowerConversion, System.Private.CoreLib]](System.String)
       pop       ebp
       ret
M01_L00:
       mov       ecx,15FB
       mov       edx,2F44000
       call      CORINFO_HELP_STRCNS
       mov       ecx,eax
       call      dword ptr ds:[8B47408]
       int       3
; Total bytes of code 39

System.Tests.Perf_String.ToUpperInvariant(s: "TeSt")

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 22.307563959955505 > 21.191137149939482.
IsChangePoint: Marked as a change because one of 11/24/2022 8:21:24 PM, 11/29/2022 1:43:29 AM falls between 11/20/2022 12:28:28 PM and 11/29/2022 1:43:29 AM.
IsRegressionStdDev: Marked as regression because -19.939166764474805 (T) = (0 -22.472871857454024) / Math.Sqrt((0.052746839463876226 / (40)) + (0.16172320981211596 / (15))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (40) + (15) - 2, .025) and -0.10815470797653229 = (20.27954372768855 - 22.472871857454024) / 20.27954372768855 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```### Baseline Jit Disasm

```assembly
; System.Tests.Perf_String.ToUpperInvariant(System.String)
       cmp       [edx],dl
       mov       ecx,ds:[68E1388]
       cmp       [ecx],ecx
       call      dword ptr ds:[83BBA68]; System.Globalization.TextInfo.ToUpper(System.String)
       ret
; Total bytes of code 17

; System.Globalization.TextInfo.ToUpper(System.String)
       push      ebp
       mov       ebp,esp
       test      edx,edx
       je        short M01_L00
       call      dword ptr ds:[0AD1F1E0]; System.Globalization.TextInfo.ChangeCaseCommon[[System.Globalization.TextInfo+ToUpperConversion, System.Private.CoreLib]](System.String)
       pop       ebp
       ret
M01_L00:
       mov       ecx,15FB
       mov       edx,674000
       call      CORINFO_HELP_STRCNS
       mov       ecx,eax
       call      dword ptr ds:[82573F0]
       int       3
; Total bytes of code 39

Compare Jit Disasm

; System.Tests.Perf_String.ToUpperInvariant(System.String)
       cmp       [edx],dl
       mov       ecx,ds:[710138C]
       cmp       [ecx],ecx
       call      dword ptr ds:[8B7BA98]; System.Globalization.TextInfo.ToUpper(System.String)
       ret
; Total bytes of code 17
; System.Globalization.TextInfo.ToUpper(System.String)
       push      ebp
       mov       ebp,esp
       test      edx,edx
       je        short M01_L00
       call      dword ptr ds:[0B42F1E0]; System.Globalization.TextInfo.ChangeCaseCommon[[System.Globalization.TextInfo+ToUpperConversion, System.Private.CoreLib]](System.String)
       pop       ebp
       ret
M01_L00:
       mov       ecx,15FB
       mov       edx,2E64000
       call      CORINFO_HELP_STRCNS
       mov       ecx,eax
       call      dword ptr ds:[8A27408]
       int       3
; Total bytes of code 39

System.Tests.Perf_String.ToLowerInvariant(s: "TEST")

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 22.3944121502011 > 21.263453484685247.
IsChangePoint: Marked as a change because one of 11/24/2022 8:21:24 PM, 11/29/2022 1:43:29 AM falls between 11/20/2022 12:28:28 PM and 11/29/2022 1:43:29 AM.
IsRegressionStdDev: Marked as regression because -23.098305815554287 (T) = (0 -22.453831343781868) / Math.Sqrt((0.19178260901473113 / (40)) + (0.030123444831090702 / (15))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (40) + (15) - 2, .025) and -0.09271272814819023 = (20.548704856613284 - 22.453831343781868) / 20.548704856613284 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```### Baseline Jit Disasm

```assembly
; System.Tests.Perf_String.ToLowerInvariant(System.String)
       cmp       [edx],dl
       mov       ecx,ds:[70C1388]
       cmp       [ecx],ecx
       call      dword ptr ds:[8B6B960]; System.Globalization.TextInfo.ToLower(System.String)
       ret
; Total bytes of code 17

; System.Globalization.TextInfo.ToLower(System.String)
       push      ebp
       mov       ebp,esp
       test      edx,edx
       je        short M01_L00
       call      dword ptr ds:[0B483228]; System.Globalization.TextInfo.ChangeCaseCommon[[System.Globalization.TextInfo+ToLowerConversion, System.Private.CoreLib]](System.String)
       pop       ebp
       ret
M01_L00:
       mov       ecx,15FB
       mov       edx,2E54000
       call      CORINFO_HELP_STRCNS
       mov       ecx,eax
       call      dword ptr ds:[8A073F0]
       int       3
; Total bytes of code 39

Compare Jit Disasm

; System.Tests.Perf_String.ToLowerInvariant(System.String)
       cmp       [edx],dl
       mov       ecx,ds:[74E138C]
       cmp       [ecx],ecx
       call      dword ptr ds:[8F5B960]; System.Globalization.TextInfo.ToLower(System.String)
       ret
; Total bytes of code 17
; System.Globalization.TextInfo.ToLower(System.String)
       push      ebp
       mov       ebp,esp
       test      edx,edx
       je        short M01_L00
       call      dword ptr ds:[0B883108]; System.Globalization.TextInfo.ChangeCaseCommon[[System.Globalization.TextInfo+ToLowerConversion, System.Private.CoreLib]](System.String)
       pop       ebp
       ret
M01_L00:
       mov       ecx,15FB
       mov       edx,3314000
       call      CORINFO_HELP_STRCNS
       mov       ecx,eax
       call      dword ptr ds:[8E07408]
       int       3
; Total bytes of code 39

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@DrewScoggins DrewScoggins

Labels
arch-x86 branch-refs/heads/main kind-micro os-windows perf-regression runtime-coreclr
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@DrewScoggins @dakersnar