-
Notifications
You must be signed in to change notification settings - Fork 4.7k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Perf] Windows/x64: 13 Regressions on 7/1/2023 10:14:52 PM #88376
Comments
The diff is always 7180d7b...0d77cf0. The |
More instances: |
I'll take a look at the others. |
Report of diffs:
|
…nversion In dotnet#88376 (comment) I noticed a spurious diff in the System.Tests.Perf_Int32.ToStringHex benchmark that looked like this: ```diff G_M62987_IG13: and edi, esi mov ebx, dword ptr [rbp+88H] - mov ecx, 1 test ebx, ebx - cmovle ebx, ecx + jg SHORT G_M62987_IG15 + ;; size=12 bbWeight=1.00 PerfScore 2.50 +G_M62987_IG14: + mov ebx, 1 + ;; size=5 bbWeight=0.98 PerfScore 0.24 +G_M62987_IG15: ``` Investigating, it turns out to be caused by a check in if-conversion for detecting loops, that is using a very sharp boundary. It turns out that the integral block weights are exactly the same (in both the base and diff), but the floating point calculation ended up with exactly 1 and very close to 1 in the base/diff cases respectively. So two changes to address it: * Switch getBBWeight to multiply by BB_UNITY_WEIGHT after dividing * Switch if-conversion to have a 5% tolerance so that losing just a single count in the wrong place will not cause a decision change. Note that the check here is used as a cheap "is inside loop" check.
…nversion (#88385) In #88376 (comment) I noticed a spurious diff in the System.Tests.Perf_Int32.ToStringHex benchmark that looked like this: ```diff G_M62987_IG13: and edi, esi mov ebx, dword ptr [rbp+88H] - mov ecx, 1 test ebx, ebx - cmovle ebx, ecx + jg SHORT G_M62987_IG15 + ;; size=12 bbWeight=1.00 PerfScore 2.50 +G_M62987_IG14: + mov ebx, 1 + ;; size=5 bbWeight=0.98 PerfScore 0.24 +G_M62987_IG15: ``` Investigating, it turns out to be caused by a check in if-conversion for detecting loops, that is using a very sharp boundary. It turns out that the integral block weights are exactly the same (in both the base and diff), but the floating point calculation ended up with exactly 1 and very close to 1 in the base/diff cases respectively. So two changes to address it: * Switch getBBWeight to multiply by BB_UNITY_WEIGHT after dividing * Switch if-conversion to have a 5% tolerance so that losing just a single count in the wrong place will not cause a decision change. Note that the check here is used as a cheap "is inside loop" check.
I merged a fix for EmptyTakeSelectToArray |
Tagging subscribers to this area: @dotnet/area-system-linq Issue DetailsRun Information
Regressions in System.Linq.Tests.Perf_Enumerable
ReproGeneral Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Linq.Tests.Perf_Enumerable*' PayloadsSystem.Linq.Tests.Perf_Enumerable.EmptyTakeSelectToArrayHistogram
Description of detection logic
JIT DisasmsSystem.Linq.Tests.Perf_Enumerable.Contains_ElementNotFound(input: ICollection)Histogram
Description of detection logic
JIT DisasmsDocsProfiling workflow for dotnet/runtime repository Run Information
Regressions in System.Tests.Perf_Int32
ReproGeneral Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Int32*' PayloadsSystem.Tests.Perf_Int32.ToStringHex(value: -2147483648)Histogram
Description of detection logic
JIT DisasmsSystem.Tests.Perf_Int32.ToStringHex(value: 2147483647)Histogram
Description of detection logic
JIT DisasmsDocsProfiling workflow for dotnet/runtime repository Run Information
Regressions in System.Text.Perf_Ascii
ReproGeneral Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Perf_Ascii*' PayloadsSystem.Text.Perf_Ascii.ToUpperInPlace_Bytes(Size: 128)Histogram
Description of detection logic
JIT DisasmsSystem.Text.Perf_Ascii.ToLowerInPlace_Chars(Size: 128)Histogram
Description of detection logic
JIT DisasmsSystem.Text.Perf_Ascii.ToUpperInPlace_Chars(Size: 128)Histogram
Description of detection logic
JIT DisasmsSystem.Text.Perf_Ascii.ToLowerInPlace_Bytes(Size: 128)Histogram
Description of detection logic
JIT DisasmsSystem.Text.Perf_Ascii.Equals_Bytes_Chars(Size: 128)Histogram
Description of detection logic
JIT DisasmsDocsProfiling workflow for dotnet/runtime repository Run Information
Regressions in System.Globalization.Tests.StringSearch
ReproGeneral Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Globalization.Tests.StringSearch*' PayloadsSystem.Globalization.Tests.StringSearch.IsPrefix_FirstHalf(Options: (en-US, OrdinalIgnoreCase, False))Histogram
Description of detection logic
JIT DisasmsSystem.Globalization.Tests.StringSearch.IsPrefix_DifferentFirstChar(Options: (en-US, OrdinalIgnoreCase, False))Histogram
Description of detection logic
JIT DisasmsDocsProfiling workflow for dotnet/runtime repository Run Information
Regressions in System.Tests.Perf_Enum
ReproGeneral Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Enum*' PayloadsSystem.Tests.Perf_Enum.GetName_Generic_FlagsHistogram
Description of detection logic
JIT DisasmsDocsProfiling workflow for dotnet/runtime repository Run Information
Regressions in Benchstone.BenchI.TreeInsert
ReproGeneral Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Benchstone.BenchI.TreeInsert*' PayloadsBenchstone.BenchI.TreeInsert.TestHistogram
Description of detection logic
JIT DisasmsDocsProfiling workflow for dotnet/runtime repository
|
The |
Run Information
Regressions in System.Linq.Tests.Perf_Enumerable
Test Report
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
Baseline
Compare
System.Linq.Tests.Perf_Enumerable.EmptyTakeSelectToArray
Histogram
Description of detection logic
JIT Disasms
System.Linq.Tests.Perf_Enumerable.Contains_ElementNotFound(input: ICollection)
Histogram
Description of detection logic
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Regressions in System.Tests.Perf_Int32
Test Report
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
Baseline
Compare
System.Tests.Perf_Int32.ToStringHex(value: -2147483648)
Histogram
Description of detection logic
JIT Disasms
System.Tests.Perf_Int32.ToStringHex(value: 2147483647)
Histogram
Description of detection logic
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Regressions in System.Text.Perf_Ascii
Test Report
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
Baseline
Compare
System.Text.Perf_Ascii.ToUpperInPlace_Bytes(Size: 128)
Histogram
Description of detection logic
JIT Disasms
System.Text.Perf_Ascii.ToLowerInPlace_Chars(Size: 128)
Histogram
Description of detection logic
JIT Disasms
System.Text.Perf_Ascii.ToUpperInPlace_Chars(Size: 128)
Histogram
Description of detection logic
JIT Disasms
System.Text.Perf_Ascii.ToLowerInPlace_Bytes(Size: 128)
Histogram
Description of detection logic
JIT Disasms
System.Text.Perf_Ascii.Equals_Bytes_Chars(Size: 128)
Histogram
Description of detection logic
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Regressions in System.Globalization.Tests.StringSearch
Test Report
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
Baseline
Compare
System.Globalization.Tests.StringSearch.IsPrefix_FirstHalf(Options: (en-US, OrdinalIgnoreCase, False))
Histogram
Description of detection logic
JIT Disasms
System.Globalization.Tests.StringSearch.IsPrefix_DifferentFirstChar(Options: (en-US, OrdinalIgnoreCase, False))
Histogram
Description of detection logic
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Regressions in System.Tests.Perf_Enum
Test Report
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
Baseline
Compare
System.Tests.Perf_Enum.GetName_Generic_Flags
Histogram
Description of detection logic
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Regressions in Benchstone.BenchI.TreeInsert
Test Report
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
Baseline
Compare
Benchstone.BenchI.TreeInsert.Test
Histogram
Description of detection logic
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
The text was updated successfully, but these errors were encountered: