-
Notifications
You must be signed in to change notification settings - Fork 4.7k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Perf] Linux/x64: 27 Regressions on 8/17/2022 6:09:21 PM #74395
Comments
Run Information
Regressions in System.Text.Json.Tests.Perf_Segment
Reprogit clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Json.Tests.Perf_Segment*' PayloadsHistogramSystem.Text.Json.Tests.Perf_Segment.ReadSingleSegmentSequenceByN(numberOfBytes: 8192, TestCase: Json40KB)
Description of detection logic
Description of detection logic
Description of detection logic
DocsProfiling workflow for dotnet/runtime repository
Regressions in System.Memory.Span<Byte>
Reprogit clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Memory.Span<Byte>*' PayloadsHistogramSystem.Memory.Span<Byte>.IndexOfValue(Size: 512)
Description of detection logic
Description of detection logic
Description of detection logic
DocsProfiling workflow for dotnet/runtime repository Run Information
Regressions in System.IO.Tests.Perf_Path
Reprogit clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.IO.Tests.Perf_Path*' PayloadsHistogramSystem.IO.Tests.Perf_Path.GetFileNameWithoutExtension
Description of detection logic
DocsProfiling workflow for dotnet/runtime repository Run Information
Regressions in System.Collections.ContainsTrue<Int32>
Reprogit clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.ContainsTrue<Int32>*' PayloadsHistogramSystem.Collections.ContainsTrue<Int32>.ImmutableArray(Size: 512)
Description of detection logic
DocsProfiling workflow for dotnet/runtime repository |
Something in this date range: 543bcc5...a250bfe |
I couldn't figure out the best area label to add to this issue. If you have write-permissions please help me learn by adding exactly one area label. |
Tagging subscribers to 'arch-wasm': @lewing Issue DetailsThere are some large wasm regressions here; as much as a 10 x slow down. Unfortunately, there seems to be some kind of glitch in the reporting, and the narrowest I can get the change range is this: 543bcc5...a250bfe. @lewing @radical @pavelsavara Any likely candidates in that date range? Automated issue reporting below. Run Information
Regressions in System.Text.RegularExpressions.Tests.Perf_Regex_Common
Reprogit clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Common*' PayloadsHistogramSystem.Text.RegularExpressions.Tests.Perf_Regex_Common.Uri_IsNotMatch(Options: None)
Description of detection logic
Description of detection logic
Description of detection logic
Description of detection logic
Description of detection logic
Description of detection logic
DocsProfiling workflow for dotnet/runtime repository
Regressions in System.Text.Json.Tests.Perf_Reader
Reprogit clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Json.Tests.Perf_Reader*' PayloadsHistogramSystem.Text.Json.Tests.Perf_Reader.ReadSpanEmptyLoop(IsDataCompact: True, TestCase: Json40KB)
Description of detection logic
Description of detection logic
Description of detection logic
Description of detection logic
Description of detection logic
DocsProfiling workflow for dotnet/runtime repository Run Information
Regressions in System.Tests.Perf_String
Reprogit clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_String*' PayloadsHistogramSystem.Tests.Perf_String.Format_MultipleArgs
Description of detection logic
DocsProfiling workflow for dotnet/runtime repository Run Information
Regressions in System.Tests.Perf_Environment
Reprogit clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_Environment*' PayloadsHistogramSystem.Tests.Perf_Environment.ExpandEnvironmentVariables
Description of detection logic
Description of detection logic
DocsProfiling workflow for dotnet/runtime repository Run Information
Regressions in System.Memory.ReadOnlySpan
Reprogit clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Memory.ReadOnlySpan*' PayloadsHistogramSystem.Memory.ReadOnlySpan.IndexOfString(input: "StrIng", value: "string", comparisonType: OrdinalIgnoreCase)
Description of detection logic
DocsProfiling workflow for dotnet/runtime repository Run Information
Regressions in System.Collections.ContainsFalse<Int32>
Reprogit clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.ContainsFalse<Int32>*'
### Histogram
System.Collections.ContainsFalse<Int32>.ICollection(Size: 512)
Description of detection logic
Description of detection logic
Description of detection logic
DocsProfiling workflow for dotnet/runtime repository Run Information
Regressions in Microsoft.Extensions.Configuration.Xml.XmlConfigurationProviderBenchmarks
Reprogit clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'Microsoft.Extensions.Configuration.Xml.XmlConfigurationProviderBenchmarks*' PayloadsHistogramMicrosoft.Extensions.Configuration.Xml.XmlConfigurationProviderBenchmarks.Load(FileName: "repeated.xml")
Description of detection logic
DocsProfiling workflow for dotnet/runtime repository
|
The span indexof regressions are pretty brutal |
@adamsitnik we think this might be related to the Vector128 changes |
Also regressed the mono interpreter pretty badly dotnet/perf-autofiling-issues#7976 |
we can see the impact of #73768 very clearly in other aot json tests but the span here is large in perf pipeline and the other vectorization might have caused issues as well. |
The usage of generic structs like:
probably causes wasm to fall back to the interpreter for these methods, since the AOT compiler cannot determine the instantiations that will be used at runtime. Imho such code should be avoided in low level BCL code, its better to have some code duplication. |
This isn't really practical, IMO, for us or our customers who are going to do the same. We have many features that are built on and around generics and generic specialization. Much like with Many of these algorithms are very long and very complex. It's not just a small amount of duplication, but large amounts of duplication in some of our most critical code (where having one place to review/update is often core/critical due to its complexity). |
Notably, in the case of WASM/AOT could recognize the pattern of |
in the mean time string handling is now slower than net6 in many cases and it happened post the rc1 branch |
Notably in WASM. Most other targets, including Arm64, have gotten significantly faster (often 2x or more). The “best” fix for .NET 7 given that and the short timeframe is likely to ifdef for WASM or similar. I don’t think reverting it is the right choice given the gains we see on almost all our other targets. For .NET 8 I think this really needs a more in depth look from the WASM side on how generics can be better supported. That could be one of the suggestions above or something as “simple” as some attribute that tells WASM/AOT what to specialize for in an opt-in manner. I don’t believe duplicating code or maintaining the status quo is viable, especially given the prevalence of genetics in .NET and all the newer features and existing perf patterns that encourage users to write generic and generic specialized algorithms |
Historically when doing this kind of vectorization work we have assumed it's reasonable to measure only on CoreCLR x64 and Arm64 and I assume that's what @adamsitnik did. Maybe it's fine to rely on the perf lab to catch anything Mono specific - it's just that timeframes are tighter at the moment. If #if is feasible that seems reasonable as me in the short term? |
@vargaz Why is the AOT compiler not able to determine the instantiations that will be used at runtime in this case? The instantiation chain for these generic helper methods always uses a small set of concrete types. There is no ambiguity about what this will be instantiated with. |
So what happens is that the AOT compiler creates a shared instantiation for bytes and enums whose base type is bytes, and that ends up being called, and then it cannot inline the call to DontNegate.NegateIfNeeded () into that. This appears fixable, but not in the net7 timeframe. |
I think both if possible. |
We should start with a fix in main and then backport it to rc2. |
resolved for net7 in #75996 |
There are some large wasm regressions here; as much as a 10 x slow down. Unfortunately, there seems to be some kind of glitch in the reporting, and the narrowest I can get the change range is this: 543bcc5...a250bfe.
@lewing @radical @pavelsavara Any likely candidates in that date range?
Automated issue reporting below.
Run Information
Regressions in System.Text.RegularExpressions.Tests.Perf_Regex_Common
Test Report
Repro
Payloads
Baseline
Compare
Histogram
System.Text.RegularExpressions.Tests.Perf_Regex_Common.Uri_IsNotMatch(Options: None)
Description of detection logic
Description of detection logic
Description of detection logic
Description of detection logic
Description of detection logic
Description of detection logic
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Regressions in System.Text.Json.Tests.Perf_Reader
Test Report
Repro
Payloads
Baseline
Compare
Histogram
System.Text.Json.Tests.Perf_Reader.ReadSpanEmptyLoop(IsDataCompact: True, TestCase: Json40KB)
Description of detection logic
Description of detection logic
Description of detection logic
Description of detection logic
Description of detection logic
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Regressions in System.Tests.Perf_String
Test Report
Repro
Payloads
Baseline
Compare
Histogram
System.Tests.Perf_String.Format_MultipleArgs
Description of detection logic
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Regressions in System.Tests.Perf_Environment
Test Report
Repro
Payloads
Baseline
Compare
Histogram
System.Tests.Perf_Environment.ExpandEnvironmentVariables
Description of detection logic
Description of detection logic
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Regressions in System.Memory.ReadOnlySpan
Test Report
Repro
Payloads
Baseline
Compare
Histogram
System.Memory.ReadOnlySpan.IndexOfString(input: "StrIng", value: "string", comparisonType: OrdinalIgnoreCase)
Description of detection logic
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Regressions in System.Collections.ContainsFalse<Int32>
Test Report
Repro
System.Collections.ContainsFalse<Int32>.ICollection(Size: 512)
Description of detection logic
Description of detection logic
Description of detection logic
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Regressions in Microsoft.Extensions.Configuration.Xml.XmlConfigurationProviderBenchmarks
Test Report
Repro
Payloads
Baseline
Compare
Histogram
Microsoft.Extensions.Configuration.Xml.XmlConfigurationProviderBenchmarks.Load(FileName: "repeated.xml")
Description of detection logic
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
The text was updated successfully, but these errors were encountered: