Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[perf] System.Runtime.Intrinsics.** performance regressions in Mono AOT x64 microbenchmarks from Preview 5 to Preview 7 #89245

Closed
kotlarmilos opened this issue Jul 20, 2023 · 5 comments
Labels
Milestone

Comments

@kotlarmilos
Copy link
Member

kotlarmilos commented Jul 20, 2023

Description

The following performance regressions occurred between the Preview 5 snapshot (May 2nd, 2023) and Preview 7 snapshot (July 19th, 2023).

The configuration used for the benchmarks is Mono AOT with LLVM on x64 ubuntu 18.04.

Name Baseline Value Compare Value % Difference
System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt64>.DivideBenchmark 18.89 20.22 7.04
System.Runtime.Intrinsics.Tests.Perf_Vector128Of<SByte>.GetHashCodeBenchmark 98.32 105.83 7.64
@kotlarmilos
Copy link
Member Author

@vargaz @fanyang-mono There are a number of Vector128 improvements in #89238 and I assume we can close this issue.

@fanyang-mono
Copy link
Member

fanyang-mono commented Jul 20, 2023

I don't understand your last comment. I searched in #89238. And these two microbenchmarks also showed as regressions. How that could help close this issue?

@kotlarmilos
Copy link
Member Author

kotlarmilos commented Jul 21, 2023

There are a lot of Vector128 improvements and a few regressions. Sometimes, the regressions are considered as a tradeoff where general cases improved but edge cases regressed. Not sure if it is the case here, but if not, we should mitigate it.

@fanyang-mono
Copy link
Member

There are two things here:

  1. These Vector128 microbenchmarks are quite independent from each other. Even though most of them got improved, the regressed ones still worth looking into.
  2. Like I mentioned in [perf] System.Numerics.** performance regressions in Mono AOT x64 microbenchmarks from Preview 5 to Preview 7 #89246, we should collect results with average of a couple data points. These looks like false positive to me. The performance of these two microbenchmarks didn't change.

@kotlarmilos
Copy link
Member Author

Thanks for the info. Here are the last 10 runs with the baseline before the regression, and the targets after the recovery.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of.DivideBenchmark

Run Baseline Target
1 18,78173253 18,79884798
2 18,76230461 18,62760552
3 18,76230461 18,69527468
4 18,79260984 19,9226462
5 19,01008983 18,65647022
6 19,12448913 18,83510237
7 18,85043033 20,09960835
8 18,7581046 20,14199486
9 18,73615193 20,15739381
10 18,74319966 20,15808063
avg 18,83214171 19,40930246

System.Runtime.Intrinsics.Tests.Perf_Vector128Of.GetHashCodeBenchmark

Run Baseline Target
1 97,80791568 105,8302407
2 97,87356477 103,1638385
3 98,30399123 104,7245511
4 98,89306389 104,3972979
5 98,09571083 96,46227839
6 97,78725048 104,6662993
7 97,79220034 103,1119935
8 102,149487 104,1949933
9 102,0973694 104,143184
10 102,3439054 104,327521
avg 99,3144459 103,5022198

In both cases it looks like noise, thus closing this issues.

@ghost ghost locked as resolved and limited conversation to collaborators Aug 20, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Projects
None yet
Development

No branches or pull requests

2 participants