-
Notifications
You must be signed in to change notification settings - Fork 632
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
.Net 6 and 8 slower than .Net 472 #929
Comments
I think it would be very helpful if you tried to setup a code sample or even better something running under https://benchmarkdotnet.org/articles/guides/getting-started.html. At the very least share relevant pieces of your code, the team will have no idea on how to begin to attempt to reproduce this as is, so more information is likely needed. |
This is very strange |
Thanks! |
Without knowing the details, perhaps https://github.com/bchavez/Bogus can be of some assistance to create test data. |
Here are benchmark results. I have also added a benchmark class. I had to rename the field names etc. but it's basically a simple search of 5-gram field. .Net 6 full is slightly better than 472, but empty test is much slower. Also, I observed that 6 and 8 results are all over the place, sometimes the same method repeated 3 times in a row gives 2x difference in execution time. This can also be seen from the result, Stdev is quite big for .Net 6 and 8 (especially for an empty test). This lead us to believe that there might be a problem in runtime or garbage collector, so I run the same benchmark using server GC, and now the results are great with 6 and even better with 8. So, out immediate problem seems to be solved, but you perhaps that's something to look for. As for test index, I can't share it publicly (and, besides, it's quite large, test case is 900MB), but perhaps there's a way to share it privately. Thanks, Tom Workstation GC
Server GC
|
Great, this gets us a starting point. But, we will need more to narrow this down more than target framework to have any useful place to start (after all, we are maintaining over 600,000 lines of code). One thing that would be helpful to know is whether you are benchmarking using the released components, and if not, which tag(s) or branch(es) you are compiling from source with. We have gone several hundred commits beyond the last release, so this may be relevant. But now that you have created some benchmarks, would you consider contributing them back to us by forking the branch for #310? It is difficult for us to tell which components in the codebase are extremely common and which are extremely rare. The best we came up with was to benchmark the code in the demos, but I suspect this is still a far cry from a common real world combination of components that we should be using to determine whether a change we make is for the better or worse in terms of performance. We still haven't seen any code, and that is the crucial bit. You don't technically need to provide an index, just the code to generate one and the code you are using to search that exhibits the behavior you are seeing. The data to generate the index can be completely based on a random seed so it generates the same data every time it is run. That way you don't have to share any real production data, and you don't have to provide a large index to us, just the code to generate the same index every time it runs. While @jeme suggested a 3rd party package to generate test data with, do note that RandomizedTesting.Generators has several methods for generating simple or realistic strings. It is also important to base the implementation off of |
Ups, seems that I forgot to attach benchmark code and full results. I have replaced field names and removed input, but you can see what I'm measuring. |
Is there an existing issue for this?
Describe the bug
I have a quite large index with 5-gram field on very large strings (tens or hundreds characters).
The program searches this index and looks for matches using NGramPhraseQuery. I usually have to look for a match for few hundred or thousand input queries.
I tested with .Net framework 4.72 , .Net 6 and .Net 8.
Results for .Net 6 and 8 are 2-4 times slower than .Net framework 4.72 for the same input.
Did you see this behaviour before, or have an idea what to look for?
Thanks, Tom
Expected Behavior
No response
Steps To Reproduce
No response
Exceptions (if any)
No response
Lucene.NET Version
4.8.0-beta00016
.NET Version
4.7.2 and 8.0.100
Operating System
Windows 10
Anything else?
No response
The text was updated successfully, but these errors were encountered: