Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix #11764, faster hashing of (u)int #12407

Merged
merged 2 commits into from
Oct 15, 2019
Merged

fix #11764, faster hashing of (u)int #12407

merged 2 commits into from
Oct 15, 2019

Conversation

narimiran
Copy link
Member

Running https://github.com/cwpearson/nim-issue-11764/blob/master/slow_set.nim on my machine, which is the test case for #11764 produces the following before/after results:

Before this PR:

(1) time 0.017153174
(2) time 1.512243921
(3) time 18.05979435
(4) time 17.861573436
(5) time 11.350780583
(6) time 0.003636462999999424
(7) time 16.871593051
(8) time 0.002521137000002227

After this PR:

(1) time 0.022618518
(2) time 0.02331171199999999
(3) time 0.01813793399999999
(4) time 0.01471349299999999
(5) time 0.008311381999999992
(6) time 0.01135809600000001
(7) time 0.02215919299999999
(8) time 0.002485686000000015

Summary: this PR makes HashSets and HashTables about three orders of magnitude faster for large amounts of (u)int keys, because it prevents huge amounts of collisions happening with the old algorithm (where the hash of an int was that int) for consecutive ints.

@Araq
Copy link
Member

Araq commented Oct 10, 2019

Ok, but use * for uint which cannot overflow.

@mratsim
Copy link
Collaborator

mratsim commented Oct 11, 2019

Delegating review to @jangko we have critical slowness issue with HashSet[uint64] but I'm back at home only on Oct 22.

@jangko
Copy link
Contributor

jangko commented Oct 12, 2019

I can confirm the slow excl previously observed with this code is now able to run fast even in debug mode.

@narimiran
Copy link
Member Author

I can confirm the slow excl previously observed with this code is now able to run fast even in debug mode.

Thanks for testing it!


Since this breaks two packages in our test set (which rely on order of keys in HashTable), and who knows how many more other packages, and even though I can send fixes to these two packages and others can easily be fixed, I'm of opinion that such breakages should not happen between two patch versions (1.0.0 and 1.0.2), so I'm retracting [backport] tag from this PR.

@narimiran narimiran changed the title [backport] fix #11764, faster hashing of (u)int fix #11764, faster hashing of (u)int Oct 12, 2019
@narimiran narimiran force-pushed the fix-11764 branch 3 times, most recently from 318f0a1 to 7aaac13 Compare October 14, 2019 11:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants