Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

default to 128bit hashing for collision checks #98

Merged
merged 9 commits into from
Nov 14, 2019

Conversation

karlmcguire
Copy link
Contributor

@karlmcguire karlmcguire commented Nov 1, 2019

If a user Set 30,000,000 items per second for 30 days straight and no items are evicted, the probability of a collision is 0.000000000008 or 8 x 10-12.

If there is a collision, Ristretto will notice and not process the request (Get/Set/Del).


This change is Reviewable

Copy link
Contributor

@manishrjain manishrjain left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:lgtm: Got a few comments.

Reviewed 7 of 10 files at r1, 1 of 3 files at r2, 2 of 2 files at r3.
Reviewable status: all files reviewed, 5 unresolved discussions (waiting on @jarifibrahim and @karlmcguire)


store.go, line 103 at r3 (raw file):

}

func (m *lockedMap) Get(hashes [2]uint64) (interface{}, bool) {

key, conflict


store.go, line 116 at r3 (raw file):

}

func (m *lockedMap) Set(hashes [2]uint64, value interface{}) {

Set(key, conflict, value)


store.go, line 127 at r3 (raw file):

		return
	}
	if (item.hashes[1] != hashes[1]) && hashes[1] != 0 {

I'd flip the order.


store.go, line 138 at r3 (raw file):

}

func (m *lockedMap) Del(hashes [2]uint64) {

key, hash or key, conflict


z/z.go, line 32 at r3 (raw file):

	case string:
		raw := []byte(k)
		return [2]uint64{MemHash(raw), xxhash.Sum64(raw)}

I'd investigate a way to reuse MemHash. Add a TODO for later.

Copy link
Contributor Author

@karlmcguire karlmcguire left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed 7 of 7 files at r4.
Reviewable status: all files reviewed, 5 unresolved discussions (waiting on @jarifibrahim and @manishrjain)


store.go, line 103 at r3 (raw file):

Previously, manishrjain (Manish R Jain) wrote…

key, conflict

Done.


store.go, line 116 at r3 (raw file):

Previously, manishrjain (Manish R Jain) wrote…

Set(key, conflict, value)

Done.


store.go, line 127 at r3 (raw file):

Previously, manishrjain (Manish R Jain) wrote…

I'd flip the order.

Done.


store.go, line 138 at r3 (raw file):

Previously, manishrjain (Manish R Jain) wrote…

key, hash or key, conflict

Done.


z/z.go, line 32 at r3 (raw file):

Previously, manishrjain (Manish R Jain) wrote…

I'd investigate a way to reuse MemHash. Add a TODO for later.

Done.

Copy link
Contributor Author

@karlmcguire karlmcguire left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Dismissed @manishrjain from 5 discussions.
Reviewable status: :shipit: complete! all files reviewed, all discussions resolved (waiting on @jarifibrahim)

@karlmcguire karlmcguire merged commit 99d1bbb into master Nov 14, 2019
@karlmcguire karlmcguire deleted the karl/collision-fix branch November 14, 2019 17:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

Successfully merging this pull request may close these issues.

2 participants