-
Notifications
You must be signed in to change notification settings - Fork 1.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Bug Fix: Handle multi-part lists properly in ReadPostingList #4574
Conversation
If a key has a startUid > 0 it should not be read by ReadPostingList. Instead, when the main key is accessed, the iterator should be advanced so that these keys are skipped over.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Reviewable status: 0 of 2 files reviewed, 2 unresolved discussions (waiting on @manishrjain and @martinmr)
posting/list.go, line 491 at r1 (raw file):
func (l *List) Iterate(readTs uint64, afterUid uint64, f func(obj *pb.Posting) error) error { if l == nil { return nil
I'd create an error called errorNilList
, and return it for all the funcs.
posting/mvcc.go, line 151 at r2 (raw file):
// Trying to read a single part of a multi part list. This type of list // should be read once using the canonical list (with startUid equal to zero). return nil, nil
No nil error here. Should have a pre-defined error.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Reviewable status: 0 of 2 files reviewed, 2 unresolved discussions (waiting on @manishrjain)
posting/list.go, line 491 at r1 (raw file):
Previously, manishrjain (Manish R Jain) wrote…
I'd create an error called
errorNilList
, and return it for all the funcs.
Removed the special nil handling in favor of not returning nil lists in ReadPostingList
posting/mvcc.go, line 151 at r2 (raw file):
Previously, manishrjain (Manish R Jain) wrote…
No nil error here. Should have a pre-defined error.
Done.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Reviewable status: 0 of 2 files reviewed, 3 unresolved discussions (waiting on @manishrjain and @martinmr)
a discussion (no related file):
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Reviewed 2 of 2 files at r3.
Reviewable status: all files reviewed, 4 unresolved discussions (waiting on @manishrjain and @martinmr)
posting/list.go, line 1136 at r3 (raw file):
func (l *List) Facets(readTs uint64, param *pb.FacetParams, langs []string, listType bool) ([]*pb.Facets, error) { l.RLock()
revert all changes here.
posting/mvcc.go, line 150 at r3 (raw file):
if err != nil { return nil, errors.Wrapf(err, "while reading posting list with key [%v]", key) }
Comment here that we used to have a bug here, as caught by . This PR fixes that.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Reviewable status: 0 of 2 files reviewed, 4 unresolved discussions (waiting on @manishrjain and @martinmr)
posting/list.go, line 1136 at r3 (raw file):
Previously, manishrjain (Manish R Jain) wrote…
revert all changes here.
Done.
posting/mvcc.go, line 150 at r3 (raw file):
Previously, manishrjain (Manish R Jain) wrote…
Comment here that we used to have a bug here, as caught by . This PR fixes that.
Done.
This PR changes ReadPostingList to skip over the parts of a multi-part list so that they are not accessed the next time ReadPostingList is called. A multi-part posting list should only be accessed via the main key. Accessing the posting list via one of the other keys was causing issues during rollup and adding spurious keys to the database, which eventually caused reads to return nil in the Jepsen uid-set test.
* Revert "Disable splits in release 1.2 (#4672)" This reverts commit 1846c68. * Bug Fix: Handle multi-part lists properly in ReadPostingList (#4574) This PR changes ReadPostingList to skip over the parts of a multi-part list so that they are not accessed the next time ReadPostingList is called. A multi-part posting list should only be accessed via the main key. Accessing the posting list via one of the other keys was causing issues during rollup and adding spurious keys to the database, which eventually caused reads to return nil in the Jepsen uid-set test. * Add checks to verify expected number of list splits in list_test.go (#4609) (cherry picked from commit 9ca59f8) Co-authored-by: Martin Martinez Rivera <[email protected]>
This PR changes ReadPostingList to skip over the parts of a multi-part list so that they are not accessed the next time ReadPostingList is called. A multi-part posting list should only be accessed via the main key. Accessing the posting list via one of the other keys was causing issues during rollup and adding spurious keys to the database, which eventually caused reads to return nil in the Jepsen uid-set test.
This change is