-
Notifications
You must be signed in to change notification settings - Fork 272
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Implement roaring64_bitmap_internal_validate #588
Conversation
Either here or as part of another PR, we should add this to our fuzzer... https://github.com/RoaringBitmap/CRoaring/blob/master/fuzz/croaring_fuzzer.c |
* Returns true if the bitmap is consistent. It may be useful to call this | ||
* after deserializing bitmaps from untrusted sources. If | ||
* roaring64_bitmap_internal_validate returns true, then the bitmap is | ||
* consistent and can be trusted not to cause crashes or memory corruption. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Given what the docs describe, I feel that there is some slight mismatch with what we do in roaring64_bitmap_portable_deserialize_safe
, because there we construct 32-bit roaring bitmaps and, importantly, don't call internal_validate
on them. Not sure what the best solution is here.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
That's a good point. I think the way we interact with those bitmaps is safe (at least for now):
- we don't do anything with the containers themselves
- roaring_bitmap_portable_deserialize_safe will always return a consistent count of containers (if the bitmap says it has N containers, there should be N keys, typecodes, containers)
- we don't (currently) rely on the container keys being ordered
- This does mean we could create a valid roaring64 bitmap from an invalid serialization (if a sub-bitmap serialization had unsorted keys, we'll end up with them in the right place because we insert them individually into the ART)
- freeing the bitmap must always be safe
It does mean that, like roaring_bitmap_portable_deserialize_safe
, despite the name containing "safe", it's absolutely possible to get a segfault interacting with the returned roaring64 bitmap if you feed it junk unless you call validate on it (or have external proof that the input is validly the output of serializing a roaring64 bitmap)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
That's a fair assessment, we can leave the wording as-is then.
7f24b10
to
badcd83
Compare
Add calls to validate the bitmap around the unit tests
That's technically undefined behaviour, and shows up in ubsan
badcd83
to
729dc9e
Compare
729dc9e
to
a6c3d8d
Compare
Add calls to validate the bitmap around the unit tests