You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Related to the continuation of StringView/BinaryView support in #6163
In arrow_data, the ByteView type is used to encapsulate this structure from the spec:
Notably, the spec dictates that all of these values must be signed integers. However, we're using u32.
The arrow-rs builder for GenericByteViewArray doesn't seem to have any range checks on the block, offset and len values for the view structure, which means, I think, you can happily construct a StringView array with arrow-rs, and then attempt to pass it to PyArrow or Java over IPC and have it fail at runtime.
Describe your question
Should we either
be using i32 instead of u32 internally
be adding constraints on the builder methods to ensure that we don't allow adding strings > 2GB
Has someone noticed this before and addressed it and it's not actually a problem
The text was updated successfully, but these errors were encountered:
I think the original version of the spec used unsigned which was what the implementation was then based off, I don't see an issue with using i32 or clamping u32
Which part is this question about
Related to the continuation of StringView/BinaryView support in #6163
In
arrow_data
, theByteView
type is used to encapsulate this structure from the spec:Notably, the spec dictates that all of these values must be signed integers. However, we're using u32.
The arrow-rs builder for GenericByteViewArray doesn't seem to have any range checks on the block, offset and len values for the view structure, which means, I think, you can happily construct a StringView array with arrow-rs, and then attempt to pass it to PyArrow or Java over IPC and have it fail at runtime.
Describe your question
Should we either
The text was updated successfully, but these errors were encountered: