ARROW-10962: [FlightRPC][Java] fill in empty body buffer if needed #8963
Conversation
|
Requesting review from @praveenbingo and @liyafan82 as they seem to have merged the last few Java PRs into the arrow repo. The background here is that we are working on flight integration tests in Rust (e.g. #9049) and we found an edge case in the flight protobuf integration. |
|
Sorry for my late response. Will take a look late today. |
There was a problem hiding this comment.
Expanded to 'compatibility'
There was a problem hiding this comment.
nit: do you think we also need a test case for which a schema has an empty body?
There was a problem hiding this comment.
Yup, I added a test for this as well.
There was a problem hiding this comment.
Maybe we need to assign the schema to an object and verify that its body is null?
There was a problem hiding this comment.
The schema itself has no body. I've added a test to check the parsed message.
There was a problem hiding this comment.
We can extract a common method for the logic of writing the message to a ByteArrayOutputStream?
There was a problem hiding this comment.
Do we need to check the message body here?
There was a problem hiding this comment.
I've added assertions here.
liyafan82
left a comment
liyafan82
left a comment
There was a problem hiding this comment.
LGTM. Will merge soon if there are no more comments.
|
#9049 is still blocked on getting this PR merged. Any chance you can do so @liyafan82 ? |
|
🎉 thanks @liyafan82 -- I didn't do anything, it was all @lidavidm . Thanks @lidavidm for the efforts! |
This PR has a few refactorings and then the main commit contains a new Flight integration test client and server 🎉 The middleware scenario tests are currently skipped because they will fail until `tonic` can be updated to a version containing [a fix having to do with trailers](hyperium/tonic#510); this is tracked in [ARROW-10961](https://issues.apache.org/jira/browse/ARROW-10961). Some Rust <-> Java integration tests will fail until [this PR is merged](#8963); I'm happy to rebase once that goes in, but I wanted to get code review started on this. Thank you!! Closes #9049 from carols10cents/rust-flight-integration Lead-authored-by: Carol (Nichols || Goulding) <[email protected]> Co-authored-by: Jake Goulding <[email protected]> Signed-off-by: Neville Dipale <[email protected]>
This PR has a few refactorings and then the main commit contains a new Flight integration test client and server 🎉 The middleware scenario tests are currently skipped because they will fail until `tonic` can be updated to a version containing [a fix having to do with trailers](hyperium/tonic#510); this is tracked in [ARROW-10961](https://issues.apache.org/jira/browse/ARROW-10961). Some Rust <-> Java integration tests will fail until [this PR is merged](apache/arrow#8963); I'm happy to rebase once that goes in, but I wanted to get code review started on this. Thank you!! Closes #9049 from carols10cents/rust-flight-integration Lead-authored-by: Carol (Nichols || Goulding) <[email protected]> Co-authored-by: Jake Goulding <[email protected]> Signed-off-by: Neville Dipale <[email protected]>
Fixes compatibility with standard Protobuf implementations, which may omit serialization of an empty field. In the case of an empty record batch, this would cause the Java client to fail a precondition check since there would be no body buffer instead of a single empty body buffer. Closes apache#8963 from lidavidm/arrow-10962 Authored-by: David Li <[email protected]> Signed-off-by: liyafan82 <[email protected]>
3 participants
Fixes compatibility with standard Protobuf implementations, which may omit serialization of an empty field. In the case of an empty record batch, this would cause the Java client to fail a precondition check since there would be no body buffer instead of a single empty body buffer.