Kafka codec: precompute request size before serialization, so we do n…#6862
Merged
mattklein123 merged 2 commits intoenvoyproxy:masterfrom May 10, 2019
Merged
Conversation
…ot have buffer copying Signed-off-by: Adam Kotwasinski <adam.kotwasinski@gmail.com>
Contributor
Author
|
/wait until tests pass |
Contributor
Author
|
@mattklein123 Could you please take a look when you have a moment? Thank you! |
mattklein123
requested changes
May 8, 2019
Member
mattklein123
left a comment
mattklein123
left a comment
There was a problem hiding this comment.
LGTM with small comments. Thank you!
/wait
| * Computes the size of this request, if it were to be serialized. | ||
| * @return serialized size of request | ||
| */ | ||
| virtual size_t computeSize() const PURE; |
Member
There was a problem hiding this comment.
Prefer explicit types. uint64_t or uint32_t ? Same elsewhere.
| : AbstractRequest{request_header}, data_{data} {}; | ||
|
|
||
| /** | ||
| * Compute the size of request, what includes both the request header and its real data. |
| EncodingContext encoder{-1}; | ||
| encoder.encode(data_len, output_); // Encode data length into result. | ||
| output_.add(data_buffer); // Copy encoded data into result. | ||
| const int32_t size = htobe32(message.computeSize()); |
Member
There was a problem hiding this comment.
unsigned? Same elsewhere where applicable.
|
🙀 Error while processing event: |
…stead of size_t Signed-off-by: Adam Kotwasinski <adam.kotwasinski@gmail.com>
Contributor
Author
|
/wait until tests pass |
Contributor
Author
|
@mattklein123 |
Member
|
@adamkotwasinski in the future please make sure the waiting label is cleared when you are ready for me to look at it otherwise the PR doesn't show up in my typical workflow. Thank you. |
mpuncel
added a commit
to mpuncel/envoy
that referenced
this pull request
May 10, 2019
* master: (88 commits) upstream: Null-deref on TCP health checker if setsockopt fails (envoyproxy#6793) ci: switch macOS CI to azure pipelines (envoyproxy#6889) os syscalls lib: break apart syscalls used for hot restart (envoyproxy#6880) Kafka codec: precompute request size before serialization, so we do n… (envoyproxy#6862) upstream: move static and strict_dns clusters to dedicated files (envoyproxy#6886) Rollforward of api: Add total_issued_requests to Upstream Locality and Endpoint Stats. (envoyproxy#6692) (envoyproxy#6784) fix explicit constructor in copy-initialization (envoyproxy#6884) stats: use tag iterator rather than constructing the tag-array and searching that. (envoyproxy#6853) common: use unscoped build target in generate_version_linkstamp (envoyproxy#6877) Addendum to envoyproxy#6778 (envoyproxy#6882) ci: add minimum Linux build for Azure Pipelines (envoyproxy#6881) grpc: utilities for inter-converting grpc::ByteBuffer and Buffer::Instance. (envoyproxy#6732) upstream: allow excluding hosts from lb calculations until initial health check (envoyproxy#6794) stats: prevent unused counters from leaking across hot restart (envoyproxy#6850) network filters: add `injectDataToFilterChain(data, end_stream)` method to network filter callbacks (envoyproxy#6750) delete things that snuck back in (envoyproxy#6873) config: scoped rds (2b): support delta APIs in ConfigProvider framework (envoyproxy#6781) string == string! (envoyproxy#6868) config: add mssing imports to delta_subscription_state (envoyproxy#6869) protobuf: add missing default case to enum (envoyproxy#6870) ... Signed-off-by: Michael Puncel <mpuncel@squareup.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
…ot have buffer copying
Description:
Current request serialization code uses a temporary buffer to compute size of request (it's a non-trivial operation, driven by request's api version present in
EncodingContext).This PR finally adds
.computeSizemethod to request and underlying structures, it's pretty similar to serialization after all (use template overload for primitive type/array/string OR just callcomputeSizeon "rich object").Identified in #4950 (comment)
The alternative was to use
Buffer::Instance::prepend, but then we require the user to provide empty buffer to codec.Risk Level: Low (new Kafka codec that's not active anyways)
Testing: automated tests
Docs Changes: n/a
Release Notes: n/a