improve query logging efficiency by passing io.Writer along#4122
Merged
sougou merged 2 commits intovitessio:masterfrom Aug 12, 2018
Merged
improve query logging efficiency by passing io.Writer along#4122sougou merged 2 commits intovitessio:masterfrom
sougou merged 2 commits intovitessio:masterfrom
Conversation
Extract the common functionality for formatting bind variables out from both vtgate and vttablet's query logging modules into a sqltypes.FormatBindVariables helper. Signed-off-by: Michael Demmer <mdemmer@slack-corp.com>
Change the internal interface for query logging so that it passes an io.Writer into the formatter function as opposed to producing a temporary string and calling io.WriteBytes. This should reduce some GC pressure and enable more efficient buffer management under heavy workloads. Signed-off-by: Michael Demmer <mdemmer@slack-corp.com>
Member
|
I'm not sure what your appetite / ability is for external platforms, but we monitor our GC with https://cloud.google.com/profiler/ and it works great. |
sougou
approved these changes
Aug 12, 2018
Contributor
sougou
left a comment
sougou
left a comment
There was a problem hiding this comment.
This is a good cleanup. So, it doesn't need a noticeable performance impact.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Change the internal interface for query logging so that it passes an io.Writer into the formatter function as opposed to producing a temporary string and calling io.WriteBytes.
This should reduce some GC pressure and enable more efficient buffer management under heavy workloads.
Along the way extract the common functionality for formatting bind variables out from both vtgate and vttablet's query logging modules into a sqltypes.FormatBindVariables helper.
Testing
On a couple of quick before-and-after tests in our dev environment I couldn't discern any meaningful CPU performance impact, but I wasn't sure how to measure the GC impact. In principle I can't think of any reason why this would be worse, though it might not actually be meaningfully better.
If this looks reasonable then we (Slack) can try it out in a more systematic way to see if it has any measurable effect.