[improve] [txn] [PIP-160] Protocol changes and configuration changes for transaction batch log #16617
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
@liangyepianzhou @congbobo184 @codelipenghui Could you take a look, please? |
congbobo184
added
doc-required
conf/broker.conf
Outdated
| transactionLogBatchedWriteMaxRecords= 512; | ||
| # If enabled the feature that transaction log batch, this attribute means bytes size in a batch. | ||
| transactionLogBatchedWriteMaxSize= 1024 * 1024 * 4; |
There was a problem hiding this comment.
I think we can't user 1024 * 1024 * 4 here?
conf/broker.conf
Outdated
| # persist into a single BK entry. This will make Pulsar transactions work more efficiently, aka batched log. | ||
| # see: https://github.com/apache/pulsar/issues/15370 | ||
| transactionLogBatchedWriteEnabled = false; |
There was a problem hiding this comment.
Suggested change
| transactionLogBatchedWriteEnabled = false; | |
| transactionLogBatchedWriteEnabled=false; |
Please check all.
|
@poorbarcode Please update the PR title as the PR really changed. |
congbobo184
reviewed
Jul 15, 2022
conf/broker.conf
Outdated
Comment on lines
1540
to
1562
| # persist into a single BK entry. This will make Pulsar transactions work more efficiently, aka batched log. | ||
| # see: https://github.com/apache/pulsar/issues/15370 | ||
| transactionLogBatchedWriteEnabled = false; | ||
| # If enabled the feature that transaction log batch, this attribute means maximum log records count in a batch. | ||
| transactionLogBatchedWriteMaxRecords= 512; | ||
| # If enabled the feature that transaction log batch, this attribute means bytes size in a batch. | ||
| transactionLogBatchedWriteMaxSize= 1024 * 1024 * 4; | ||
| # If enabled the feature that transaction log batch, this attribute means maximum wait time(in millis) for the first | ||
| # record in a batch | ||
| transactionLogBatchedWriteMaxDelayInMillis= 1; | ||
| # Provide a mechanism allowing the Pending Ack Store to aggregate multiple records into a batched record and persist | ||
| # into a single BK entry. This will make Pulsar transactions work more efficiently, aka batched log. | ||
| # see: https://github.com/apache/pulsar/issues/15370 | ||
| transactionPendingAckBatchedWriteEnabled = false; | ||
| # If enabled the feature that transaction pending ack log batch, this attribute means maximum log records count in a | ||
| # batch. | ||
| transactionPendingAckBatchedWriteMaxRecords= 512; | ||
| # If enabled the feature that transaction pending ack log batch, this attribute means bytes size in a batch. | ||
| transactionPendingAckBatchedWriteMaxSize= 1024 * 1024 * 4; | ||
| # If enabled the feature that transaction pending ack log batch, this attribute means maximum wait time(in millis) for | ||
| # the first record in a batch. | ||
| transactionPendingAckBatchedWriteMaxDelayInMillis= 1; |
There was a problem hiding this comment.
move to
under
Line 1437 in cfabdf8
poorbarcode
changed the title
Already Fixed. |
poorbarcode
changed the title
@codelipenghui Thanks much. This is a good way to solve the errors caused by carelessness. Already fixed. |
liangyepianzhou
approved these changes
Jul 15, 2022
| @FieldContext( | ||
| category = CATEGORY_SERVER, | ||
| doc = "If enabled the feature that transaction log batch, this attribute means maximum wait time(in millis)" | ||
| + " for the first record in a batch, default 1." |
There was a problem hiding this comment.
Suggested change
| + " for the first record in a batch, default 1." | |
| + " for the first record in a batch, default 1 millisecond." |
| @FieldContext( | ||
| category = CATEGORY_SERVER, | ||
| doc = "If enabled the feature that transaction pending ack log batch, this attribute means maximum wait" | ||
| + " time(in millis) for the first record in a batch, default 1." |
There was a problem hiding this comment.
Suggested change
| + " time(in millis) for the first record in a batch, default 1." | |
| + " time(in millisecond) for the first record in a batch, default 1 millisecond." |
codelipenghui
approved these changes
Jul 15, 2022
congbobo184
approved these changes
Jul 15, 2022
mattisonchao
approved these changes
Jul 16, 2022
Already fixed. Thanks |
|
/pulsarbot run-failure-checks |
Doc status update@poorbarcode has updated relevant docs in a follow-up PR #16764. |
Anonymitaet
added
doc-complete
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Master Issue: #15370
Motivation
see #15370
Modifications
I will complete proposal #15370 with these pull requests( current pull request is the step-2 ):
TxnLogBufferedWriterDocumentation
doc-requireddoc-not-neededdocdoc-complete