Adds a check to automatically adjust for batch size limit enforced by aws #664
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
This looks great! I'll do some testing locally, but I think it should go pretty smoothly. |
|
I verified locally that this will correctly subdivide message batches that exceed the SQS batch message payload limit. |
|
Thanks for your contribution, @correnarsprowls! |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This addresses #663
The change to
batch_sendwill ensure that a batch is never larger than the 256kb limit set by SQS. If the batch size is too large then it will recurse until the batch size is small enough. If a single message is larger than the batch size limit then it will display the original error message but the rest of the batch will succeed.These changes were tested using LocalStack.