[8.x](backport #41142) [filebeat][GCS] - Fixed failed job handling and removed false-positive error logs #41167
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…e error logs (#41142) [filebeat][GCS] fix failed job handling and remove false-positive error logs In cases where GCS report an error, we were logging the error but not otherwise making use of the information. When GCS was reporting that a requested object does note exist this was causing unnecessary log-spam and not updating the set of failed jobs to remove the object, resulting in future futile re-attempts to collect the object. When any other error was reported, the set of failed jobs was not being updated, resulting in continued re-attempts, even past the maximum retry count. This fixes both cases by differentiating the two situations, logging only at debug level when the object is reported to be missing and removing the object from the failed rework set, and logging at error and increasing the failure count for all other error cases. (cherry picked from commit fbff318)
botelastic
bot
added
the
needs_team
|
This pull request doesn't have a |
ShourieG
approved these changes
Oct 8, 2024
Closed
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Type of change
Proposed commit message
In cases where GCS reports an error, we were logging the error but not
otherwise making use of the information. When GCS was reporting that a
requested object does not exist this was causing unnecessary log-spam and
not updating the set of failed jobs to remove the object, resulting in future
futile re-attempts to collect the object. When any other error was reported,
the set of failed jobs was not being updated, resulting in continued
re-attempts, even past the maximum retry count.
This fixes both cases by differentiating the two situations, logging only at
debug level when the object is reported to be missing and removing the object
from the failed rework set, and logging at error and increasing the failure
count for all other error cases.
Checklist
CHANGELOG.next.asciidocorCHANGELOG-developer.next.asciidoc.Disruptive User Impact
Author's Checklist
How to test this PR locally
Related issues
Use cases
Screenshots
Logs
This is an automatic backport of pull request #41142 done by [Mergify](https://mergify.com).