[native] Clean old tasks based on termination time#21368
Merged
spershin merged 1 commit intoprestodb:masterfrom Nov 13, 2023
Merged
[native] Clean old tasks based on termination time#21368spershin merged 1 commit intoprestodb:masterfrom
spershin merged 1 commit intoprestodb:masterfrom
Conversation
Currently we clean old tasks based on end time. End time is set when the task finishes processing. Significant time may pass between that point and when downstream workers finish consuming results, when the task is terminated. In such situations the task is cleaned up immediately after being terminated, leading to race conditions with in flight status requests. Using the termination time is safer and a closer analog to what is used in Java.
26 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Currently we clean old tasks based on end time. End time is set when the task finishes processing. Significant time may pass between that point and when downstream workers finish consuming results, when the task is terminated. In such situations the task is cleaned up immediately after being terminated, leading to race conditions with in flight status requests.
Using the termination time is safer and a closer analog to what is used in Java.