Changing watch to maintain a context rather than recalculating state summary each time #218
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…ry each time Currently whenever we want to summary of the events we watch. We loop through every job we have seen events for and sum up which state they are currently in. This becomes very slow when you have 50000+ jobs, as each event you'll process all 50000 jobs Now we hold a summary of the current state, so it isn't recalculated on each event. - Each event just updates the current state - Getting a summary of the current state is just a case of looking it up, rather than recalculating We could do more optimisations to make starting watching a big jobset faster: - Start from current (This may make the summary confusing however, as it won't include all jobs in the jobset) - We could not print anything until we catch up with the current state - Or just print every 1s rather than every event Printing takes most of the time now. So skipping past until current saves a LOT of time
jankaspar
approved these changes
Nov 1, 2019
svc-gh-ghzonetrans-p
pushed a commit
that referenced
this pull request
Sep 5, 2024
d80tb7
added a commit
that referenced
this pull request
Sep 5, 2024
Co-authored-by: Christopher Martin <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Currently whenever we want to summary of the events we watch. We loop through every job we have seen events for and sum up which state they are currently in.
This becomes very slow when you have 50000+ jobs, as each event you'll process all 50000 jobs
Now we hold a summary of the current state, so it isn't recalculated on each event.
We could do more optimisations to make starting watching a big jobset faster:
Printing takes most of the time now. So skipping past until current saves a LOT of time