Disk space and number of items in the store #229
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
tmcgilchrist
approved these changes
May 11, 2023
Member
tmcgilchrist
left a comment
tmcgilchrist
left a comment
There was a problem hiding this comment.
Changes look good, please address the code comments.
MisterDA
reviewed
May 11, 2023
Member
|
@mtelvers you might want to grab aabae0f. Ideally |
Co-authored-by: Tim McGilchrist <[email protected]>
Co-authored-by: Tim McGilchrist <[email protected]>
Co-authored-by: Tim McGilchrist <[email protected]>
Co-authored-by: Antonin Décimo <[email protected]>
benmandrew
added a commit
to benmandrew/opam-repository
that referenced
this pull request
Jan 31, 2024
CHANGES: - Support OBuilder Docker backend on Windows and Linux (@MisterDA ocurrent/ocluster#143) - Support OBuilder FreeBSD worker (@tmcgilchrist ocurrent/ocluster#236) - Specify upper bound on number of items in OBuilder store (@mtelvers ocurrent/ocluster#229) - Add Prometheus statistics for OBuilder cache hit rate (@mtelvers ocurrent/ocluster#238)
nberth
pushed a commit
to nberth/opam-repository
that referenced
this pull request
Jun 18, 2024
CHANGES: - Support OBuilder Docker backend on Windows and Linux (@MisterDA ocurrent/ocluster#143) - Support OBuilder FreeBSD worker (@tmcgilchrist ocurrent/ocluster#236) - Specify upper bound on number of items in OBuilder store (@mtelvers ocurrent/ocluster#229) - Add Prometheus statistics for OBuilder cache hit rate (@mtelvers ocurrent/ocluster#238)
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR introduces two new features:
It adds the command line option
--obuilder-prune-item-thresholdwhich specifies an upper bound on the number of result items/steps which will be held in the obuilder cache store. It is possible to specify both--obuilder-prune-thresholdand the new--obuilder-prune-item-thresholdat the same time which will limit both the number of items and the maximum percentage free and it will prune until both conditions are satisfied. It is noted that some stores perform badly when the number of items becomes very large.The free disk space calculation is now passed to the OBuilder Store via
Builder.dfrather than usingExtUnix.All.statvfsto determine the amount of free disk space. As a consequence, we do not need the variablet.root. On macOS with OpenZFS,statvfs(and macOS's owndf) returns the space used by the current ZFS volume only and does not include the sub-volumes. Instead, we must usezpoolto get the actual space used. I have extended this to BTRFS as well but the difference is negligible.