Closed
Conversation
facebook-github-bot
added
the
CLA Signed
ndmitchell
reviewed
Oct 6, 2022
http_archive/http_archive.bzl
Outdated
| def _type(ctx: "context") -> str.type: | ||
| typ = value_or(ctx.attrs.type, "tar.gz") | ||
| if typ not in ("tar.gz", "tar.zst"): | ||
| if typ not in _FLAGS.keys(): |
Contributor
There was a problem hiding this comment.
I think not in _FLAGS will do the same test, but in a hash map, without allocating an array of values first.
Contributor
Author
There was a problem hiding this comment.
Right, good point. Fixed.
Contributor
|
@ndmitchell has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator. |
facebook-github-bot
pushed a commit
to facebook/buck2
that referenced
this pull request
Oct 6, 2022
Summary: Adds support for `tar.xz` archives to `http_archive`. The user needs to set the attribute `type = "tar.xz"` to make use of this. X-link: facebook/buck2-prelude#3 Reviewed By: arlyon Differential Revision: D40138919 Pulled By: arlyon fbshipit-source-id: 3cbecdbde522ec5fc56acee4c75cb6caf62c8e27
facebook-github-bot
pushed a commit
that referenced
this pull request
Dec 19, 2024
…cross all host platforms Summary: ### Motivation My team has a concrete need for buck to generate 100% matching zip files for the same sets of inputs on all host platforms (macOS, Linux, Windows). Current limitations: 1. File order can be different on file system with different case sensitivity. 2. Windows can't write correct posix mode (i.e. permissions) for any entries. Although the entries themselves might fully match, those discrepancies result in different metadata, which results in a different zip file. See D67149264 for an in-depth explanation of the use case that requires this level of determinism. ### Tentative solution #1 In D66386385, I made it so the asset generation rule was only executable from Linux. Paired with buck cross builds, it made so that outputs from macOS and Linux matched, but did not work on Windows [due to some lower level buck problem](https://fb.workplace.com/groups/930797200910874/posts/1548299102494011) (still unresolved). ### Tentative solution #2 In D66404381, I wrote my own Python script to create zip files. I got all the files and metadata to match everywhere, but I could not get around differences in the compression results. Decided not to pursue because compression is important for file size. ### Tentative solution #3 In D67149264, I duplicated and tweaked buck's zip binary. It did work, but IanChilds rightfully pointed out that I'd be making maintenance on those libraries more difficult and that the team is even planning on deleting those, at some point. ### Tentative solution #4 (this diff!) IanChilds advised me to try to fix buck itself to produce consistent results, so this is me giving it a try. Because the root problem could not have been done in a backwards compatible way (the file permissions, specifically; see inlined comment), I decided to use an argument to control whether the zip tool should strive to produce a deterministic output or not, at the expense of some loss of metadata. The changes are simple and backwards compatible, but any feedback on the root problem, idea and execution are welcome. Reviewed By: christolliday Differential Revision: D67301945 fbshipit-source-id: c42ef7a52efd235b43509337913d905bcbaf3782
avdv
pushed a commit
to avdv/buck2-prelude
that referenced
this pull request
Aug 25, 2025
…gets Module sub targets
meta-codesync bot
pushed a commit
that referenced
this pull request
Nov 19, 2025
Summary: Context: https://fb.workplace.com/groups/pyreqa/permalink/25336338796029280/ This is currently a soft error in buck, this fixes #3 and #4 by buck command hits shown in data: https://fburl.com/scuba/buck2_builds/r6tjb3q1 Reviewed By: stroxler, JakobDegen Differential Revision: D87397708 fbshipit-source-id: fbaba078c5fc70c1074e04907eb4effe2b6a6350
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds support for
tar.xzarchives tohttp_archive. The user needs to set the attributetype = "tar.xz"to make use of this.