Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[bazel] make rust_rules deps cacheable #12515

Closed
3 tasks done
Tracked by #12449
timothytrippel opened this issue May 6, 2022 · 1 comment · Fixed by #12526
Closed
3 tasks done
Tracked by #12449

[bazel] make rust_rules deps cacheable #12515

timothytrippel opened this issue May 6, 2022 · 1 comment · Fixed by #12526
Assignees
Labels

Comments

@timothytrippel
Copy link
Contributor

timothytrippel commented May 6, 2022

The rust_rules attempt to download the rust_linux_x86_64 external repository using the ctx.download action. Unfortunately, it seems this action does not first check in the repository cache for files before downloading, where as the ctx.download_and_extract action does. This is vaguely hinted at in the bazel repository rules documentation. Specifically, the description for the sha256 parameter of the download action only states: "the expected SHA-256 hash of the file downloaded. This must match the SHA-256 hash of the file downloaded ..."; while the the description for the sha256 parameter of the download_and_extract action additionally states: "... If provided, the repository cache will first be checked for a file with the given hash; a download will only be attempted if the file was not found in the cache. After a successful download, the file will be added to the cache."

Moreover, after testing out a patch of the rules_rust repository that replaces the ctx.download action with the ctx.download_and_extract action the rust_linux_x86_64 repository files are then found in the pre-populated repository cache (prepared with the prep-bazel-airgapped-builds.sh script), instead of downloaded from the network.

To fix this issue and support airgapped SW builds, the following item must be addressed:

  • fork and patch the rules_rust repo (see this commit)

Addressed in: #12526

  • use the forked / patched rules_rust repo in the opentitan bazel workspace
  • update the prep-bazel-airgapped-builds.sh to prefetch and cache the rust_linux_x86_64 external repository
@timothytrippel timothytrippel self-assigned this May 6, 2022
timothytrippel added a commit to timothytrippel/opentitan that referenced this issue May 6, 2022
As described in lowRISC#12515, the upstream `rules_rust` repository attempted
to download several dependencies in a manner that did not first check
the repository cache before going to the network. This made supporting
an airgapped build environment problematic. To fix this issue, the
`rules_rust` repository was forked and patched. Please follow this link
for the commit message that describes the patch in detail:
lowRISC/rules_rust@be0d6ca

This commit update the bazel dependencies to use our patched
`rust_rules` instead of the upstream ones. Additionally, this commit
updates the `prep-bazel-airgapped-builds.sh` script to pre-fetch
additional dependencies to pre-populate the bazel repository cache with
additional required dependencies to support airgapped builds.

This fixes lowRISC#12515.

Signed-off-by: Timothy Trippel <[email protected]>
@timothytrippel
Copy link
Contributor Author

CC: @cfrantz

timothytrippel added a commit that referenced this issue May 9, 2022
As described in #12515, the upstream `rules_rust` repository attempted
to download several dependencies in a manner that did not first check
the repository cache before going to the network. This made supporting
an airgapped build environment problematic. To fix this issue, the
`rules_rust` repository was forked and patched. Please follow this link
for the commit message that describes the patch in detail:
lowRISC/rules_rust@be0d6ca

This commit update the bazel dependencies to use our patched
`rust_rules` instead of the upstream ones. Additionally, this commit
updates the `prep-bazel-airgapped-builds.sh` script to pre-fetch
additional dependencies to pre-populate the bazel repository cache with
additional required dependencies to support airgapped builds.

This fixes #12515.

Signed-off-by: Timothy Trippel <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant