Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Support new-style digest functions #18731

Closed
wants to merge 1 commit into from
Closed

Support new-style digest functions #18731

wants to merge 1 commit into from

Conversation

tylerwilliams
Copy link
Contributor

Support new-style digest functions.

This PR adds support for new-style digest functions to the remote execution
library code. The remote-apis spec says:

// * `digest_function` is a lowercase string form of a `DigestFunction.Value`
//   enum, indicating which digest function was used to compute `hash`. If the
//   digest function used is one of MD5, MURMUR3, SHA1, SHA256, SHA384, SHA512,
//   or VSO, this component MUST be omitted. In that case the server SHOULD
//   infer the digest function using the length of the `hash` and the digest
//   functions announced in the server's capabilities.

This is a partial commit for #18658.

@tylerwilliams tylerwilliams marked this pull request as ready for review June 21, 2023 05:35
@tylerwilliams tylerwilliams requested a review from a team as a code owner June 21, 2023 05:35
@github-actions github-actions bot added awaiting-review PR is awaiting review from an assigned reviewer team-Remote-Exec Issues and PRs for the Execution (Remote) team labels Jun 21, 2023
coeuvre
coeuvre requested changes Jun 21, 2023
View reviewed changes
Copy link
Member

@coeuvre coeuvre left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we please add a test case for URL of the new style digest function?

@tylerwilliams tylerwilliams force-pushed the b3_remote_exec branch 2 times, most recently from 65db605 to 10f1965 Compare June 21, 2023 22:03
@tylerwilliams
Copy link
Contributor Author

tylerwilliams commented Jun 21, 2023 •
edited by brentleyjones
Loading

Can we please add a test case for URL of the new style digest function?

Done.

coeuvre
coeuvre reviewed Jun 22, 2023
View reviewed changes
src/main/java/com/google/devtools/build/lib/remote/ByteStreamUploader.java
return digestFunction.getNumber() <= 7;
}

private String buildUploadResourceName(
Copy link
Member

@coeuvre coeuvre Jun 22, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You may also want to change GrpcCacheClient#getResourceName, maybe in another PR.

Copy link
Contributor Author

@tylerwilliams tylerwilliams Jun 22, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You may also want to change GrpcCacheClient#getResourceName, maybe in another PR.

Yeah it's a bit cyclical; I'm happy to add support for it here so that download and upload are both kept in sync -- which I've done.

Adding another test for the download path will be simple once there is a DigestHashFunction for BLAKE3. Will send that in a followup after adding the hasher.

@coeuvre coeuvre added awaiting-PR-merge PR has been approved by a reviewer and is ready to be merge internally and removed awaiting-review PR is awaiting review from an assigned reviewer labels Jun 22, 2023
@tylerwilliams tylerwilliams mentioned this pull request Jun 26, 2023
Closed
@tylerwilliams
Copy link
Contributor Author

@coeuvre friendly ping on this -- do I need to do anything more to get this merged? Thanks!

@coeuvre
Copy link
Member

coeuvre commented Jun 27, 2023

It has been imported but is still under review internally, should be merged soon. No actions required on your side! Sorry for the delay.

@github-actions github-actions bot removed the awaiting-PR-merge PR has been approved by a reviewer and is ready to be merge internally label Jun 27, 2023
@brentleyjones brentleyjones deleted the b3_remote_exec branch June 27, 2023 12:47
@brentleyjones brentleyjones mentioned this pull request Aug 7, 2023
Merged
@iancha1992
Copy link
Member

The changes in this PR have been included in Bazel 6.4.0 RC1. Please test out the release candidate and report any issues as soon as possible. If you're using Bazelisk, you can point to the latest RC by setting USE_BAZEL_VERSION=last_rc.
Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@coeuvre coeuvre coeuvre approved these changes

Assignees
No one assigned
Labels
team-Remote-Exec Issues and PRs for the Execution (Remote) team
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@tylerwilliams @coeuvre @iancha1992