feat(uv): Support native sdist builds via RBE #770
Merged
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.

The Python ecosystem presently lacks (PEP-720) a comprehensive crossbuilding story.
As a workaround for this and issues which could arise by "faking" crossbuilds eg by lying about the current interpreter platform, we can instead lean on RBE to ignore the issue by executing sdist to bdist build actions on workers matching the target configuration.
With thanks to @fmeum for the guidance, this is achieved by using Bazel's execution groups feature. We define a dummy toolchain of no files which can be resolved only in a configuration matching the target configuration, and consequently inherits the
exec_propertiesof the target platform. Then we create an execution group which contains that dummy toolchain.ctx.actions.runcalls referencing an exec group inherit the exec properties of the chosen platform for which the toolchains in the group were resolved. Thus we have a way to enforce that the action will be placed on a host matching the target's configuration.The new e2e case exercises this behavior by forcibly marking a dependency which has no bdists (it was installed via sdist url) as being native and thus requiring platform-specific builds which must occur as platform-specific actions. This behavior can be observed in the following query
The image index depends on both configurations of the binary, each of which has a dependency on the "native" source build of
cowsay. Consequently we expect to see twoPySdistBuildactions,In order to make performing these builds more efficient in RBE, this patch also reworks the sdist build helper so that we use Python to unzip the sdist archive in the same action step as we perform the build. In RBE this eliminates the time and transit overhead of using one action to run
tar.bzlto unzip the sdist only to re-pack the resulting tree artifact and shovel it to a different worker for the subsequent sdist build action.Changes are visible to end-users: yes
The UV extension will now attempt to place sdist builds of native extensions on the target platform via RBE.
Test plan
See #771 for the test case code.
Inspect the build plan for the image index, which should show one sdist build on arm64 and one on amd64.