WIP switch GPU workers to image that uses multi engine generic worker #700
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
bhearsum
force-pushed
the
gpu-multi
branch
8 times, most recently
from
cf39689 to
ea4c6ae
Compare
|
It looks like these new workers largely work fine. We are hitting taskcluster/taskcluster#7128, which will need to be fixed before this is landable, but all of the GPU tasks seem to work well within Docker on this new image. |
bhearsum
force-pushed
the
gpu-multi
branch
2 times, most recently
from
109dade to
44827e9
Compare
taskcluster/taskcluster#7428 has this patch. We'll need a new worker image when that's done to pick it up. I also want to wait on mozilla-releng/fxci-config#244 to land to avoid some unnecessary confusion in this repo. |
bhearsum
force-pushed
the
gpu-multi
branch
6 times, most recently
from
26b3d9b to
996ab9e
Compare
The new image we're upgrading GPU workers to uses Ubuntu 24.04, which makes it incompatible with various parts of the pipeline (mostly due to Python package pinning). As it turns out, the easiest way to fix this is to dockerize the GPU tasks. We need slight updates to GPU task payloads to accommodate this. This will fix mozilla#391.
Without this we end up with these files being inaccessible in subsequent tasks.
This has always been needed, but it was found on the host system on the previous image.
bhearsum
force-pushed
the
gpu-multi
branch
11 times, most recently
from
7792ffc to
a111917
Compare
bhearsum
added a commit
to bhearsum/fxci-config
that referenced
this pull request
Jan 17, 2025
…GPU workers And also, use the newly minted dated image for them, which appears to be working well in mozilla/translations#700.
bhearsum
added a commit
to bhearsum/fxci-config
that referenced
this pull request
Jan 17, 2025
…GPU workers And also, use the newly minted dated image for them, which appears to be working well in mozilla/translations#700.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This will be necessary for #466 due to the "simple" engine not supporting chain of trust.
This initial work is just for some sanity checking that nothing obvious breaks. Aside from tasks in automation, we also need to see how interactive tasks behave, and if they meet the needs (most notably, see if we can have root in interactive tasks if needed).