Skip to content

Fix ucx-py version, use UCX 1.17.0 in pip devcontainers#4562

Merged
rapids-bot[bot] merged 4 commits intorapidsai:branch-24.10from
bdice:fix-ucx-py-version
Jul 31, 2024
Merged

Fix ucx-py version, use UCX 1.17.0 in pip devcontainers#4562
rapids-bot[bot] merged 4 commits intorapidsai:branch-24.10from
bdice:fix-ucx-py-version

Conversation

@bdice
Copy link
Contributor

@bdice bdice commented Jul 29, 2024

Fixes ucx-py dependency in dependencies.yaml and update-version.sh.

Updates to UCX 1.17.0 in pip devcontainers. (context: #4562 (comment)).

@bdice bdice requested review from a team as code owners July 29, 2024 22:49
@bdice bdice requested a review from AyodeAwe July 29, 2024 22:49
@bdice bdice changed the base branch from branch-24.08 to branch-24.10 July 29, 2024 22:50
)
for DEP in "${DEPENDENCIES[@]}"; do
for FILE in dependencies.yaml conda/environments/*.yaml python/cugraph-{pyg,dgl}/conda/*.yaml; do
for FILE in dependencies.yaml conda/environments/*.yaml python/cugraph-{pyg,dgl}/conda/*.yaml; do
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I switched the iteration order of this double-for loop. That way we only find-replace ucx-py one time instead of once for every other dependency.

sed_runner "/-.* ucx-py==/ s/==.*/==${NEXT_UCX_PY_VERSION}.*,>=0.0.0a0/g" "${FILE}"
done
for FILE in python/**/pyproject.toml python/**/**/pyproject.toml; do
sed_runner "/-.* ucx-py\(-cu[[:digit:]]\{2\}\)\{0,1\}==/ s/==.*/==${NEXT_UCX_PY_VERSION}.*,>=0.0.0a0/g" "${FILE}"
Copy link
Contributor Author

@bdice bdice Jul 29, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The important change is that the ucx-py logic was not handling -cu11 / -cu12 suffixes. Now it does.

@jameslamb jameslamb added improvement Improvement / enhancement to an existing function non-breaking Non-breaking change labels Jul 30, 2024
@jameslamb
Copy link
Member

jameslamb commented Jul 30, 2024

I merged latest branch-24.10 into this, now that the forward-mergers have been completed.

A few jobs immediately failed, like this:

GitHub Actions has encountered an internal error when running your job.

(build link)

A manual re-run fixed that for the triage job, so I guess it was just something temporary.

The checks jobs failed with similar GitHub internal errors (build link). Hopefully a manual restart will fix those too.

@jameslamb
Copy link
Member

pip (but not conda) devcontainers builds here were failing like this:

🏗️ build container
About to run devcontainer build --workspace-folder /home/runner/_work/cugraph/cugraph --config /home/runner/_work/cugraph/cugraph/.devcontainer/cuda12.5-pip/devcontainer.json --image-name undefined:latest
[2024-07-30T15:02:04.599Z] @devcontainers/cli 0.67.0. Node.js v16.20.2. linux 6.5.0-1023-aws x64.

Error: Dev container build failed: Command failed: docker inspect --type image rapidsai/devcontainers:24.10-cpp-cuda12.5-ucx1.15.0-openmpi-ubuntu22.04 (exit code: undefined)
An error occurred building the container.
Error: Command failed: docker inspect --type image rapidsai/devcontainers:24.10-cpp-cuda12.5-ucx1.15.0-openmpi-ubuntu22.04

(build link)

Looks like that's because there aren't any ucx1.15.0 devcontainers images for 24.10: https://hub.docker.com/r/rapidsai/devcontainers/tags?page=&page_size=&ordering=&name=24.10-cpp-cuda12.5.

That matches what I see at the most recent devcontainers release run on 24.10: https://github.com/rapidsai/devcontainers/actions/runs/10101081673

Since we want to be using UCX 1.17.0 anyway (rapidsai/build-planning#77), I've just updated to that here: 6e61cd6

I should have done that in this repo after rapidsai/devcontainers#338 anyway. I'll go propose similar changes across the other repos using the devcontainers UCX feature (see rapidsai/devcontainers#273 for reference list).

cc @pentschev @trxcllnt

@jameslamb jameslamb changed the title Fix ucx-py version Fix ucx-py version, use UCX 1.17 in pip devcontainers Jul 30, 2024
@jameslamb jameslamb changed the title Fix ucx-py version, use UCX 1.17 in pip devcontainers Fix ucx-py version, use UCX 1.17.0 in pip devcontainers Jul 30, 2024
@jameslamb jameslamb removed request for a team July 30, 2024 16:02
@jameslamb
Copy link
Member

/merge

@rapids-bot rapids-bot bot merged commit 4dd8fca into rapidsai:branch-24.10 Jul 31, 2024
raydouglass added a commit that referenced this pull request Aug 8, 2024
Back ports the `update-version.sh` fixes from
#4562
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci improvement Improvement / enhancement to an existing function non-breaking Non-breaking change

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants