Skip to content

Replace conda-mambabuild with conda-build - #752

Merged
rapids-bot[bot] merged 3 commits into
rapidsai:branch-25.04from
jakirkham:use_cb
Feb 27, 2025
Merged

Replace conda-mambabuild with conda-build#752
rapids-bot[bot] merged 3 commits into
rapidsai:branch-25.04from
jakirkham:use_cb

Conversation

@jakirkham

Copy link
Copy Markdown
Member

These are now equivalent in behavior. However support for conda-mambabuild is being dropped. So switch to conda-build.

xref: rapidsai/build-planning#149

These are now equivalent in behavior. However support for
`conda-mambabuild` is being dropped. So switch to `conda-build`.
The default behavior of `conda-build` is to use packages from its local
cache. Given this, we don't need this flag. It also causes conda-build
to look in a different place than we typically use for the package cache
on CI, which causes its own problems.
Comment thread ci/build_python.sh Outdated
Comment on lines -18 to -19
rapids-conda-retry build \
--use-local \

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As conda-build always uses its local cache, the --use-local flag is unneeded

Additionally it appears conda-build has a bug that causes it to look inside its package cache before it is created, which causes the following CI error

UnavailableInvalidChannel: HTTP 404 UNAVAILABLE OR INVALID for channel opt/conda/conda-bld <file:///opt/conda/conda-bld>

The channel is not accessible or is invalid.

You will need to adjust your conda configuration to proceed.
Use `conda config --show channels` to view your configuration's current state,
and use `conda config --show-sources` to view config file locations.

As of conda 4.3, a valid channel must contain a `noarch/repodata.json` and
associated `noarch/repodata.json.bz2` file, even if `noarch/repodata.json` is
empty. Use `conda index /opt/conda/conda-bld`, or create `noarch/repodata.json`
and associated `noarch/repodata.json.bz2`.

Typically we don't use /opt/conda/conda-bld, but use our own package cache location. So this isn't even the right place for it to check

Since we don't need this flag here (nor use it elsewhere), it is easy enough to drop, which fixes all of these issues

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Works for me. I looked through the blame to check why it was added originally... looks like you added it, I think to be sure that the build of integration uses the rapids-xgboost just built in the same build:

#669 (comment)

So I checked CI logs here to be sure this was working as expected and not falling back to a previous rapids-xgboost nightly.

It is:

BUILD START: ['rapids-xgboost-25.04.00a-cuda11_py310_250227_g07bf842_11.conda']
...
BUILD START: ['rapids-25.04.00a-cuda11_py310_250227_g07bf842_11.conda']
...
    rapids-xgboost:                   25.04.00a-cuda11_py310_250227_g07bf842_11        file:///tmp/conda-bld-output
...

(build link)

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch!

Maybe there was a past conda-build issue we were working around with the test phase

@jakirkham
jakirkham requested a review from bdice February 27, 2025 04:43
@jakirkham

Copy link
Copy Markdown
Member Author

The test-conda-nightly env failures are unrelated and appear to be due to issue: rapidsai/rapids-logger#27

@jameslamb jameslamb left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Even if that rapids-logger change was made, you'd still run into issues like what you're facing here because that project isn't built every night (https://github.com/rapidsai/rapids-logger/actions/workflows/build.yaml).

It'd be better to just add rapids-logger to this exclusion list:

EXCLUDED_PACKAGES = {
# These packages are not built every night:
"rapids",
"rapids-xgboost",

@jameslamb
jameslamb self-requested a review February 27, 2025 23:05
@jameslamb

Copy link
Copy Markdown
Member

/merge

@rapids-bot
rapids-bot Bot merged commit 0e9041d into rapidsai:branch-25.04 Feb 27, 2025
@jakirkham
jakirkham deleted the use_cb branch February 27, 2025 23:14
@jakirkham

Copy link
Copy Markdown
Member Author

Thanks James! 🙏

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants