-
Notifications
You must be signed in to change notification settings - Fork 83
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
ci: Use uv for all pip installs #2444
Conversation
Need astral-sh/uv#1526 or an alternative for this to work first. Edit: Can do this for the time being: - name: Set up Python
uses: actions/setup-python@v5
with:
python-version: ${{ matrix.python-version }}
- name: Set the VIRTUAL_ENV variable for uv to work
run: |
echo "VIRTUAL_ENV=${Python_ROOT_DIR}" >> $GITHUB_ENV
- name: Install dependencies
run: |
python -m pip install uv
python -m uv pip install --upgrade <stuff> which comes from astral-sh/uv#1386 (comment). |
Also needs astral-sh/uv#1531 so will have to wait for the next release of |
Given astral-sh/uv#1531 (comment) we also need astral-sh/uv#313 to go in. |
Okay with astral-sh/uv#1526 in and using the
pattern over
pattern I think this should be good. edit: No, |
Reported failure in astral-sh/uv#2062 |
Also blocked by astral-sh/uv#2685 |
d5d84c6
to
01f001b
Compare
So this is really blocked by |
01f001b
to
f6d34f3
Compare
With the release of ...
"papermill>=2.5.0",
"scrapbook>=0.5.0",
... astral-sh/uv#2685 is sidestepped, but the issue with TensorFlow dependency resolution in astral-sh/uv#2062 is still blocking. |
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## main #2444 +/- ##
=======================================
Coverage 98.21% 98.21%
=======================================
Files 69 69
Lines 4543 4543
Branches 804 804
=======================================
Hits 4462 4462
Misses 48 48
Partials 33 33
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. |
* Loosen the install requirements for papermill, scrapbook, and sphinxcontrib-bibtex to use lower bounds instead of compatible release constraints. * Required for PR #2444
This reverts commit 44fdb31.
1c2e4ef
to
d4f6e7c
Compare
if: matrix.python-version != '3.8' | ||
run: | | ||
python -m pip install uv | ||
uv pip install --system --upgrade pip setuptools wheel | ||
uv pip install --system --upgrade ".[all,test]" | ||
|
||
# c.f. https://github.com/astral-sh/uv/issues/2062 | ||
- name: Install dependencies (Python 3.8) | ||
if: matrix.python-version == '3.8' |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
A bit of a hack, but until astral-sh/uv#2062 gets resolves, we might as well speed up everything else.
# Need to use --extra-index-url as dependencies aren't on scientific-python-nightly-wheels package index. | ||
# Need to use --pre as dev releases will need priority over stable releases. | ||
python -m pip install \ | ||
uv pip install --system \ |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Double check this as pip
and uv
handle --extra-index-url
differently. c.f. scientific-python/upload-nightly-action#76
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yeah, this needed to get flipped to
uv pip install --system \
--pre \
--index-url https://pypi.org/simple/ \
--extra-index-url https://pypi.anaconda.org/scientific-python-nightly-wheels/simple \
matplotlib
as uv
give priority to --extra-index-url
.
$ pip list
Package Version
--------------- -------------------------
contourpy 1.3.0.dev1
cycler 0.12.1
fonttools 4.51.0
kiwisolver 1.4.5
matplotlib 3.10.0.dev114+g4cbef2d4c0
numpy 2.1.0.dev0
packaging 24.0
pillow 10.3.0
pip 24.0
pyparsing 3.1.2
python-dateutil 2.9.0.post0
setuptools 69.5.1
six 1.16.0
uv 0.1.39
wheel 0.43.0
* uv pip install --upgrade is going to try to upgrade numpy along with scipy, so to avoid this, don't use --upgrade when installing scipy from the nightly wheel index.
# uv wants to upgrade dependencies (numpy) to a dev release too, so don't --upgrade | ||
uv pip install --system --index-url https://pypi.anaconda.org/scientific-python-nightly-wheels/simple scipy |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This was previously forcing an upgrade of numpy
with uv
as well, but just want to test scipy
which should work with numpy
v1
or v2
, and trying to test numpy
v2
will break jax
at the moment.
Okay, I think this is finally working as expected across all workflows now. |
Some examples of install time drops:
🚀 |
I'm going to approve and merge this myself. As always, PRs approved by a single core dev can be reverted as needed by the rest of the dev team. |
Description
Use
uv
to try to speed up the installs of all the Python dependencies.Fall back to using
pip
for Python 3.8 until astral-sh/uv#2062 is resolved.Apply subtle changes to install commands in .github/workflows/dependencies-head.yml.
uv pip install --upgrade
will try to upgrade all dependencies of the target package as well, which for the dependencies-head workflow isn't the goal. So remove the--upgrade
from calls that also install from the scientific-python-nightly-wheels package index when testing only particular packages.up pip
andpip
have different behavior with regards to--extra-index-url
, asuv pip
gives--extra-index-url
priority over--index-url
, wherepip
does not give priority to either. Use this withuv pip
to give priority to the scientific-python-nightly-wheels package index.Checklist Before Requesting Reviewer
Before Merging
For the PR Assignees: