Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: uv to replace poetry #3900

Merged
merged 29 commits into from
Sep 25, 2024
Merged

chore: uv to replace poetry #3900

merged 29 commits into from
Sep 25, 2024

Conversation

phact
Copy link
Collaborator

@phact phact commented Sep 24, 2024

uv is a relatively new python package manager with great performance. Langflow has a lot of dependencies that result in slow poetry install / lock times which uv can alleviate.

This WIP PR gets the core features like build and unit tests working.

Would be great to get nightlies working with uv.

package publish with uv is pending.

The current config supports both poetry and uv. We can remove poetry as we gain confidence with uv.

@dosubot dosubot bot added size:L This PR changes 100-499 lines, ignoring generated files. python Pull requests that update Python code labels Sep 24, 2024
Copy link

This pull request is automatically being deployed by Amplify Hosting (learn more).

Access this pull request here: https://pr-3900.dmtpw4p5recq1.amplifyapp.com

@phact phact changed the title UV to replace poetry uv to replace poetry Sep 24, 2024
scripts/update_dependencies.py Outdated Show resolved Hide resolved
Makefile Outdated Show resolved Hide resolved
@dosubot dosubot bot added lgtm This PR has been approved by a maintainer size:XL This PR changes 500-999 lines, ignoring generated files. and removed size:L This PR changes 100-499 lines, ignoring generated files. labels Sep 25, 2024
autofix-ci bot and others added 5 commits September 25, 2024 17:57
- Replace `poetry-core` with `hatchling` in build-system requirements
- Update `langflow-base` dependency to version `0.0.96`
- Add `tool.hatch.build.targets.wheel` configuration
- Adjust `tool.uv.sources` paths for `langflow-frontend` and `langflow-base`
@ogabrielluiz ogabrielluiz changed the title uv to replace poetry chore: uv to replace poetry Sep 25, 2024
@ogabrielluiz ogabrielluiz merged commit fbb097d into langflow-ai:main Sep 25, 2024
30 checks passed
requires-python = ">=3.10,<3.13"
# Define your main dependencies here
dependencies = [
"langflow-base==0.0.96", # Local editable package
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How does this work if I run python -m langflow run in my repo - does it use the pinned 0.0.96 version from pypi or the path from the tool.uv.sources above?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good eyes, I think we want:
"langflow-base @ file://./src/backend/base", # Local editable package

which is what I had originally in here but I don't know if we have a good way of keeping it from getting accidentally committed. uv seems to replace it with the 0.0.96 base automatically when we build.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was fixed. It has to be the version but then add it as a workspace package

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This file is still needed to update dependencies for the poetry version during docker builds - https://github.com/langflow-ai/langflow/blob/main/docker/build_and_push.Dockerfile#L54

Can we bring this back? Was it causing issues?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ogabrielluiz is the plan to update docker too?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

- name: Run unit tests
uses: nick-fields/retry@v3
with:
timeout_minutes: 12
max_attempts: 2
command: make unit_tests async=false args="--splits ${{ matrix.splitCount }} --group ${{ matrix.group }}"
- name: Minimize uv cache
run: uv cache prune --ci
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see the install uv -> python -> restore -> install project .. -> minimize steps repeated a few times. Can we move that to a sub workflow?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

good idea want to take a stab at it?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not sure if this is addressed here #3919

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll add this once we merge 3919

diogocabral pushed a commit to headlinevc/langflow that referenced this pull request Nov 26, 2024
* uv sync works

* fist stab at Makefile

* uv treatment for langflow-base

* sqlmodel to 0.0.18

* add reinstall_backend to Makefile

* makefile - reinstall_backend fix and unit_test dependency

* fix dev dependencies

* fix dev dependencies

* fix dev dependencies

* lock

* Makefile

* [autofix.ci] apply automated fixes

* Update Makefile

Co-authored-by: Gabriel Luiz Freitas Almeida <[email protected]>

* delete update_dependencies

* fix lint

* Remove Poetry lock check from GitHub Actions workflow

* Switch to 'uv' for dependency management and caching in style-check workflow

* Update style-check workflow to use '--only-dev' flag for Ruff check

* Integrate 'uv' package setup and caching in GitHub Actions workflows

* Update version check in GitHub Actions to use 'uv tree' for langflow-base

* Remove redundant poetry environment setup in GitHub Actions workflow

* Add step to minimize uv cache in GitHub Actions workflow

* Update GitHub Actions workflow to use 'uv' for dependency management and caching

* Remove redundant script execution from build_langflow target in Makefile

* [autofix.ci] apply automated fixes

* Switch build system from Poetry to Hatchling and update dependencies

- Replace `poetry-core` with `hatchling` in build-system requirements
- Update `langflow-base` dependency to version `0.0.96`
- Add `tool.hatch.build.targets.wheel` configuration
- Adjust `tool.uv.sources` paths for `langflow-frontend` and `langflow-base`

* update lock

* Switch build system from Poetry to Hatchling in pyproject.toml

* Add langchain-unstructured dependency to pyproject.toml

---------

Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: Gabriel Luiz Freitas Almeida <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ignore-for-release lgtm This PR has been approved by a maintainer python Pull requests that update Python code size:XL This PR changes 500-999 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants