Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ci: update release workflows to use uv #3919

Merged
merged 13 commits into from
Sep 27, 2024
Merged
Show file tree
Hide file tree
Changes from 5 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
50 changes: 29 additions & 21 deletions .github/workflows/nightly_build.yml
Original file line number Diff line number Diff line change
Expand Up @@ -29,23 +29,31 @@ jobs:
with:
persist-credentials: true

- name: Set up Python ${{ env.PYTHON_VERSION }} + Poetry ${{ env.POETRY_VERSION }}
uses: "./.github/actions/poetry_caching"
- name: Install uv
uses: astral-sh/setup-uv@v3
with:
python-version: ${{ env.PYTHON_VERSION }}
poetry-version: ${{ env.POETRY_VERSION }}
cache-key: ${{ runner.os }}-poetry-${{ env.POETRY_VERSION }}-${{ hashFiles('**/poetry.lock') }}

- name: Install dependencies
run: |
poetry env use ${{ env.PYTHON_VERSION }}
poetry install
enable-cache: true
cache-dependency-glob: "uv.lock"
- name: "Set up Python"
uses: actions/setup-python@v5
with:
python-version-file: "pyproject.toml"
- name: Restore uv cache
uses: actions/cache@v4
with:
path: /tmp/.uv-cache
key: uv-${{ runner.os }}-${{ hashFiles('uv.lock') }}
restore-keys: |
uv-${{ runner.os }}-${{ hashFiles('uv.lock') }}
uv-${{ runner.os }}
- name: Install the project
run: uv sync --dev

- name: Generate main nightly tag
id: generate_main_tag
run: |
# NOTE: This outputs the tag with the `v` prefix.
MAIN_TAG="$(poetry run python ./scripts/ci/pypi_nightly_tag.py main)"
MAIN_TAG="$(uv run python ./scripts/ci/pypi_nightly_tag.py main)"
echo "main_tag=$MAIN_TAG" >> $GITHUB_OUTPUT
echo "main_tag=$MAIN_TAG"

Expand All @@ -64,7 +72,7 @@ jobs:
if: ${{ steps.check_main_tag.outputs.main_tag_exists == 'false' }}
run: |
# NOTE: This outputs the tag with the `v` prefix.
BASE_TAG="$(poetry run python ./scripts/ci/pypi_nightly_tag.py base)"
BASE_TAG="$(uv run python ./scripts/ci/pypi_nightly_tag.py base)"
echo "base_tag=$BASE_TAG" >> $GITHUB_OUTPUT
echo "base_tag=$BASE_TAG"

Expand All @@ -82,17 +90,17 @@ jobs:
# project-name is updated, which does not have dependencies installed.
BASE_TAG="${{ steps.generate_base_tag.outputs.base_tag }}"
echo "Updating base project version to $BASE_TAG"
poetry run python ./scripts/ci/update_pyproject_name.py langflow-base-nightly base
poetry run python ./scripts/ci/update_pyproject_version.py $BASE_TAG base
uv run python ./scripts/ci/update_pyproject_name.py langflow-base-nightly base
uv run python ./scripts/ci/update_pyproject_version.py $BASE_TAG base

# This updates the dependency of langflow-base to langflow-base-nightly in {project_root}/pyproject.toml
poetry run python ./scripts/ci/update_lf_base_dependency.py $BASE_TAG
uv run python ./scripts/ci/update_lf_base_dependency.py $BASE_TAG

# Use the main tag created earlier
MAIN_TAG="${{ steps.generate_main_tag.outputs.main_tag }}"
echo "Updating main project version to $MAIN_TAG"
poetry run python ./scripts/ci/update_pyproject_version.py $MAIN_TAG main
poetry run python ./scripts/ci/update_pyproject_name.py langflow-nightly main
uv run python ./scripts/ci/update_pyproject_version.py $MAIN_TAG main
uv run python ./scripts/ci/update_pyproject_name.py langflow-nightly main

git add pyproject.toml src/backend/base/pyproject.toml
git commit -m "Update version and project name"
Expand Down Expand Up @@ -122,10 +130,10 @@ jobs:
working-directory: src/backend/base
run: |
# If the main tag already exists, we need to retrieve the base version from the main tag codebase.
version=$(poetry version --short)
echo "base_tag=v$version" >> $GITHUB_OUTPUT
echo "base_tag=v$version"
version=$(uv tree | grep 'langflow-base' | awk '{print $3}')
echo "base_tag=$version" >> $GITHUB_OUTPUT
echo "base_tag=$version"

- name: Set Base Tag
id: set_base_tag
run: |
Expand Down
55 changes: 36 additions & 19 deletions .github/workflows/release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -57,17 +57,25 @@ jobs:
skipped: ${{ steps.check-version.outputs.skipped }}
steps:
- uses: actions/checkout@v4
- name: Install poetry
run: pipx install poetry==${{ env.POETRY_VERSION }}
- name: Set up Python 3.12
- name: Install uv
uses: astral-sh/setup-uv@v3
with:
enable-cache: true
cache-dependency-glob: "uv.lock"
- name: "Set up Python"
uses: actions/setup-python@v5
with:
python-version: "3.12"
cache: "poetry"
- name: Set up Nodejs 20
uses: actions/setup-node@v4
python-version-file: "pyproject.toml"
- name: Restore uv cache
uses: actions/cache@v4
with:
node-version: "20"
path: /tmp/.uv-cache
key: uv-${{ runner.os }}-${{ hashFiles('uv.lock') }}
restore-keys: |
uv-${{ runner.os }}-${{ hashFiles('uv.lock') }}
uv-${{ runner.os }}
- name: Install the project
run: uv sync --dev
- name: Check Version
id: check-version
run: |
Expand Down Expand Up @@ -103,10 +111,10 @@ jobs:
echo "Server terminated successfully"
fi
- name: Publish to PyPI
if: steps.check-version.outputs.skipped == 'false'
env:
POETRY_PYPI_TOKEN_PYPI: ${{ secrets.PYPI_API_TOKEN }}
run: make publish base=true
run: |
make publish base=true
- name: Upload Artifact
if: steps.check-version.outputs.skipped == 'false'
uses: actions/upload-artifact@v4
Expand All @@ -123,17 +131,25 @@ jobs:
version: ${{ steps.check-version.outputs.version }}
steps:
- uses: actions/checkout@v4
- name: Install poetry
run: pipx install poetry==${{ env.POETRY_VERSION }}
- name: Set up Python 3.12
- name: Install uv
uses: astral-sh/setup-uv@v3
with:
enable-cache: true
cache-dependency-glob: "uv.lock"
- name: "Set up Python"
uses: actions/setup-python@v5
with:
python-version: "3.12"
cache: "poetry"
- name: Set up Nodejs 20
uses: actions/setup-node@v4
python-version-file: "pyproject.toml"
- name: Restore uv cache
uses: actions/cache@v4
with:
node-version: "20"
path: /tmp/.uv-cache
key: uv-${{ runner.os }}-${{ hashFiles('uv.lock') }}
restore-keys: |
uv-${{ runner.os }}-${{ hashFiles('uv.lock') }}
uv-${{ runner.os }}
- name: Install the project
run: uv sync --dev
# If pre-release is true, we need to check if ["a", "b", "rc", "dev", "post"] is in the version string
# if the version string is incorrect, we need to exit the workflow
- name: Check if pre-release
ogabrielluiz marked this conversation as resolved.
Show resolved Hide resolved
Expand Down Expand Up @@ -183,7 +199,8 @@ jobs:
- name: Publish to PyPI
env:
POETRY_PYPI_TOKEN_PYPI: ${{ secrets.PYPI_API_TOKEN }}
run: make publish main=true
run: |
make publish main=true
ogabrielluiz marked this conversation as resolved.
Show resolved Hide resolved
- name: Upload Artifact
uses: actions/upload-artifact@v4
with:
Expand Down
63 changes: 35 additions & 28 deletions .github/workflows/release_nightly.yml
Original file line number Diff line number Diff line change
Expand Up @@ -60,24 +60,35 @@ jobs:
with:
ref: ${{ inputs.nightly_tag_main }}
persist-credentials: true
- name: Install poetry
run: |
pipx install poetry==${{ env.POETRY_VERSION }}
- name: Set up Python 3.12
- name: Install uv
uses: astral-sh/setup-uv@v3
with:
enable-cache: true
cache-dependency-glob: "uv.lock"
- name: "Set up Python"
uses: actions/setup-python@v5
with:
python-version: "3.12"
cache: "poetry"
python-version-file: "pyproject.toml"
- name: Set up Nodejs 20
uses: actions/setup-node@v4
with:
node-version: "20"
- name: Restore uv cache
uses: actions/cache@v4
with:
path: /tmp/.uv-cache
key: uv-${{ runner.os }}-${{ hashFiles('uv.lock') }}
restore-keys: |
uv-${{ runner.os }}-${{ hashFiles('uv.lock') }}
uv-${{ runner.os }}
- name: Install the project
run: uv sync --dev

- name: Verify Nightly Name and Version
working-directory: src/backend/base
run: |
name=$(poetry version | cut -d' ' -f1)
version=v$(poetry version --short)
name=$(uv tree | grep 'langflow-base' | awk '{print $2}')
version=$(uv tree | grep 'langflow-base' | awk '{print $3}' | sed 's/^v//')
ogabrielluiz marked this conversation as resolved.
Show resolved Hide resolved
if [ "$name" != "langflow-base-nightly" ]; then
echo "Name $name does not match langflow-base-nightly. Exiting the workflow."
exit 1
Expand Down Expand Up @@ -133,29 +144,25 @@ jobs:
with:
ref: ${{ inputs.nightly_tag_main }}

- name: Install poetry
run: pipx install poetry==${{ env.POETRY_VERSION }}
- name: Set up Python 3.12
- name: Install uv
uses: astral-sh/setup-uv@v3
with:
enable-cache: true
cache-dependency-glob: "uv.lock"
- name: "Set up Python"
uses: actions/setup-python@v5
with:
python-version: "3.12"
cache: "poetry"
- name: Set up Nodejs 20
uses: actions/setup-node@v4
python-version-file: "pyproject.toml"
- name: Restore uv cache
uses: actions/cache@v4
with:
node-version: "20"
- name: Verify Nightly Name and Version
ogabrielluiz marked this conversation as resolved.
Show resolved Hide resolved
run: |
name=$(poetry version | cut -d' ' -f1)
version=v$(poetry version --short)
if [ "$name" != "langflow-nightly" ]; then
echo "Name $name does not match langflow-nightly. Exiting the workflow."
exit 1
fi
if [ "$version" != "${{ inputs.nightly_tag_main }}" ]; then
echo "Version $version does not match nightly tag ${{ inputs.nightly_tag_main }}. Exiting the workflow."
exit 1
fi
path: /tmp/.uv-cache
key: uv-${{ runner.os }}-${{ hashFiles('uv.lock') }}
restore-keys: |
uv-${{ runner.os }}-${{ hashFiles('uv.lock') }}
uv-${{ runner.os }}
- name: Install the project
run: uv sync --dev

- name: Wait for PyPI Propagation
run: sleep 300 # wait for 5 minutes to ensure PyPI propagation of base
Expand Down
12 changes: 4 additions & 8 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -416,20 +416,16 @@ update: ## update dependencies
uv sync --upgrade

publish_base:
#TODO: replace with uvx twine upload dist/*
cd src/backend/base && poetry publish --skip-existing
cd src/backend/base && uv publish --skip-existing

publish_langflow:
#TODO: replace with uvx twine upload dist/*
poetry publish
uv publish

publish_base_testpypi:
#TODO: replace with uvx twine upload dist/*
cd src/backend/base && poetry publish --skip-existing -r test-pypi
cd src/backend/base && uv publish --skip-existing -r test-pypi

publish_langflow_testpypi:
#TODO: replace with uvx twine upload dist/*
poetry publish -r test-pypi
uv publish -r test-pypi
ogabrielluiz marked this conversation as resolved.
Show resolved Hide resolved

publish: ## build the frontend static files and package the project and publish it to PyPI
@echo 'Publishing the project'
Expand Down
Loading