chore: update docker images to use uv (langflow-ai#3916)

* fix dev dependencies * fix dev dependencies * update lock * Add langchain-unstructured dependency to pyproject.toml * Switch to uv-based Docker image and streamline build process - Use `ghcr.io/astral-sh/uv:python3.12-bookworm-slim` as the base image - Simplify environment variable setup and remove unnecessary ones - Install project dependencies using `uv sync` with cache mounts - Add and copy necessary files for the build process - Update npm installation and frontend build steps - Adjust entrypoint and runtime configurations - Remove redundant user setup and streamline CMD execution * Refactor Dockerfile to use multi-stage builds for optimized image size - Introduce a builder stage to install dependencies and build the frontend. - Use npm ci instead of npm install for more reliable builds. - Copy built frontend assets to the final image. - Transition to a runtime stage with a slimmer Python base image. - Ensure the final image only contains necessary runtime dependencies. * Switch base image to `ghcr.io/astral-sh/uv:python3.12-bookworm-slim` and update Dockerfile - Replace `python:3.12.3-slim` with `ghcr.io/astral-sh/uv:python3.12-bookworm-slim` as the base image. - Enable bytecode compilation and set link mode to copy. - Update dependency installation process using `uv sync`. - Simplify frontend build and copy process. - Add OCI labels for image metadata. - Update CMD to use `langflow-base` for running the application. * Update Dockerfile to use uv base image and optimize dependency installation - Switch base image to `ghcr.io/astral-sh/uv:python3.12-bookworm-slim` - Combine apt-get commands and clean up to reduce image size - Replace Poetry with uv for dependency management and caching * Refactor Dockerfile to use multi-stage build for optimized image size - Introduce a builder stage using `ghcr.io/astral-sh/uv:python3.12-bookworm-slim` - Add a runtime stage using `python:3.12.3-slim` - Copy the virtual environment from the builder stage to the runtime stage - Remove the `uv` entrypoint to avoid invoking it by default * Update Dockerfile: Fix indentation and remove redundant ENTRYPOINT reset * ci: update release workflows to use uv (langflow-ai#3919) * Update nightly build workflow to use 'uv' for dependency management and caching - Replace Poetry with 'uv' for dependency installation and project setup - Add steps to install 'uv' and set up caching for 'uv.lock' - Modify Python setup to use version specified in 'pyproject.toml' - Remove Node.js setup steps * Update release workflow to use uv for dependency management - Replace Poetry with uv for dependency installation and caching - Update Python setup to use version specified in pyproject.toml - Add steps to restore uv cache and install project dependencies using uv - Adjust publish steps to maintain functionality with new setup * Replace 'poetry publish' with 'uv publish' in Makefile for consistency * Update nightly build workflow to use 'uv' for dependency management and caching * Set up Node.js 20 in release_nightly workflow and update version verification logic * Update Makefile to use `uv sync --frozen` for backend dependencies installation * Update Makefile to pass arguments to 'uv build' and remove '--skip-existing' from publish commands - Modified 'build_langflow' target to accept arguments. - Removed '--skip-existing' from 'publish_base' and 'publish_base_testpypi' commands. - Added TODO comments for updating test-pypi repository usage. * Remove --skip-existing flag from uv publish commands and add TODO comments for test-pypi updates * new lock * Update CI workflow to remove version prefix and add build args - Removed 'v' prefix from version extraction in nightly release workflow. - Added '--no-sources' argument to the build command in the distribution step. * Update CI workflow to use 'uv' for versioning and publishing - Replace 'poetry' with 'uv' for version extraction in release checks * Update CI workflow to use UV_PUBLISH_TOKEN for PyPI publishing * Add verification step for 'langflow-nightly' name and version in CI workflow - Corrected awk commands for extracting 'langflow-base' name and version. - Added a new step to verify 'langflow-nightly' name and version against expected values. * feat: Update dev.Dockerfile to use 'uv' for dependency management and caching * update dockerfiles with --no-editable --------- Co-authored-by: phact <[email protected]> Co-authored-by: Jordan Frazier <[email protected]>
headlinevc · Nov 26, 2024 · 57ebf2e · 57ebf2e
1 parent f9bbac1
commit 57ebf2e
Show file tree

Hide file tree

Showing 8 changed files with 234 additions and 242 deletions.
diff --git a/.github/workflows/nightly_build.yml b/.github/workflows/nightly_build.yml
@@ -29,23 +29,31 @@ jobs:
         with:
           persist-credentials: true
 
-      - name: Set up Python ${{ env.PYTHON_VERSION }} + Poetry ${{ env.POETRY_VERSION }}
-        uses: "./.github/actions/poetry_caching"
+      - name: Install uv
+        uses: astral-sh/setup-uv@v3
         with:
-          python-version: ${{ env.PYTHON_VERSION }}
-          poetry-version: ${{ env.POETRY_VERSION }}
-          cache-key: ${{ runner.os }}-poetry-${{ env.POETRY_VERSION }}-${{ hashFiles('**/poetry.lock') }}
-
-      - name: Install dependencies
-        run: |
-          poetry env use ${{ env.PYTHON_VERSION }}
-          poetry install
+          enable-cache: true
+          cache-dependency-glob: "uv.lock"
+      - name: "Set up Python"
+        uses: actions/setup-python@v5
+        with:
+          python-version-file: "pyproject.toml"
+      - name: Restore uv cache
+        uses: actions/cache@v4
+        with:
+          path: /tmp/.uv-cache
+          key: uv-${{ runner.os }}-${{ hashFiles('uv.lock') }}
+          restore-keys: |
+            uv-${{ runner.os }}-${{ hashFiles('uv.lock') }}
+            uv-${{ runner.os }}
+      - name: Install the project
+        run: uv sync --dev
 
       - name: Generate main nightly tag
         id: generate_main_tag
         run: |
           # NOTE: This outputs the tag with the `v` prefix.
-          MAIN_TAG="$(poetry run python ./scripts/ci/pypi_nightly_tag.py main)"
+          MAIN_TAG="$(uv run python ./scripts/ci/pypi_nightly_tag.py main)"
           echo "main_tag=$MAIN_TAG" >> $GITHUB_OUTPUT
           echo "main_tag=$MAIN_TAG"
 
@@ -64,7 +72,7 @@ jobs:
         if: ${{ steps.check_main_tag.outputs.main_tag_exists == 'false' }}
         run: |
           # NOTE: This outputs the tag with the `v` prefix.
-          BASE_TAG="$(poetry run python ./scripts/ci/pypi_nightly_tag.py base)"
+          BASE_TAG="$(uv run python ./scripts/ci/pypi_nightly_tag.py base)"
           echo "base_tag=$BASE_TAG" >> $GITHUB_OUTPUT
           echo "base_tag=$BASE_TAG"
 
@@ -82,17 +90,17 @@ jobs:
           # project-name is updated, which does not have dependencies installed.
           BASE_TAG="${{ steps.generate_base_tag.outputs.base_tag }}"
           echo "Updating base project version to $BASE_TAG"
-          poetry run python ./scripts/ci/update_pyproject_name.py langflow-base-nightly base
-          poetry run python ./scripts/ci/update_pyproject_version.py $BASE_TAG base
+          uv run python ./scripts/ci/update_pyproject_name.py langflow-base-nightly base
+          uv run python ./scripts/ci/update_pyproject_version.py $BASE_TAG base
 
           # This updates the dependency of langflow-base to langflow-base-nightly in {project_root}/pyproject.toml
-          poetry run python ./scripts/ci/update_lf_base_dependency.py $BASE_TAG
+          uv run python ./scripts/ci/update_lf_base_dependency.py $BASE_TAG
 
           # Use the main tag created earlier
           MAIN_TAG="${{ steps.generate_main_tag.outputs.main_tag }}"
           echo "Updating main project version to $MAIN_TAG"
-          poetry run python ./scripts/ci/update_pyproject_version.py $MAIN_TAG main
-          poetry run python ./scripts/ci/update_pyproject_name.py langflow-nightly main
+          uv run python ./scripts/ci/update_pyproject_version.py $MAIN_TAG main
+          uv run python ./scripts/ci/update_pyproject_name.py langflow-nightly main
 
           git add pyproject.toml src/backend/base/pyproject.toml
           git commit -m "Update version and project name"
@@ -122,10 +130,10 @@ jobs:
         working-directory: src/backend/base
         run: |
           # If the main tag already exists, we need to retrieve the base version from the main tag codebase.
-          version=$(poetry version --short)
-          echo "base_tag=v$version" >> $GITHUB_OUTPUT
-          echo "base_tag=v$version"
-      
+          version=$(uv tree | grep 'langflow-base' | awk '{print $3}')
+          echo "base_tag=$version" >> $GITHUB_OUTPUT
+          echo "base_tag=$version"
+
       - name: Set Base Tag
         id: set_base_tag
         run: |

diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml
@@ -57,17 +57,25 @@ jobs:
       skipped: ${{ steps.check-version.outputs.skipped }}
     steps:
       - uses: actions/checkout@v4
-      - name: Install poetry
-        run: pipx install poetry==${{ env.POETRY_VERSION }}
-      - name: Set up Python 3.12
+      - name: Install uv
+        uses: astral-sh/setup-uv@v3
+        with:
+          enable-cache: true
+          cache-dependency-glob: "uv.lock"
+      - name: "Set up Python"
         uses: actions/setup-python@v5
         with:
-          python-version: "3.12"
-          cache: "poetry"
-      - name: Set up Nodejs 20
-        uses: actions/setup-node@v4
+          python-version-file: "pyproject.toml"
+      - name: Restore uv cache
+        uses: actions/cache@v4
         with:
-          node-version: "20"
+          path: /tmp/.uv-cache
+          key: uv-${{ runner.os }}-${{ hashFiles('uv.lock') }}
+          restore-keys: |
+            uv-${{ runner.os }}-${{ hashFiles('uv.lock') }}
+            uv-${{ runner.os }}
+      - name: Install the project
+        run: uv sync --dev
       - name: Check Version
         id: check-version
         run: |
@@ -103,10 +111,10 @@ jobs:
             echo "Server terminated successfully"
           fi
       - name: Publish to PyPI
-        if: steps.check-version.outputs.skipped == 'false'
         env:
           POETRY_PYPI_TOKEN_PYPI: ${{ secrets.PYPI_API_TOKEN }}
-        run: make publish base=true
+        run: |
+          make publish base=true
       - name: Upload Artifact
         if: steps.check-version.outputs.skipped == 'false'
         uses: actions/upload-artifact@v4
@@ -123,23 +131,31 @@ jobs:
       version: ${{ steps.check-version.outputs.version }}
     steps:
       - uses: actions/checkout@v4
-      - name: Install poetry
-        run: pipx install poetry==${{ env.POETRY_VERSION }}
-      - name: Set up Python 3.12
+      - name: Install uv
+        uses: astral-sh/setup-uv@v3
+        with:
+          enable-cache: true
+          cache-dependency-glob: "uv.lock"
+      - name: "Set up Python"
         uses: actions/setup-python@v5
         with:
-          python-version: "3.12"
-          cache: "poetry"
-      - name: Set up Nodejs 20
-        uses: actions/setup-node@v4
+          python-version-file: "pyproject.toml"
+      - name: Restore uv cache
+        uses: actions/cache@v4
         with:
-          node-version: "20"
+          path: /tmp/.uv-cache
+          key: uv-${{ runner.os }}-${{ hashFiles('uv.lock') }}
+          restore-keys: |
+            uv-${{ runner.os }}-${{ hashFiles('uv.lock') }}
+            uv-${{ runner.os }}
+      - name: Install the project
+        run: uv sync --dev
       # If pre-release is true, we need to check if  ["a", "b", "rc", "dev", "post"] is in the version string
       # if the version string is incorrect, we need to exit the workflow
       - name: Check if pre-release
         if: inputs.pre_release == 'true'
         run: |
-          version=$(poetry version --short)
+          version=$(uv tree | grep 'langflow' | awk '{print $2}' | sed 's/^v//')
           if [[ "${version}" =~ ^([0-9]+\.)?([0-9]+\.)?[0-9]+((a|b|rc|dev|post)([0-9]+))$ ]]; then
             echo "Pre-release version detected. Continuing with the release."
           else
@@ -149,7 +165,7 @@ jobs:
       - name: Check Version
         id: check-version
         run: |
-          version=$(poetry version --short)
+          version=$(uv tree | grep 'langflow' | awk '{print $2}' | sed 's/^v//')
           last_released_version=$(curl -s "https://pypi.org/pypi/langflow/json" | jq -r '.releases | keys | .[]' | sort -V | tail -n 1)
           if [ "$version" = "$last_released_version" ]; then
             echo "Version $version is already released. Skipping release."
@@ -182,8 +198,9 @@ jobs:
           fi
       - name: Publish to PyPI
         env:
-          POETRY_PYPI_TOKEN_PYPI: ${{ secrets.PYPI_API_TOKEN }}
-        run: make publish main=true
+          UV_PUBLISH_TOKEN: ${{ secrets.PYPI_API_TOKEN }}
+        run: |
+          make publish main=true
       - name: Upload Artifact
         uses: actions/upload-artifact@v4
         with:

diff --git a/.github/workflows/release_nightly.yml b/.github/workflows/release_nightly.yml
@@ -60,24 +60,35 @@ jobs:
         with:
           ref: ${{ inputs.nightly_tag_main }}
           persist-credentials: true
-      - name: Install poetry
-        run: |
-          pipx install poetry==${{ env.POETRY_VERSION }}
-      - name: Set up Python 3.12
+      - name: Install uv
+        uses: astral-sh/setup-uv@v3
+        with:
+          enable-cache: true
+          cache-dependency-glob: "uv.lock"
+      - name: "Set up Python"
         uses: actions/setup-python@v5
         with:
-          python-version: "3.12"
-          cache: "poetry"
+          python-version-file: "pyproject.toml"
       - name: Set up Nodejs 20
         uses: actions/setup-node@v4
         with:
           node-version: "20"
+      - name: Restore uv cache
+        uses: actions/cache@v4
+        with:
+          path: /tmp/.uv-cache
+          key: uv-${{ runner.os }}-${{ hashFiles('uv.lock') }}
+          restore-keys: |
+            uv-${{ runner.os }}-${{ hashFiles('uv.lock') }}
+            uv-${{ runner.os }}
+      - name: Install the project
+        run: uv sync --dev
 
       - name: Verify Nightly Name and Version
         working-directory: src/backend/base
         run: |
-          name=$(poetry version | cut -d' ' -f1)
-          version=v$(poetry version --short)
+          name=$(uv tree | grep 'langflow-base' | awk '{print $1}')
+          version=$(uv tree | grep 'langflow-base' | awk '{print $2}')
           if [ "$name" != "langflow-base-nightly" ]; then
             echo "Name $name does not match langflow-base-nightly. Exiting the workflow."
             exit 1
@@ -133,35 +144,43 @@ jobs:
         with:
           ref: ${{ inputs.nightly_tag_main }}
 
-      - name: Install poetry
-        run: pipx install poetry==${{ env.POETRY_VERSION }}
-      - name: Set up Python 3.12
+      - name: Install uv
+        uses: astral-sh/setup-uv@v3
+        with:
+          enable-cache: true
+          cache-dependency-glob: "uv.lock"
+      - name: "Set up Python"
         uses: actions/setup-python@v5
         with:
-          python-version: "3.12"
-          cache: "poetry"
-      - name: Set up Nodejs 20
-        uses: actions/setup-node@v4
+          python-version-file: "pyproject.toml"
+      - name: Restore uv cache
+        uses: actions/cache@v4
         with:
-          node-version: "20"
+          path: /tmp/.uv-cache
+          key: uv-${{ runner.os }}-${{ hashFiles('uv.lock') }}
+          restore-keys: |
+            uv-${{ runner.os }}-${{ hashFiles('uv.lock') }}
+            uv-${{ runner.os }}
+      - name: Install the project
+        run: uv sync --dev
+
       - name: Verify Nightly Name and Version
         run: |
-          name=$(poetry version | cut -d' ' -f1)
-          version=v$(poetry version --short)
+          name=$(uv tree | grep 'langflow' | awk '{print $1}')
+          version=$(uv tree | grep 'langflow' | awk '{print $2}')
           if [ "$name" != "langflow-nightly" ]; then
             echo "Name $name does not match langflow-nightly. Exiting the workflow."
             exit 1
           fi
-          if [ "$version" != "${{ inputs.nightly_tag_main }}" ]; then
-            echo "Version $version does not match nightly tag ${{ inputs.nightly_tag_main }}. Exiting the workflow."
+          if [ "$version" != "${{ inputs.nightly_tag_base }}" ]; then
+            echo "Version $version does not match nightly tag ${{ inputs.nightly_tag_base }}. Exiting the workflow."
             exit 1
           fi
-
       - name: Wait for PyPI Propagation
         run: sleep 300 # wait for 5 minutes to ensure PyPI propagation of base
 
       - name: Build project for distribution
-        run: make build main=true
+        run: make build main=true args="--no-sources"
       - name: Test CLI
         run: |
           python -m pip install dist/*.whl

diff --git a/Makefile b/Makefile
@@ -322,7 +322,7 @@ ifdef main
 	make install_frontendci
 	make build_frontend
 	make build_langflow_base
-	make build_langflow
+	make build_langflow args="$(args)"
 endif
 
 build_langflow_base:
@@ -334,7 +334,7 @@ build_langflow_backup:
 
 build_langflow:
 	uv lock --no-upgrade
-	uv build
+	uv build $(args)
 ifdef restore
 	mv pyproject.toml.bak pyproject.toml
 	mv uv.lock.bak uv.lock
@@ -403,20 +403,18 @@ update: ## update dependencies
 	uv sync --upgrade
 
 publish_base:
-	#TODO: replace with uvx twine upload dist/*
-	cd src/backend/base && poetry publish --skip-existing
+	cd src/backend/base && uv publish
 
 publish_langflow:
-	#TODO: replace with uvx twine upload dist/*
-	poetry publish
+	uv publish
 
 publish_base_testpypi:
-	#TODO: replace with uvx twine upload dist/*
-	cd src/backend/base && poetry publish --skip-existing -r test-pypi
+	# TODO: update this to use the test-pypi repository
+	cd src/backend/base && uv publish -r test-pypi
 
 publish_langflow_testpypi:
-	#TODO: replace with uvx twine upload dist/*
-	poetry publish -r test-pypi
+	# TODO: update this to use the test-pypi repository
+	uv publish -r test-pypi
 
 publish: ## build the frontend static files and package the project and publish it to PyPI
 	@echo 'Publishing the project'