Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: update docker images to use uv #3916

Merged
merged 14 commits into from
Sep 30, 2024
Merged
Show file tree
Hide file tree
Changes from 8 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
103 changes: 45 additions & 58 deletions docker/build_and_push.Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -9,91 +9,78 @@

# 1. use python:3.12.3-slim as the base image until https://github.com/pydantic/pydantic-core/issues/1292 gets resolved
# 2. do not add --platform=$BUILDPLATFORM because the pydantic binaries must be resolved for the final architecture
FROM python:3.12.3-slim as builder-base

ENV PYTHONDONTWRITEBYTECODE=1 \
\
# pip
PIP_DISABLE_PIP_VERSION_CHECK=on \
PIP_DEFAULT_TIMEOUT=100 \
\
# poetry
# https://python-poetry.org/docs/configuration/#using-environment-variables
POETRY_VERSION=1.8.2 \
# make poetry install to this location
POETRY_HOME="/opt/poetry" \
# make poetry create the virtual environment in the project's root
# it gets named `.venv`
POETRY_VIRTUALENVS_IN_PROJECT=true \
# do not ask any interactive question
POETRY_NO_INTERACTION=1 \
\
# paths
# this is where our requirements + virtual environment will live
PYSETUP_PATH="/opt/pysetup" \
VENV_PATH="/opt/pysetup/.venv"
# Use a Python image with uv pre-installed
FROM ghcr.io/astral-sh/uv:python3.12-bookworm-slim AS builder

# Install the project into `/app`
WORKDIR /app

# Enable bytecode compilation
ENV UV_COMPILE_BYTECODE=1

# Copy from the cache instead of linking since it's a mounted volume
ENV UV_LINK_MODE=copy

RUN apt-get update \
&& apt-get install --no-install-recommends -y \
# deps for installing poetry
curl \
# deps for building python deps
build-essential npm \
build-essential \
# npm
npm \
# gcc
gcc \
&& apt-get clean \
&& rm -rf /var/lib/apt/lists/*

RUN --mount=type=cache,target=/root/.cache \
curl -sSL https://install.python-poetry.org | python3 -
RUN --mount=type=cache,target=/root/.cache/uv \
--mount=type=bind,source=uv.lock,target=uv.lock \
--mount=type=bind,source=README.md,target=README.md \
--mount=type=bind,source=pyproject.toml,target=pyproject.toml \
--mount=type=bind,source=src/backend/base/README.md,target=src/backend/base/README.md \
--mount=type=bind,source=src/backend/base/uv.lock,target=src/backend/base/uv.lock \
--mount=type=bind,source=src/backend/base/pyproject.toml,target=src/backend/base/pyproject.toml \
uv sync --frozen --no-install-project --no-dev

ADD ./src /app/src

COPY src/frontend /tmp/src/frontend
WORKDIR /tmp/src/frontend
RUN --mount=type=cache,target=/root/.npm \
npm ci \
&& npm run build \
&& cp -r build /app/src/backend/langflow/frontend \
&& rm -rf /tmp/src/frontend
jordanrfrazier marked this conversation as resolved.
Show resolved Hide resolved

WORKDIR /app
COPY pyproject.toml poetry.lock README.md ./
COPY src/ ./src
COPY scripts/ ./scripts
RUN python -m pip install requests --user && cd ./scripts && python update_dependencies.py
jordanrfrazier marked this conversation as resolved.
Show resolved Hide resolved

# 1. Install the dependencies using the current poetry.lock file to create reproducible builds
# 2. Do not install dev dependencies
# 3. Install all the extras to ensure all optionals are installed as well
# 4. --sync to ensure nothing else is in the environment
# 5. Build the wheel and install "langflow" package (mainly for version)

# Note: moving to build and installing the wheel will make the docker images not reproducible.
RUN $POETRY_HOME/bin/poetry lock --no-update \
# install current lock file with fixed dependencies versions \
# do not install dev dependencies \
&& $POETRY_HOME/bin/poetry install --without dev --sync -E deploy -E couchbase -E cassio \
&& $POETRY_HOME/bin/poetry build -f wheel \
&& $POETRY_HOME/bin/poetry run pip install dist/*.whl
ADD ./pyproject.toml /app/pyproject.toml
ADD ./uv.lock /app/uv.lock


RUN --mount=type=cache,target=/root/.cache/uv \
uv sync --frozen --no-dev
FROM python:3.12.3-slim AS runtime
ogabrielluiz marked this conversation as resolved.
Show resolved Hide resolved

COPY --from=builder /app/.venv /app/.venv
# Place executables in the environment at the front of the path
ENV PATH="/app/.venv/bin:$PATH"
# Reset the entrypoint, don't invoke `uv`
ENTRYPOINT []
jordanrfrazier marked this conversation as resolved.
Show resolved Hide resolved
################################
# RUNTIME
# Setup user, utilities and copy the virtual environment only
################################
# 1. use python:3.12.3-slim as the base image until https://github.com/pydantic/pydantic-core/issues/1292 gets resolved
FROM python:3.12.3-slim as runtime

RUN apt-get -y update \
&& apt-get install --no-install-recommends -y \
curl \
&& apt-get clean \
&& rm -rf /var/lib/apt/lists/*

LABEL org.opencontainers.image.title=langflow
LABEL org.opencontainers.image.authors=['Langflow']
LABEL org.opencontainers.image.licenses=MIT
LABEL org.opencontainers.image.url=https://github.com/langflow-ai/langflow
LABEL org.opencontainers.image.source=https://github.com/langflow-ai/langflow

RUN useradd user -u 1000 -g 0 --no-create-home --home-dir /app/data
COPY --from=builder-base --chown=1000 /app/.venv /app/.venv
ENV PATH="/app/.venv/bin:${PATH}"

USER user
WORKDIR /app

ENV LANGFLOW_HOST=0.0.0.0
ENV LANGFLOW_PORT=7860

CMD ["python", "-m", "langflow", "run"]
CMD ["langflow", "run"]
116 changes: 43 additions & 73 deletions docker/build_and_push_base.Dockerfile
Original file line number Diff line number Diff line change
@@ -1,54 +1,26 @@


# syntax=docker/dockerfile:1
# Keep this syntax directive! It's used to enable Docker BuildKit

# Based on https://github.com/python-poetry/poetry/discussions/1879?sort=top#discussioncomment-216865
# but I try to keep it updated (see history)

################################
# PYTHON-BASE
# Sets up all our shared environment variables
# BUILDER-BASE
# Used to build deps + create our virtual environment
################################

# use python:3.12.3-slim as the base image until https://github.com/pydantic/pydantic-core/issues/1292 gets resolved
FROM python:3.12.3-slim as python-base

# python
ENV PYTHONUNBUFFERED=1 \
# prevents python creating .pyc files
PYTHONDONTWRITEBYTECODE=1 \
\
# pip
PIP_DISABLE_PIP_VERSION_CHECK=on \
PIP_DEFAULT_TIMEOUT=100 \
\
# poetry
# https://python-poetry.org/docs/configuration/#using-environment-variables
POETRY_VERSION=1.8.2 \
# make poetry install to this location
POETRY_HOME="/opt/poetry" \
# make poetry create the virtual environment in the project's root
# it gets named `.venv`
POETRY_VIRTUALENVS_IN_PROJECT=true \
# do not ask any interactive question
POETRY_NO_INTERACTION=1 \
\
# paths
# this is where our requirements + virtual environment will live
PYSETUP_PATH="/opt/pysetup" \
VENV_PATH="/opt/pysetup/.venv"
# 1. use python:3.12.3-slim as the base image until https://github.com/pydantic/pydantic-core/issues/1292 gets resolved
# 2. do not add --platform=$BUILDPLATFORM because the pydantic binaries must be resolved for the final architecture
# Use a Python image with uv pre-installed
FROM ghcr.io/astral-sh/uv:python3.12-bookworm-slim

# Install the project into `/app`
WORKDIR /app

# prepend poetry and venv to path
ENV PATH="$POETRY_HOME/bin:$VENV_PATH/bin:$PATH"
# Enable bytecode compilation
ENV UV_COMPILE_BYTECODE=1

# Copy from the cache instead of linking since it's a mounted volume
ENV UV_LINK_MODE=copy

################################
# BUILDER-BASE
# Used to build deps + create our virtual environment
################################
FROM python-base as builder-base
RUN apt-get update \
&& apt-get install --no-install-recommends -y \
# deps for installing poetry
Expand All @@ -61,41 +33,39 @@ RUN apt-get update \
gcc \
&& apt-get clean \
&& rm -rf /var/lib/apt/lists/*
# Install the project's dependencies using the lockfile and settings
RUN --mount=type=cache,target=/root/.cache/uv \
--mount=type=bind,source=src/backend/base/README.md,target=src/backend/base/README.md \
--mount=type=bind,source=src/backend/base/uv.lock,target=src/backend/base/uv.lock \
--mount=type=bind,source=src/backend/base/pyproject.toml,target=src/backend/base/pyproject.toml \
cd src/backend/base && uv sync --frozen --no-install-project --no-dev

ADD ./src /app/src

COPY src/frontend /tmp/src/frontend
WORKDIR /tmp/src/frontend
RUN npm install \
&& npm run build \
&& cp -r build /app/src/backend/base/langflow/frontend \
&& rm -rf /tmp/src/frontend

WORKDIR /app/src/backend/base
RUN --mount=type=cache,target=/root/.cache/uv \
uv sync --frozen --no-dev

# Place executables in the environment at the front of the path
ENV PATH="/app/src/backend/base/.venv/bin:$PATH"
# Reset the entrypoint, don't invoke `uv`
ENTRYPOINT []

LABEL org.opencontainers.image.title=langflow
LABEL org.opencontainers.image.authors=['Langflow']
LABEL org.opencontainers.image.licenses=MIT
LABEL org.opencontainers.image.url=https://github.com/langflow-ai/langflow
LABEL org.opencontainers.image.source=https://github.com/langflow-ai/langflow

RUN --mount=type=cache,target=/root/.cache \
curl -sSL https://install.python-poetry.org | python3 -

# Now we need to copy the entire project into the image
COPY pyproject.toml poetry.lock ./
COPY src/frontend/package.json /tmp/package.json
RUN cd /tmp && npm install
WORKDIR /app
COPY src/frontend ./src/frontend
RUN rm -rf src/frontend/node_modules
RUN cp -a /tmp/node_modules /app/src/frontend
COPY scripts ./scripts
COPY Makefile ./
COPY README.md ./
RUN cd src/frontend && npm run build
COPY src/backend ./src/backend
RUN cp -r src/frontend/build src/backend/base/langflow/frontend
RUN rm -rf src/backend/base/dist
RUN useradd -m -u 1000 user && \
mkdir -p /app/langflow && \
chown -R user:user /app && \
chmod -R u+w /app/langflow

# Update PATH with home/user/.local/bin
ENV PATH="/home/user/.local/bin:${PATH}"
RUN cd src/backend/base && $POETRY_HOME/bin/poetry build

# Copy virtual environment and built .tar.gz from builder base

USER user
# Install the package from the .tar.gz
RUN python -m pip install /app/src/backend/base/dist/*.tar.gz --user

ENV LANGFLOW_HOST=0.0.0.0
ENV LANGFLOW_PORT=7860

CMD ["python", "-m", "langflow", "run"]
CMD ["langflow-base", "run"]
14 changes: 7 additions & 7 deletions docker/dev.Dockerfile
Original file line number Diff line number Diff line change
@@ -1,20 +1,20 @@
FROM python:3.12-bookworm
FROM ghcr.io/astral-sh/uv:python3.12-bookworm-slim
ENV TZ=UTC

WORKDIR /app

RUN apt update -y
RUN apt install \
RUN apt-get update && apt-get install -y \
build-essential \
curl \
npm \
-y
&& apt-get clean \
&& rm -rf /var/lib/apt/lists/*

COPY . /app

RUN pip install poetry
RUN poetry config virtualenvs.create false
RUN poetry install --no-interaction --no-ansi
# Install dependencies using uv
RUN --mount=type=cache,target=/root/.cache/uv \
uv pip install --system -r requirements.txt

jordanrfrazier marked this conversation as resolved.
Show resolved Hide resolved
EXPOSE 7860
EXPOSE 3000
Expand Down
34 changes: 34 additions & 0 deletions uv.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading