Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: actualiza a la nueva versión de uv #47

Merged
merged 18 commits into from
Sep 3, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .dockerignore
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
.venv
17 changes: 14 additions & 3 deletions .github/workflows/pre-commit-main.yml
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,18 @@ jobs:
with:
# required to grab the history of the PR
fetch-depth: 0
- uses: actions/setup-python@v5
with:
python-version: "3.11"
# Not needed since uv installs python automatically
# - uses: actions/setup-python@v5
# with:
# python-version-file: ".python-version"
- name: Set up uv
run: curl -LsSf https://astral.sh/uv/0.4.3/install.sh | sh
# Needed for mypy, since the pre-commit hook is run in "system" mode
- name: Install dependencies
run: |
uv sync --dev --frozen
# Workaround to allow pre-commit to find the venv where dependencies are installed
- run: |
echo "$PWD/.venv/bin" >> $GITHUB_PATH
uv pip install pip
- uses: pre-commit/[email protected]
16 changes: 13 additions & 3 deletions .github/workflows/pre-commit-pr.yml
Original file line number Diff line number Diff line change
Expand Up @@ -15,9 +15,19 @@ jobs:
with:
# required to grab the history of the PR
fetch-depth: 0
- uses: actions/setup-python@v5
with:
python-version: "3.11"
# Not needed since uv installs python automatically
# - uses: actions/setup-python@v5
# with:
# python-version-file: ".python-version"
- name: Set up uv
run: curl -LsSf https://astral.sh/uv/0.4.3/install.sh | sh
# Needed for mypy, since the pre-commit hook is run in "system" mode
- name: Install dependencies
run: uv sync --dev --frozen
# Workaround to allow pre-commit to find the venv where dependencies are installed
- run: |
echo "$PWD/.venv/bin" >> $GITHUB_PATH
uv pip install pip
- uses: pre-commit/[email protected]
with:
# Ejecutar solo en ficheros que hayan cambiado https://github.com/pre-commit/action/issues/7
Expand Down
16 changes: 4 additions & 12 deletions .github/workflows/pytest.yml
Original file line number Diff line number Diff line change
Expand Up @@ -27,17 +27,9 @@ jobs:
- uses: actions/setup-python@v5
with:
python-version: ${{ matrix.python-version }}
- name: Set up uv
run: curl -LsSf https://astral.sh/uv/0.4.3/install.sh | sh
- name: Test with pytest
run: |
pip install uv
uv pip install --system -r requirements-dev.txt
uv pip install --system pytest-cov
uv pip install --system -e .
pytest --cov=./ --cov-report=xml
- name: Upload coverage to Codecov
uses: codecov/codecov-action@v4
with:
token: ${{ secrets.CODECOV_TOKEN }}
name: codecov-umbrella
fail_ci_if_error: false
verbose: true
uv sync --dev --frozen
uv run pytest
33 changes: 6 additions & 27 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
@@ -1,14 +1,11 @@
# See https://pre-commit.com for more information
# See https://pre-commit.com/hooks.html for more hooks
default_language_version:
python: python3.11

ci:
autoupdate_commit_msg: "chore: update pre-commit hooks"

repos:
- repo: https://github.com/pre-commit/pre-commit-hooks
rev: v4.4.0
rev: v4.6.0
hooks:
- id: check-added-large-files
args: [--maxkb=50000]
Expand All @@ -32,50 +29,32 @@ repos:
hooks:
- id: python-use-type-annotations
- repo: https://github.com/astral-sh/ruff-pre-commit
rev: v0.1.0
rev: v0.6.1
hooks:
- id: ruff
types_or: [python, pyi, jupyter]
args: [--fix, --show-fixes, --exit-non-zero-on-fix]
- id: ruff-format
types_or: [python, pyi, jupyter]
- repo: https://github.com/pre-commit/mirrors-mypy
rev: v1.6.1
rev: v1.11.1
hooks:
- id: mypy
language: system
args: ["--install-types", "--non-interactive"]
additional_dependencies: [numpy]
# Esto tiene la desventaja de que siempre se va a ejecutar mypy
# sobre todos los ficheros del repo (incluidos en la conf de mypy)
# Se puede sustituir por `files:`
# Ref: https://www.kianmeng.org/2022/09/excluding-files-with-mypy-hook-for-pre.html
pass_filenames: false
- repo: https://github.com/codespell-project/codespell
rev: v2.2.4
rev: v2.3.0
hooks:
- id: codespell
additional_dependencies:
- tomli
- repo: https://github.com/pre-commit/mirrors-prettier
rev: v3.1.0
rev: v4.0.0-alpha.8
hooks:
- id: prettier
types_or: [markdown, yaml]
- repo: https://github.com/astral-sh/uv-pre-commit
rev: 0.1.28
hooks:
- id: pip-compile
name: pip-compile requirements.in
args: [-o, requirements.txt, requirements.in]
- id: pip-compile
name: pip-compile requirements-dev.in
args:
[
-o,
requirements-dev.txt,
-c,
requirements.txt,
requirements.in,
requirements-dev.in,
]
files: ^(requirements\.(in|txt)|requirements-dev\.(in|txt))$
1 change: 1 addition & 0 deletions .python-version
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
3.11
43 changes: 27 additions & 16 deletions Dockerfile
Original file line number Diff line number Diff line change
@@ -1,24 +1,35 @@
# From https://fastapi.tiangolo.com/deployment/docker/
# From https://docs.astral.sh/uv/guides/integration/docker/
FROM python:3.11-slim

#RUN apt-get -yq update && apt-get -yqq install ssh git
# uv installer requires curl (and certificates) to download the release archive
# ssh and git are required by setuptools_scm for automatic versioning using git
RUN apt-get -yq update && apt-get -yqq install --no-install-recommends ssh git curl ca-certificates

WORKDIR /code
COPY ./requirements.txt /code/requirements.txt
# Download the latest installer
ADD https://astral.sh/uv/0.4.3/install.sh /uv-installer.sh

ENV PIP_ROOT_USER_ACTION=ignore \
PIP_DISABLE_PIP_VERSION_CHECK=1 \
PIP_NO_CACHE_DIR=1
# Run the installer then remove it
RUN sh /uv-installer.sh && rm /uv-installer.sh

RUN pip install uv
RUN uv pip install --system --no-cache -r /code/requirements.txt
# Ensure the installed binary is on the `PATH`
ENV PATH="/root/.cargo/bin/:$PATH"

COPY ./template /code/template
COPY ./pyproject.toml /code/pyproject.toml
# Change the working directory to the `app` directory
WORKDIR /app

# Solo necesario si queremos instalar la librería y obtener la version
# automaticamente. Ver: https://pypi.org/project/setuptools-scm/
#RUN --mount=source=.git,target=.git,type=bind \
# pip install -e .
# Copy the lockfile and `pyproject.toml` into the image
COPY uv.lock /app/uv.lock
COPY pyproject.toml /app/pyproject.toml

CMD ["uvicorn", "template.api:app", "--host", "0.0.0.0", "--port", "80"]
# Install dependencies
RUN uv sync --frozen --no-cache --no-dev --no-install-project

# Copy the project into the image
COPY . /app

# Sync the project into a new environment
# Mounting the git directory is required by setuptools_scm: https://setuptools-scm.readthedocs.io/en/stable/usage/
RUN --mount=source=.git,target=.git,type=bind \
uv sync --frozen --no-cache --no-dev

CMD ["uv", "run", "uvicorn", "template.api:app", "--host", "0.0.0.0", "--port", "80"]
36 changes: 12 additions & 24 deletions Makefile
Original file line number Diff line number Diff line change
@@ -1,43 +1,31 @@
.ONESHELL:
.PHONY: env install compile sync hooks ruff test mypy docker build run debug push
.PHONY: install hooks hooks-update ruff test mypy docker build run debug push

SHELL=/bin/bash
DOCKER_IMG_NAME=ghcr.io/komorebi-ai/python-template
DOCKER_CONTAINER=template
GH_USER=GITHUB_USERNAME
GH_TOKEN_FILE=GITHUB_TOKEN_PATH

env:
conda create -n python-template python=3.11

install:
pip install uv
uv pip install -r requirements-dev.txt
uv pip install -e .[dev]

compile:
pip install uv
uv pip compile -o requirements.txt requirements.in
uv pip compile -o requirements-dev.txt -c requirements.txt requirements.in requirements-dev.in

sync:
pip install uv
uv pip sync requirements-dev.txt
uv pip install -e .[dev]
curl -LsSf https://astral.sh/uv/install.sh | sh
uv run pre-commit install

hooks:
pre-commit install
pre-commit run --all-files
uv run pre-commit run --all-files

hooks-update:
uv run pre-commit autoupdate

ruff:
ruff format .
ruff check --fix --show-fixes .
uv run ruff format .
uv run ruff check --fix --show-fixes .

test:
python -m pytest
uv run pytest

mypy:
mypy --install-types --non-interactive
uv run mypy --install-types --non-interactive

docker: build run

Expand All @@ -47,7 +35,7 @@ build:
# https://stackoverflow.com/questions/26564825/what-is-the-meaning-of-a-double-dollar-sign-in-bash-makefile
run:
[ "$$(docker ps -a | grep $(DOCKER_CONTAINER))" ] && docker stop $(DOCKER_CONTAINER) && docker rm $(DOCKER_CONTAINER)
docker run -d --restart=unless-stopped --name $(DOCKER_CONTAINER) -p 80:80 $(DOCKER_IMG_NAME)
docker run -d --restart=unless-stopped --name $(DOCKER_CONTAINER) -p 5000:80 $(DOCKER_IMG_NAME)

debug:
docker run -it $(DOCKER_IMG_NAME) /bin/bash
Expand Down
63 changes: 44 additions & 19 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,13 +1,12 @@
# Template

![testing workflow](https://github.com/Komorebi-AI/python-template/actions/workflows/pytest.yml/badge.svg)
[![codecov](https://codecov.io/gh/Komorebi-AI/python-template/branch/main/graph/badge.svg?token=DQXLBOUDPX)](https://codecov.io/gh/Komorebi-AI/python-template)
[![Checked with mypy](http://www.mypy-lang.org/static/mypy_badge.svg)](http://mypy-lang.org/)
[![pre-commit](https://img.shields.io/badge/pre--commit-enabled-brightgreen?logo=pre-commit&logoColor=white)](https://github.com/pre-commit/pre-commit)

Template for Python libraries. The associated development guide can be found [here](https://github.com/Komorebi-AI/docs/blob/main/python_dev.md).

## Components
## Structure

The repository is structured into the following directories:

Expand All @@ -22,60 +21,86 @@ Conveniently, a set of workflows via Github Actions are already installed:

Tools:

- [uv](https://docs.astral.sh/uv/)
- [ruff](https://docs.astral.sh/ruff/)
- [mypy](https://mypy.readthedocs.io/)
- [pytest](https://docs.pytest.org/en/)
- [uv](https://github.com/astral-sh/uv)
- [pre-commit](https://pre-commit.com/)
- [prettier](https://prettier.io/)
- [codespell](https://github.com/codespell-project/codespell)

## Installation

Install a specific version of the package with `pip`
### Application

Install package and pinned dependencies with the [`uv`](https://docs.astral.sh/uv/) package manager:

1. Install `uv`. See instructions for Windows, Linux or MacOS [here](https://docs.astral.sh/uv/getting-started/installation/).

2. Clone repository

3. Install package and dependencies in a virtual environment:

```{bash}
uv sync
```

4. Run any command or Python script with `uv run`, for instance:

```{bash}
uv run template/main.py
```

### Library

Install a specific version of the package with `pip` or `uv pip`:

```{bash}
pip install git+ssh://[email protected]/Komorebi-AI/[email protected]
```

## Setup development environment
## Setup development environment (Unix)

Create isolated environment with required Python version. This assumes that you have `conda` installed ([see instructions](https://docs.anaconda.com/free/miniconda/)):
Install [`uv`](https://docs.astral.sh/uv/getting-started/installation/) and pre-commit hooks:

```{bash}
make env
make install
```

Then, activate the environment:
`uv` will automatically create a virtual environment with the specified Python version in `.python-version` and install the dependencies from `uv.lock` (both standard and dev dependencies). It will also install the package in editable mode.

### Adding new dependencies

Add dependencies with:

```{bash}
conda activate python-template
uv add <PACKAGE>
```

Install dependencies:
Add dev dependencies with:

```{bash}
make install
uv add --dev <PACKAGE>
```

Install and run pre-commit hooks:
Remove dependency with:

```{bash}
make hooks
uv remove <PACKAGE>
```

### Adding new dependencies

The `requirements.txt` is generated automatically with `uv` and it should not be edited manually. Add abstract dependencies to `requirements.in` and `requirements-dev.in`. If necessary, add version requirements but try to be as flexible as possible. Then, update the `requirements.txt` file with:
In all cases `uv` will automatically update the `uv.lock` file and sync the virtual environment. This can also be done manually with:

```{bash}
make compile
uv sync
```

Sync the local environment with the `requirements-dev.txt` file:
## Run pre-commit hooks

Hooks are run on modified files before any commit. To run them manually on all files use:

```{bash}
make sync
make hooks
```

## Run linter and formatter
Expand Down
Loading