Compatibility Issue with logbook and setuptools in Docker Environment

[tomduese]

Dagster version

Dagster Version: 1.7.15

What's the issue?

The error appears to be related to the logbook package trying to use setuptools.command.test, which seems to be removed in setuptools. This results in a failure to build the Docker image when using the latest version of setuptools.

see setuptools changelog

v72.0.0 (28 Jul 2024)

Deprecations and Removals

The test command has been removed. Users relying on ‘setup.py test’ will need to migrate to another test runner or pin setuptools before this version. (#931)

What did you expect to happen?

this has never happened before and occurrs perfectly in line with the removal of setup.py test yesterday

How to reproduce?

Steps to Reproduce:

Dockerfile configuration:

FROM python:3.11-slim
RUN pip3 install --upgrade pip wheel setuptools
RUN pip install \
    dagster==1.7.15 \
    dagster-graphql==1.7.15 \
    dagit==1.7.15 \
    dagster-postgres==0.23.15 \
    dagster-docker==0.23.15 \
    dbt-postgres==1.8.2

13.04 Collecting logbook<1.6,>=1.5 (from dbt-core>=1.8.0a1->dbt-postgres==1.8.2)
13.06   Downloading Logbook-1.5.3.tar.gz (85 kB)
13.14   Preparing metadata (setup.py): started
13.35   Preparing metadata (setup.py): finished with status 'error'
13.36   error: subprocess-exited-with-error
13.36   
13.36   × python setup.py egg_info did not run successfully.
13.36   │ exit code: 1
13.36   ╰─> [6 lines of output]
13.36       Traceback (most recent call last):
13.36         File "<string>", line 2, in <module>
13.36         File "<pip-setuptools-caller>", line 34, in <module>
13.36         File "/tmp/pip-install-4ju949gb/logbook_71c16caab73c4571a18703879902f577/setup.py", line 64, in <module>
13.36           from setuptools.command.test import test as TestCommand
13.36       ModuleNotFoundError: No module named 'setuptools.command.test'
13.36       [end of output]

Deployment type

Docker Compose

Deployment details

Environment:

• Docker Image: python:3.11-slim
• Dagster Version: 1.7.15
• dbt-postgres Version: 1.8.2
• setuptools Version: Latest (as of the issue date)
• logbook Version: 1.5.3

Additional information

Temporary Workaround:

Disabling the upgrade of setuptools in the Dockerfile resolves the issue:

FROM python:3.11-slim
RUN pip3 install --upgrade pip wheel 
RUN pip install \
    dagster==1.7.15 \
    dagster-graphql==1.7.15 \
    dagit==1.7.15 \
    dagster-postgres==0.23.15 \
    dagster-docker==0.23.15 \
    dbt-postgres==1.8.2

I would love to continue using the latest version of setuptools :-)

Request:

Could you consider updating the dependencies, particularly the logbook package, to a newer version that is compatible with the latest setuptools? This will help ensure that users can use the latest versions of build tools without encountering compatibility issues.

Thank you for your attention to this matter.

Message from the maintainers

Impacted by this issue? Give it a 👍! We factor engagement into prioritization.

[gibsondan]

Hi @tomduese - It looks like that pin is coming from dbt-core rather than dagster (we don't have a logbook dependency) - you may want to file this in the dbt repo instead.

[gibsondan]

dbt-labs/dbt-core#10495 is the relevant dbt-core issue.

[gibsondan]

It appears the offending version of setuptools has been yanked: https://pypi.org/project/setuptools/72.0.0/

[tomduese]

thanks for this fast respone, you are completele right - I will write an issue in dbt-core :-)