pip-compile logfire with opentelemetry-instrumentation depending on setuptools

[robert-moyai]

Question

Dont know if this is an issue or not so opened a question. I'm trying to instrument a fastapi app. That means that you need to pip install logfire[fastapi], which pip installs logfire and opentelemetry-instrumentation-fastapi. But looking at the requirements that I am building from requirements.in I see the following warning:

# The following packages are considered to be unsafe in a requirements file:
setuptools==73.0.1 \
    --hash=sha256:b208925fcb9f7af924ed2dc04708ea89791e24bde0d3020b27df0e116088b34e \
    --hash=sha256:d59a3e788ab7e012ab2c4baed1b376da6366883ee20d7a5fc426816e3d7b1193
    # via opentelemetry-instrumentation

Which when trying to instrument the app (logfire.instrument_fastapi(app)) is giving the following error below. Now I am running the app in bazel. I know that bazels sandbox environment is highly restrictive, and certain packages might be expecting files that are not properly bundled or accessible in this environment. But the Lorem ipsum.txt file being referenced in the error is part of a test or example data within the jaraco.text package, which should not typically be required at runtime. Now I do seem to see the file as existing in setuptools: https://github.com/pypa/setuptools/blob/main/setuptools/_vendor/jaraco/text/Lorem%20ipsum.txt.

Traceback (most recent call last):
  File "/private/var/tmp/_bazel_user/5cf272d3549b214b63c8a304b6dc81fa/execroot/_main/bazel-out/darwin_x86_64-fastbuild/bin/src/chats/entrypoints/app.runfiles/_main/src/chats/entrypoints/app.py", line 20, in <module>
    logfire.instrument_fastapi(app)
  File "/private/var/tmp/_bazel_user/5cf272d3549b214b63c8a304b6dc81fa/execroot/_main/bazel-out/darwin_x86_64-fastbuild/bin/src/chats/entrypoints/app.runfiles/rules_python~~pip~pip_311_logfire/site-packages/logfire/_internal/main.py", line 867, in instrument_fastapi
    from .integrations.fastapi import instrument_fastapi
  File "/private/var/tmp/_bazel_user/5cf272d3549b214b63c8a304b6dc81fa/execroot/_main/bazel-out/darwin_x86_64-fastbuild/bin/src/chats/entrypoints/app.runfiles/rules_python~~pip~pip_311_logfire/site-packages/logfire/_internal/integrations/fastapi.py", line 23, in <module>
    from opentelemetry.instrumentation.fastapi import FastAPIInstrumentor
  File "/private/var/tmp/_bazel_user/5cf272d3549b214b63c8a304b6dc81fa/execroot/_main/bazel-out/darwin_x86_64-fastbuild/bin/src/chats/entrypoints/app.runfiles/rules_python~~pip~pip_311_opentelemetry_instrumentation_fastapi/site-packages/opentelemetry/instrumentation/fastapi/__init__.py", line 199, in <module>
    from opentelemetry.instrumentation.instrumentor import BaseInstrumentor
  File "/private/var/tmp/_bazel_user/5cf272d3549b214b63c8a304b6dc81fa/execroot/_main/bazel-out/darwin_x86_64-fastbuild/bin/src/chats/entrypoints/app.runfiles/rules_python~~pip~pip_311_opentelemetry_instrumentation/site-packages/opentelemetry/instrumentation/instrumentor.py", line 27, in <module>
    from opentelemetry.instrumentation.dependencies import (
  File "/private/var/tmp/_bazel_user/5cf272d3549b214b63c8a304b6dc81fa/execroot/_main/bazel-out/darwin_x86_64-fastbuild/bin/src/chats/entrypoints/app.runfiles/rules_python~~pip~pip_311_opentelemetry_instrumentation/site-packages/opentelemetry/instrumentation/dependencies.py", line 4, in <module>
    from pkg_resources import (
  File "/private/var/tmp/_bazel_user/5cf272d3549b214b63c8a304b6dc81fa/execroot/_main/bazel-out/darwin_x86_64-fastbuild/bin/src/chats/entrypoints/app.runfiles/rules_python~~pip~pip_311_setuptools/site-packages/pkg_resources/__init__.py", line 98, in <module>
    from jaraco.text import drop_comment, join_continuation, yield_lines
  File "/private/var/tmp/_bazel_user/5cf272d3549b214b63c8a304b6dc81fa/execroot/_main/bazel-out/darwin_x86_64-fastbuild/bin/src/chats/entrypoints/app.runfiles/rules_python~~pip~pip_311_setuptools/site-packages/setuptools/_vendor/jaraco/text/__init__.py", line 231, in <module>
    files(__name__).joinpath('Lorem ipsum.txt').read_text(encoding='utf-8')
  File "/private/var/tmp/_bazel_user/5cf272d3549b214b63c8a304b6dc81fa/external/rules_python~~python~python_3_11_x86_64-apple-darwin/lib/python3.11/pathlib.py", line 1058, in read_text
    with self.open(mode='r', encoding=encoding, errors=errors) as f:
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/private/var/tmp/_bazel_user/5cf272d3549b214b63c8a304b6dc81fa/external/rules_python~~python~python_3_11_x86_64-apple-darwin/lib/python3.11/pathlib.py", line 1044, in open
    return io.open(self, mode, buffering, encoding, errors, newline)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
FileNotFoundError: [Errno 2] No such file or directory: '/private/var/tmp/_bazel_user/5cf272d3549b214b63c8a304b6dc81fa/execroot/_main/bazel-out/darwin_x86_64-fastbuild/bin/src/chats/entrypoints/app.runfiles/rules_python~~pip~pip_311_setuptools/site-packages/setuptools/_vendor/jaraco/text/Lorem ipsum.txt'
Logfire project URL: https://logfire.pydantic.dev/company/company-app

Generally I feel that the opentelemetry-instrumentation dependency on setuptools is degrading you build stability.

[alexmojaki]

This sounds like a problem with setuptools (cc @jaraco) or maybe opentelemetry-instrumentation, I don't think there's much that logfire can do.

[robert-moyai]

Yeah I agree the whole dependency flow made me a bit confused as to where to actually raise this. But thanks for linking it through Alex!

[jaraco]

See pypa/setuptools#3072 and pypa/setuptools#4487. It's a known deficiency in bazel, and something that it sounds like bazel might have a fix for soon. In the meantime, downstream packagers and users need to employ a workaround.

[robert-moyai]

Ok thanks a lot for that I'll raise it