[BUG] setuptools >= 64.0.0 with editable install from subdirectory breaks mypy

[rtandy]

setuptools version

setuptools >= 64.0.0, up to at least 65.0.2

Python version

Python 3.10.6

OS

macOS Big Sur 11.6.8 (20G730)

Additional environment information

No response

Description

Reproducer: https://github.com/rtandy/setuptools-editable-subdirectory
(Reduced reproducer. In our original project (private) the Python module is one subdirectory of a larger repository, and the app that consumes the module has its own separate repository.)

With setuptools 64.0.0 and later, pip install -e git+https://github.com/<repo>#egg=<package>&subdirectory=<subdir> changed its behaviour in some way that breaks mypy. At time of writing, the latest available setuptools 65.0.2 did not fix the problem.

Python itself is able to load the module:

$ pipenv run python3
Python 3.10.6 (main, Aug 11 2022, 13:49:01) [Clang 13.0.0 (clang-1300.0.29.30)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import mylib.example
>>> mylib.example.example()
'Hello'

however, mypy fails:

$ pipenv run mypy .
app.py:1: error: Cannot find implementation or library stub for module named "mylib.example"
app.py:1: note: See https://mypy.readthedocs.io/en/stable/running_mypy.html#missing-imports
app.py:1: error: Cannot find implementation or library stub for module named "mylib"
Found 2 errors in 1 file (checked 1 source file)

I also noticed that pip freeze no longer shows the subdirectory for the editable module:

$ pipenv run pip freeze
WARNING: Could not find a Python project for directory /Users/ryan/.local/share/virtualenvs/app-FRm2CBtr/src/test-library (tried all parent directories)
mypy==0.971
mypy-extensions==0.4.3
-e git+https://github.com/rtandy/setuptools-editable-subdirectory@412e934070173ea904daad6e99125e4afc2a1275#egg=test_library
tomli==2.0.1
typing_extensions==4.3.0

I don't think this is a Pipenv bug. It appears to run the pip command I'd expect (including subdirectory) and running pip install -e git+https://... manually has the same results.

Thank you for maintaining setuptools!

Expected behavior

The branch older-setuptools shows the working behaviour with setuptools < 64.

$ git checkout older-setuptools
$ pipenv update --dev
[...]
$ pipenv run mypy .
Success: no issues found in 1 source file
$ pipenv run pip freeze
mypy==0.971
mypy-extensions==0.4.3
-e git+https://github.com/rtandy/setuptools-editable-subdirectory@0b624539211adf7b5d584426f7cf3dba49d7f62b#egg=test_library&subdirectory=library
tomli==2.0.1
typing_extensions==4.3.0

How to Reproduce

git clone https://github.com/rtandy/setuptools-editable-subdirectory
cd setuptools-editable-subdirectory/app
pipenv install --dev
pipenv run mypy .
pipenv run pip freeze

Output

$ pipenv run mypy .
app.py:1: error: Cannot find implementation or library stub for module named "mylib.example"
app.py:1: note: See https://mypy.readthedocs.io/en/stable/running_mypy.html#missing-imports
app.py:1: error: Cannot find implementation or library stub for module named "mylib"
Found 2 errors in 1 file (checked 1 source file)
$ pipenv run pip freeze
WARNING: Could not find a Python project for directory /Users/ryan/.local/share/virtualenvs/app-FRm2CBtr/src/test-library (tried all parent directories)
mypy==0.971
mypy-extensions==0.4.3
-e git+https://github.com/rtandy/setuptools-editable-subdirectory@158b003430fa340a75364bd84e17a7aef4f6291c#egg=test_library
tomli==2.0.1
typing_extensions==4.3.0

[abravalheri]

Hi @rtandy, thank you very much for reporting this.

I believe this problem with mypy has already been reported in #3518, right?
Can we close this issue as a duplicate?

I cannot say anything about pip freeze or pipenv freeze... PEP 660 explicitly allows backends to implement editable installs via import hooks, so I don't know how those projects deal with it...

[rtandy]

You're right, it's a duplicate. I guess I missed it because I was looking for bugs about subdirectory installs. Thank you for the quick response, sorry for the noise!