Namespace packages problem with typestubs

[ljodal]

Bug Report

I'm experiencing a weird issue where enabling --namespace-packages triggers mypy to complain about the unittest.mock package. It works fine without --namespace-packages (except, you know, namespace packages don't work).

<snip>
LOG:  Metadata not found for unittest.mock
LOG:  Parsing /Users/sigurd/Developer/<snip>/.venv/lib/python3.10/site-packages/mypy/typeshed/stdlib/unittest/mock.pyi (unittest.mock)
<snip>
LOG:  Metadata not found for unittest
LOG:  Parsing /Users/sigurd/Developer/<snip>/.venv/lib/python3.10/site-packages/mypy/typeshed/stdlib/unittest/__init__.pyi (unittest)
<snip>
LOG:  Metadata not found for unittest.case
LOG:  Parsing /Users/sigurd/Developer/<snip>/.venv/lib/python3.10/site-packages/mypy/typeshed/stdlib/unittest/case.pyi (unittest.case)
LOG:  Metadata not found for unittest.loader
LOG:  Parsing /Users/sigurd/Developer/<snip>/.venv/lib/python3.10/site-packages/mypy/typeshed/stdlib/unittest/loader.pyi (unittest.loader)
LOG:  Metadata not found for unittest.main
LOG:  Parsing /Users/sigurd/Developer/<snip>/.venv/lib/python3.10/site-packages/mypy/typeshed/stdlib/unittest/main.pyi (unittest.main)
LOG:  Metadata not found for unittest.result
LOG:  Parsing /Users/sigurd/Developer/<snip>/.venv/lib/python3.10/site-packages/mypy/typeshed/stdlib/unittest/result.pyi (unittest.result)
LOG:  Metadata not found for unittest.runner
LOG:  Parsing /Users/sigurd/Developer/<snip>/.venv/lib/python3.10/site-packages/mypy/typeshed/stdlib/unittest/runner.pyi (unittest.runner)
LOG:  Metadata not found for unittest.signals
LOG:  Parsing /Users/sigurd/Developer/<snip>/.venv/lib/python3.10/site-packages/mypy/typeshed/stdlib/unittest/signals.pyi (unittest.signals)
LOG:  Metadata not found for unittest.suite
LOG:  Parsing /Users/sigurd/Developer/<snip>/.venv/lib/python3.10/site-packages/mypy/typeshed/stdlib/unittest/suite.pyi (unittest.suite)
LOG:  Metadata not found for unittest.async_case
LOG:  Parsing /Users/sigurd/Developer/<snip>/.venv/lib/python3.10/site-packages/mypy/typeshed/stdlib/unittest/async_case.pyi (unittest.async_case)
<snip>
LOG:  Metadata not found for mypy.typeshed.stdlib.unittest.mock
LOG:  Parsing /Users/sigurd/Developer/<snip>/.venv/lib/python3.10/site-packages/mypy/typeshed/stdlib/unittest/mock.pyi (mypy.typeshed.stdlib.unittest.mock)
LOG:  Build finished in 11.806 seconds with 5984 modules, and 1 errors
.venv/lib/python3.10/site-packages/mypy/typeshed/stdlib/unittest/mock.pyi: error: Source file found twice under different module names: "unittest.mock" and "mypy.typeshed.stdlib.unittest.mock"

To Reproduce

I haven't been able to reproduce this outside of our project yet (which is not open-source). It's a fairly large Django project.

I'm hoping someone can point me in the direction where things are going wrong so I can dig deeper to figure out what's going wrong.

Expected Behavior

Mypy should not complain about stubs

Actual Behavior

Mypy exits with an error.

Your Environment

• Mypy version used: 0.982
• Mypy command-line flags: --namespace-packages
• Mypy configuration options from mypy.ini (and other config files):

# pyproject.toml
[tool.mypy]
python_version = "3.10"
strict_optional = true
ignore_missing_imports = true
follow_imports = "normal"
allow_untyped_globals = true
plugins = ["mypy_django_plugin.main", "pydantic.mypy"]
show_error_codes = true
warn_redundant_casts = true
warn_unused_ignores = true

[tool.django-stubs]
django_settings_module = "<snip>.settings"

[tool.pydantic-mypy]
init_forbid_extra = true
init_typed = true
warn_required_dynamic_aliases = true
warn_untyped_fields = true

[[tool.mypy.overrides]]
module =  [
  ...
]
ignore_errors = false
check_untyped_defs = true
disallow_any_unimported = true
disallow_incomplete_defs = true
disallow_subclassing_any = true
disallow_untyped_calls = true
disallow_untyped_decorators = true
disallow_untyped_defs = true
no_implicit_optional = true
warn_return_any = true

# Ignore typing errors in admin, migrations and tests
[[tool.mypy.overrides]]
module = [
   ...
]
ignore_errors = true

# The files below are written to conform to slightly stricter typing.
[[tool.mypy.overrides]]
module = [
    ...
]
check_untyped_defs = true
disallow_untyped_calls = true
disallow_untyped_defs = true

[[tool.mypy.overrides]]
module = [
  ...
]
check_untyped_defs = true
disallow_untyped_defs = true

• Python version used: 3.10.8

[hauntsaninja]

Does it reproduce without the mypy plugins?

[ljodal]

Yes, it does

[tony]

Python 3.10.1
mypy 0.990

Log

TRACE: Looking for posixpath at posixpath.meta.json
LOG:  Could not load cache for posixpath: posixpath.meta.json
LOG:  Metadata not found for posixpath
LOG:  Parsing /home/user/project/.env/lib/python3.10/site-packages/mypy/typeshed/stdlib/posixpath.pyi (posixpath)
TRACE: Looking for pkg2.anotherapp.constants at pkg2/anotherapp/constants.meta.json
LOG:  Could not load cache for pkg2.anotherapp.constants: pkg2/anotherapp/constants.meta.json
LOG:  Metadata not found for pkg2.anotherapp.constants
LOG:  Parsing /home/user/project/src/pkg2/anotherapp/constants.py (pkg2.anotherapp.constants)
TRACE: Looking for project.mypkg.impersonation at project/mypkg/impersonation.meta.json
LOG:  Could not load cache for project.mypkg.impersonation: project/mypkg/impersonation.meta.json
LOG:  Metadata not found for project.mypkg.impersonation
LOG:  Parsing /home/user/project/src/project/mypkg/impersonation.py (project.mypkg.impersonation)
TRACE: Looking for faker.factory at faker/factory.meta.json
LOG:  Could not load cache for faker.factory: faker/factory.meta.json
LOG:  Metadata not found for faker.factory
LOG:  Parsing /home/user/project/.env/lib/python3.10/site-packages/faker/factory.py (faker.factory)
TRACE: Looking for faker.generator at faker/generator.meta.json
LOG:  Could not load cache for faker.generator: faker/generator.meta.json
LOG:  Metadata not found for faker.generator
LOG:  Parsing /home/user/project/.env/lib/python3.10/site-packages/faker/generator.py (faker.generator)
TRACE: Looking for faker.proxy at faker/proxy.meta.json
LOG:  Could not load cache for faker.proxy: faker/proxy.meta.json
LOG:  Metadata not found for faker.proxy
LOG:  Parsing /home/user/project/.env/lib/python3.10/site-packages/faker/proxy.py (faker.proxy)
TRACE: Looking for freezegun.api at freezegun/api.meta.json
LOG:  Could not load cache for freezegun.api: freezegun/api.meta.json
LOG:  Metadata not found for freezegun.api
LOG:  Parsing /home/user/project/.env/lib/python3.10/site-packages/freezegun/api.pyi (freezegun.api)
TRACE: Looking for pytest_mock.plugin at pytest_mock/plugin.meta.json
LOG:  Could not load cache for pytest_mock.plugin: pytest_mock/plugin.meta.json
LOG:  Metadata not found for pytest_mock.plugin
LOG:  Parsing /home/user/project/.env/lib/python3.10/site-packages/pytest_mock/plugin.py (pytest_mock.plugin)
TRACE: Looking for mypy.typeshed.stdlib.unittest.mock at mypy/typeshed/stdlib/unittest/mock.meta.json
LOG:  Could not load cache for mypy.typeshed.stdlib.unittest.mock: mypy/typeshed/stdlib/unittest/mock.meta.json
LOG:  Metadata not found for mypy.typeshed.stdlib.unittest.mock
LOG:  Parsing /home/user/project/.env/lib/python3.10/site-packages/mypy/typeshed/stdlib/unittest/mock.pyi (mypy.typeshed.stdlib.unittest.mock)
TRACE: Looking for urllib3.util.url at urllib3/util/url.meta.json
LOG:  Could not load cache for urllib3.util.url: urllib3/util/url.meta.json
LOG:  Metadata not found for urllib3.util.url
LOG:  Parsing /home/user/project/.env/lib/python3.10/site-packages/urllib3-stubs/util/url.pyi (urllib3.util.url)
TRACE: Looking for http.client at http/client.meta.json
LOG:  Could not load cache for http.client: http/client.meta.json
LOG:  Metadata not found for http.client
LOG:  Parsing /home/user/project/.env/lib/python3.10/site-packages/mypy/typeshed/stdlib/http/client.pyi (http.client)
TRACE: Looking for requests.adapters at requests/adapters.meta.json
LOG:  Could not load cache for requests.adapters: requests/adapters.meta.json
LOG:  Metadata not found for requests.adapters
LOG:  Parsing /home/user/project/.env/lib/python3.10/site-packages/requests-stubs/adapters.pyi (requests.adapters)
TRACE: Looking for responses.matchers at responses/matchers.meta.json
LOG:  Could not load cache for responses.matchers: responses/matchers.meta.json
LOG:  Metadata not found for responses.matchers
LOG:  Parsing /home/user/project/.env/lib/python3.10/site-packages/responses/matchers.py (responses.matchers)
TRACE: Looking for responses.registries at responses/registries.meta.json
LOG:  Could not load cache for responses.registries: responses/registries.meta.json
LOG:  Metadata not found for responses.registries
LOG:  Parsing /home/user/project/.env/lib/python3.10/site-packages/responses/registries.py (responses.registries)
TRACE: Looking for urllib3.response at urllib3/response.meta.json
LOG:  Could not load cache for urllib3.response: urllib3/response.meta.json
LOG:  Metadata not found for urllib3.response
LOG:  Parsing /home/user/project/.env/lib/python3.10/site-packages/urllib3-stubs/response.pyi (urllib3.response)
TRACE: Looking for unittest.mock at unittest/mock.meta.json
LOG:  Could not load cache for unittest.mock: unittest/mock.meta.json
LOG:  Metadata not found for unittest.mock
LOG:  Parsing /home/user/project/.env/lib/python3.10/site-packages/mypy/typeshed/stdlib/unittest/mock.pyi (unittest.mock)
LOG:  Build finished in 1.181 seconds with 622 modules, and 1 errors
.env/lib/python3.10/site-packages/mypy/typeshed/stdlib/unittest/mock.pyi: error: Source file found twice under different module names: "mypy.typeshed.stdlib.unittest.mock" and "unittest.mock"

Third party libraries

• responses: 0.21.0
• faker: 13.15.0
• freezegun: 1.2.1
• pytest-mock: 3.8.2

The trace mentions requests-stubs and urllib3-stubs but I don't have them in my environment.

Internal usage

In my code, I have imports from unittest.mock in two different ways:

from unittest import mock

And also:

from unittest.mock import patch

[ljodal]

I've traced this back to getsentry/responses#556, updating to responses==0.22.0 fixes the problem

[hauntsaninja]

Oh lol, wow that would explain it