Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

AssertionError in import_proxifier.create_module while importing from django.utils.six.moves.urllib.parse #1

Open
jayvdb opened this issue Jul 16, 2020 · 5 comments

Comments

@jayvdb
Copy link
Contributor

jayvdb commented Jul 16, 2020

Running Django 3.0.8 and django-utils-six-2.0 and django-compat-patcher master , when using https://github.com/andersinno/django-form-designer-ai I encounter an assertion error in compat_patcher_core/import_proxifier.py create_module
when it tries to load https://github.com/andersinno/django-form-designer-ai/blob/master/form_designer/views.py#L9

from django.utils.six.moves.urllib.parse import urlencode

The first assertion is at

assert module.__name__ == self.real_name, module.__name__

module.__name__ reported by the assertion is six.moves.urllib_parse

But if I stupidly comment out that assertion, there are several assertions which occur during exec_module phase of module loading.

In case it helps andersinno/django-form-designer-ai#38 gets that app roughly working.

@jayvdb
Copy link
Contributor Author

jayvdb commented Oct 5, 2020

Occurring again at https://travis-ci.org/github/jayvdb/bluebottle/jobs/732751957#L6827

    module = import_module(module_path)
  File "/opt/python/3.8.3/lib/python3.8/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
  File "<frozen importlib._bootstrap>", line 991, in _find_and_load
  File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 783, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/home/travis/virtualenv/python3.8.3/lib/python3.8/site-packages/corsheaders/middleware.py", line 6, in <module>
    from django.utils.six.moves.urllib.parse import urlparse
  File "<frozen importlib._bootstrap>", line 991, in _find_and_load
  File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 657, in _load_unlocked
  File "<frozen importlib._bootstrap>", line 556, in module_from_spec
  File "/home/travis/virtualenv/python3.8.3/lib/python3.8/site-packages/compat_patcher_core/import_proxifier.py", line 124, in create_module
    assert module.__name__ == self.real_name, module.__name__
AssertionError: six.moves.urllib_parse

@jayvdb
Copy link
Contributor Author

jayvdb commented Oct 5, 2020

Bypassing that one

  File "/home/travis/virtualenv/python3.8.3/lib/python3.8/site-packages/django/utils/module_loading.py", line 17, in import_string
    module = import_module(module_path)
  File "/opt/python/3.8.3/lib/python3.8/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
  File "<frozen importlib._bootstrap>", line 991, in _find_and_load
  File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 783, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/home/travis/virtualenv/python3.8.3/lib/python3.8/site-packages/corsheaders/middleware.py", line 6, in <module>
    from django.utils.six.moves.urllib.parse import urlparse
  File "<frozen importlib._bootstrap>", line 991, in _find_and_load
  File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
  File "/home/travis/virtualenv/python3.8.3/lib/python3.8/site-packages/compat_patcher_core/import_proxifier.py", line 129, in exec_module
    assert module.__name__ in (self.alias_name, self.real_name), module.__name__
AssertionError: six.moves.urllib_parse

@jayvdb
Copy link
Contributor Author

jayvdb commented Oct 5, 2020

Removing that assert solves the problem.
Obviously just removing them isnt the best solution, and the problem wont be limited to Django and its inclusion of six as a subpackage. Many other packages vendor six, and other packages.

One way to keep the assert is to add config to identify imports to allow bypassing these asserts.

It might be possible to detect that this import doesnt need to be proxied. Possibly django-compat-patcher can declare that there should be no patching of django.utils.six.

@pakal
Copy link
Owner

pakal commented Jan 13, 2021

Sorry, I completely missed enabling notifications on this repository ^^'

Double-patching django.utils.six seems is unnecessary, the project maintainer should just remove django-utils-six-2.0, or disable the "fix_deletion_utils_six" fixer of DCP.

I couldn't reproduce the problem since when i try it, the patched django.utils.six has a proper "name" on Python3.8. Weird.

However, this error is very unclear, and indeed I should auto-test what happens when the compat-patcher-core tries to inject an already-injected module. I don't know if the patcher should fail, or warn and proceed, or warn and abort... or maybe it can be left as an option for the specific fixer.

@pakal
Copy link
Owner

pakal commented Jan 19, 2021

Ok I managed to reproduce it, I removed assertions that were wrong due to the interfering custom module importer of six.

Latest django-compat-patcher 0.10, with compat-patcher-core 1.2, should solve this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants