64 bit pip installs 32 bit DLLs (PyQt5 and PySide2) #8649
Labels
resolution: wrong project
Should be reported elsewhere
Comments
triage-new-issues
bot
added
the
S: needs triage
|
Can you show the logs of |
|
|
Sorry, I forgot to mention the command should be run in a clean environment (so the packages are actually installed). Running this in the existing environment would do nothing since all the packages are installed “correctly” from pip’s perspective. |
|
No worries. I've uninstalled Python and manually cleaned the AppData directories before reinstalling. This is as clean as I think that I can get it. The DLLs are still x86 though. |
|
So it seems that pip is convinced your Python installation is 32-bit. Can you check the output of the following Python code? import distutils.util
distutils.util.get_platform()Also, can you provide the result of:
|
Not surprising that it's 64 bit since python.exe and pip3.exe were given that way by dumpbin. I got the same path to python.exe as well. I suspect you'll want to see this from a clean install, so I'll do that next. But the result of doing it like this leads me to expect that it's going to be the same thing. |
|
I did a full uninstall and manually cleaned by %ProgramFiles% and %AppData% folders for files that pip might have cached. Then ran And from Python No difference. I'm starting to wonder if the installer or python or pip itself creates any registry keys that are not being removed on uninstall. I believe that I had, some time ago, installed python and that it may have been the x86 version. I did find under HKLM/Software/WOW6432Node/Python/PyLauncer that my prior install has left some keys. Unless PyLauncher is 32 bit for the amd64 installer too. |
|
Please provide the output of |
|
|
So the problem is that your Python is reporting that it's compatible with "win32" wheels. That's very weird - my copy of Python correctly reports tags like Where did you get your copy of Python from? Is it the python.org distribution? I'm wondering if there's a problem with 3.8.5 🙁 I'm still on 3.8.3. |
|
Yes, it was Python.org and I believe it was on this page: https://www.python.org/downloads/release/python-385/ The "Windows x86-64 executable installer" |
|
OK, so I upgraded my Python using that version, and I still get the correct tags. Something is definitely broken with your environment, but I'm not sure what. One further possibility. Can you do >>> import distutils.util
>>> print(distutils.util.__file__)Also, can you provide the output of |
[Edit: forgot the next part you asked for] |
|
This is so weird. What does this give you? from pip._internal.utils.compatibility_tags import get_supported
get_supported()Also, does the following returns anything? |
|
OK, I'm baffled. Your pip installation is somehow broken, but I'm at a loss to know how. How about That should give exactly the same result as And as a clutching-at-straws possibility: That's all I've got left... |
And the next command you suggested does not output anything |
|
So the support library is also reporting correctly. The only theory I can come up now is maybe you have some configs or environment variables set that manually overrides the default platform. But |
|
Just checked with "set" on the command line. No pip or python environment variables from what I can tell. |
|
@JesseMaurais Can you make changes to your installed copy of pip? That would mean editing a file in If you can, then could you modify def make_target_python(options):
# type: (Values) -> TargetPython
print(options) # <--- Please add this print statement
target_python = TargetPython(
platform=options.platform,
py_version_info=options.python_version,
abi=options.abi,
implementation=options.implementation,
)
return target_pythonIf you can then re-run Don't forget to undo the change you made once you get the debug output! 🙂 |
|
I believe that I've discovered the problem. in .../Lib/distutils/util.py we have the code for the platform def get_platform():
if os.name == 'nt':
TARGET_TO_PLAT = {
'x86' : 'win32',
'x64' : 'win-amd64',
'arm' : 'win-arm32',
}
return TARGET_TO_PLAT.get(os.environ.get('VSCMD_ARG_TGT_ARCH')) or get_host_platform()
else:
return get_host_platform()Which checks the environment variable
And I think Kevin Puetz (puetzk) is correct. Using this variable to determine the platform dependencies for Python packages is not reliable; it can grab packages that do not match the architecture against which Python was built and so scripts using such packages will not run in the interpreter for which they were installed. I ran afoul of this because I installed the packages in my Visual Studio Developers Command Prompt and because my Visual Studio installation was x86. This may have been obvious had I not cropped out the header whenever pasting text output from the prompt. And as for the environment of that same console Whereas if I run from the vanilla console And that is the correct answer. Maybe this would also have been more obvious had I not been running the dev console as admin, since it lands you in C:\Windows\System32 by default, whereas with a default user it lands in you the VS folder under Program Files (x86) for some reason. Also I had probably been using the vanilla console when I ran some of your tests that gave the correct answer and the dev console when I installed the packages. The fault there was mine. My work around for now is to install using vanilla |
|
Nice bit of debugging there, and thank you particularly for taking the time to report back here on what you found! So in conclusion:
Glad you found a workaround, but I agree this should be fixed. Setuptools is in the process of incorporating/merging distutils (pypa/setuptools#2143), and traditionally has been the place where distutils gets "developed" anyway (new features and fixes), so this is probably worth raising there. /cc @pypa/setuptools-developers - what's the best way to report this now? Basically a copy of https://bugs.python.org/issue38989 |
|
You're right, it isn't a pip bug. But I think pip might avoid it by using I mean in ...\Lib\site-packages\pip_vendor\packaging\tags.py(678): def _generic_platforms():
# type: () -> Iterator[str]
yield _normalize_string(distutils.util.get_platform())And again in ...\Lib\distutils\util.py(97): def get_platform():
if os.name == 'nt':
TARGET_TO_PLAT = {
'x86' : 'win32',
'x64' : 'win-amd64',
'arm' : 'win-arm32',
}
return TARGET_TO_PLAT.get(os.environ.get('VSCMD_ARG_TGT_ARCH')) or get_host_platform()
else:
return get_host_platform()As long as
|
|
Indeed, that seems reasonable - it would be a change to the It's not obvious what the difference between I've opened pypa/packaging#327 for this. One thing that I'm still slightly confused by. When you attempted the installs you were using the VS developer prompt. When you did the debug checks we asked you to try, I assume you were using a "normal" command prompt, as otherwise we would have seen a platform of "win32". Is that right? I assume it is, but I'm just tying up the loose ends here 🙂 |
|
Yes, that my bad. |
No worries, we were all baffled, no reason to think that would be relevant. (Famous last words, I know 🙂) |
For now, please report and submit patches to pypa/distutils. Anything patched there will get readily merged into Setuptools and is fairly straightforward to port to CPython if appropriate. |
|
Nice debugging @JesseMaurais! ^>^ Is anyone clear on what the next steps on this issue are? |
pradyunsg
added
OS: windows
triage-new-issues
bot
removed
S: needs triage
|
Someone needs to raise issues (or better, PRs) against pypa/distutils for the root cause, and pypa/packaging for the possible workaround discussed above. I don't know if @JesseMaurais intends to do this. I don't, personally. I don't believe there's a pip bug here, so this particular issue could probably be closed. |
|
Reading the BPO issue, it seems like the behaviour change to I’ll take the liberty to close this and redirect further discussion to pypa/packaging#327. |
uranusjr
added
resolution: wrong project
|
pypa/packaging#327 (comment) is fixed by pypa/packaging#396, but it needs a release and an update to the vendored pin before pip works again in VsDevCmd prompts. |
This was referenced Apr 28, 2021
Closed
1 task
1 task
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Environment
Installed Python for all users in custom location (%ProgramFiles%\Python38)
Description
Cannot run PyQt5 or PySide2 applications written in Python, either those I make myself or those also installed via pip.
Expected behavior
That 64 bit pip installs 64 bit DLLs so that Python scripts do not fail to load them. The code "from PyQt5.QtWidgets import QApplication" (or with PySide2 instead) should return without error.
How to Reproduce
Install packages PyQt5 and PySide2 with command "pip3 install pyqt5 pyside2" which I did from a console with admin privileges, though I don't think that matters.
Output
The "not a valid Win32 application" error is one I've seen before so I went to the site-packages folder and found the Qt DLLs that are installed with PyQt5 and PySide2. Running dumpbin on them to determine their architecture, found
Where the machine (x86) is the relevant information. Similarly for PySide2, found
Then I double checked that the Python and pip executables were matching. They were not.
Here we have machine (x64) and obviously a 64 bit program cannot load a 32 bit DLL (or vice versa). In fact no Qt app written in Python with these packages will even launch, exactly as I've observed. If pip cannot install 64 bit DLLs for Qt then it should not install these packages at all.
The text was updated successfully, but these errors were encountered: