Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

add libffi-7.dll to pypy on windows #2141

Merged
merged 3 commits into from
Jul 12, 2021
Merged

add libffi-7.dll to pypy on windows #2141

merged 3 commits into from
Jul 12, 2021

Conversation

mattip
Copy link
Contributor

@mattip mattip commented Jul 11, 2021 •
edited
Loading

Thanks for contributing, make sure you address all the checklists (for details on how see

development documentation)!

  • ran the linter to address style issues (tox -e fix_lint)
  • wrote descriptive pull request text
  • ensured there are test(s) validating the fix
  • added news fragment in docs/changelog folder
  • updated/extended the documentation

This is a continuation of PyPy issue 1922 where virtualenv has the shared objects needed to run PyPy hardcoded, and PyPy has not yet adopted CPython's launchers on windows. I am not sure how best to solve this. I was hoping a change in PyPy's venv module would have helped, but it seems not.

The immediate problem is that PyPy wishes to add libffi-7.dll to the windows build for the next release (in 6 weeks or so?). I have tested this fix locally for the yet-as-unmerged-to-master change in PyPy's sources.

@codecov
Copy link

codecov bot commented Jul 11, 2021 •
edited
Loading

Codecov Report

Merging #2141 (1eb033b) into main (17caadc) will not change coverage.
The diff coverage is 100.00%.

❗ Current head 1eb033b differs from pull request most recent head 587e3d2. Consider uploading reports for the commit 587e3d2 to get more accurate results
Impacted file tree graph

@@           Coverage Diff           @@
##             main    #2141   +/-   ##
=======================================
  Coverage   93.72%   93.72%           
=======================================
  Files          88       88           
  Lines        4382     4382           
=======================================
  Hits         4107     4107           
  Misses        275      275
Flag Coverage Δ
tests 93.72% <100.00%> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
...ualenv/create/via_global_ref/builtin/pypy/pypy2.py 98.70% <100.00%> (ø)
...ualenv/create/via_global_ref/builtin/pypy/pypy3.py 100.00% <100.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 17caadc...587e3d2. Read the comment docs.

gaborbernat
gaborbernat requested changes Jul 11, 2021
View reviewed changes
src/virtualenv/create/via_global_ref/builtin/pypy/pypy2.py
@@ -112,7 +112,7 @@ def modules(cls):

@classmethod
def _shared_libs(cls):
return ["libpypy-c.dll"]
return ["libpypy-c.dll", "libffi-7.dll"]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wouldn't this need to be version limited to newer pypy versions?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

no, the code to copy ignores missing files

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, 🤔 wonder why 😆

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

virtualenv/src/virtualenv/create/via_global_ref/builtin/pypy/common.py

Lines 45 to 49 in 17caadc

python_dir = Path(interpreter.system_executable).resolve().parent
for libname in cls._shared_libs():
src = python_dir / libname
if src.exists():
yield src

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will it work without this DLL? Should we make it hard dependency rather than soft that it is now?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Currently this DLL is soft depdency (so assuming that's not a bug), if in the future this is not the case we should make it hard dependency and version guard the hard dependency part.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would be nice to avoid this code altogether in favor of a more robust solution

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That wasn't my question. Can we say a usecase where pypy would work without this DLL with the current less robust solution? If yes, we can keep it a soft dependency. If not, it must be hard depedency.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am not sure what you mean, sorry. PyPy v7.3.5 will not have the libffi-7.dll and will work without it. Once the branch to add the dll gets merged, any future PyPy versions will require the dll. I am hopeful that branch will be merged before the PyPy v7.3.6 release, but it is dependent on a new release of virtualenv with this PR or one similar.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On the other hand, I can envision a scenario where a packager like conda uses some OS-level utilities to avoid copying the dll. In this case even though the dlls are required, they are not copied by this code into the new virtualenv.

@mattip mattip mentioned this pull request Jul 11, 2021
Merged
5 tasks
@mattip
Copy link
Contributor Author

mattip commented Jul 11, 2021

Is the pypy2 windows test run known to be flaky or did I mess something up?

gaborbernat
gaborbernat requested changes Jul 11, 2021
View reviewed changes
docs/changelog/2141.misc.rst Outdated Show resolved Hide resolved
src/virtualenv/create/via_global_ref/builtin/pypy/pypy2.py
@@ -112,7 +112,7 @@ def modules(cls):

@classmethod
def _shared_libs(cls):
return ["libpypy-c.dll"]
return ["libpypy-c.dll", "libffi-7.dll"]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will it work without this DLL? Should we make it hard dependency rather than soft that it is now?

@gaborbernat
Fix changelog
587e3d2 
Signed-off-by: Bernát Gábor <[email protected]>
@gaborbernat gaborbernat merged commit 63e884e into pypa:main Jul 12, 2021
@mattip
Copy link
Contributor Author

mattip commented Jul 12, 2021

Thanks

@gaborbernat
Copy link
Contributor

Released via https://virtualenv.pypa.io/en/latest/changelog.html#v20-5-0-2021-07-13

@mattip
Copy link
Contributor Author

mattip commented Jul 13, 2021

Wow, that was fast, thanks!

@mattip mattip mentioned this pull request Dec 29, 2021
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@gaborbernat gaborbernat gaborbernat requested changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@mattip @gaborbernat