Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

AST parsing of setup.py fails on non-addition type binOps #204

Closed
techalchemy opened this issue Mar 6, 2020 · 0 comments · Fixed by #209
Closed

AST parsing of setup.py fails on non-addition type binOps #204

techalchemy opened this issue Mar 6, 2020 · 0 comments · Fixed by #209
Labels
bug Something isn't working

Comments

@techalchemy
Copy link
Member

For specific cases in complex setup.py parsing, ast_unparse_setup_py fails due to its lack of adequate mapping of ast.binOps operators.

One good example is when parsing the setup.py that accompanies PyYAML:

>>> ast_parse_setup_py("/tmp/pkgs/PyYAML-5.3/setup.py")
...
  File "/home/techalchemy/.virtualenvs/tempenv-74f7279776e1c/lib/python3.8/site-packages/requirementslib/models/setup_info.py", line 664, in visit_BinOp
    left = ast_unparse(node.left, initial_mapping=True)
  File "/home/techalchemy/.virtualenvs/tempenv-74f7279776e1c/lib/python3.8/site-packages/requirementslib/models/setup_info.py", line 693, in ast_unparse
    unparsed = [unparse(el) for el in item.elts]
  File "/home/techalchemy/.virtualenvs/tempenv-74f7279776e1c/lib/python3.8/site-packages/requirementslib/models/setup_info.py", line 693, in <listcomp>
    unparsed = [unparse(el) for el in item.elts]
  File "/home/techalchemy/.virtualenvs/tempenv-74f7279776e1c/lib/python3.8/site-packages/requirementslib/models/setup_info.py", line 695, in ast_unparse
    unparsed = tuple([unparse(el) for el in item.elts])
  File "/home/techalchemy/.virtualenvs/tempenv-74f7279776e1c/lib/python3.8/site-packages/requirementslib/models/setup_info.py", line 695, in <listcomp>
    unparsed = tuple([unparse(el) for el in item.elts])
  File "/home/techalchemy/.virtualenvs/tempenv-74f7279776e1c/lib/python3.8/site-packages/requirementslib/models/setup_info.py", line 794, in ast_unparse
    return unparsed
UnboundLocalError: local variable 'unparsed' referenced before assignment

local variable 'unparsed' referenced before assignment

This is currently a release blocker for pypa/pipenv#3369
@techalchemy techalchemy added the bug Something isn't working label Mar 6, 2020
@techalchemy techalchemy mentioned this issue Mar 6, 2020
Closed
30 tasks
techalchemy added a commit that referenced this issue Mar 11, 2020
@techalchemy
Fix binary operator mapping in setup.py ast parser
d262489 
- Fix handling of `ast.binOp` nodes during unmapping of `setup.py`
  files when parsing -- add mapping for all binary operators and
  execute translated functions using the unparsed left and right hand
  sides of the binary operator
- Add equivalent handling for `ast.Compare` types
- Add `importlib.import_module` attempts to `ast.Compare` evaluations
  when encountering `ast.Attribute` types on either side of the
  comparison
- Add handling for `ast.IfExp` type to evaluate truth values of the
  expression and return `node.body` if the expression is `True`,
  otherwise `node.orelse`
- Add equivalence mapping for `ast.Ellipsis` type which becomes
  `ast.Constant` in python `>= 3.8`
- Fixes #204
- Fixes #206
- Fixes #207

Signed-off-by: Dan Ryan <[email protected]>
@techalchemy techalchemy mentioned this issue Mar 11, 2020
Merged
techalchemy added a commit that referenced this issue Mar 11, 2020
@techalchemy
Fix binary operator mapping in setup.py ast parser
7a6cd44 
- Fix handling of `ast.binOp` nodes during unmapping of `setup.py`
  files when parsing -- add mapping for all binary operators and
  execute translated functions using the unparsed left and right hand
  sides of the binary operator
- Add equivalent handling for `ast.Compare` types
- Add `importlib.import_module` attempts to `ast.Compare` evaluations
  when encountering `ast.Attribute` types on either side of the
  comparison
- Add handling for `ast.IfExp` type to evaluate truth values of the
  expression and return `node.body` if the expression is `True`,
  otherwise `node.orelse`
- Add equivalence mapping for `ast.Ellipsis` type which becomes
  `ast.Constant` in python `>= 3.8`
- Fixes #204
- Fixes #206
- Fixes #207

Signed-off-by: Dan Ryan <[email protected]>
techalchemy added a commit that referenced this issue Mar 31, 2020
@techalchemy
Fix binary operator mapping in setup.py ast parser
65c70fa 
- Fix handling of `ast.binOp` nodes during unmapping of `setup.py`
  files when parsing -- add mapping for all binary operators and
  execute translated functions using the unparsed left and right hand
  sides of the binary operator
- Add equivalent handling for `ast.Compare` types
- Add `importlib.import_module` attempts to `ast.Compare` evaluations
  when encountering `ast.Attribute` types on either side of the
  comparison
- Add handling for `ast.IfExp` type to evaluate truth values of the
  expression and return `node.body` if the expression is `True`,
  otherwise `node.orelse`
- Add equivalence mapping for `ast.Ellipsis` type which becomes
  `ast.Constant` in python `>= 3.8`
- Fixes #204
- Fixes #206
- Fixes #207

Signed-off-by: Dan Ryan <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Fix binary operator mapping in setup.py ast parser sarugaku/requirementslib
1 participant
@techalchemy