[gyp_installer] Usage of GYP's Command Expansions leads to an AttributeError exception #851

cher-nov · 2019-06-30T18:28:01Z

Notes

This is seems to be caused by the next piece of code:
https://github.com/bincrafters/gyp/blob/cc91d1b0dc0375a9087974a8fd09fe9539a6ab3e/pylib/gyp/input.py#L898-L905

The problem is universal_newlines=True. The Python3 documentation states next:

If encoding or errors are specified, or text is true, the file objects stdin, stdout and stderr are opened in text mode with the specified encoding and errors, as described above in Frequently Used Arguments. The universal_newlines argument is equivalent to text and is provided for backwards compatibility. By default, file objects are opened in binary mode.

Note that there's also many universal_newlines=True in the other places, introduced in by the bincrafters/gyp@067a33b.

Also note that there's much more Python3 compatibility fixes appeared in the official repo, so it's maybe have sense to rebase onto it (not replacing with it!) the Bincrafters version (moreover, their version seems to retain compatibility with Python2).

Steps to reproduce

Run on Linux: python3 gyp/gyp_main.py --depth=. --format=gypd lemme_die.gyp
Contents of lemme_die.gyp:

{
    'conditions': [
      [ '"<!(uname -m)" == "i686"', {

      }],
    ],
}

Logs

Exception: 'str' object has no attribute 'decode'
Traceback (most recent call last):
  File "share/gyp/pylib/gyp/input.py", line 487, in CallLoadTargetBuildFile
    includes, depth, check, False)
  File "share/gyp/pylib/gyp/input.py", line 408, in LoadTargetBuildFile
    build_file_data, PHASE_EARLY, variables, build_file_path)
  File "share/gyp/pylib/gyp/input.py", line 1286, in ProcessVariablesAndConditionsInDict
    build_file)
  File "share/gyp/pylib/gyp/input.py", line 1301, in ProcessVariablesAndConditionsInList
    ProcessVariablesAndConditionsInDict(item, phase, variables, build_file)
  File "share/gyp/pylib/gyp/input.py", line 1260, in ProcessVariablesAndConditionsInDict
    ProcessConditionsInDict(the_dict, phase, variables, build_file)
  File "share/gyp/pylib/gyp/input.py", line 1139, in ProcessConditionsInDict
    variables, build_file)
  File "share/gyp/pylib/gyp/input.py", line 1260, in ProcessVariablesAndConditionsInDict
    ProcessConditionsInDict(the_dict, phase, variables, build_file)
  File "share/gyp/pylib/gyp/input.py", line 1133, in ProcessConditionsInDict
    build_file)
  File "share/gyp/pylib/gyp/input.py", line 1057, in EvalCondition
    cond_expr, true_dict, false_dict, phase, variables, build_file)
  File "share/gyp/pylib/gyp/input.py", line 1071, in EvalSingleCondition
    build_file)
  File "share/gyp/pylib/gyp/input.py", line 904, in ExpandVariables
    p_stdout = p_stdout.decode()
AttributeError: 'str' object has no attribute 'decode'

Package and Environment Details

Package Name/Version: gyp_installer/20190423
Operating System+version: Linux Xubuntu 18.04.2 LTS
Compiler+version: gcc-7.4
Conan version: conan 1.16.1
Python version: Python 3.6.8

Conan configuration

[settings]
arch=x86_64
arch_build=x86_64
build_type=Release
compiler=gcc
compiler.libcxx=libstdc++
compiler.version=7
os=Linux
os_build=Linux

The text was updated successfully, but these errors were encountered:

Croydon · 2019-06-30T21:07:48Z

We certainty shouldn't maintain our own fork of gyp. Node.js relies on gyp heavily, and they are discussing right now if they want to maintain gyp in their org. If they decide for that, we should switch to this. If not, looking for alternatives maintainers or considering freezing it.

I believe gyp_installer got only added because it is a dependency of another package?

cher-nov · 2019-07-01T17:16:25Z

I believe gyp_installer got only added because it is a dependency of another package?

I don't know actually, but this was extremely useful at least for me to install it from Conan.

uilianries · 2019-07-01T19:03:37Z

Yeah, it would be nice having another alternative. @SSE4 spent a lot of time fixing our fork. If we could use node version I would be happy, at least they are updating more frequently than us.

SSE4 · 2019-07-02T07:59:25Z

if they have added python3 compatibility to the official repo, we may try to use that, that's great news

cher-nov · 2019-07-02T08:23:10Z

if they have added python3 compatibility to the official repo, we may try to use that, that's great news

They have (if we're about Google), but its Ninja generator doesn't work properly on Windows and fails on msvs_emulation.py:985 (_ExtractImportantEnvironment) with the exception TypeError: argument should be integer or bytes-like object, not 'str'. This also fails on POSIX systems, reporting gyp: Variable uname -m must expand to a string or list of strings; found a bytes in case of the aforementioned example. But I think that their Python3 port could be merged with Bincrafters one.

SSE4 · 2019-07-02T08:27:31Z

okay, and what about nodejs/npm, do they use their own fork of gyp or just Google's repository?

cher-nov · 2019-07-02T08:32:07Z

It seems that they use a their own fork:
https://github.com/nodejs/node-gyp/commits/master/gyp
It also seems that it's Python3-compatible and updated regularly.

SSE4 · 2019-07-02T08:34:09Z

okay then, let's try to migrate to node-gyp, for me it looks like the best option for now

SSE4 · 2019-07-02T08:35:04Z

for @Croydon : gyp installer was added because some package use it as build system, for instance libuv, may be few others.

Croydon · 2019-07-02T08:38:52Z

Google is barely maintaining GYP, it's pretty much dead for them as they have moved on to other build systems.

The Node.js community has still several discussion ongoing about the future of their build system, but in the short term they are likely going to migrate to another fork which has Python 3 support, see: nodejs/node#26620

So we could use https://github.com/refack/GYP

See nodejs/node#26620 and bincrafters/community#851 for further reasoning

Croydon · 2019-07-02T16:47:47Z

Python 2 works with this GYP fork. Python 3 fails currently with this error https://ci.appveyor.com/project/bincrafters/conan-gyp-installer/build/job/2pin2muby1qphq74#L412

even though they are supposed to have fixed that https://github.com/refack/GYP/blame/master/gyp/generator/xcodeproj_file.py#L160

maybe we should try to push @uilianries patch upstream as he removed the cmp parameter completely, which seems to be necessary bincrafters/gyp@c606b59#diff-b71148dfd080925ebc604279f4d3fe23R1416

Croydon · 2019-07-02T16:51:04Z

Actually, they should merge https://chromium.googlesource.com/external/gyp/+/aca1e2c3d346d704adfa60944e6b4dd06f4728be

Croydon · 2019-07-02T16:54:03Z

Created issue: refack/GYP3#48

Croydon · 2019-07-03T00:24:37Z

@cher-nov You spoke out against replacing our fork with the upstream version, why?

cher-nov · 2019-07-03T04:24:55Z

@cher-nov You spoke out against replacing our fork with the upstream version, why?

Because of this: #851 (comment).

Croydon · 2020-10-22T06:04:03Z

We decided a while ago to archive our gyp recipe.

https://github.com/bincrafters/conan-gyp_installer

Unmaintained recipe.
If you desire a Conan GYP package, I recommend to package https://github.com/nodejs/gyp-next which is an actively maintained,
backwards compatible fork of the abandoned original with Python 3 support

cher-nov added the bug label Jun 30, 2019

Croydon added a commit to bincrafters/conan-gyp_installer that referenced this issue Jul 2, 2019

Change to Refack's GYP fork

832fa65

See nodejs/node#26620 and bincrafters/community#851 for further reasoning

Croydon closed this as completed Oct 22, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[gyp_installer] Usage of GYP's Command Expansions leads to an AttributeError exception #851

[gyp_installer] Usage of GYP's Command Expansions leads to an AttributeError exception #851

cher-nov commented Jun 30, 2019 •

edited

Loading

Croydon commented Jun 30, 2019

cher-nov commented Jul 1, 2019

uilianries commented Jul 1, 2019

SSE4 commented Jul 2, 2019

cher-nov commented Jul 2, 2019 •

edited

Loading

SSE4 commented Jul 2, 2019

cher-nov commented Jul 2, 2019

SSE4 commented Jul 2, 2019

SSE4 commented Jul 2, 2019

Croydon commented Jul 2, 2019

Croydon commented Jul 2, 2019

Croydon commented Jul 2, 2019

Croydon commented Jul 2, 2019

Croydon commented Jul 3, 2019

cher-nov commented Jul 3, 2019

Croydon commented Oct 22, 2020 •

edited

Loading

[gyp_installer] Usage of GYP's Command Expansions leads to an AttributeError exception #851

[gyp_installer] Usage of GYP's Command Expansions leads to an AttributeError exception #851

Comments

cher-nov commented Jun 30, 2019 • edited Loading

Notes

Steps to reproduce

Logs

Package and Environment Details

Conan configuration

Croydon commented Jun 30, 2019

cher-nov commented Jul 1, 2019

uilianries commented Jul 1, 2019

SSE4 commented Jul 2, 2019

cher-nov commented Jul 2, 2019 • edited Loading

SSE4 commented Jul 2, 2019

cher-nov commented Jul 2, 2019

SSE4 commented Jul 2, 2019

SSE4 commented Jul 2, 2019

Croydon commented Jul 2, 2019

Croydon commented Jul 2, 2019

Croydon commented Jul 2, 2019

Croydon commented Jul 2, 2019

Croydon commented Jul 3, 2019

cher-nov commented Jul 3, 2019

Croydon commented Oct 22, 2020 • edited Loading

cher-nov commented Jun 30, 2019 •

edited

Loading

cher-nov commented Jul 2, 2019 •

edited

Loading

Croydon commented Oct 22, 2020 •

edited

Loading