[v14.x] Backport Python 3.10 compatibility patches #40689

richardlau · 2021-11-01T11:50:09Z

Backport of #40296.

Required to fix the Windows CI with v14.x-staging. Conflicts were due to v14.x supporting fallback to Python 2 if Python 3 is not available.

richardlau · 2021-11-01T11:54:02Z

Apparently the jinja patches will need work to remain compatible with Python 2.
e.g. https://ci.nodejs.org/job/node-test-commit-linux/nodes=fedora-latest-x64/43511/console

11:51:43 Traceback (most recent call last):
11:51:43   File "../../deps/v8/third_party/inspector_protocol/code_generator.py", line 718, in <module>
11:51:43     main()
11:51:43   File "../../deps/v8/third_party/inspector_protocol/code_generator.py", line 601, in main
11:51:43     jinja_env = initialize_jinja_env(jinja_dir, config.protocol.output, config)
11:51:43   File "../../deps/v8/third_party/inspector_protocol/code_generator.py", line 190, in initialize_jinja_env
11:51:43     import jinja2
11:51:43   File "/home/iojs/build/workspace/node-test-commit-linux/deps/v8/third_party/jinja2/__init__.py", line 33, in <module>
11:51:43     from jinja2.environment import Environment, Template
11:51:43   File "/home/iojs/build/workspace/node-test-commit-linux/deps/v8/third_party/jinja2/environment.py", line 16, in <module>
11:51:43     from jinja2.defaults import BLOCK_START_STRING, \
11:51:43   File "/home/iojs/build/workspace/node-test-commit-linux/deps/v8/third_party/jinja2/defaults.py", line 32, in <module>
11:51:43     from jinja2.tests import TESTS as DEFAULT_TESTS
11:51:43   File "/home/iojs/build/workspace/node-test-commit-linux/deps/v8/third_party/jinja2/tests.py", line 13, in <module>
11:51:43     from collections.abc import Mapping
11:51:43 ImportError: No module named abc
11:51:43 make[2]: *** [tools/v8_gypfiles/v8_base_without_compiler.target.mk:30:

PR-URL: nodejs#40296 Fixes: nodejs#40294 Reviewed-By: Jiawen Geng <[email protected]> Reviewed-By: James M Snell <[email protected]> Reviewed-By: Richard Lau <[email protected]> Reviewed-By: Christian Clauss <[email protected]> Reviewed-By: Michaël Zasso <[email protected]>

nodejs-github-bot · 2021-11-01T12:23:45Z

CI: https://ci.nodejs.org/job/node-test-pull-request/40615/
V8 CI: https://ci.nodejs.org/job/node-test-commit-v8-linux/4364/

richardlau · 2021-11-01T12:37:43Z

V8 builds are failing with
e.g. https://ci.nodejs.org/job/node-test-commit-v8-linux/4364/nodes=rhel7-s390x,v8test=v8test/console

12:32:18 ERROR at //testing/gmock/BUILD.gn:11:1: Dependency not allowed.
12:32:18 source_set("gmock") {
12:32:18 ^--------------------
12:32:18 The item //testing/gmock:gmock
12:32:18 can not depend on //third_party/googletest:gtest_config
12:32:18 because it is not in //third_party/googletest:gtest_config's visibility list: [
12:32:18   //third_party/googletest:*
12:32:18 ]
12:32:18 
12:32:18 make: *** [Makefile:272: v8] Error 1

I've tried runs on v14.x-staging and v14.x (e.g. without this PR) and the same failure is present 😞.
v14.x-staging: https://ci.nodejs.org/job/node-test-commit-v8-linux/4365/nodes=rhel7-s390x,v8test=v8test/console
v14.x: https://ci.nodejs.org/job/node-test-commit-v8-linux/4366/nodes=rhel7-s390x,v8test=v8test/console

targos · 2021-11-01T12:44:37Z

Didn't we fix these errors already?

targos · 2021-11-01T12:46:21Z

Or could it be a problem with the GN version that's in CI?

targos · 2021-11-01T12:48:56Z

I think the last time V8 CI was executed for v14.x is in #39990: https://ci.nodejs.org/job/node-test-commit-v8-linux/4257/

richardlau · 2021-11-01T14:56:23Z

Or could it be a problem with the GN version that's in CI?

Yes, most probably by the looks of things. It looks like s390x is the only platform failing this way. When compared to ppc64_le, s390x is on a newer version of gn:

$ ansible -m shell -a "/home/iojs/build-tools/gn --version" test-ibm-rhel7-s390x-?
test-ibm-rhel7-s390x-3 | CHANGED | rc=0 >>
1938 (0153d36)
test-ibm-rhel7-s390x-2 | CHANGED | rc=0 >>
1938 (0153d36)
test-ibm-rhel7-s390x-4 | CHANGED | rc=0 >>
1938 (0153d36)
test-ibm-rhel7-s390x-1 | CHANGED | rc=0 >>
1938 (0153d36)
$ ansible  -m shell -a "/home/iojs/build-tools/gn --version" test-osuosl-centos7-ppc64_le-?
test-osuosl-centos7-ppc64_le-4 | CHANGED | rc=0 >>
1930 (c0a2d23)
test-osuosl-centos7-ppc64_le-3 | CHANGED | rc=0 >>
1930 (c0a2d23)
test-osuosl-centos7-ppc64_le-1 | CHANGED | rc=0 >>
1930 (c0a2d23)
test-osuosl-centos7-ppc64_le-2 | CHANGED | rc=0 >>
1930 (c0a2d23)
$

Looking at the gn commit history

0153d36 Corrects the generation of .stamp file names. by Filip Filmar · 5 weeks ago
de86ec4 Escape spaces in "libs" by Nico Weber · 7 weeks ago
07e2e1b Fix typos in README.md by Ted Pudlik · 7 weeks ago
46b572c Fix configs visibility not always working by Tomasz Śniatowski · 8 weeks ago
69ec4fc C++ modernization improvements. by Brett Wilson · 3 months ago
32aa9d9 Minor style updates. by Brett Wilson · 3 months ago
45aa842 Enable GN to build & run on z/OS by Gaby Baghdadi · 6 months ago
eea3906 Fix a typo in the doc of "generated_file" by Dangyi Liu · 3 months ago
c0a2d23 Update doc/help to say --ninja-executable works for VS Flags by Shezan Baig · 4 months ago

https://gn.googlesource.com/gn/+/46b572ce4ceedfe57f4f84051bd7da624c98bf01 ("Fix configs visibility not always working") sounds like a plausible cause -- that is the newer gn is enforcing visibility rules that were previously not working. I'll try rolling back gn on the s390x machines.

richardlau · 2021-11-01T15:14:04Z

V8 CI: https://ci.nodejs.org/job/node-test-commit-v8-linux/4367/ (✔️)

richardlau · 2021-11-01T17:59:36Z

Looks like Windows CI is timing out on similar tests to the timeouts on master (refs #40684).

nodejs-github-bot · 2021-11-23T15:38:28Z

CI: https://ci.nodejs.org/job/node-test-pull-request/41058/

nodejs-github-bot · 2021-11-24T01:41:01Z

CI: https://ci.nodejs.org/job/node-test-pull-request/41064/ (✔️)

richardlau · 2021-11-24T11:53:43Z

Landed in 0682370...233279d.

PR-URL: #40296 Backport-PR-URL: #40689 Fixes: #40294 Reviewed-By: Jiawen Geng <[email protected]> Reviewed-By: James M Snell <[email protected]> Reviewed-By: Richard Lau <[email protected]> Reviewed-By: Christian Clauss <[email protected]> Reviewed-By: Michaël Zasso <[email protected]>

This comment has been minimized.

Sign in to view

targos approved these changes Nov 1, 2021

View reviewed changes

targos and others added 3 commits November 1, 2021 08:15

richardlau force-pushed the v14.x-py3.10 branch from 5839ebb to 31c42ae Compare November 1, 2021 12:21

This comment has been minimized.

Sign in to view

This was referenced Nov 2, 2021

CI Reliability 2021-11-02 nodejs/reliability#105

Open

CI Reliability 2021-11-03 nodejs/reliability#106

Open

CI Reliability 2021-11-04 nodejs/reliability#107

Open

This was referenced Nov 5, 2021

CI Reliability 2021-11-05 nodejs/reliability#108

Open

CI Reliability 2021-11-06 nodejs/reliability#109

Open

This comment has been minimized.

Sign in to view

github-actions bot mentioned this pull request Nov 24, 2021

CI Reliability 2021-11-24 nodejs/reliability#127

Open

45 tasks

richardlau closed this Nov 24, 2021

richardlau deleted the v14.x-py3.10 branch November 24, 2021 11:53

github-actions bot mentioned this pull request Nov 25, 2021

CI Reliability 2021-11-25 nodejs/reliability#128

Open

38 tasks

This was referenced Nov 26, 2021

CI Reliability 2021-11-26 nodejs/reliability#129

Open

CI Reliability 2021-11-27 nodejs/reliability#130

Open

richardlau mentioned this pull request Oct 28, 2022

build: support Python 3.11 #45191

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[v14.x] Backport Python 3.10 compatibility patches #40689

[v14.x] Backport Python 3.10 compatibility patches #40689

richardlau commented Nov 1, 2021

This comment has been minimized.

richardlau commented Nov 1, 2021

nodejs-github-bot commented Nov 1, 2021 •

edited by richardlau

Loading

richardlau commented Nov 1, 2021

targos commented Nov 1, 2021

targos commented Nov 1, 2021

targos commented Nov 1, 2021

richardlau commented Nov 1, 2021

richardlau commented Nov 1, 2021 •

edited

Loading

This comment has been minimized.

This comment has been minimized.

richardlau commented Nov 1, 2021

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

nodejs-github-bot commented Nov 23, 2021

nodejs-github-bot commented Nov 24, 2021 •

edited by richardlau

Loading

richardlau commented Nov 24, 2021

[v14.x] Backport Python 3.10 compatibility patches #40689

[v14.x] Backport Python 3.10 compatibility patches #40689

Conversation

richardlau commented Nov 1, 2021

This comment has been minimized.

richardlau commented Nov 1, 2021

nodejs-github-bot commented Nov 1, 2021 • edited by richardlau Loading

richardlau commented Nov 1, 2021

targos commented Nov 1, 2021

targos commented Nov 1, 2021

targos commented Nov 1, 2021

richardlau commented Nov 1, 2021

richardlau commented Nov 1, 2021 • edited Loading

This comment has been minimized.

This comment has been minimized.

richardlau commented Nov 1, 2021

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

nodejs-github-bot commented Nov 23, 2021

nodejs-github-bot commented Nov 24, 2021 • edited by richardlau Loading

richardlau commented Nov 24, 2021

nodejs-github-bot commented Nov 1, 2021 •

edited by richardlau

Loading

richardlau commented Nov 1, 2021 •

edited

Loading

nodejs-github-bot commented Nov 24, 2021 •

edited by richardlau

Loading