Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Compiling 10.15.3 failing intermittently in custom build steps #27025

Closed
danielgindi opened this issue Mar 31, 2019 · 4 comments
Closed

Compiling 10.15.3 failing intermittently in custom build steps #27025

danielgindi opened this issue Mar 31, 2019 · 4 comments
Labels
build Issues and PRs related to build files or the CI. windows Issues and PRs related to the Windows platform.

Comments

@danielgindi
Copy link

  • Version: 10.15.3
  • Platform: Windows x64

Running vcbuild.bat on a clean clone - fails, due to obj/global_intermediate/node_protocol.pdl not existing, or other such file in global_intermediate.
A second run on vcbuild.bat succeeds!

I've narrowed it down to the default /m:2.
The custom build steps in the vcxproj are not configured in a way that they depend on each other. The only dependency defined is their order in the project file, and they are configured as single items in the project with a custom build step on each of them.

The result is that with multiple CLs running, the ordering is messed up. Because the factor is 2 concurrent CLs, a second run always succeeds.

I've also tried a workaround with NUMBER_OF_PROCESSORS=1 in the env, but the shell does not allow such an override.
@bnoordhuis bnoordhuis mentioned this issue Mar 31, 2019
Closed
bnoordhuis added a commit to bnoordhuis/io.js that referenced this issue Mar 31, 2019
@bnoordhuis
build: fix inspector dependency resolution
de0ea90 
It was reported that parallel builds on Windows sometimes error because
of missing intermediate files.

On closer inspection I noticed that some files are copied from src/ to
the intermediate build directory in a way where they don't participate
in dependency resolution. Put another way, the build system doesn't
know to wait for the copy to complete because we don't tell it to.

Fix that by not copying around files but instead making the script that
processes them a little smarter about where to find them and where to
store the results.

Fixes: nodejs#27025
@bnoordhuis bnoordhuis added windows Issues and PRs related to the Windows platform. build Issues and PRs related to build files or the CI. labels Mar 31, 2019
@bnoordhuis
Copy link
Member

@danielgindi Can you check if #27026 fixes the issue for you? Thanks.

@danielgindi
Copy link
Author

@bnoordhuis that did it! thanks :-)
will this be backported to 10.x (or even 10.15.3)?

@danielgindi
Copy link
Author

@bnoordhuis You might want to gitignore: src/inspector/node_protocol.json

@danielgindi
Copy link
Author

@bnoordhuis Sorry, but it still fails occassionaly:

  v8_inspector_compress_protocol_json
  Traceback (most recent call last):
    File "tools\compress_json.py", line 15, in <module>
      fp = open(sys.argv[1])
  IOError: [Errno 2] No such file or directory: 'C:\\Users\\DANIEL~1.SIL\\AppData\\Local\\Temp\\gxQEEFVlmG053ZmQZIQe\\n
  ode\\Release\\obj\\global_intermediate\\concatenated_protocol.json'

BethGriggs pushed a commit that referenced this issue Apr 9, 2019
@bnoordhuis @BethGriggs
build: fix inspector dependency resolution
19a30f3 
It was reported that parallel builds on Windows sometimes error because
of missing intermediate files.

On closer inspection I noticed that some files are copied from src/ to
the intermediate build directory in a way where they don't participate
in dependency resolution. Put another way, the build system doesn't
know to wait for the copy to complete because we don't tell it to.

Fix that by not copying around files but instead making the script that
processes them a little smarter about where to find them and where to
store the results.

PR-URL: #27026
Fixes: #27025
Reviewed-By: Eugene Ostroukhov <[email protected]>
Reviewed-By: Richard Lau <[email protected]>
Signed-off-by: Beth Griggs <[email protected]>
BethGriggs pushed a commit that referenced this issue Apr 10, 2019
@bnoordhuis @BethGriggs
build: fix inspector dependency resolution
875dbb0 
It was reported that parallel builds on Windows sometimes error because
of missing intermediate files.

On closer inspection I noticed that some files are copied from src/ to
the intermediate build directory in a way where they don't participate
in dependency resolution. Put another way, the build system doesn't
know to wait for the copy to complete because we don't tell it to.

Fix that by not copying around files but instead making the script that
processes them a little smarter about where to find them and where to
store the results.

PR-URL: #27026
Fixes: #27025
Reviewed-By: Eugene Ostroukhov <[email protected]>
Reviewed-By: Richard Lau <[email protected]>
Signed-off-by: Beth Griggs <[email protected]>
BethGriggs pushed a commit that referenced this issue Apr 10, 2019
@bnoordhuis @BethGriggs
build: fix inspector dependency resolution
b502873 
It was reported that parallel builds on Windows sometimes error because
of missing intermediate files.

On closer inspection I noticed that some files are copied from src/ to
the intermediate build directory in a way where they don't participate
in dependency resolution. Put another way, the build system doesn't
know to wait for the copy to complete because we don't tell it to.

Fix that by not copying around files but instead making the script that
processes them a little smarter about where to find them and where to
store the results.

PR-URL: #27026
Fixes: #27025
Reviewed-By: Eugene Ostroukhov <[email protected]>
Reviewed-By: Richard Lau <[email protected]>
Signed-off-by: Beth Griggs <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
build Issues and PRs related to build files or the CI. windows Issues and PRs related to the Windows platform.
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

build: fix inspector dependency resolution bnoordhuis/io.js
2 participants
@bnoordhuis @danielgindi