Node required to be installed outside of Bazel #54

promiseofcake · 2018-01-17T21:30:19Z

It seems like node is externally required (installed outside of Bazel) to process '@yarn_modules//' targets. Is there a way we can use a fetched node for this similarly to as we do for go-related builds in bazelbuild/rules_go?

The text was updated successfully, but these errors were encountered:

pcj · 2018-01-17T21:33:06Z

Can you elaborate? Node should be installed automatically. I'm not aware of how you're using @yarn_modules// that would require a node install outside of bazel.

promiseofcake · 2018-01-17T21:43:44Z

Sure, so I have node installed via nvm on my machine (for development outside of Bazel), and when I removed it I started having issues with my builds. Here is how we are using this repo now:

# WORKSPACE
git_repository(
    name = "org_pubref_rules_node",
    commit = "2b4b38dcb76eba98419c932c98bd22fe6e322c9f",
    remote = "https://github.com/pubref/rules_node.git",
)

git_repository(
    name = "org_pubref_rules_protobuf",
    remote = "https://github.com/pubref/rules_protobuf",
    commit = "563b674a2ce6650d459732932ea2bc98c9c9a9bf", # Nov 28, 2017
)

load("@org_pubref_rules_node//node:rules.bzl", "node_repositories", "yarn_modules")
node_repositories()

# required to initialize dependencies previously out of our control via @org_pubref_rules_node
yarn_modules(
    name = "yarn_modules",
    deps = {
        "google-protobuf": "3.4.0",
        "grpc": "1.6.0"
    },
)
node_proto_repositories()

# BUILD
node_proto_library(
    name = "foojs_grpc",
    imports = ["protobufs/imports"],
    protos = ["protos/foo/foo.proto"],
    verbose = 1,
    with_grpc = True,
)

node_binary(
    name = "foojs",
    main = "index.js",
    deps = [
        ":foojs_grpc",
        "@yarn_modules//:_all_",
    ],
)

Output

bazel build //foo/js:foojs
...........
DEBUG: /private/var/tmp/_bazel_lucas/4e96d1a6d41518319f74f637163517c2/external/bazel_tools/tools/build_defs/pkg/pkg.bzl:197:9: @//foo/js:foojs_bundle: you provided a non dictionary to the pkg_tar `files` attribute. This attribute was renamed to `srcs`. Consider renaming it in your BUILD file.
ERROR: /path/to/foo/js/BUILD:3:1: no such package '@yarn_modules//': Traceback (most recent call last):
    File "/private/var/tmp/_bazel_lucas/4e96d1a6d41518319f74f637163517c2/external/org_pubref_rules_node/node/internal/yarn_modules.bzl", line 59
        execute(ctx, [node, yarn_js, "install"], qui...)
    File "/private/var/tmp/_bazel_lucas/4e96d1a6d41518319f74f637163517c2/external/org_pubref_rules_node/node/internal/yarn_modules.bzl", line 4, in execute
        fail((" ".join(cmds) + ("failed: \nST...))))
/private/var/tmp/_bazel_lucas/4e96d1a6d41518319f74f637163517c2/external/node/bin/node /private/var/tmp/_bazel_lucas/4e96d1a6d41518319f74f637163517c2/external/yarn/bin/yarn.js installfailed:
STDOUT:
yarn install v1.0.1
info No lockfile found.
[1/4] Resolving packages...
[2/4] Fetching packages...
[3/4] Linking dependencies...
[4/4] Building fresh packages...
info Visit https://yarnpkg.com/en/docs/cli/install for documentation about this command.

STDERR:
warning package.json: No license field
warning [email protected]: No license field
error /private/var/tmp/_bazel_lucas/4e96d1a6d41518319f74f637163517c2/external/yarn_modules/node_modules/grpc: Command failed.
Exit code: 127
Command: ./node_modules/.bin/node-pre-gyp install --fallback-to-build --library=static_library
Arguments:
Directory: /private/var/tmp/_bazel_lucas/4e96d1a6d41518319f74f637163517c2/external/yarn_modules/node_modules/grpc
Output:
env: node: No such file or directory
 and referenced by '//foo/js:foojs'
ERROR: Analysis of target '//foo/js:foojs' failed; build aborted: no such package '@yarn_modules//': Traceback (most recent call last):
    File "/private/var/tmp/_bazel_lucas/4e96d1a6d41518319f74f637163517c2/external/org_pubref_rules_node/node/internal/yarn_modules.bzl", line 59
        execute(ctx, [node, yarn_js, "install"], qui...)
    File "/private/var/tmp/_bazel_lucas/4e96d1a6d41518319f74f637163517c2/external/org_pubref_rules_node/node/internal/yarn_modules.bzl", line 4, in execute
        fail((" ".join(cmds) + ("failed: \nST...))))
/private/var/tmp/_bazel_lucas/4e96d1a6d41518319f74f637163517c2/external/node/bin/node /private/var/tmp/_bazel_lucas/4e96d1a6d41518319f74f637163517c2/external/yarn/bin/yarn.js installfailed:
STDOUT:
yarn install v1.0.1
info No lockfile found.
[1/4] Resolving packages...
[2/4] Fetching packages...
[3/4] Linking dependencies...
[4/4] Building fresh packages...
info Visit https://yarnpkg.com/en/docs/cli/install for documentation about this command.

STDERR:
warning package.json: No license field
warning [email protected]: No license field
error /private/var/tmp/_bazel_lucas/4e96d1a6d41518319f74f637163517c2/external/yarn_modules/node_modules/grpc: Command failed.
Exit code: 127
Command: ./node_modules/.bin/node-pre-gyp install --fallback-to-build --library=static_library
Arguments:
Directory: /private/var/tmp/_bazel_lucas/4e96d1a6d41518319f74f637163517c2/external/yarn_modules/node_modules/grpc
Output:
env: node: No such file or directory
INFO: Elapsed time: 21.171s
FAILED: Build did NOT complete successfully (7 packages loaded)

Relink Node via NVM

export NVM_DIR="/Users/lucas/.nvm"
[ -s "$NVM_DIR/nvm.sh" ] && \. "$NVM_DIR/nvm.sh"

Then it succeeds.

promiseofcake · 2018-01-17T21:45:48Z

Also attempting to just build the yarn_module targets I get the same error:

bazel build @yarn_modules//:_all_
...........
ERROR: Skipping '@yarn_modules//:_all_': no such package '@yarn_modules//': Traceback (most recent call last):
	File "/private/var/tmp/_bazel_lucas/4e96d1a6d41460169f74f268073930c2/external/org_pubref_rules_node/node/internal/yarn_modules.bzl", line 59
		execute(ctx, [node, yarn_js, "install"], qui...)
	File "/private/var/tmp/_bazel_lucas/4e96d1a6d41460169f74f268073930c2/external/org_pubref_rules_node/node/internal/yarn_modules.bzl", line 4, in execute
		fail((" ".join(cmds) + ("failed: \nST...))))
/private/var/tmp/_bazel_lucas/4e96d1a6d41460169f74f268073930c2/external/node/bin/node /private/var/tmp/_bazel_lucas/4e96d1a6d41460169f74f268073930c2/external/yarn/bin/yarn.js installfailed:
STDOUT:
yarn install v1.0.1
info No lockfile found.
[1/4] Resolving packages...
[2/4] Fetching packages...
[3/4] Linking dependencies...
[4/4] Building fresh packages...
info Visit https://yarnpkg.com/en/docs/cli/install for documentation about this command.

STDERR:
warning package.json: No license field
warning [email protected]: No license field
error /private/var/tmp/_bazel_lucas/4e96d1a6d41460169f74f268073930c2/external/yarn_modules/node_modules/grpc: Command failed.
Exit code: 127
Command: ./node_modules/.bin/node-pre-gyp install --fallback-to-build --library=static_library
Arguments:
Directory: /private/var/tmp/_bazel_lucas/4e96d1a6d41460169f74f268073930c2/external/yarn_modules/node_modules/grpc
Output:
env: node: No such file or directory
WARNING: Target pattern parsing failed.
ERROR: no such package '@yarn_modules//': Traceback (most recent call last):
	File "/private/var/tmp/_bazel_lucas/4e96d1a6d41460169f74f268073930c2/external/org_pubref_rules_node/node/internal/yarn_modules.bzl", line 59
		execute(ctx, [node, yarn_js, "install"], qui...)
	File "/private/var/tmp/_bazel_lucas/4e96d1a6d41460169f74f268073930c2/external/org_pubref_rules_node/node/internal/yarn_modules.bzl", line 4, in execute
		fail((" ".join(cmds) + ("failed: \nST...))))
/private/var/tmp/_bazel_lucas/4e96d1a6d41460169f74f268073930c2/external/node/bin/node /private/var/tmp/_bazel_lucas/4e96d1a6d41460169f74f268073930c2/external/yarn/bin/yarn.js installfailed:
STDOUT:
yarn install v1.0.1
info No lockfile found.
[1/4] Resolving packages...
[2/4] Fetching packages...
[3/4] Linking dependencies...
[4/4] Building fresh packages...
info Visit https://yarnpkg.com/en/docs/cli/install for documentation about this command.

STDERR:
warning package.json: No license field
warning [email protected]: No license field
error /private/var/tmp/_bazel_lucas/4e96d1a6d41460169f74f268073930c2/external/yarn_modules/node_modules/grpc: Command failed.
Exit code: 127
Command: ./node_modules/.bin/node-pre-gyp install --fallback-to-build --library=static_library
Arguments:
Directory: /private/var/tmp/_bazel_lucas/4e96d1a6d41460169f74f268073930c2/external/yarn_modules/node_modules/grpc
Output:
env: node: No such file or directory
INFO: Elapsed time: 15.231s
FAILED: Build did NOT complete successfully (0 packages loaded)

promiseofcake · 2018-01-17T21:54:19Z

Hm maybe the issue is one of the transitive dependencies from the yarn_modules has a node command as part of their instal stanza in their package.json, related to #40?

pcj · 2018-01-17T21:57:57Z

OK I see yeah that looks like a bug. The node-pre-gyp or related must have a shebang script line line #!/usr/bin/env node. We need to modify the PATH to prepend the bazel-installed one prior to calling the node yarn.js part.

I'll plan to fix this later this afternoon or tonight unless you want to do it first.

pcj · 2018-01-17T21:59:04Z

Same symptom as #40 but in a different place.

pcj · 2018-01-18T01:28:58Z

Looks like you're on darwin, yes? Unable to repro on my linux laptop.

promiseofcake · 2018-01-18T02:09:23Z

Yup Darwin, hm, is there a place I can should look to try to add debugging to try to figure out what is going on?

promiseofcake · 2018-01-18T14:43:37Z

@pcj -- so I think the problem was I was using a hash that did not have #41 merged in. I was using2b4b38dcb76eba98419c932c98bd22fe6e322c9f, which is the linked hash from the HEAD of pubref/rules_protobuf. I will bump my rules_node version and it should then be solved, perhaps if nothing is blocking I will modify pubref/rules_protobuf to reference that latest hash as well.

promiseofcake closed this as completed Jan 18, 2018

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Node required to be installed outside of Bazel #54

Node required to be installed outside of Bazel #54

promiseofcake commented Jan 17, 2018

pcj commented Jan 17, 2018

promiseofcake commented Jan 17, 2018

promiseofcake commented Jan 17, 2018

promiseofcake commented Jan 17, 2018

pcj commented Jan 17, 2018

pcj commented Jan 17, 2018

pcj commented Jan 18, 2018

promiseofcake commented Jan 18, 2018

promiseofcake commented Jan 18, 2018

Node required to be installed outside of Bazel #54

Node required to be installed outside of Bazel #54

Comments

promiseofcake commented Jan 17, 2018

pcj commented Jan 17, 2018

promiseofcake commented Jan 17, 2018

promiseofcake commented Jan 17, 2018

promiseofcake commented Jan 17, 2018

pcj commented Jan 17, 2018

pcj commented Jan 17, 2018

pcj commented Jan 18, 2018

promiseofcake commented Jan 18, 2018

promiseofcake commented Jan 18, 2018