npm install fails for latest 0.5.0 release

[trevor-e]

I tried switching my local forked install back to the upstream 0.5.0 version you just released (thanks btw!), and now I'm getting this error:

➜  npm i [email protected]
npm error code 1
npm error path <project>/node_modules/tree-sitter-swift
npm error command failed
npm error command sh -c node-gyp-build
npm error TOUCH Release/obj.target/node_modules/node-addon-api/node_addon_api_except.stamp
npm error   ACTION binding_gyp_tree_sitter_swift_binding_target_wait_for_tree_sitter node_modules/tree-sitter-cli
npm error gyp info it worked if it ends with ok
npm error gyp info using [email protected]
npm error gyp info using [email protected] | darwin | arm64
npm error gyp info find Python using Python version 3.12.2 found at "/opt/homebrew/opt/[email protected]/bin/python3.12"
npm error gyp info spawn /opt/homebrew/opt/[email protected]/bin/python3.12
npm error gyp info spawn args [
npm error gyp info spawn args '/Users/telkins/.nvm/versions/node/v20.12.0/lib/node_modules/npm/node_modules/node-gyp/gyp/gyp_main.py',
npm error gyp info spawn args 'binding.gyp',
npm error gyp info spawn args '-f',
npm error gyp info spawn args 'make',
npm error gyp info spawn args '-I',
npm error gyp info spawn args '<project>/node_modules/tree-sitter-swift/build/config.gypi',
npm error gyp info spawn args '-I',
npm error gyp info spawn args '/Users/telkins/.nvm/versions/node/v20.12.0/lib/node_modules/npm/node_modules/node-gyp/addon.gypi',
npm error gyp info spawn args '-I',
npm error gyp info spawn args '/Users/telkins/Library/Caches/node-gyp/20.12.0/include/node/common.gypi',
npm error gyp info spawn args '-Dlibrary=shared_library',
npm error gyp info spawn args '-Dvisibility=default',
npm error gyp info spawn args '-Dnode_root_dir=/Users/telkins/Library/Caches/node-gyp/20.12.0',
npm error gyp info spawn args '-Dnode_gyp_dir=/Users/telkins/.nvm/versions/node/v20.12.0/lib/node_modules/npm/node_modules/node-gyp',
npm error gyp info spawn args '-Dnode_lib_file=/Users/telkins/Library/Caches/node-gyp/20.12.0/<(target_arch)/node.lib',
npm error gyp info spawn args '-Dmodule_root_dir=<project>/node_modules/tree-sitter-swift',
npm error gyp info spawn args '-Dnode_engine=v8',
npm error gyp info spawn args '--depth=.',
npm error gyp info spawn args '--no-parallel',
npm error gyp info spawn args '--generator-output',
npm error gyp info spawn args 'build',
npm error gyp info spawn args '-Goutput_dir=.'
npm error gyp info spawn args ]
npm error gyp info spawn make
npm error gyp info spawn args [ 'BUILDTYPE=Release', '-C', 'build' ]
npm error make: *** No rule to make target `../node_modules/tree-sitter-cli', needed by `aab630bda7ca2536d7051bf3c2d7e450d17166c1.intermediate'.  Stop.
npm error gyp ERR! build error
npm error gyp ERR! stack Error: `make` failed with exit code: 2
npm error gyp ERR! stack at ChildProcess.<anonymous> (/Users/telkins/.nvm/versions/node/v20.12.0/lib/node_modules/npm/node_modules/node-gyp/lib/build.js:209:23)
npm error gyp ERR! System Darwin 23.2.0
npm error gyp ERR! command "/Users/telkins/.nvm/versions/node/v20.12.0/bin/node" "/Users/telkins/.nvm/versions/node/v20.12.0/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js" "rebuild"
npm error gyp ERR! cwd <project>/node_modules/tree-sitter-swift
npm error gyp ERR! node -v v20.12.0
npm error gyp ERR! node-gyp -v v10.1.0
npm error gyp ERR! not ok

npm error A complete log of this run can be found in: /Users/telkins/.npm/_logs/2024-05-06T19_55_53_273Z-debug-0.log

Oddly this doesn't happen when I install the package locally with something like "file:../ which I see is what the test-npm-package project is doing. Running both the install and postinstall NPM scripts work for me too.

[trevor-e]

I narrowed it down to the custom actions being done in binding.gyp:

      "actions": [
          {
	      "action_name": "wait_for_tree_sitter",
	      "action": ["node", "scripts/wait-for-tree-sitter.js"],
	      "inputs": [],
	      "outputs": ["node_modules/tree-sitter-cli"]
	  },
          {
	      "action_name": "generate_header_files",
	      "inputs": [
	          "grammar.js",
		  "node_modules/tree-sitter-cli"
	      ],
	      "outputs": [
	          "src/grammar.json",
		  "src/node-types.json",
		  "src/parser.c",
		  "src/tree_sitter",
	      ],
	      "action": ["tree-sitter", "generate", "--no-bindings"],
	  }
      ]

I see the node-gyp build step produces a Makefile that tries to call tree-sitter-cli and fails, which is what the error message in the OP is referring to. I tried a few things but couldn't figure out a way to fix this using the custom GYP setup going on in this repo, as a workaround I'm using a fork in trevor-e#3 which reverts gitignoring some of the generated files. I see in the README that you won't accept PRs that check in the generated files so this seems unlikely to get merged.

[alex-pinkus]

This is very strange. I added that action because without it, npm install was flaky; the core issue is that tree-sitter-cli downloads its binary asynchronously at install time. But it seems like something doesn't work quite right...

I wonder if I should just switch NPM publishing over to the with-generated-files branch.

[trevor-e]

Yea I think switching to the 'with-generated-files' branch, at least for publishing, is probably the easiest. If you release 0.5.0 on GitHub I'm happy to test that out before publishing it to npm.

[alex-pinkus]

ok, just tagged the 0.5.0 release and updated the generated files branch. Let me know how your testing goes. I added a binding.gyp to that branch that doesn't try to generate files (since it doesn't need to 😛). Let me know how your testing goes and if there are any other files that need to be added.

[trevor-e]

Sorry for the delay! I tried the with-generated-files branch and I'm getting a new error:

npm error ../bindings/node/binding.cc:3:10: fatal error: 'nan.h' file not found
npm error #include "nan.h"

The binding.cc file looks stale since it's still referencing nan.h and not napi.h. I checked out the branch locally, ran tree-sitter generate, and see there are a bunch of changes that weren't committed. I think the write-generated-grammar.sh script might be out of date and needs to add some extra files like you said.