ENOTCACHED / cache mode is 'only-if-cached' but no cached response available

[ixxie]

I'm not sure this isn't related to npmlock2nix, but now that my build works I get this:

[...]
building
90m..................] / rollbackFailedOptional: verb npm-session 7c5d0f9d32bc49c
> [email protected] preinstall /build/myproject
> /build/myproject/node_modules/.hooks/preinstall
npm ERR! code ENOTCACHED install loadAllDepsIntoIdealTree
npm ERR! request to https://registry.npmjs.org/polka failed: cache mode is 'only-if-cached' but no cached response available.

npm ERR! A complete log of this run can be found in:
npm ERR!     /build/myproject/.npm/_logs/2020-12-26T15_18_40_514Z-debug.log

builder for '/nix/store/fxhblr5b6rbzbz6mxj5y414f8b1pcz60-TODO-0.0.1.drv' failed with exit code 1
error: build of '/nix/store/fxhblr5b6rbzbz6mxj5y414f8b1pcz60-TODO-0.0.1.drv' failed

Is it possible that this is related to svanderburg/node2nix#106 (and also svanderburg/node2nix#70) and that this is because of bundled package-lock.json files in some packages in the dependency tree?

If this is so, I guess npmlock2nix needs to do something like svanderburg/node2nix#106 (comment)

[andir]

I think is might be related with the other projects but neither of their solutions are likey to work here. Can you pust the relevant parts of your package.json and package-lock.json? My guess is that it is just another variant of the git references. I've multiple branches that add better/different support of them but all of them require massive cleanup.

[ixxie]

You can reproduce this with the Sapper template: npx degit "sveltejs/sapper-template#rollup" sappertemplate

I used a vanilla npmlock2nix shell.nix like this:

{ pkgs ? import <nixpkgs> {}, npmlock2nix ? import <npmlock2nix> { inherit pkgs; } }:
npmlock2nix.shell {
  src = ./.;
  nodejs = pkgs.nodejs-14_x;
}

Based on the error message in the post above, it seems that polka is the culprit.

[ixxie]

Any ideas about even a hacky/temporary workaround for this? I currently cannot use npmlock2nix for my project anymore due to this issue...

[dhess]

This issue is unfortunately fairly common, in my brief experience so far with npmlock2nix. The really bad bit is that it can hit you out of the blue when you add a new dependency, meaning that everything might work fine for a few weeks and then, suddenly, you've hit a a dead end.

[dhess]

On that note, I'm not sure whether this is the same issue or not, but if you try to set up Tailwind CSS for use with a create-react-app app as advised in the Tailwind docs here, you get the following error:

error: builder for '/nix/store/0pn16y1r4r02sfp8hfk81ca4g33j8r8p-react-nix-craco-0.1.0.drv' failed with exit code 1;
       last 10 log lines:
       > 90m..................] / rollbackFailedOptional: verb npm-session fa56488e4e6b8c7
       > > [email protected] preinstall /private/tmp/nix-build-react-nix-craco-0.1.0.drv-0
       > > /private/tmp/nix-build-react-nix-craco-0.1.0.drv-0/node_modules/.hooks/preinstall
       > patching script interpreter paths in .
       > npm ERR! code ENOTCACHEDrver: sill fetchPackage[0m
       > npm ERR! request to https://registry.npmjs.org/@tailwindcss%2fpostcss7-compat failed: cache mode is 'only-if-cached' but no cached response available.
       > 
       > npm ERR! A complete log of this run can be found in:
       > npm ERR!     /private/tmp/nix-build-react-nix-craco-0.1.0.drv-0/.npm/_logs/2021-08-06T01_52_58_999Z-debug.log
       > 
       For full logs, run 'nix log /nix/store/0pn16y1r4r02sfp8hfk81ca4g33j8r8p-react-nix-craco-0.1.0.drv'.
error: 1 dependencies of derivation '/nix/store/l0dca36c585yy6y20cagbh7ll9ksfcc9-nix-shell-env.drv' failed to build

Unfortunately, I don't know how to work around this, as create-react-app requires this special compatibility release of tailwindcss.

[happysalada]

I ran into the same error on my project, it seems to be a dependency that was pulled out of npm.
If you remove your package-lock.json and recreate it (with npm install), for me it removed the problematic dependency.

unfortunately I got the same error with another dependency after that.

[deliciouslytyped]

I may have gained some insight into the internals of this, but not much. Nag me to take some notes at some point.
https://github.com/npm/pacote appears to be related (and is part of an npm checkout).
Do note that this code has churned a bit between node 6 and 7.

Here's some log from where we got with peeking at the above mentioned issue today, but it doesn't seem to be the same as a related looking issue we've been fighting;

38 verbose stack Error: request to https://registry.npmjs.org/polka failed: cache mode is 'only-if-cached' but no cached response available.
38 verbose stack     at /nix/store/dpn8hvlq7v703kz3x3npx6522x89kb57-nodejs-16.3.0/lib/node_modules/npm/node_modules/make-fetch-happen/index.js:133:21
38 verbose stack     at async Arborist.[nodeFromEdge] (/nix/store/dpn8hvlq7v703kz3x3npx6522x89kb57-nodejs-16.3.0/lib/node_modules/npm/node_modules/@npmcli/arborist/lib/arborist/build-ideal-tree.js:934:19)
38 verbose stack     at async Arborist.[buildDepStep] (/nix/store/dpn8hvlq7v703kz3x3npx6522x89kb57-nodejs-16.3.0/lib/node_modules/npm/node_modules/@npmcli/arborist/lib/arborist/build-ideal-tree.js:880:11)
38 verbose stack     at async Arborist.buildIdealTree (/nix/store/dpn8hvlq7v703kz3x3npx6522x89kb57-nodejs-16.3.0/lib/node_modules/npm/node_modules/@npmcli/arborist/lib/arborist/build-ideal-tree.js:218:7)
38 verbose stack     at async Promise.all (index 1)
38 verbose stack     at async Arborist.reify (/nix/store/dpn8hvlq7v703kz3x3npx6522x89kb57-nodejs-16.3.0/lib/node_modules/npm/node_modules/@npmcli/arborist/lib/arborist/reify.js:136:5)
38 verbose stack     at async Install.install (/nix/store/dpn8hvlq7v703kz3x3npx6522x89kb57-nodejs-16.3.0/lib/node_modules/npm/lib/install.js:150:5)  

This log is using a lock file generated by npm 6 so that npmlock2nix can handle it, and then we use npm 7 because the output seems easier to make sense of and use in debugging.

[dhess]

FYI, I have yet to encounter this with https://github.com/serokell/nix-npm-buildpackage (edit not with npm v6, anyway; see my later comment below).

[evanjs]

Unfortunately, I have encountered this issue with nix-npm-buildpackage, at least using NPM v7.

Under the thread for serokell/nix-npm-buildpackage#33, @yorickvP linked npm/cli#2160, which seems to point to a fix for pacote being pushed with npm/pacote@500a34f.

Based on the versions associated with nodejs/npm, this fix should be available as part of pacote 11.3.2 / nodejs 7.11.0 if I understand correctly.

Still totally lost with where that leaves us with all these projects, as they seem to be mostly up to date as far as nodejs/npm versions go.

[dhess]

Unfortunately, I have encountered this issue with nix-npm-buildpackage, at least using NPM v7.

Yes, I have as well with npm v7, but not with npm v6.

[yorickvP]

Hi! When I looked into it, I think my conclusion was that npm v7 now also requires the cache to exist during the npm prune step. Copying the cache into that step might help.

[milahu]

good news : ) should be fixed in #93 (comment)

[dhess]

good news : ) should be fixed in #93 (comment)

I tried your branch here: milahu@3970a1b

I don't get ENOTCACHED anymore, but I do get this failure when trying to build a React project:

 nix-build -A packages.x86_64-darwin.frontend
warning: Git tree '/Users/dhess/git/primer-frontend-web' is dirty
these 2 derivations will be built:
  /nix/store/aafdmp6zklgan5nw863w29q33lp0m94j-primer-frontend-web-0.1.0.drv
  /nix/store/lwsbgvhylbvw2hfpa12py680f091xvc3-primer-frontend-web-0.1.0.drv
building '/nix/store/aafdmp6zklgan5nw863w29q33lp0m94j-primer-frontend-web-0.1.0.drv'...
patching sources
configuring
no configure script, doing nothing
building
run 'npm install' ...
[ .................] - fetchMetadata: sill install loadAllDepsIntoIdealTree
error: builder for '/nix/store/aafdmp6zklgan5nw863w29q33lp0m94j-primer-frontend-web-0.1.0.drv' failed with exit code 1;
       last 6 log lines:
       > patching sources
       > configuring
       > no configure script, doing nothing
       > building
       > run 'npm install' ...
       > [ .................] - fetchMetadata: sill install loadAllDepsIntoIdealTree
       For full logs, run 'nix log /nix/store/aafdmp6zklgan5nw863w29q33lp0m94j-primer-frontend-web-0.1.0.drv'.
error: 1 dependencies of derivation '/nix/store/lwsbgvhylbvw2hfpa12py680f091xvc3-primer-frontend-web-0.1.0.drv' failed to build

The project works with nix-npm-buildpackage.

[milahu]

maybe thats too silent. can you test d0c89d1?

[dhess]

maybe thats too silent. can you test d0c89d1?

Sorry, no luck — back to ENOTCACHED:

 nix-build -A packages.x86_64-darwin.frontend                                                        ~/git/primer-frontend-web
warning: Git tree '/Users/dhess/git/primer-frontend-web' is dirty
these 2 derivations will be built:
  /nix/store/gskhdrfcz08fhkq5bm03ny52flprrw62-primer-frontend-web-0.1.0.drv
  /nix/store/xs4aakg1v6580sa0kxr428fvaghi58rl-primer-frontend-web-0.1.0.drv
building '/nix/store/gskhdrfcz08fhkq5bm03ny52flprrw62-primer-frontend-web-0.1.0.drv'...
patching sources
configuring
no configure script, doing nothing
building
90m..................] / rollbackFailedOptional: verb npm-session 707adbebf5cc376
> [email protected] preinstall /private/tmp/nix-build-primer-frontend-web-0.1.0.drv-0
> /private/tmp/nix-build-primer-frontend-web-0.1.0.drv-0/node_modules/.hooks/preinstall
patching script interpreter paths in .
npm ERR! code ENOTCACHED install loadAllDepsIntoIdealTree
npm ERR! request to https://registry.npmjs.org/@tailwindcss%2fpostcss7-compat failed: cache mode is 'only-if-cached' but no cached response available.

npm ERR! A complete log of this run can be found in:
npm ERR!     /private/tmp/nix-build-primer-frontend-web-0.1.0.drv-0/.npm/_logs/2021-09-03T00_46_27_051Z-debug.log

error: builder for '/nix/store/gskhdrfcz08fhkq5bm03ny52flprrw62-primer-frontend-web-0.1.0.drv' failed with exit code 1;
       last 10 log lines:
       > 90m..................] / rollbackFailedOptional: verb npm-session 707adbebf5cc376
       > > [email protected] preinstall /private/tmp/nix-build-primer-frontend-web-0.1.0.drv-0
       > > /private/tmp/nix-build-primer-frontend-web-0.1.0.drv-0/node_modules/.hooks/preinstall
       > patching script interpreter paths in .
       > npm ERR! code ENOTCACHED install loadAllDepsIntoIdealTree
       > npm ERR! request to https://registry.npmjs.org/@tailwindcss%2fpostcss7-compat failed: cache mode is 'only-if-cached' but no cached response available.
       > 
       > npm ERR! A complete log of this run can be found in:
       > npm ERR!     /private/tmp/nix-build-primer-frontend-web-0.1.0.drv-0/.npm/_logs/2021-09-03T00_46_27_051Z-debug.log
       > 
       For full logs, run 'nix log /nix/store/gskhdrfcz08fhkq5bm03ny52flprrw62-primer-frontend-web-0.1.0.drv'.
error: 1 dependencies of derivation '/nix/store/xs4aakg1v6580sa0kxr428fvaghi58rl-primer-frontend-web-0.1.0.drv' failed to build

[milahu]

can you share your package.json and package-lock.json files?

[dhess]

can you share your package.json and package-lock.json files?

Not for that particular project, but here's the one it's based on, where I first encountered this issue. With milahu@3970a1b (and master), this commit works fine: https://github.com/hackworthltd/react-nix-craco-public/commit/d184c089e36f40188d9f5f6b4b493b2f736226c8

But once you try to add Tailwind & its dependencies, it breaks with the same error that I showed above: https://github.com/hackworthltd/react-nix-craco-public/commit/e4dd29ae364d0625c81ee0abfc61e96397bd7e56

[dhess]

intead of "cliui": "^6.0.0", it should say "cliui": "file:///nix/store/xxxxxxxxxxxx-cliui-6.0.0.tgz",

Hmm, are you sure that's a good solution? @andir and I discovered that Nix store paths were being written into the package-lock.json here: #85 (comment) and that didn't seem like a good idea, a) because of the 211-character path length limitation (which can apparently be worked around), and b) because it wouldn't be a very useful package-lock.json file for people who aren't using Nix.

[milahu]

Hmm, are you sure that's a good solution?

no ... the problem started with tailwindcss which is declared as

    "tailwindcss": "npm:@tailwindcss/postcss7-compat@^2.2.9",

    "tailwindcss": {
      "version": "npm:@tailwindcss/[email protected]",
      "resolved": "https://registry.npmjs.org/@tailwindcss/postcss7-compat/-/postcss7-compat-2.2.9.tgz",

in the patched lockfile

    "tailwindcss": {
      "resolved": "file:///nix/store/zcggqgvhw919c209c7f4hpfq3ki6a1bw-postcss7-compat-2.2.9.tgz",
      "version": "@tailwindcss/[email protected]"
    },

the patched version should be 2.2.9 to make it work (workaround, maybe its a bug in npm)

          echo patch lockfile for debug
          #sed -i 's|"@tailwindcss/postcss7-compat@^2.2.9"|"npm:@tailwindcss/[email protected]"|' package.json
          #sed -i 's|"version":"@tailwindcss/[email protected]"|"version":"npm:@tailwindcss/[email protected]"|' package-lock.json
          #npm verb stack Error: request to https://registry.npmjs.org/@tailwindcss%2fpostcss7-compat failed: cache mode is 'only-if-cached' but no cached response available.

          sed -i 's|"@tailwindcss/postcss7-compat@^2.2.9"|"2.2.9"|' package.json
          sed -i 's|"version":"@tailwindcss/[email protected]"|"version":"2.2.9"|' package-lock.json
          # works

          echo "use npm version $(npm --version) in $(${which}/bin/which npm)"
          # use npm version 6.14.14 in /nix/store/pvnvfvimzldwbx08pj53iaf3z1wmwl94-nodejs-14.17.5/bin/npm
          echo "run 'npm install' ..."
          npm install --offline --nodedir=${nodeSource nodejs} --progress=false # --loglevel=verbose

it wouldn't be a very useful package-lock.json file for people who aren't using Nix.

the patched json files are only used in npmlock2nix

[dhess]

The Tailwind version string comes straight from the documentation here: https://tailwindcss.com/docs/guides/create-react-app

It works fine with nix-npm-buildpackage.

[milahu]

please try 47e3003

with "patching dependencies" i mean the function patchDependency in npmlock2nix/internal.nix

[dhess]

please try 47e3003

Closer, but still no cigar. It works fine with https://github.com/hackworthltd/react-nix-craco-public/commit/aff06f32d4befcca2a633c95a4295fec1f332d82

However, with the more complicated (private) repo I tested yesterday, it fails with this log: https://gist.github.com/dhess/084878f17db5859c0bf70f251e235731

(Edit I recreated the package-lock.json file from scratch and the error is the same.)

(Edit #2 Exactly the same problem with npm 7 (using nodejs-16_x from nixpkgs-unstable).)

[milahu]

probably another bug in npmlock2nix

a different version of babel-loader was detected higher up in the tree

-> why is version 8.1.0 not installed deep in /build/node_modules/react-scripts/node_modules/

possible reason: a github version github:owner/repo is replaced with wildcard version *

to debug, add this to npmlock2nix/internal.nix

        cat package-lock.json | ${jq}/bin/jq -r | grep -C3 babel-loader
        cat package.json | ${jq}/bin/jq -r | grep babel-loader

in the first buildPhase, before npm install --offline
add jq (for json pretty-printing) to dependencies in internal.nix line 1

then compare with your input files package-lock.json and package.json

# nix-shell -p jq
cat package-lock.json | jq -r | grep -C3 babel-loader
cat package.json | jq -r | grep babel-loader

[dhess]

I guess there's no real harm in making the package.json and package-lock.json files from the project available to you, which would make this a lot easier, right? :) So here you go:

package.json: https://gist.github.com/dhess/0ba93bd8685b14f88672ae8068a775b2

package-lock.json: https://gist.githubusercontent.com/dhess/1cced868803bbeda16d1b27a02216dbc/raw/346a84a241d4c2e15460df731199977d3eba4071/gistfile1.txt

[milahu]

could be a bug in npm 6

patched lockfile, as produced by npmlock2nix

{
  "dependencies": {
    "react-scripts": {
      "version": "4.0.3",
      "resolved": "file:///nix/store/jfkjd4w2nfzwa93q97h4db6r47m06b3a-react-scripts-4.0.3.tgz",
      "dependencies": {
        "babel-loader": {
          "version": "8.1.0",
          "resolved": "file:///nix/store/873g019wn2x550f4sdssjdl7nlgy9n3g-babel-loader-8.1.0.tgz"
        },
      },
      "requires": {
        "babel-loader": "8.1.0"
      }
    }
  }
}

workaround in npmlock2nix:

in react-scripts.requires, replace "babel-loader": "8.1.0"
with "babel-loader": "file:///nix/store/873g019wn2x550f4sdssjdl7nlgy9n3g-babel-loader-8.1.0.tgz"

thats how i fixed the ENOTCACHED bug with my github dependencies

[dhess]

Thanks for your hard work on this!

I think I'll continue using nix-npm-buildpackage for the time being, since it works without any workarounds required.

[milahu]

weird, with nix-npm-buildpackage i also get the warning from react-scripts

this warning "should" be fixed by npm install --legacy-bundling
which disables all deduplication and creates a "deep" node_modules

i say "should" cos npm install --legacy-bundling does not work >: (
i still get a "flat" node_modules, with a global /node_modules/babel-loader

"dear npm, are my commands a joke to you?" ...

your case with "flat" node_modules

= deduplication on, note the "deduped" annotations

++ npm ls babel-loader
[email protected] /build/react-nix-craco-public
+-- @storybook/[email protected]
| `-- @storybook/[email protected]
|   `-- @storybook/[email protected]
|     `-- [email protected]  deduped
+-- @storybook/[email protected]
| +-- @storybook/[email protected]
| | `-- @storybook/[email protected]
| |   `-- @storybook/[email protected]
| |     `-- [email protected]  deduped
| `-- @storybook/[email protected]
|   `-- [email protected]
`-- [email protected]
  `-- [email protected]

"global" version of babel-loader, shared by @storybook/builder-webpack4 and @storybook/manager-webpack4

++ head node_modules/babel-loader/package.json
{
  "name": "babel-loader",
  "version": "8.2.2",

"local" version of babel-loader, only for react-scripts

++ head node_modules/react-scripts/node_modules/babel-loader/package.json
{
  "name": "babel-loader",
  "version": "8.1.0",

we install packages by symlinks, so we have no need for deduplication (and its side-effects)
nope, currently we copy packages to node_modules

maybe this can be solved with buildInputs for dependencies and nativeBuildInputs for devDependencies
we could use hardlinks like pnpm or fallback to copy if hardlinks dont work

[jmgilman]

I'm getting this issue with the wrappy package (v1.0.2). It looks like #93 was merged, so it doesn't appear to solve the issue in my case. nix-npm-buildpackage also fails with the same error.

Here is my lockfile.

[LucianU]

I'm also getting the issue while trying to install gatsby-cli on aarch64-darwin with nixpkgs-22.05-darwin.

[nomeata]

Yup, running into this due to wrappy. @jmgilman , did you find a solution or work-around?

[jmgilman]

9/10 times it's related to package locks that are missing hashes. This is especially prevalent in older versions of npm. I'd have to go track down the issue again, but I did eventually find out that, against all common sense, hashes are technically optional in lock files.

[nomeata]

Thanks! My work-around was using node2nix, which somehow did the job.

[klarkc]

Happened with me:

  npm.v1.node_modules { src = inputs.simplecrypto; } + /node_modules;

package-lock.json

npm ERR! code ENOTCACHED
npm ERR! request to https://registry.npmjs.org/aes-js failed: cache mode is 'only-if-cached' but no cached response available.

[vec715]

any updates on this? still actual in 2024 on 24.05