Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Unable to install npm package from private registry #25924

Closed
bavoco opened this issue Sep 28, 2024 · 9 comments · Fixed by #26473
Closed

Unable to install npm package from private registry #25924

bavoco opened this issue Sep 28, 2024 · 9 comments · Fixed by #26473
Assignees
@marvinhagemeister
Labels
bug Something isn't working correctly install node compat

Comments

@bavoco
Copy link

bavoco commented Sep 28, 2024

Version: Deno 2.0.0-rc.7
OS: Windows 11

package.json

{
  "name": "repro",
  "version": "1.0.0",
  "description": "",
  "main": "index.js",
  "scripts": {},
  "type": "module",
  "author": "",
  "license": "ISC",
  "dependencies": {
    "@myscope/packagename": "^14.6.2"
  }
}

deno.json

{}

.npmrc

@myscope:registry=https://some.gitlab.server.com/api/v4/packages/npm/
//some.gitlab.server.com/api/v4/packages/npm/:_authToken=supersecret

When I run deno install, the output is:

error: Failed caching npm package '@myscope/[email protected]'.

Caused by:
    Could not find npm package tarball at: https://some.gitlab.server.com/api/v4/projects/12345/packages/npm/@myscope/packagename/-/@myscope/packagename-14.6.2.tgz

The resulting node_modules folder looks like:
image

Notice that it did download the dependencies of the package I was trying to install. So although it says it could not find a package tarball, it definitely did find something at the url and it is using the authtoken from the npmrc correcly otherwise it would have had a 404.

The output of `DENO_DIR=C:/TEMP/deno DENO_LOG=debug deno install` is: 
DEBUG RS - deno_config::deno_json:780 - Config file found at 'C:\Users\me.me\Downloads\repro\deno.json'
DEBUG RS - deno_config::workspace::discovery:263 - package.json file found at 'C:\Users\me.me\Downloads\repro\package.json'
DEBUG RS - deno::args:931 - Finished config loading.
DEBUG RS - deno::npm::managed::resolution:282 - Running npm resolution.
DEBUG RS - hyper_util::client::legacy::connect::dns:122 - resolving host="some.gitlab.server.com"
DEBUG RS - hyper_util::client::legacy::connect::http:643 - connecting to 10.170.97.45:443
DEBUG RS - hyper_util::client::legacy::connect::http:646 - connected to 10.170.97.45:443
DEBUG RS - h2::client:1281 - binding client connection
DEBUG RS - h2::client:1286 - client connection bound
DEBUG RS - h2::codec::framed_write:213 - send frame=Settings { flags: (0x0), enable_push: 0, initial_window_size: 2097152, max_frame_size: 16384, max_header_list_size: 16384 }
DEBUG RS - h2::proto::connection:138 - Connection; peer=Client
DEBUG RS - hyper_util::client::legacy::pool:396 - pooling idle connection for ("https", some.gitlab.server.com)
DEBUG RS - h2::codec::framed_write:213 - send frame=WindowUpdate { stream_id: StreamId(0), size_increment: 5177345 }
DEBUG RS - h2::codec::framed_write:213 - send frame=Headers { stream_id: StreamId(1), flags: (0x5: END_HEADERS | END_STREAM) }                           
DEBUG RS - h2::codec::framed_read:405 - received frame=Settings { flags: (0x0), max_concurrent_streams: 128, initial_window_size: 65536, max_frame_size: 16777215 }
DEBUG RS - h2::codec::framed_write:213 - send frame=Settings { flags: (0x1: ACK) }
DEBUG RS - h2::codec::framed_read:405 - received frame=WindowUpdate { stream_id: StreamId(0), size_increment: 2147418112 }
DEBUG RS - h2::codec::framed_read:405 - received frame=Settings { flags: (0x1: ACK) }
DEBUG RS - h2::proto::settings:56 - received settings ACK; applying Settings { flags: (0x0), enable_push: 0, initial_window_size: 2097152, max_frame_size: 16384, max_header_list_size: 16384 }
DEBUG RS - h2::codec::framed_read:405 - received frame=Headers { stream_id: StreamId(1), flags: (0x4: END_HEADERS) }
DEBUG RS - h2::codec::framed_read:405 - received frame=Data { stream_id: StreamId(1), flags: (0x1: END_STREAM) }
DEBUG RS - deno_npm::resolution::graph:927 - <package-req> - Resolved @myscope/packagename@^14.6.2 to @myscope/[email protected]
DEBUG RS - hyper_util::client::legacy::connect::dns:122 - resolving host="registry.npmjs.org"
DEBUG RS - hyper_util::client::legacy::connect::dns:122 - resolving host="registry.npmjs.org"
DEBUG RS - hyper_util::client::legacy::connect::http:643 - connecting to 104.16.31.34:443
DEBUG RS - hyper_util::client::legacy::connect::http:643 - connecting to 104.16.31.34:443                                                                
DEBUG RS - hyper_util::client::legacy::connect::http:646 - connected to 104.16.31.34:443                                                                 
DEBUG RS - hyper_util::client::legacy::connect::http:646 - connected to 104.16.31.34:443                                                                 
DEBUG RS - h2::client:1281 - binding client connection
DEBUG RS - h2::client:1286 - client connection bound
DEBUG RS - h2::codec::framed_write:213 - send frame=Settings { flags: (0x0), enable_push: 0, initial_window_size: 2097152, max_frame_size: 16384, max_header_list_size: 16384 }
DEBUG RS - h2::proto::connection:138 - Connection; peer=Client
DEBUG RS - hyper_util::client::legacy::pool:396 - pooling idle connection for ("https", registry.npmjs.org)
DEBUG RS - h2::codec::framed_read:405 - received frame=Settings { flags: (0x0), max_concurrent_streams: 100, initial_window_size: 65536, max_frame_size: 16777215 }
DEBUG RS - h2::codec::framed_write:213 - send frame=Settings { flags: (0x1: ACK) }
DEBUG RS - h2::codec::framed_read:405 - received frame=WindowUpdate { stream_id: StreamId(0), size_increment: 2147418112 }
DEBUG RS - h2::codec::framed_write:213 - send frame=WindowUpdate { stream_id: StreamId(0), size_increment: 5177345 }
DEBUG RS - hyper_util::client::legacy::pool:270 - reuse idle connection for ("https", registry.npmjs.org)
DEBUG RS - h2::codec::framed_write:213 - send frame=Headers { stream_id: StreamId(1), flags: (0x5: END_HEADERS | END_STREAM) }
DEBUG RS - h2::codec::framed_write:213 - send frame=Headers { stream_id: StreamId(3), flags: (0x5: END_HEADERS | END_STREAM) }
DEBUG RS - h2::client:1281 - binding client connection                                                                                                   
DEBUG RS - h2::client:1286 - client connection bound                                                                                                     
DEBUG RS - h2::codec::framed_write:213 - send frame=Settings { flags: (0x0), enable_push: 0, initial_window_size: 2097152, max_frame_size: 16384, max_header_list_size: 16384 }                                                                                                                                   
DEBUG RS - h2::proto::connection:138 - Connection; peer=Client
DEBUG RS - h2::codec::framed_read:405 - received frame=Settings { flags: (0x0), max_concurrent_streams: 100, initial_window_size: 65536, max_frame_size: 16777215 }
DEBUG RS - h2::codec::framed_write:213 - send frame=Settings { flags: (0x1: ACK) }
DEBUG RS - h2::codec::framed_read:405 - received frame=WindowUpdate { stream_id: StreamId(0), size_increment: 2147418112 }
DEBUG RS - h2::codec::framed_write:213 - send frame=WindowUpdate { stream_id: StreamId(0), size_increment: 5177345 }
DEBUG RS - h2::codec::framed_write:213 - send frame=GoAway { error_code: NO_ERROR, last_stream_id: StreamId(0) }
DEBUG RS - h2::proto::connection:428 - Connection::poll; connection error error=GoAway(b"", NO_ERROR, Library)
DEBUG RS - h2::codec::framed_read:405 - received frame=Settings { flags: (0x1: ACK) }
DEBUG RS - h2::proto::settings:56 - received settings ACK; applying Settings { flags: (0x0), enable_push: 0, initial_window_size: 2097152, max_frame_size: 16384, max_header_list_size: 16384 }
DEBUG RS - h2::codec::framed_read:405 - received frame=Headers { stream_id: StreamId(3), flags: (0x4: END_HEADERS) }
DEBUG RS - h2::codec::framed_read:405 - received frame=Data { stream_id: StreamId(3) }
DEBUG RS - h2::codec::framed_read:405 - received frame=Data { stream_id: StreamId(3) }                                                                   
DEBUG RS - h2::codec::framed_read:405 - received frame=Data { stream_id: StreamId(3) }                                                                   
DEBUG RS - h2::codec::framed_read:405 - received frame=Data { stream_id: StreamId(3), flags: (0x1: END_STREAM) }                                         
DEBUG RS - h2::codec::framed_read:405 - received frame=Headers { stream_id: StreamId(1), flags: (0x4: END_HEADERS) }
DEBUG RS - h2::codec::framed_read:405 - received frame=Data { stream_id: StreamId(1) }
DEBUG RS - h2::codec::framed_read:405 - received frame=Data { stream_id: StreamId(1) }
DEBUG RS - h2::codec::framed_read:405 - received frame=Data { stream_id: StreamId(1) }                                                                   
DEBUG RS - h2::codec::framed_read:405 - received frame=Data { stream_id: StreamId(1) }                                                                   
DEBUG RS - h2::codec::framed_read:405 - received frame=Data { stream_id: StreamId(1) }                                                                   
DEBUG RS - h2::codec::framed_read:405 - received frame=Data { stream_id: StreamId(1) }                                                                   
DEBUG RS - h2::codec::framed_read:405 - received frame=Data { stream_id: StreamId(1) }
DEBUG RS - h2::codec::framed_read:405 - received frame=Data { stream_id: StreamId(1) }
DEBUG RS - h2::codec::framed_read:405 - received frame=Data { stream_id: StreamId(1) }
DEBUG RS - h2::codec::framed_read:405 - received frame=Data { stream_id: StreamId(1) }
DEBUG RS - h2::codec::framed_read:405 - received frame=Data { stream_id: StreamId(1) }                                                                   
DEBUG RS - h2::codec::framed_read:405 - received frame=Data { stream_id: StreamId(1) }                                                                   
DEBUG RS - h2::codec::framed_read:405 - received frame=Data { stream_id: StreamId(1) }                                                                   
DEBUG RS - h2::codec::framed_read:405 - received frame=Data { stream_id: StreamId(1) }                                                                   
DEBUG RS - h2::codec::framed_read:405 - received frame=Data { stream_id: StreamId(1) }
DEBUG RS - h2::codec::framed_read:405 - received frame=Data { stream_id: StreamId(1) }
DEBUG RS - h2::codec::framed_read:405 - received frame=Data { stream_id: StreamId(1) }
DEBUG RS - h2::codec::framed_read:405 - received frame=Data { stream_id: StreamId(1) }
DEBUG RS - h2::codec::framed_read:405 - received frame=Data { stream_id: StreamId(1) }
DEBUG RS - h2::codec::framed_read:405 - received frame=Data { stream_id: StreamId(1) }
DEBUG RS - h2::codec::framed_read:405 - received frame=Data { stream_id: StreamId(1) }
DEBUG RS - h2::codec::framed_read:405 - received frame=Data { stream_id: StreamId(1) }
DEBUG RS - h2::codec::framed_read:405 - received frame=Data { stream_id: StreamId(1) }
DEBUG RS - h2::codec::framed_read:405 - received frame=Data { stream_id: StreamId(1) }
DEBUG RS - h2::codec::framed_read:405 - received frame=Data { stream_id: StreamId(1), flags: (0x1: END_STREAM) }
DEBUG RS - deno_npm::resolution::graph:927 - @myscope/[email protected] - Resolved lit-html@^3.2.0 to [email protected]                                          
DEBUG RS - deno_npm::resolution::graph:927 - @myscope/[email protected] - Resolved vlq@^2.0.4 to [email protected]                                                    
DEBUG RS - hyper_util::client::legacy::pool:270 - reuse idle connection for ("https", registry.npmjs.org)                                                
DEBUG RS - h2::codec::framed_write:213 - send frame=Headers { stream_id: StreamId(5), flags: (0x5: END_HEADERS | END_STREAM) }                           
DEBUG RS - h2::codec::framed_read:405 - received frame=Headers { stream_id: StreamId(5), flags: (0x4: END_HEADERS) }
DEBUG RS - h2::codec::framed_read:405 - received frame=Data { stream_id: StreamId(5) }
DEBUG RS - h2::codec::framed_read:405 - received frame=Data { stream_id: StreamId(5) }                                                                   
DEBUG RS - h2::codec::framed_read:405 - received frame=Data { stream_id: StreamId(5) }                                                                   
DEBUG RS - h2::codec::framed_read:405 - received frame=Data { stream_id: StreamId(5) }                                                                   
DEBUG RS - h2::codec::framed_read:405 - received frame=Data { stream_id: StreamId(5), flags: (0x1: END_STREAM) }                                         
DEBUG RS - deno_npm::resolution::graph:927 - [email protected] - Resolved @types/trusted-types@^2.0.2 to @types/[email protected]                         
DEBUG RS - deno::util::fs:580 - Acquiring file lock at C:\Users\me.me\Downloads\repro\node_modules\.deno\.deno.lock                        
DEBUG RS - deno::util::fs:600 - Acquired file lock at C:\Users\me.me\Downloads\repro\node_modules\.deno\.deno.lock                         
DEBUG RS - hyper_util::client::legacy::pool:270 - reuse idle connection for ("https", some.gitlab.server.com)                                          
DEBUG RS - hyper_util::client::legacy::pool:270 - reuse idle connection for ("https", registry.npmjs.org)
DEBUG RS - h2::codec::framed_write:213 - send frame=Headers { stream_id: StreamId(3), flags: (0x5: END_HEADERS | END_STREAM) }
DEBUG RS - h2::codec::framed_write:213 - send frame=Headers { stream_id: StreamId(7), flags: (0x5: END_HEADERS | END_STREAM) }
DEBUG RS - h2::codec::framed_read:405 - received frame=Headers { stream_id: StreamId(7), flags: (0x4: END_HEADERS) }
DEBUG RS - h2::codec::framed_read:405 - received frame=Data { stream_id: StreamId(7) }
DEBUG RS - h2::codec::framed_read:405 - received frame=Data { stream_id: StreamId(7) }
DEBUG RS - h2::codec::framed_read:405 - received frame=Data { stream_id: StreamId(7) }
DEBUG RS - h2::codec::framed_read:405 - received frame=Data { stream_id: StreamId(7) }
DEBUG RS - h2::codec::framed_read:405 - received frame=Data { stream_id: StreamId(7) }                                                                   
DEBUG RS - h2::codec::framed_read:405 - received frame=Data { stream_id: StreamId(7) }
DEBUG RS - h2::codec::framed_read:405 - received frame=Data { stream_id: StreamId(7) }                                                                   
DEBUG RS - h2::codec::framed_read:405 - received frame=Data { stream_id: StreamId(7) }                                                                   
DEBUG RS - h2::codec::framed_read:405 - received frame=Data { stream_id: StreamId(7) }                                                                   
DEBUG RS - h2::codec::framed_read:405 - received frame=Data { stream_id: StreamId(7) }                                                                   
DEBUG RS - h2::codec::framed_read:405 - received frame=Data { stream_id: StreamId(7) }
DEBUG RS - h2::codec::framed_read:405 - received frame=Data { stream_id: StreamId(7) }
DEBUG RS - h2::codec::framed_read:405 - received frame=Data { stream_id: StreamId(7) }
DEBUG RS - h2::codec::framed_read:405 - received frame=Data { stream_id: StreamId(7) }
DEBUG RS - h2::codec::framed_read:405 - received frame=Data { stream_id: StreamId(7) }
DEBUG RS - h2::codec::framed_read:405 - received frame=Data { stream_id: StreamId(7) }
DEBUG RS - h2::codec::framed_read:405 - received frame=Data { stream_id: StreamId(7) }
DEBUG RS - h2::codec::framed_read:405 - received frame=Data { stream_id: StreamId(7) }
DEBUG RS - h2::codec::framed_read:405 - received frame=Data { stream_id: StreamId(7) }
DEBUG RS - h2::codec::framed_read:405 - received frame=Data { stream_id: StreamId(7) }
DEBUG RS - h2::codec::framed_read:405 - received frame=Data { stream_id: StreamId(7) }
DEBUG RS - h2::codec::framed_read:405 - received frame=Data { stream_id: StreamId(7) }
DEBUG RS - h2::codec::framed_read:405 - received frame=Data { stream_id: StreamId(7) }
DEBUG RS - h2::codec::framed_read:405 - received frame=Data { stream_id: StreamId(7) }
DEBUG RS - h2::codec::framed_read:405 - received frame=Data { stream_id: StreamId(7) }
DEBUG RS - h2::codec::framed_read:405 - received frame=Data { stream_id: StreamId(7) }
DEBUG RS - h2::codec::framed_read:405 - received frame=Data { stream_id: StreamId(7) }
DEBUG RS - h2::codec::framed_read:405 - received frame=Data { stream_id: StreamId(7) }
DEBUG RS - h2::codec::framed_read:405 - received frame=Data { stream_id: StreamId(7) }
DEBUG RS - h2::codec::framed_read:405 - received frame=Data { stream_id: StreamId(7) }
DEBUG RS - h2::codec::framed_read:405 - received frame=Data { stream_id: StreamId(7) }
DEBUG RS - h2::codec::framed_read:405 - received frame=Data { stream_id: StreamId(7) }
DEBUG RS - h2::codec::framed_read:405 - received frame=Data { stream_id: StreamId(7) }
DEBUG RS - h2::codec::framed_read:405 - received frame=Data { stream_id: StreamId(7) }
DEBUG RS - h2::codec::framed_read:405 - received frame=Data { stream_id: StreamId(7) }
DEBUG RS - h2::codec::framed_read:405 - received frame=Data { stream_id: StreamId(7) }
DEBUG RS - h2::codec::framed_read:405 - received frame=Data { stream_id: StreamId(7) }
DEBUG RS - h2::codec::framed_read:405 - received frame=Data { stream_id: StreamId(7) }
DEBUG RS - h2::codec::framed_read:405 - received frame=Data { stream_id: StreamId(7) }
DEBUG RS - h2::codec::framed_read:405 - received frame=Data { stream_id: StreamId(7), flags: (0x1: END_STREAM) }
DEBUG RS - h2::codec::framed_read:405 - received frame=Headers { stream_id: StreamId(3), flags: (0x4: END_HEADERS) }
DEBUG RS - h2::codec::framed_read:405 - received frame=Data { stream_id: StreamId(3), flags: (0x1: END_STREAM) }
error: Failed caching npm package '@myscope/[email protected]'.                                                                                               
                                                                                                                                                         
Caused by:                                                                                                                                               
    Could not find npm package tarball at: https://some.gitlab.server.com/api/v4/projects/12345/packages/npm/@myscope/packagename/-/@myscope/packagename-14.6.2.tgz
@bartlomieju bartlomieju added bug Something isn't working correctly node compat labels Sep 28, 2024
@bartlomieju bartlomieju added this to the 2.0.0 milestone Sep 28, 2024
@bartlomieju
Copy link
Member

Hi @bavoco, could you please provide more details on this issue?

Is there any chance you could share the actual scope name and package name? It's somewhat hard to figure out what's going wrong if I'm not sure which parts were edited in the output you provided.

Notice that it did download the dependencies of the package I was trying to install. So although it says it could not find a package tarball, it definitely did find something at the url and it is using the authtoken from the npmrc correcly otherwise it would have had a 404.

So if I get this right - the dependencies of your private package were installed correctly, but the installation of the package itself is erroring when retrieving the tarball?

@bavoco
Copy link
Author

bavoco commented Oct 1, 2024

I agree this isn't much to go on, I was hoping using an empty deno cache dir and debug log would be enough information.

I'll try to reproduce this with a public GitLab server later today

@bavoco
Copy link
Author

bavoco commented Oct 1, 2024

I'm unable to reproduce by publishing a public (and private) package to gitlab.com.
Tried same version number, same dependencies, added roughly the same amount of files to it.

However I can consistently reproduce this issue, also on 2.0.0-rc.9, with the business GitLab server.
As shown in the log above, deno install correctly finds the dependencies of the package and caches those, but fails caching the package itself. npm install works fine.

Any ideas I can try?

@bartlomieju
Copy link
Member

@bavoco do you have proxies involved in this setup? I'd assume so because of the business GitLab server.

@bavoco
Copy link
Author

bavoco commented Oct 1, 2024

I don't know if the requests are being proxied.
I tried another GitLab server (on the business network) and deno install works for one of the packages there.

To see if it's caused by the tar, I pointed to the tar file in the package.json and that doesn't install it. Does Deno support specifiying a tar in the package.json?

@bavoco
Copy link
Author

bavoco commented Oct 1, 2024

For now I can work around the issue with a fetch and @std/tar (very nice that it supports web streams)

@bartlomieju
Copy link
Member

To see if it's caused by the tar, I pointed to the tar file in the package.json and that doesn't install it. Does Deno support specifiying a tar in the package.json?

@bavoco not currently, no.

For now I can work around the issue with a fetch and @std/tar (very nice that it supports web streams)

That's great to hear.

I'm eager to look into this issue as soon as you can provide some more info. I'm gonna remove it from Deno 2.0 milestone for now.

@bartlomieju bartlomieju removed this from the 2.0.0 milestone Oct 2, 2024
@chris-clyde
Copy link

I got the same issue. We have a private repo in gitlab and when i try to install with the same .npmrc that works with node i get the same message as op.

We dont have any proxy envolved.

What kind of information do you need? @bartlomieju

@bartlomieju bartlomieju mentioned this issue Oct 23, 2024
Merged
@bartlomieju
Copy link
Member

FYI you can upgrade to canary version to get this fix immediately (deno upgrade canary) or wait until stable Deno v2.0.3 that should be released tonight.

bartlomieju added a commit that referenced this issue Oct 25, 2024
@marvinhagemeister @bartlomieju
fix: .npmrc settings not being passed to install/add command (#26473)
f4ef7b0 
We weren't passing the resolved npmrc settings to the install commands.
This lead us to always fall back to the default registry url instead of
using the one from npmrc.

Fixes #26139
Fixes #26033
Fixes #25924
Fixes #25822
Fixes #26152

---------

Co-authored-by: Bartek Iwańczuk <[email protected]>
@JarekToro JarekToro mentioned this issue Jan 21, 2025
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@marvinhagemeister marvinhagemeister

Labels
bug Something isn't working correctly install node compat
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

fix: .npmrc settings not being passed to install/add command denoland/deno
4 participants
@marvinhagemeister @bartlomieju @bavoco @chris-clyde