[BUG] "npm install" fails with private registry https://host/path/ but works with https://host:443/path/

[heidemn]

What / Why

It seems that NPM makes a difference between https://artifactory.ourcompany.com/artifactory/api/npm/npm/
and
https://artifactory.ourcompany.com:443/artifactory/api/npm/npm/
(URL redacted)

Only when using the explicit port notation with ":443", the "npm install" command succeeds.

When

• When using NPM with private registry, using HTTPS + default port 443.

Current Behavior

• See below.
• Interesting: "npm login" always works. (I believe also "npm publish" worked.)
  It only makes a difference for "npm install", "npm ci" & co (commands that download stuff).

Steps to Reproduce

• Not sure if this is an NPM bug or an Artifactory bug.
• The behavior below is reproducible both on Windows + Ubuntu 18.04.
• Linux: Node v12.14.1 / NPM 6.14.4
• Windows: Node v12.14.0 / NPM 6.13.4

### 1. FAILS: (port omitted)
$ npm login --registry=https://artifactory.ourcompany.com/artifactory/api/npm/npm/ --always-auth --scope=@ourcompany
(login succeeds)
$ npm install @ourcompany/[email protected]
npm ERR! code E401
npm ERR! Unable to authenticate, need: Basic realm="Artifactory Realm"

### 2. WORKS: (port explicitly specified as ":443" in the URL)
$ npm login --registry=https://artifactory.ourcompany.com:443/artifactory/api/npm/npm/ --always-auth --scope=@ourcompany
$ npm install @ourcompany/[email protected]
(success!)
....

Expected Behavior

• Private registry URL with/without explicit port spec should behave the same way.

References

• Possibly related: [BUG] Fails to install npm packages from private github repos in docker since v6.11.0 #950

[heidemn]

Debug logs:
URLs, scope, package name and username redacted, but URL structure kept intact
(same protocol, same port, same number of subdomains, same TLD).

Failure, using implicit port:

+ rm -f /home/company/.npmrc
+ rm -rf /home/company/.npm /home/company/.cache/node-gyp/
+ npm -ddd login --registry=https://artifactory.ourcompany.com/artifactory/api/npm/npm/ --always-auth --scope=@company
npm info it worked if it ends with ok
npm verb cli [
npm verb cli   '/usr/bin/node',
npm verb cli   '/usr/bin/npm',
npm verb cli   '-ddd',
npm verb cli   'login',
npm verb cli   '--registry=https://artifactory.ourcompany.com/artifactory/api/npm/npm/',
npm verb cli   '--always-auth',
npm verb cli   '--scope=@company'
npm verb cli ]
npm info using [email protected]
npm info using [email protected]
npm verb npm-session 31e22eefd9fa4e28
npm verb web login before first POST
npm http fetch POST 401 https://artifactory.ourcompany.com/artifactory/api/npm/npm/-/v1/login 251ms
npm verb web login not supported, trying couch 
Username: Password: Email: (this IS public) npm verb login before first PUT {
npm verb login   _id: 'org.couchdb.user:ciuser',
npm verb login   name: 'ciuser',
npm verb login   password: 'XXXXX',
npm verb login   type: 'user',
npm verb login   roles: [],
npm verb login   date: '2020-04-27T09:36:25.423Z'
npm verb login }
npm http fetch PUT 201 https://artifactory.ourcompany.com/artifactory/api/npm/npm/-/user/org.couchdb.user:ciuser 141ms
npm info login Authorized user ciuser
Logged in as ciuser to scope @company on https://artifactory.ourcompany.com/artifactory/api/npm/npm/.
npm verb exit [ 0, true ]
npm timing npm Completed in 16522ms
npm info ok 
+ echo 'Testing if @company/utils can be installed...'
Testing if @company/utils can be installed...
+ rm -rf /tmp/npm-test
+ mkdir -p /tmp/npm-test
+ cd /tmp/npm-test
+ npm -ddd i @company/[email protected]
npm info it worked if it ends with ok
npm verb cli [ '/usr/bin/node', '/usr/bin/npm', '-ddd', 'i', '@company/[email protected]' ]
npm info using [email protected]
npm info using [email protected]
npm verb npm-session 8e249167351a312c
npm sill install loadCurrentTree
npm sill install readLocalPackageData
npm http fetch GET 200 https://artifactory.ourcompany.com/artifactory/api/npm/npm/@company%2futils 279ms
npm http fetch GET 401 https://artifactory.ourcompany.com:443/artifactory/api/npm/npm/@company/utils/-/@company/utils-0.0.1.tgz 29ms
npm sill fetchPackageMetaData error for @company/[email protected] Unable to authenticate, need: Basic realm="Artifactory Realm"
npm timing stage:rollbackFailedOptional Completed in 0ms
npm timing stage:runTopLevelLifecycles Completed in 345ms
npm verb stack Error: Unable to authenticate, need: Basic realm="Artifactory Realm"
npm verb stack     at /usr/lib/node_modules/npm/node_modules/npm-registry-fetch/check-response.js:94:17
npm verb stack     at processTicksAndRejections (internal/process/task_queues.js:94:5)
npm verb statusCode 401
npm verb pkgid @company/[email protected]
npm verb cwd /tmp/npm-test
npm verb Linux 4.15.0-96-generic
npm verb argv "/usr/bin/node" "/usr/bin/npm" "-ddd" "i" "@company/[email protected]"
npm verb node v12.14.1
npm verb npm  v6.14.4
npm ERR! code E401
npm ERR! Unable to authenticate, need: Basic realm="Artifactory Realm"
npm verb exit [ 1, true ]
npm timing npm Completed in 545ms

npm ERR! A complete log of this run can be found in:
npm ERR!     /home/company/.npm/_logs/2020-04-27T09_36_26_306Z-debug.log

Success, using explicit port 443:

+ rm -f /home/company/.npmrc
+ rm -rf /home/company/.npm /home/company/.cache/node-gyp/
+ npm -ddd login --registry=https://artifactory.company.com:443/artifactory/api/npm/npm/ --always-auth --scope=@company
npm info it worked if it ends with ok
npm verb cli [
npm verb cli   '/usr/bin/node',
npm verb cli   '/usr/bin/npm',
npm verb cli   '-ddd',
npm verb cli   'login',
npm verb cli   '--registry=https://artifactory.company.com:443/artifactory/api/npm/npm/',
npm verb cli   '--always-auth',
npm verb cli   '--scope=@company'
npm verb cli ]
npm info using [email protected]
npm info using [email protected]
npm verb npm-session de8fd78aea55919c
npm verb web login before first POST
npm http fetch POST 401 https://artifactory.company.com:443/artifactory/api/npm/npm/-/v1/login 272ms
npm verb web login not supported, trying couch 
Username: Password: Email: (this IS public) npm verb login before first PUT {
npm verb login   _id: 'org.couchdb.user:ciuser',
npm verb login   name: 'ciuser',
npm verb login   password: 'XXXXX',
npm verb login   type: 'user',
npm verb login   roles: [],
npm verb login   date: '2020-04-27T09:35:43.855Z'
npm verb login }
npm http fetch PUT 201 https://artifactory.company.com:443/artifactory/api/npm/npm/-/user/org.couchdb.user:ciuser 172ms
npm info login Authorized user ciuser
Logged in as ciuser to scope @company on https://artifactory.company.com:443/artifactory/api/npm/npm/.
npm verb exit [ 0, true ]
npm timing npm Completed in 17321ms
npm info ok 
+ echo 'Testing if @company/utils can be installed...'
Testing if @company/utils can be installed...
+ rm -rf /tmp/npm-test
+ mkdir -p /tmp/npm-test
+ cd /tmp/npm-test
+ npm -ddd i @company/[email protected]
npm info it worked if it ends with ok
npm verb cli [ '/usr/bin/node', '/usr/bin/npm', '-ddd', 'i', '@company/[email protected]' ]
npm info using [email protected]
npm info using [email protected]
npm verb npm-session b7a814071e534830
npm sill install loadCurrentTree
npm sill install readLocalPackageData
npm http fetch GET 200 https://artifactory.company.com:443/artifactory/api/npm/npm/@company%2futils 312ms
npm http fetch GET 200 https://artifactory.company.com:443/artifactory/api/npm/npm/@company/utils/-/@company/utils-0.0.1.tgz 112ms
npm sill pacote version manifest for @company/[email protected] fetched in 442ms
npm timing stage:loadCurrentTree Completed in 464ms
npm sill install loadIdealTree
npm sill install cloneCurrentTreeToIdealTree
npm timing stage:loadIdealTree:cloneCurrentTree Completed in 0ms
npm sill install loadShrinkwrap
npm timing stage:loadIdealTree:loadShrinkwrap Completed in 1ms
npm sill install loadAllDepsIntoIdealTree
npm sill resolveWithNewModule @company/[email protected] checking installable status
npm http fetch GET 200 https://registry.npmjs.org/@babel%2fpolyfill 194ms
npm http fetch GET 200 https://registry.npmjs.org/@babel/polyfill/-/polyfill-7.8.7.tgz 97ms
npm sill pacote range manifest for @babel/polyfill@^7.6.0 fetched in 300ms
npm sill resolveWithNewModule @babel/[email protected] checking installable status
npm http fetch GET 200 https://registry.npmjs.org/core-js 56ms
npm http fetch GET 200 https://registry.npmjs.org/regenerator-runtime 235ms
npm http fetch GET 200 https://registry.npmjs.org/core-js/-/core-js-2.6.11.tgz 174ms
npm sill pacote range manifest for core-js@^2.6.5 fetched in 240ms
npm WARN deprecated [email protected]: core-js@<3 is no longer maintained and not recommended for usage due to the number of issues. Please, upgrade your dependencies to the actual version of core-js@3.
npm sill resolveWithNewModule [email protected] checking installable status
npm http fetch GET 200 https://registry.npmjs.org/regenerator-runtime/-/regenerator-runtime-0.13.5.tgz 37ms
npm sill pacote range manifest for regenerator-runtime@^0.13.4 fetched in 274ms
npm sill resolveWithNewModule [email protected] checking installable status
npm timing stage:loadIdealTree:loadAllDepsIntoIdealTree Completed in 583ms
npm timing stage:loadIdealTree Completed in 587ms
npm sill currentTree npm-test
npm sill idealTree npm-test
npm sill idealTree ├── @babel/[email protected]
npm sill idealTree ├── @company/[email protected]
npm sill idealTree ├── [email protected]
npm sill idealTree └── [email protected]
npm sill install generateActionsToTake
npm timing stage:generateActionsToTake Completed in 4ms
npm sill diffTrees action count 4
npm sill diffTrees add [email protected]
npm sill diffTrees add [email protected]
npm sill diffTrees add @babel/[email protected]
npm sill diffTrees add @company/[email protected]
npm sill decomposeActions action count 32
npm sill decomposeActions fetch [email protected]
npm sill decomposeActions extract [email protected]
npm sill decomposeActions preinstall [email protected]
npm sill decomposeActions build [email protected]
npm sill decomposeActions install [email protected]
npm sill decomposeActions postinstall [email protected]
npm sill decomposeActions finalize [email protected]
npm sill decomposeActions refresh-package-json [email protected]
npm sill decomposeActions fetch [email protected]
npm sill decomposeActions extract [email protected]
npm sill decomposeActions preinstall [email protected]
npm sill decomposeActions build [email protected]
npm sill decomposeActions install [email protected]
npm sill decomposeActions postinstall [email protected]
npm sill decomposeActions finalize [email protected]
npm sill decomposeActions refresh-package-json [email protected]
npm sill decomposeActions fetch @babel/[email protected]
npm sill decomposeActions extract @babel/[email protected]
npm sill decomposeActions preinstall @babel/[email protected]
npm sill decomposeActions build @babel/[email protected]
npm sill decomposeActions install @babel/[email protected]
npm sill decomposeActions postinstall @babel/[email protected]
npm sill decomposeActions finalize @babel/[email protected]
npm sill decomposeActions refresh-package-json @babel/[email protected]
npm sill decomposeActions fetch @company/[email protected]
npm sill decomposeActions extract @company/[email protected]
npm sill decomposeActions preinstall @company/[email protected]
npm sill decomposeActions build @company/[email protected]
npm sill decomposeActions install @company/[email protected]
npm sill decomposeActions postinstall @company/[email protected]
npm sill decomposeActions finalize @company/[email protected]
npm sill decomposeActions refresh-package-json @company/[email protected]
npm sill install executeActions
npm sill doSerial global-install 32
npm verb correctMkdir /home/company/.npm/_locks correctMkdir not in flight; initializing
npm verb lock using /home/company/.npm/_locks/staging-33a8750bb22d0887.lock for /tmp/npm-test/node_modules/.staging
npm sill doParallel extract 4
npm sill extract [email protected]
npm sill extract [email protected]
npm sill extract @babel/[email protected]
npm sill extract @company/[email protected]
npm sill tarball trying core-js@^2.6.5 by hash: sha512-5wjnpaT/3dV+XB4borEsnAYQchn00XSgTAWKDkEqv+K8KevjbzmofK6hfJ9TZIlpj2N0xQpazy7PiRQiWHqzWg==
npm sill tarball trying regenerator-runtime@^0.13.4 by hash: sha512-ZS5w8CpKFinUzOwW3c83oPeVXoNsrLsaCoLtJvAClH135j/R77RuymhiSErhm2lKcwSCIpmvIWSbDkIfAqKQlA==
npm sill tarball trying @babel/polyfill@^7.6.0 by hash: sha512-LeSfP9bNZH2UOZgcGcZ0PIHUt1ZuHub1L3CVmEyqLxCeDLm4C5Gi8jRH8ZX2PNpDhQCo0z6y/+DIs2JlliXW8w==
npm sill tarball trying @company/[email protected] by hash: sha1-qsNN3z+T4EaZVs31LDn0wKwzsoI=
npm sill extract regenerator-runtime@^0.13.4 extracted to /tmp/npm-test/node_modules/.staging/regenerator-runtime-f9b22426 (50ms)
npm sill extract @babel/polyfill@^7.6.0 extracted to /tmp/npm-test/node_modules/.staging/@babel/polyfill-298b23c4 (63ms)
npm sill extract @company/[email protected] extracted to /tmp/npm-test/node_modules/.staging/@company/utils-8ff1b5c3 (96ms)
npm timing audit submit Completed in 321ms
npm http fetch POST 200 https://registry.npmjs.org/-/npm/v1/security/audits/quick 320ms
npm timing audit body Completed in 1ms
npm sill extract core-js@^2.6.5 extracted to /tmp/npm-test/node_modules/.staging/core-js-0b3fc6c9 (410ms)
npm timing action:extract Completed in 412ms
npm sill doReverseSerial unbuild 32
npm sill doSerial remove 32
npm sill doSerial move 32
npm sill doSerial finalize 32
npm sill finalize /tmp/npm-test/node_modules/core-js
npm sill finalize /tmp/npm-test/node_modules/regenerator-runtime
npm sill finalize /tmp/npm-test/node_modules/@babel/polyfill
npm sill finalize /tmp/npm-test/node_modules/@company/utils
npm timing action:finalize Completed in 4ms
npm sill doParallel refresh-package-json 4
npm sill refresh-package-json /tmp/npm-test/node_modules/core-js
npm sill refresh-package-json /tmp/npm-test/node_modules/regenerator-runtime
npm sill refresh-package-json /tmp/npm-test/node_modules/@babel/polyfill
npm sill refresh-package-json /tmp/npm-test/node_modules/@company/utils
npm timing action:refresh-package-json Completed in 14ms
npm sill doParallel preinstall 4
npm sill preinstall [email protected]
npm info lifecycle [email protected]~preinstall: [email protected]
npm sill preinstall [email protected]
npm info lifecycle [email protected]~preinstall: [email protected]
npm sill preinstall @babel/[email protected]
npm info lifecycle @babel/[email protected]~preinstall: @babel/[email protected]
npm sill preinstall @company/[email protected]
npm info lifecycle @company/[email protected]~preinstall: @company/[email protected]
npm timing action:preinstall Completed in 1ms
npm sill doSerial build 32
npm sill build [email protected]
npm info linkStuff [email protected]
npm sill linkStuff [email protected] has /tmp/npm-test/node_modules as its parent node_modules
npm sill build [email protected]
npm info linkStuff [email protected]
npm sill linkStuff [email protected] has /tmp/npm-test/node_modules as its parent node_modules
npm sill build @babel/[email protected]
npm info linkStuff @babel/[email protected]
npm sill linkStuff @babel/[email protected] has /tmp/npm-test/node_modules as its parent node_modules
npm sill build @company/[email protected]
npm info linkStuff @company/[email protected]
npm sill linkStuff @company/[email protected] has /tmp/npm-test/node_modules as its parent node_modules
npm timing action:build Completed in 1ms
npm sill doSerial global-link 32
npm sill doParallel update-linked 0
npm sill doSerial install 32
npm sill install [email protected]
npm info lifecycle [email protected]~install: [email protected]
npm sill install [email protected]
npm info lifecycle [email protected]~install: [email protected]
npm sill install @babel/[email protected]
npm info lifecycle @babel/[email protected]~install: @babel/[email protected]
npm sill install @company/[email protected]
npm info lifecycle @company/[email protected]~install: @company/[email protected]
npm timing action:install Completed in 1ms
npm sill doSerial postinstall 32
npm sill postinstall [email protected]
npm info lifecycle [email protected]~postinstall: [email protected]

> [email protected] postinstall /tmp/npm-test/node_modules/core-js
> node -e "try{require('./postinstall')}catch(e){}"

�[96mThank you for using core-js (�[94m https://github.com/zloirock/core-js �[96m) for polyfilling JavaScript standard library!�[0m

�[96mThe project needs your help! Please consider supporting of core-js on Open Collective or Patreon: �[0m
�[96m>�[94m https://opencollective.com/core-js �[0m
�[96m>�[94m https://www.patreon.com/zloirock �[0m

�[96mAlso, the author of core-js (�[94m https://github.com/zloirock �[96m) is looking for a good job -)�[0m

npm verb lifecycle [email protected]~postinstall: unsafe-perm in lifecycle true
npm verb lifecycle [email protected]~postinstall: PATH: /usr/lib/node_modules/npm/node_modules/npm-lifecycle/node-gyp-bin:/tmp/npm-test/node_modules/core-js/node_modules/.bin:/tmp/npm-test/node_modules/.bin:/home/company/.local/bin:/home/company/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/company/cloudadmin:/home/company/.local/bin:/usr/local/go/bin:/company/cloudadmin:/home/company/.local/bin:/usr/local/go/bin
npm verb lifecycle [email protected]~postinstall: CWD: /tmp/npm-test/node_modules/core-js
npm sill lifecycle [email protected]~postinstall: Args: [ '-c', `node -e "try{require('./postinstall')}catch(e){}"` ]
npm sill lifecycle [email protected]~postinstall: Returned: code: 0  signal: null
npm sill postinstall [email protected]
npm info lifecycle [email protected]~postinstall: [email protected]
npm sill postinstall @babel/[email protected]
npm info lifecycle @babel/[email protected]~postinstall: @babel/[email protected]
npm sill postinstall @company/[email protected]
npm info lifecycle @company/[email protected]~postinstall: @company/[email protected]
npm timing action:postinstall Completed in 56ms
npm verb unlock done using /home/company/.npm/_locks/staging-33a8750bb22d0887.lock for /tmp/npm-test/node_modules/.staging
npm timing stage:executeActions Completed in 525ms
npm timing stage:rollbackFailedOptional Completed in 0ms
npm timing stage:runTopLevelLifecycles Completed in 1585ms
npm sill saveTree npm-test
npm sill saveTree └─┬ @company/[email protected]
npm sill saveTree   └─┬ @babel/[email protected]
npm sill saveTree     ├── [email protected]
npm sill saveTree     └── [email protected]
npm sill install saveToDependencies
npm WARN saveError ENOENT: no such file or directory, open '/tmp/npm-test/package.json'
npm info lifecycle undefined~preshrinkwrap: undefined
npm info lifecycle undefined~shrinkwrap: undefined
npm notice created a lockfile as package-lock.json. You should commit this file.
npm info lifecycle undefined~postshrinkwrap: undefined
npm WARN enoent ENOENT: no such file or directory, open '/tmp/npm-test/package.json'
npm verb enoent This is related to npm not being able to find a file.
npm verb enoent 
npm WARN npm-test No description
npm WARN npm-test No repository field.
npm WARN npm-test No README data
npm WARN npm-test No license field.

npm sill install printInstalled
+ @company/[email protected]
added 4 packages from 3 contributors and audited 4 packages in 1.602s
found 0 vulnerabilities

npm verb exit [ 0, true ]
npm timing npm Completed in 1819ms
npm info ok 

[heidemn]

~/.npmrc (failure, implicit port)

@company:registry=https://artifactory.ourcompany.com/artifactory/api/npm/npm/
//artifactory.ourcompany.com/artifactory/api/npm/npm/:_authToken=ey...

~/.npmrc (success, explicit port 443)

@company:registry=https://artifactory.ourcompany.com:443/artifactory/api/npm/npm/
//artifactory.ourcompany.com:443/artifactory/api/npm/npm/:_authToken=ey...

[heidemn]

@isaacs how can we find out if this is an NPM bug, or an Artifactory bug?
Did you already have a chance to look at this issue?

[joshbranham]

We are hitting this as well, are you using nginx in front of your Artifactory as well? Also which Artifactory version?

Artifactory Professional
6.11.7 rev 61107900

[heidemn]

Hi Josh, we're using Artifactory 7.2.1.
We reported the issue to JFrog as #127779, and found little interest on their side.
If it's important for you, maybe contact them as well.

[joshbranham]

@heidemn do you have a public link to that issue I could use? Searching their Jira returned nothing with that number

[heidemn]

@joshbranham no it's not a public issue.
But this Github issue now already contains more information, so you don't really need it.
(When I contacted JFrog, I had not yet found the workaround with explicit port 443.)

[darcyclarke]

npm v6 is no longer in active development; We will continue to push security releases to v6 at our team's discretion as-per our Support Policy.

If your bug is preproducible on v7, please re-file this issue using our new issue template.

If your issue was a feature request, please consider opening a new RRFC or RFC. If your issue was a question or other idea that was not CLI-specific, consider opening a discussion on our feedback repo

Closing: This is an automated message.