Scoped packages don't work due to missing authorization header

[KidkArolis]

Do you want to request a feature or report a bug?
Bug

What is the current behavior?
Running yarn fails when we try installing @scoped/packages. I get Error: https://yarn-scopes-issue-hplobdybcq.now.sh/@foo/bar/-/@foo/bar-1.0.0.tgz: invalid tar file.

If the current behavior is a bug, please provide the steps to reproduce.
Follow the instructions in this repo to easily reproduce on your machine.
https://github.com/KidkArolis/yarn-scopes-issue

What is the expected behavior?
Running yarn works and installs scoped packages.

Please mention your node.js, yarn and operating system version.
Node 6, MacOS, yarn 0.16.1.

Other comments
This issue is being fixed via #1666. I've opened this issue in case alternative designs should be discussed as well as a central point for solving this issue, since there's also a dupe PR at #1561.

[KidkArolis]

tl;dr - authorization header is not being sent when downloading a scoped package tarball

[pb6]

Interesting as this bug is still present 2.5 years later in 1.16... :) Only with different err message:
error An unexpected error occurred: "https://our_repo_host/repository/npm-components/@.../aaa/-/aaa-1.2.3.tgz: Request failed "401 Unauthorized"".
When connecting to Nexus hosted repo. Interestingly, for metadata this works just fine (authorization header is present, while for package itself it's not).

So when debugging isRequestToRegistry function, I get:
this.reporter.verbose(${request.host} === ${registry.host});
verbose 10.124 our_registry_host === registry.npmjs.org
this.reporter.verbose((${requestToRegistryHost} || ${requestToYarn}) && (${requestToRegistryPath} || ${customHostSuffixInUse}));
verbose 10.124 (false || false) && (true || false)
verbose 10.124 isToRegistry: false

.npmrc looks like:

@scope:registry=https://our_repo_host/repository/npm-components/
//our_repo_host/repository/npm-components/:_auth=...=
//our_repo_host/repository/npm-components/:email=yarn@coocoocoocoo.com
//our_repo_host/repository/npm-components/:always-auth=true

[marc0s]

I got a similar error with 1.22.19. yarn is not sending the Authorization header when running the HTTP GET request to the tarball's URL.
My .yarnrc file has both the URL for the scoped packages and the authorization token:

"@scope:registry" "https://gitlab.com/api/v4/packages/npm/"
"//gitlab.com/api/v4/packages/npm/:_authToken" "REDACTED"

And in the verbose output we can find a log message like the following:

verbose 0.481132317 Error: https://gitlab.com/api/v4/projects/REDACTED/packages/npm/@scope/package/-/@scope/package-8.0.1.tgz: Request failed "404 Not Found"

After manually editing yarn's cli.js for inspecting sent headers, we observe:

"headers":{"User-Agent":"yarn/1.22.19 npm/? node/v14.19.1 linux x64","Accept":"application/vnd.npm.install-v1+json; q=1.0, application/json; q=0.8, */*","Accept-Encoding":"gzip"}

If I hardcode my Authorization header in there, the yarn add command runs successfully.

Please note that Gitlab's registry always returns 404 for unauthorized requests to tarballs.

If this comment should be instead a new issue, please tell me.

Thanks