Bad response status: 401 Unauthorized #174

meirhazonAnyVision · 2018-10-02T06:43:13Z

Hello,
When I do a docker pull from our repos using docker pull everything works fine.
When using lstags docker the pull doesnt work.
Please help.

root@cassandra-001 config]# docker pull gcr.io/anyvision-training/templates
Using default tag: latest
Trying to pull repository gcr.io/anyvision-training/templates ...
latest: Pulling from gcr.io/anyvision-training/templates
Digest: sha256:7fc087838e4da276debc2b20ba57c865d25192a39597b7dcc572c9fd396f97f9
Status: Image is up to date for gcr.io/anyvision-training/templates:latest

root@cassandra-001 config]# docker run --rm -it -v /var/run/docker.sock:/var/run/docker.sock ivanilves/lstags -p gcr.io/anyvision-training/templates
INFO[0000] ANALYZE gcr.io/anyvision-training/templates
Bad response status: 401 Unauthorized >> https://gcr.io/v2/anyvision-training/templates/tags/list

root@cassandra-001 lstags]# docker run --rm -it -v /var/run/docker.sock:/var/run/docker.sock ivanilves/lstags -j /root/.gcp/docker-registry-ro-training.json -p gcr.io/anyvision-training/templates
open /root/.gcp/docker-registry-ro-training.json: no such file or directory

[root@cassandra-001 lstags]# ls -ltr /root/.gcp/docker-registry-ro-training.json
-rw-r--r-- 1 root root 2327 Oct 2 06:13 /root/.gcp/docker-registry-ro-training.json

The xml file is good, it is used when using the plain docker command.

Thanks,

Meir

ivanilves · 2018-10-09T18:05:23Z

Hey @meirhazon1

Thanks for opening the issue!

Sorry for not answering on time - I was vacationing 🍹

As for the issue - you run lstags inside Docker so it has no access to /root/.gcp/docker-registry-ro-training.json which belongs to the host volume and does not appear inside container by default.

So, you have two options:

run lstags binary directly from the host (it's a Golang app, it has no dependencies)
"share" the /root/.gcp/docker-registry-ro-training.json file, I think -v /root/.gcp/docker-registry-ro-training.json:/root/.gcp/docker-registry-ro-training.json option will do the trick 😉

meirhazonAnyVision · 2018-10-10T05:37:46Z

Thanks so much, I will try it out

…

On Tue, Oct 9, 2018 at 9:05 PM Ivan Ilves ***@***.***> wrote: Hey @meirhazon1 <https://github.com/meirhazon1> Thanks for opening the issue! Sorry for not answering on time - I was vacationing 🍹 As for the issue - you run lstags inside Docker so it has no access to /root/.gcp/docker-registry-ro-training.json which belongs to the host volume and does not appear inside container by default. So, you have two options: - run lstags binary directly from the host (it's a Golang app, it has no dependencies) - "share" the /root/.gcp/docker-registry-ro-training.json file, I think -v /root/.gcp/docker-registry-ro-training.json:/root/.gcp/docker-registry-ro-training.json option will do the trick 😉 — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#174 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AoC4HkCQv4Tlc-QcoMTwGy1lC229Nracks5ujOVlgaJpZM4XDa0I> .

-- [image: Employee profile] Meir Hazon Senior DevOps Engineer Integration Team M +972 546630817 <+972%20546630817> E [email protected] W www.anyvision.co [image: image.png]

ivanilvescmp · 2018-10-19T08:04:09Z

Any success @meirhazon1 ? 😉

amnk · 2018-10-29T22:48:47Z

I am facing the same issue.

In order to run lstags from container I did docker login -u oauth2accesstoken -p "$(gcloud auth print-access-token)" https://gcr.io, and then mapped resulting ~/.docker/config.json into container, so I get proper resulting json:

{
  "auths": {
    "gcr.io": {
      "auth": "xxx",
      "email": "[email protected]"
  },
  "HttpHeaders": {
    "User-Agent": "Docker-Client/18.03.0-ce (linux)"
  }
}

but for some reason I still get

Bad response status: 401 Unauthorized >> https://gcr.io/v2/*redacted*

Btw, I tried using it locally (hoping that lstags will use already authenticated docker client), but I see same issue, while docker images ... works fine. I was using the following files.yaml:

lstags:
  repositories:
    - gcr.io/xxx
    - quay.io/xxx

and invoking lstags via

./lstags -f file.yaml -j config.json

where config.json contained authentication for quay.io

ivanilves · 2018-10-31T14:00:31Z

Hi!

I confirm lstags has currently some issues with gcr.io. And this I would investigate/fix ASAP.

Also, we lack support of Docker credential helpers (including ones for GCR) which is issue #175.

ivanilves · 2018-11-02T09:37:32Z

Hey we solved GCR issue #176

Now you should be able to connect to the GCR.

However, we still do not support credential helpers, so you need to login first:

docker login -u oauth2accesstoken -p "$(gcloud auth print-access-token)" gcr.io

Thank you for your patience and would be really grateful for any kind of feedback.

ivanilves · 2019-01-13T13:42:10Z

Closing because of lack of activity. Feel free to reopen. 😉

numiralofe · 2019-04-14T20:54:35Z

hi

sorry to ask on a close issue, but makes more sense re ask than open a new issue since I am still getting the same problem / error, you mentioned:

According to your last comment:

Now you should be able to connect to the GCR.
you need to login first:
docker login -u oauth2accesstoken -p "$(gcloud auth print-access-token)" gcr.io

i have done the login first using docker command line.
docker login -u _json_key -p "$(cat mykey.json)" https://gcr.io
on my .docker/config.json i have eu.gcr.io credentials
i can do a docker push manually to gcr.

but:

./lstags --docker-json=~/.docker/config.json -r eu.gcr.io/<my-project> myregistry.com/ubuntu18.04
Bad response status: 401 Unauthorized >> https://eu.gcr.io/v2//myregistry/com/ubuntu16.04/tags/list

I am missing something ?

/lstags --version
VERSION: v1.1.4

Thanks,
Nuno

ivanilves · 2019-04-15T10:32:51Z

Hi,

sorry to ask on a close issue

You did it right - if issue not really solved - should be re-opened 💪

I am missing something ?

hard to say, work with credentials helpers and store was never a strong part of lstags 😐

Good news probably

We plan to address this issue in a new API minor release: 1.2.X Today we released a first version -1.2.0- https://github.com/ivanilves/lstags/releases/tag/v1.2.0 - could you check it out - if it works for U?

numiralofe · 2019-04-15T11:31:27Z

hi Ivan,

Yes sure no probs :)

`cat mytoken.json | docker login -u _json_key --password-stdin https://eu.gcr.io
WARNING! Your password will be stored unencrypted in ~/.docker/config.json.
Configure a credential helper to remove this warning. See
https://docs.docker.com/engine/reference/commandline/login/#credentials-store

Login Succeeded`

and:

./lstags --docker-json=~/.docker/config.json -r eu.gcr.io/epg-prd registry.epgpay.com/ubuntu16.04 INFO[0000] BATCH 1 of 1 INFO[0000] ANALYZE myregistry.com/ubuntu16.04 INFO[0000] FETCHED myregistry.com/ubuntu16.04 <STATE> <DIGEST> <(local) ID> <Created At> <IMAGE>:<TAG> ABSENT sha256:f637006dde2fe9acf5113dfe023a654d3 n/a 2019-03-18T16:05:49Z myregistry.com/ubuntu16.04:haproxy-1.6.3 INFO[0000] [PULL/PUSH] ANALYZE myregistry.com/ubuntu16.04 => eu.gcr.io/epg-prd/myregistry/com/ubuntu16.04~/.*/ Bad response status: 401 Unauthorized >> https://eu.gcr.io/v2/epg-prd/myregistry/com/ubuntu16.04/tags/list

./lstags -V VERSION: v1.2.0

not working yet :(

thanks,
Nuno

ivanilves · 2019-04-15T13:35:15Z

Could U please try this command line:

./lstags -r eu.gcr.io -R /epg-prd registry.epgpay.com/ubuntu16.04

You are using default ~/.docker/config.json so --docker-json should be resolved automatically 👌

I think authentication was confused, because it expects push registry host and push registry path prefix to be passed separately (-r and -R params) but let's check how it works for U.

numiralofe · 2019-04-15T14:14:33Z

hi Ivan,

DEBU[0000] [CollectPushTags()] collection: &{refs:[registry.epgpay.com/ubuntu16.04] repos:map[registry.epgpay.com/ubuntu16.04:0xc4200943c0] tags:map[registry.epgpay.com/ubuntu16.04:[0xc420532280 0xc420432140 0xc4205320f0 0xc420532050 0xc420532190 0xc4204320f0 0xc4205321e0 0xc420532000]]} (1 repos / 8 tags) DEBU[0000] [CollectPushTags()] push config: {Prefix:/epg-prd Registry:eu.gcr.io UpdateChanged:false PathSeparator:/} DEBU[0000] [func1():registry.epgpay.com/ubuntu16.04] 'push' reference: eu.gcr.io/epg-prd/ubuntu16.04~/.*/ INFO[0000] [PULL/PUSH] ANALYZE registry.epgpay.com/ubuntu16.04 => eu.gcr.io/epg-prd/ubuntu16.04~/.*/ Bad response status: 401 Unauthorized >> https://eu.gcr.io/v2/epg-prd/ubuntu16.04/tags/list

still same error :(

ivanilves · 2019-04-15T15:00:38Z

Sad to hear that! I will look into it.

Meanwhile could you please tell me if running authentication by a short-lived token, like this:

gcloud auth print-access-token | docker login --username oauth2accesstoken --password-stdin eu.gcr.io

changes behavior or it remains the same?

Thank you very much!

numiralofe · 2019-04-15T15:48:11Z

it works :) cool

2 questions if possible:

but whats the diff between one login mechanism and the other ?
does it have any option to purge local copied images used for the sync process ?

thanks
Nuno

ivanilves · 2019-04-15T16:49:13Z

That's better! 😉

but whats the diff between one login mechanism and the other ?

json key is a long-lived, fixed token, while oauth tokens are short-lived, temporary ones, which means it's kinda more secure, but you should run the command I've sent before each time before using docker or lstags.

💡 You should be able to use credential helpers to escape this necessity. This is my docker config:

{
  "experimental" : "enabled",
  "HttpHeaders" : {
    "User-Agent" : "Docker-Client/18.09.0 (darwin)"
  },
  "credSstore" : "osxkeychain",
  "auths" : {
    "eu.gcr.io" : {

    },
    "https://quay.io" : {

    },
    "gcr.io" : {

    },
    "https://eu.gcr.io" : {

    }
  },
  "credHelpers" : {
    "marketplace.gcr.io" : "gcloud",
    "asia.gcr.io" : "gcloud",
    "us.gcr.io" : "gcloud",
    "gcr.io" : "gcloud",
  },
  "credsStore" : "osxkeychain",
  "stackOrchestrator" : "swarm"
}

no credentials hardcoded and it works with current lstags (v1.2.0+).

does it have any option to purge local copied images used for the sync process ?

no. but sounds like a good idea to have it!

ivanilves · 2019-04-15T16:52:42Z

#184 - added issue with cleanup feature. Feel free to comment.

numiralofe · 2019-04-15T16:55:39Z

json key is a long-lived, fixed token, while oauth tokens are short-lived, temporary ones, which means it's kinda more secure, but you should run the command I've sent before each time before using docker or lstags.

sorry perhaps i did not explain myself correctly (diff between short term tokens on long ones i already knew it :) ) i was asking the diff regarding the code itself, because if i am not wrong, short tokens will be a problem in case run lstags in daemon mode, right ? (assuming that token will expire and the daemon/lstags will not auto renew)

no. but sounds like a good idea to have it!

definitely :)

thanks,
Nuno

ivanilves · 2019-04-16T06:03:17Z

i was asking the diff regarding the code itself, because if i am not wrong, short tokens will be a problem in case run lstags in daemon mode, right ?

ah. OK. There should be no difference, but looks like something is not working as expected.

I will take a look into it and will try to understand what's happening. Have no idea for now.

Yes. Short term tokens will bring you some trouble in server mode for sure!

meirhazonAnyVision · 2019-04-16T07:01:32Z

Hello everybody,
Thanks so much for your replies and help.
We have used a different approach and we didnt use Istags yet.
Istags is great and we will surely use it in the future.
Thanks so much,
Meir

ivanilves · 2019-06-28T17:37:47Z

Hey @numiralofe 👋

Sorry for this long-long wait... It's open source after all 😉

After merging #187 we released https://github.com/ivanilves/lstags/releases/tag/v1.2.1

We tested it with @vonrabbe and now JSON tokens for GCR should work.

I'm closing this issue, but feel free to reopen if it does not work for you, or if you have any concerns.

ivanilves self-assigned this Oct 9, 2018

ivanilves added the question label Oct 9, 2018

ivanilves added bug critical handicap labels Oct 31, 2018

ivanilves removed handicap question labels Oct 31, 2018

ivanilves mentioned this issue Nov 1, 2018

Fix basic GCR communication issues #176

Merged

ivanilves closed this as completed Jan 13, 2019

ivanilves reopened this Apr 15, 2019

ivanilves mentioned this issue Apr 15, 2019

feat: Support Docker credential helpers #182

Merged

ivanilves added this to the v1.2.1 milestone May 11, 2019

ivanilves closed this as completed Jun 28, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Bad response status: 401 Unauthorized #174

Bad response status: 401 Unauthorized #174

meirhazonAnyVision commented Oct 2, 2018 •

edited

Loading

ivanilves commented Oct 9, 2018

meirhazonAnyVision commented Oct 10, 2018 via email

ivanilvescmp commented Oct 19, 2018

amnk commented Oct 29, 2018 •

edited

Loading

ivanilves commented Oct 31, 2018 •

edited

Loading

ivanilves commented Nov 2, 2018

ivanilves commented Jan 13, 2019

numiralofe commented Apr 14, 2019

ivanilves commented Apr 15, 2019

numiralofe commented Apr 15, 2019 •

edited

Loading

ivanilves commented Apr 15, 2019

numiralofe commented Apr 15, 2019

ivanilves commented Apr 15, 2019

numiralofe commented Apr 15, 2019 •

edited

Loading

ivanilves commented Apr 15, 2019

ivanilves commented Apr 15, 2019

numiralofe commented Apr 15, 2019 •

edited

Loading

ivanilves commented Apr 16, 2019

meirhazonAnyVision commented Apr 16, 2019

ivanilves commented Jun 28, 2019

Bad response status: 401 Unauthorized #174

Bad response status: 401 Unauthorized #174

Comments

meirhazonAnyVision commented Oct 2, 2018 • edited Loading

ivanilves commented Oct 9, 2018

meirhazonAnyVision commented Oct 10, 2018 via email

ivanilvescmp commented Oct 19, 2018

amnk commented Oct 29, 2018 • edited Loading

ivanilves commented Oct 31, 2018 • edited Loading

ivanilves commented Nov 2, 2018

ivanilves commented Jan 13, 2019

numiralofe commented Apr 14, 2019

ivanilves commented Apr 15, 2019

Good news probably

numiralofe commented Apr 15, 2019 • edited Loading

ivanilves commented Apr 15, 2019

numiralofe commented Apr 15, 2019

ivanilves commented Apr 15, 2019

numiralofe commented Apr 15, 2019 • edited Loading

ivanilves commented Apr 15, 2019

ivanilves commented Apr 15, 2019

numiralofe commented Apr 15, 2019 • edited Loading

ivanilves commented Apr 16, 2019

meirhazonAnyVision commented Apr 16, 2019

ivanilves commented Jun 28, 2019

meirhazonAnyVision commented Oct 2, 2018 •

edited

Loading

amnk commented Oct 29, 2018 •

edited

Loading

ivanilves commented Oct 31, 2018 •

edited

Loading

numiralofe commented Apr 15, 2019 •

edited

Loading

numiralofe commented Apr 15, 2019 •

edited

Loading

numiralofe commented Apr 15, 2019 •

edited

Loading