add support for harbor2 OpenID Connect (OIDC) users

[Vad1mo]

I am trying out again docker-pushrm with now more realistic scenario.

My user vad1mo is Project Admin on Project c8n. This Role is needed in Harbor to be able to edit the Project Info in the UI.
However it seems that this role is not sufficient to update the project info via the API.

It might be also the case that we are using Harbor2 with OIDC, I verified in the API explorer that the user is able to do operations with the username/password, where password is the one taken from harbor->user->profile and not OIDC.

Status: Downloaded newer image for chko/docker-pushrm:1
DEBU root cmd init config                         
DEBU home dir: /                                  
DEBU subcommand "pushrm" called                   
DEBU Using target: c8n.io/c8n/portal:latest       
DEBU using README file: /myvol/README.md          
DEBU server: c8n.io                               
DEBU namespace: c8n                               
DEBU repo: portal                                 
DEBU tag: latest                                  
DEBU repo provider: harbor2                       
DEBU Harbor2.GetAuthident called                  
DEBU using credentials for user vad1mo from generic env var 
DEBU Using Docker creds: vad1mo ********          
DEBU Harbor2.Pushrm called                        
DEBU push readme, response body: {"errors":[{"code":"UNAUTHORIZED","message":"unauthorized"}]} 
DEBU push README, status code: 401                
DEBU error pushing README, bad status code for response: 401 Unauthorized. Server responded: "UNAUTHORIZED - unauthorized" 
ERRO error pushing readme to repo server. See error message below. Run with "--debug" for more details. 

error pushing README, bad status code for response: 401 Unauthorized. Server responded: "UNAUTHORIZED - unauthorized" 

[Vad1mo]

I tried now the api explorer and the generated CURL, It works for the user vad1mo. It seems to be an issue on the docker- pushrm side rather than harbor.

I didn't see any X-Harbor-CSRF-Token in the code, it seems that docker-pushrm needs to honor the OIDC flow.

curl -X PUT "https://c8n.io/api/v2.0/projects/c8n/repositories/portal" -H "accept: application/json" -H "X-Request-Id: 8ju7378fh3f834h8" -H "authorization: Basic XXXXXXXX" -H "Content-Type: application/json" -H "X-Harbor-CSRF-Token: XXXXXX" -d "{ \"description\": \"TEST WORKS\"}"

[christian-korneck]

thanks, that’s a good catch. I’ve indeed only tested against local harbor users so far. (Just tried on demo.goharbor.io and a local instance and it worked there). Most likely we need a csrf header.

I’ll sign up on c8n.io tonight and test it.

[christian-korneck]

@Vad1mo what kind of user/pass are you using in the above curl command (in the basic auth header)? is this a local user or an oidc user? what do you use as password? the cli secret? something else? thanks

[Vad1mo]

Yes the user is OIDC but I am using the users CLI secret from the user profile.

This is also how I used it on the API Portal to login with username + CLI secret on https://c8n.io/devcenter-api-2.0

[christian-korneck]

hmm...strange, on the c8n.io api portal I don't seem to be able to make any authenticated api calls (i.e. I'm logging in with user chris and the cli secret and trying to execute the /users/current request but getting 401 unauthorized).

[christian-korneck]

ok, it now works after I have changed the cli secret to a manually generated shorter one. Also oddly I can't change it back to the old auto-generated password (getting an error in the UI). First guess is there's a problem with the length? (UI only accepts 31 characters when setting the password but the auto-generated one was 32 characters long).

[christian-korneck]

ok, I've played around a bit and I have the impression that there's a server side issue that I can't work around client side (glad if you prove me wrong 😃).

It appears to me that you can only get a valid csrf token by logging into c8n.io in the browser (via auth0). Then - and only for the session (also time limited) - you're possessing 3 cookies, of which you need two to be able to make a successful HTTP call:

• sid (needed)
• __csrf (needed)
• _gorilla_csrf (not needed? not sure what it's for?)

When you copy them from a browser session you're able to make a curl call like:

curl -H 'Cookie: sid=XXXX; __csrf=XXXX' -H 'accept: application/json' -H 'authorization: Basic XXXX' -H 'x-harbor-csrf-token: XXXX' 'https://c8n.io/api/v2.0/users/current'

where

• sid = sid (cookie value)
• x-harbor-csrf-token = __csrf (cookie value)
• authorization: Basic = base64 of <username>:<cli-secret> (needed in addition to the session cookie)

Having said that, you can get all three cookies also using basic auth, but they don't allow you to make an api call that requires auth:

(curl -c file.txt saves the cookies from the response to file and curl -b file.txt loads them from file for the request)

Make a request and we get a sid cookie:

$ curl https://c8n.io/api/v2.0/users/current -c cookies.txt
{"errors":[{"code":"UNAUTHORIZED","message":"UnAuthorize"}]}
$ cat cookies.txt
#HttpOnly_c8n.io	FALSE	/	FALSE	0	sid	XXXX

Make another request and send the stored cookie:

$ curl https://c8n.io/api/v2.0/users/current -c cookies.txt -b cookies.txt
{"errors":[{"code":"UNAUTHORIZED","message":"UnAuthorize"}]}

The server responds with the __csrf and _gorilla_csrf cookies:

$ cat cookies.txt
c8n.io	FALSE	/	TRUE	0	__csrf	XXXX 
#HttpOnly_c8n.io	FALSE	/	TRUE	1599291933	_gorilla_csrf	XXXX
#HttpOnly_c8n.io	FALSE	/	FALSE	0	sid	XXXX

But when we try to use them (in exactly the same request as with the cookies from the browser session before), they don't work:

curl -H 'Cookie: sid=XXXX' -H 'accept: application/json' -H 'authorization: Basic XXXX' -H 'x-harbor-csrf-token: XXXX' 'https://c8n.io/api/v2.0/users/current'
{"errors":[{"code":"UNAUTHORIZED","message":"UnAuthorize"}]}

So, why does it work with the API Explorer (Harbor bundled Swagger UI)? Because it runs in the browser and sends the cookies if they exist. If you try to use the API Explorer from a browser session where you haven't logged in with auth0 (or in which the cookies have expired) it doesn't work either (even if you put the username + cli secret in the swagger "login" form).

An API call with the API Explorer looks like (not as simple as the curl command that it generates):

curl -H 'Host: c8n.io' -H 'Cookie: sid=XXXX; _gorilla_csrf=XXXX; __csrf=XXXX' -H 'user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:80.0) Gecko/20100101 Firefox/80.0' -H 'accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8' -H 'accept-language: en-US,en;q=0.5' -H 'dnt: 1' -H 'upgrade-insecure-requests: 1' -H 'cache-control: max-age=0' -H 'te: trailers' --compressed 'https://c8n.io/api/v2.0/systeminfo'

I'm not really familiar with harbor OIDC but from googling I believe to use the API with an OIDC provider you would need an OIDC token from the OIDC provider:

"You need to id token from the OIDC provider you configured." source

Here's some infos in the FAQ. And there's also an example how to get it from the Keycload OIDC provider: Looks like an API request against Keycloak's api.

So I assume that if c8n.io would somehow show users their OIDC token then users could use it with docker-pushrm. I don't think it can be used to log in with Docker, so probably a change to docker-pushrm would still be needed to enable api keys for the harbor2 provider.

I'm not sure if fetching the OIDC token is a standardized thing that is the same for all OIDC providers? (if so it might be possible to add it to docker-pushrm, i.e. by showing users an auth link that they need to open in the browser. But I assume such a webapp would need to be hosted by the owner of the realm, i.e. c8n.io).

@Vad1mo do you maybe have any idea how I could fetch the OIDC token for my c8n.io user?

[christian-korneck]

documented this as known limitation and changed issue label from bug to enhancement

[Vad1mo]

Thank you @christian-korneck for your investigation. Thats a bummer!

"Service accounts" are on the roadmap. In some issue comment it noted that this will contain some sort of API Keys.

Nevertheless, I will investigate further on how to get it working with AUTH0 for a normal user, The requested scopes openid,offline_access,profile,email should permit to get an ID Token.

[Vad1mo]

API Keys are on the Roadmap for 2.2 see goharbor/harbor#13093

[christian-korneck]

looks like harbor v2.2.0-rc1 has been released today, with 8 new APIs for robot accounts. I’ll try it soon.

[Vad1mo]

Yes, I have seen it this morning. Recalling the last demo I have seen regarding 2.2 it might still not work to update the Docs via the Robot account.

However it will be quite easy to extend the permission, as they plan it to make it more flexible allowing to define custom permission.

What we could do is to setup a test 2.2 instance quickly and see if it is working and if not crete a PR to harbor adding this capability.

[christian-korneck]

some related links (will look into this again soon'ish)
goharbor/harbor#14145 (comment)
https://github.com/goharbor/harbor/blob/main/src/common/rbac/const.go

[giosava94]

Hi, there are any news or updates about this topic? Is the development still on going?

[christian-korneck]

yes, this project is still maintained. Regarding this issue: Not sure. There has been a lot of development in Harbor but I haven't tracked it. Are you are Harbor user? Can you test?

[christian-korneck]

quick update: This issue has likely been resolved by improvments made in Harbor. I just tried on the https://demo.goharbor.io demo instance and can confirm that a robot account with the "new" update repo permission is sufficient for using docker-pushrm. And I believe robot accounts are not affected by the OIDC issues described here (that occured with regular user logins iirc).

So we probably can update the docs and close this issue.

The only reason why I'm leaving this issue open for now is that demo.goharbor.io does not use OIDC auth (and I currently don't have access to any other harbor instance with OIDC). So we should find a way to confirm this also with a quick test against a Harbor OIDC instance. If anyone could test this, I'd appreciate it. (If no one can help, it'll be a project for some future weekend for me to set up Harbor against Keycloak).

[Vad1mo]

Harbor robot accounts are completely independent of the underlying auth system.
Happy to close this issue 🎉

[christian-korneck]

Thanks, Vadim! I've now also updated the docs: link .

[giosava94]

quick update: This issue has likely been resolved by improvments made in Harbor. I just tried on the https://demo.goharbor.io demo instance and can confirm that a robot account with the "new" update repo permission is sufficient for using docker-pushrm. And I believe robot accounts are not affected by the OIDC issues described here (that occured with regular user logins iirc).

So we probably can update the docs and close this issue.

The only reason why I'm leaving this issue open for now is that demo.goharbor.io does not use OIDC auth (and I currently don't have access to any other harbor instance with OIDC). So we should find a way to confirm this also with a quick test against a Harbor OIDC instance. If anyone could test this, I'd appreciate it. (If no one can help, it'll be a project for some future weekend for me to set up Harbor against Keycloak).

Hi,
sorry for the very late response.

We are using harbor 2.11 but the permission "Update repository" is not available.

In our case our robot account has these permissions:

we can set these permissions (as you can see there is no "Update repository")

Moreover, both in version 2.10 and 2.9 we can't see that permission. Which harbor version did you use in your example?

[Vad1mo]

Not all permission are exposed in the UI, also some UI permissions are combined of low level permissions. Take a look at the REST API docs. And maybe code to truly assess the situation. I don't want to tell something wrong here.

[christian-korneck]

@giosava94 Ive tried on the public demo server of the Harbor project. You can try and see for yourself. Maybe it helps you to figure out what's different with your system. Login infos are here:
https://goharbor.io/docs/2.11.0/install-config/demo-server/

The "update repo" permissions are definitely visible in the UI (see screenshots above). Version of the demo server is always the latest (currently 2.11.0).

[giosava94]

Thank you very much. I will give it a try.