[ADR] Select an IDAM integration solution for GitLab

[Racer159]

Is your feature request related to a problem? Please describe.

As Kay I want to have GitLab integrated with Keycloak (for authentication) so that I can easily login with one account / set of credentials.

Describe the solution you'd like

We should determine a method that allows us to integrate GitLab with Keycloak in uds-core that considers the following:

• SAML integration may be required for some customers
• We should consider license reqs: https://docs.gitlab.com/ee/administration/auth/
• We should consider RMF / control requirements (engage @CloudBeard)
• Delivery should be engaged to validate the above

These should be written in a simple ADR to capture why the final solution was chosen and a follow on implementation issue should be created.

Additional context

This is skewed towards Cloud solutions.

This is needed to meet IA requirements and should be focused on authn for now.

uds-core README: https://github.com/defenseunicorns/uds-core/blob/0a83d02f5782d911e3bb63935b0cac70030e5c9b/src/keycloak/README.md?plain=1

[zachariahmiller]

Engage with @blancharda @ntwkninja and anyone else they suggest

[ntwkninja]

SAML integration may be required for some customers
We should consider license reqs: https://docs.gitlab.com/ee/administration/auth/
We should consider RMF / control requirements (engage @CloudBeard)
Delivery should be engaged to validate the above

• not tracking any specific requirements for SAML
• license == GL Premium
• RMF, it's probably safe to assume that we're not documenting anything beyond this

On a semi-related note, if there is a way to integrate Keycloak groups with GitLab groups that could be a desirable feature but likely out of scope for the early stages.

[zachariahmiller]

On a semi-related note, if there is a way to integrate Keycloak groups with GitLab groups that could be a desirable feature but likely out of scope for the early stages.

IIRC from the work @jacobbmay did researching this, SAML group mapping is an ultimate feature so that wouldn't be there out of the box.

Given our requirements for UDS-prod I believe our goal will be to automate this integration ourselves to functionally achieve the same end result, it just won't be baked in.

[jacobbmay]

Yeah SAML group sync requires at least premium: https://docs.gitlab.com/ee/user/group/saml_sso/group_sync.html

I think they have started to add group syncing features for OIDC now too (not as full featured as SAML groups yet), but looks like that also requires premium: https://docs.gitlab.com/ee/administration/auth/oidc.html#configure-users-based-on-oidc-group-membership

With premium, SAML groups sync can definitely be used to map keycloak groups to GitLab groups. I think you might be able to have it preconfigured without a paid license so it goes into effect once you activate a license, but I'm not certain.

[naveensrinivasan]

Do we need a prototype solution for the ADR? We already know that Keycloak works with GitLab.

[zachariahmiller]

@naveensrinivasan I think you can write the proposal and leave it in that state, then go do the prototyping before we accept it

[naveensrinivasan]

@naveensrinivasan I think you can write the proposal and leave it in that state, then go do the prototyping before we accept it

Isn't Keycloak already being used in GitLab? Do you think the prototype adds value?

Unless the ADR recommends using Keycloak differently, we should stick with what we're used to.

[zachariahmiller]

@naveensrinivasan It is being used but not in this specific implementation. There are multiple patterns and approaches ti automating those patterns and that is why prototyping will be needed

[zachariahmiller]

Noting this here. Less architectural but important from the perspective of the implementation path #60

[naveensrinivasan]

I made some significant changes to the ADR to address @bburky request for user deletion and auditing features in the future.

[bburky]

Thanks @naveensrinivasan, see comments in defenseunicorns/uds-software-factory#16 for more info.

Both SAML Group Sync and SCIM are paid Premium/Ultimate features:

• https://docs.gitlab.com/ee/user/group/saml_sso/group_sync.html
• https://docs.gitlab.com/ee/administration/settings/scim_setup.html

If users do not have Group Sync they need to manually administer access inside GitLab, this often means end users end up with too much access in practice when they incorrectly retain access when moving between teams. SCIM is the obvious API to implement automatic deprovisioning, which is important to disable GitLab accounts and prevent access via PAT and SSH key after users are disabled in Keycloak.

Are we okay with users not having these important security features? Do we really want to maintain custom reimplementations of these features? Can we require at least GitLab Premium, or clearly tell users their admins need to do lots of manual management if they use free GitLab?

Should we at least configure GitLab with SAML instead of OIDC so that users who have a paid GitLab license can adopt these features?

[zachariahmiller]

Thanks @naveensrinivasan, see comments in defenseunicorns/uds-software-factory#16 for more info.

Both SAML Group Sync and SCIM are paid Premium/Ultimate features:

* https://docs.gitlab.com/ee/user/group/saml_sso/group_sync.html

* https://docs.gitlab.com/ee/administration/settings/scim_setup.html

If users do not have Group Sync they need to manually administer access inside GitLab, this often means end users end up with too much access in practice when they incorrectly retain access when moving between teams. SCIM is the obvious API to implement automatic deprovisioning, which is important to disable GitLab accounts and prevent access via PAT and SSH key after users are disabled in Keycloak.

Are we okay with users not having these important security features? Do we really want to maintain custom reimplementations of these features? Can we require at least GitLab Premium, or clearly tell users their admins need to do lots of manual management if they use free GitLab?

Should we at least configure GitLab with SAML instead of OIDC so that users who have a paid GitLab license can adopt these features?

@bburky Thanks for being so responsive on this issue and the draft ADR, its very much appreciated.

I believe we are targeting premium and can make default assumptions based on that, even if it is possible for someone on their own to intentionally override things to use oidc for their environment. Concerns would be if some of the features we actually need are locked behind ultimate at which point I believe our desire as a team would be to mitigate that with tooling to solve the same concerns. Not the most ideal scenario, but neither is $100/user/month.

Its kind of unclear to me in their documentation what gaps may exist in the functionality between Premium and Ultimate in this regard, but if SAML group sync and SCIM are fully featured in Premium we should target that as our baseline config, if there are gaps we may need to bridge the gaps, but the less we have to do custom to fulfill our requirement the better. Either way, first step would be determining that beyond any doubt and making a fully informed decision from there.

CC @oates in-case im misrepresenting something here from a business standpoint.

[oates]

Correct. We will have premium for UDS Prod as it stands today. We could potentially move away from it one day but this and several other things in SWF would need to be addressed.

[zachariahmiller]

Correct. We will have premium for UDS Prod as it stands today. We could potentially move away from it one day but this and several other things in SWF would need to be addressed.

Do we have a target of how many users we are budgeted for as a premium license?

[bburky]

Ok, thanks. If we're doing a minimum of Premium for SWF, that is good to know. So far I think everything SSO was in Premium. We could also perhaps support lower as "not recommended" and "admins will need to do more things manually".

#60 was implemented using openid_connect, we may want to switch to SAML to support Group Sync and SCIM in the future. I think we could integrate SAML Group Sync with Keycloak today (but note it's only updated on interactive login, SCIM Group Sync is an open issue for GitLab). SCIM for deprovisioning is supported by GitLab, but is not supported natively by Keycloak... SCIM would still be my suggested API instead of using GitLab-specific APIs, defenseunicorns/uds-software-factory#16 discusses implementing this with custom code.

[zachariahmiller]

Ok, thanks. If we're doing a minimum of Premium for SWF, that is good to know. So far I think everything SSO was in Premium. We could also perhaps support lower as "not recommended" and "admins will need to do more things manually".

#60 was implemented using openid_connect, we may want to switch to SAML to support Group Sync and SCIM in the future. I think we could integrate SAML Group Sync with Keycloak today (but note it's only updated on interactive login, SCIM Group Sync is an open issue for GitLab). SCIM for deprovisioning is supported by GitLab, but is not supported natively by Keycloak... SCIM would still be my suggested API instead of using GitLab-specific APIs, defenseunicorns/uds-software-factory#16 discusses implementing this with custom code.

Yeah that initial implementation of oidc was to remove blockers for users that don't have the same requirements or environment considerations as we would have in a cloud hosted environment, essentially with them accepting the caveats and call outs you noted above.

We have the intent to update to SAML as the default config pending the output of this ADR and taking into account any limitations either license or implementation wise that may have to be accounted for.

For SCIM we will have to figure out the best way to implement the automation for all of our needs if there are gaps beyond what comes out of the box with keycloak (existing plugin or updating existing plugin hopefully) and gitlab's SCIM implementation, but 100% agreed that the default path there should be to utilize SCIM where feasible instead of building a custom system.

fwiw it's possible there may be a desire in the future to extend this type of functionality to work without requiring a premium gitlab license, but that is a very far forward thinking view. That would be driven by very intentional and risk weighted business decisions and is realistically way out of scope of what we need to define for a path forward for the near to mid term.

I'll provide additional feedback on the ADR later, but wanted to respond to your specific call outs here. Again, very much appreciate your promptly taking the time for a thorough review of all of this!

[bburky]

Requirements:

Authentication

Use SSO from Keycoak for user authentication in the browser. Do not support username and password login to GitLab.

Provision users

Provide a method to provision users. Such as autoprovisioning on first SSO login, or automatic pushing (e.g. SCIM) of a user list from Keycloak to GitLab. Both are recommended.

Note: it's nice to support pushed users, because this allows users within the GitLab environment to begin interacting with the new user before their first login.

Automatic provisioning (user creation) is required. There is no specific requirement between specific implementations of: autoprovisioning on SSO login, live automatic pushing, cron/scheduled automatic pushing. However, the automatic provisioning should be prompt.

Deprovision users

Note, it possible to authenticate to GitLab with Personal Access Tokens (PATs) and SSH keys, these bypass SSO because the user does not login with a browser. It is critical to promptly deactivate or delete users from GitLab to terminate access.

(By default UDS will likely disable SSH entirely. This simplifies the network access to the environment: all access into NLB → Kubernetes LoadBalancer → Istio, etc is all pure HTTP(S).)

An alternate option is to not delete the user from GitLab but instead remove all of their accesses (remove them from all groups and repos). This is not recommended, ideally deactivated users become fully unauthenticated (cannot log in) after terminating access.

Note, deleted and deactivated users do not consume a license. GitLab bills per user active in the past 30 days(?). To minimize cost, users should be deactivated promptly.

A GitLab administrator can also manually deprovision users. Automation of deprovision is not required initially, but should be targeted for a long term design.

See:

• https://docs.gitlab.com/ee/administration/settings/scim_setup.html

GitLab Groups

Within GitLab groups are often used to administer users' access to groups of repositories. These groups can be manually configured by a GitLab administrator or automated from SAML groups or SCIM groups. Automation of groups is not required initially, but should be targeted for a long term design.

See:

• https://docs.gitlab.com/ee/integration/saml.html#configure-users-based-on-saml-group-membership
• https://gitlab.com/groups/gitlab-org/-/epics/8902
• https://docs.gitlab.com/ee/administration/auth/ldap/ldap_synchronization.html (do not use LDAP username/password login, but LDAP sync is plausibly an option)

Instance admin group

A list of GitLab instance admins could be configured in Keycloak and automatically synced (e.g. SAML or SCIM) to GitLab.

This is only a nice to have, this is not required.

See: https://docs.gitlab.com/ee/integration/saml.html#administrator-groups

Audit

A complete audit implementation is out of scope of this ADR. However, please do identify how current users and groups can be queried/exported from GitLab. Also identify how to get a live streaming access log of all user activity out of GitLab (and written to e.g. stdout, a file, an s3 bucket, over net network, etc). Audit log storage storage is out of scope.