Use gitcredential helpers for git authentication

[hferentschik]

Following up on issue #5341 which used a different approach for fixing the problem around ForkAndPullRepo in helpers.go.

The idea is to find a more holistic approach of handling git authentication locally as well as in the pipeline.

To avoid writing anything to disk, git's credential helpers, in particular the cache helper, could be used. The cache helper stores the credentials for a specified time in memory. The credentials never hit the disk. The idea is to add something along this line to Gitter:

// CacheCredendials caches the credentials for the specified user, host and path for the specified number of seconds.
CacheCredendials(host string, path string, username string, password string, time int) error

// Clears potentially cached credentials for the specified host, path and user, if they exist
ClearCredentials(host string, path string, username string) error

This way, when dealing with repositories which need authentication one can call CacheCredendials once with an appropriate timeout and then call other methods of the Gitter interface without having to think about the credentials.

As a further step, one can then imagine providing our own credential helper. There is an API for that. This custom credential helper could be a standalone program or a jx command. It would then retrieve the credentials from the cluster.

This mechanism would also solve the issue around jx import adding the API token to the git remote.

See also https://git-scm.com/book/en/v2/Git-Tools-Credential-Storage

[hferentschik]

@ccojocar ^^

[ccojocar]

I think this is the way to go for git auth. We can implement a custom credentials helper for vault and one for k8s secrets, even though in the case of k8s secrets the Tekton pipeline should pre-configure the git in the build pods from the secret attached to the service account without get secrets permissions (see https://github.com/tektoncd/pipeline/blob/master/docs/auth.md#basic-authentication-git).

I would like to have a credential helper which is able to read/write git credentials in/from vault something liken (secret/git/<url>/ username: value, password: value). I think the url needs to be encoded somehow to allow fast lookup (maybe replace / and . with _). It should require for authentication either a vault token in an environment variable or a k8s service account in a file (this is usually how a service accounts are mounted into pods).

Some examples:
https://github.com/git/git/blob/master/contrib/credential/libsecret/git-credential-libsecret.c
https://github.com/kelseyhightower/hub-credential-helper/blob/master/main.go

It is also possible to query credentials form helper via git. See git credential fill and also https://git-scm.com/docs/git-credential. We can maybe explore this for git providers authentication where we can query the token from git credentials store.

[hferentschik]

There is second option which might be a bit easier to implement. We could leverage the GIT_ASKPASS env variable. We could set it either where needed or as early as a jx call gets initialized (in the Cobra setup).

We could set GIT_ASKPASS to jx itself using a command like jx step git credential.

This approach would solve authentication for all cases where jx is involved and where we make git calls from within jx.

The disadvantage of this approach is that it won't work for users using the git CLI directly. There a dedicated k8s-cred-helper is superior, since once configured it will work regardless of whether git is called as part of a jx command or standalone.

[hferentschik]

See discussion around git authentication on this PR as well #5850 (comment)

[jenkins-x-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.
If this issue is safe to close now please do so with /close.
Provide feedback via https://jenkins-x.io/community.
/lifecycle stale

[jenkins-x-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.
If this issue is safe to close now please do so with /close.
Provide feedback via https://jenkins-x.io/community.
/lifecycle stale

[hferentschik]

/remove-lifecycle stale

[jenkins-x-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.
If this issue is safe to close now please do so with /close.
Provide feedback via https://jenkins-x.io/community.
/lifecycle stale

[jenkins-x-bot]

Stale issues rot after 30d of inactivity.
Mark the issue as fresh with /remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.
If this issue is safe to close now please do so with /close.
Provide feedback via https://jenkins-x.io/community.
/lifecycle rotten

[jenkins-x-bot]

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen.
Mark the issue as fresh with /remove-lifecycle rotten.
Provide feedback via https://jenkins-x.io/community.
/close

[jenkins-x-bot]

@jenkins-x-bot: Closing this issue.

In response to this:

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen.
Mark the issue as fresh with /remove-lifecycle rotten.
Provide feedback via https://jenkins-x.io/community.
/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the jenkins-x/lighthouse repository.