Azure passthrough #433

joelddiaz · 2021-12-14T21:03:28Z

Migrate away from depending on the Azure Active Directory Graph API since it will be sunset in June 2022.

No more mint mode for Azure. So continue supporting passthrough and manual mode as the only supported modes on Azure.
The implementation is to always mark the Azure root secret as passthrough, and pivot a live running originally-mint cluster to passthrough mode.
Overwrite existing Secrets with the content found in kube-system/azure-credentials (we can assume this as removing the root creds was never supported in Azure).
Try to clean up previously minted App Registrations / Service Principals (while the Azure AD Graph API still works), but treat errors as non-fatal (just set a condition on the CredentialsRequest).
Update test cases.

xref: https://issues.redhat.com/browse/CCO-173

go get github.com/Azure/azure-sdk-for-go go mod vendor ; go mod tidy commit

go get github.com/Azure/go-autorest/autorest go get github.com/Azure/go-autorest/autorest/adal go get github.com/Azure/go-autorest/autorest/azure/auth go get github.com/Azure/go-autorest/autorest/to go get github.com/Azure/go-autorest/autorest/azure/cli go get github.com/Azure/go-autorest/autorest/validation followed by go mod vendor ; go mod tidy and committing the changes

codecov · 2021-12-14T21:07:42Z

Codecov Report

Merging #433 (bf76773) into master (64bab70) will decrease coverage by 0.21%.
The diff coverage is 82.40%.

❗ Current head bf76773 differs from pull request most recent head 32b6237. Consider uploading reports for the commit 32b6237 to get more accurate results

@@            Coverage Diff             @@
##           master     #433      +/-   ##
==========================================
- Coverage   46.39%   46.17%   -0.22%     
==========================================
  Files          94       91       -3     
  Lines        9688     9236     -452     
==========================================
- Hits         4495     4265     -230     
+ Misses       4612     4455     -157     
+ Partials      581      516      -65

Impacted Files	Coverage Δ
pkg/azure/clients.go	`0.00% <ø> (ø)`
pkg/azure/minter.go	`35.71% <55.55%> (-12.85%)`	⬇️
pkg/operator/secretannotator/azure/reconciler.go	`35.52% <75.00%> (-1.78%)`	⬇️
pkg/azure/actuator.go	`56.83% <78.26%> (+11.78%)`	⬆️
...redentialsrequest/credentialsrequest_controller.go	`54.08% <100.00%> (+3.53%)`	⬆️
pkg/azure/mock/client_generated.go	`20.72% <0.00%> (-63.97%)`	⬇️
... and 2 more

pkg/azure/actuator.go

akhil-rane · 2021-12-14T23:57:12Z

pkg/azure/actuator.go

+func (a *Actuator) syncPassthrough(ctx context.Context, cr *minterv1.CredentialsRequest, cloudCredsSecret *corev1.Secret, logger log.FieldLogger) error {
+	syncErr := a.syncCredentialSecrets(ctx, cr, cloudCredsSecret, logger)
+
+	// Since we are live pivoting from Mint to Passthrough, try to clean up the old App Registration
+	cleanupErr := a.cleanupAfterPassthroughPivot(ctx, cr, cloudCredsSecret, logger)
+	if cleanupErr != nil {
+		logger.WithError(cleanupErr).Warn("unable to clean up previously minted App Regisration/Service Principal")
 	}
+	if syncErr == nil && cleanupErr != nil {
+		// A syncErr is more important to communicate about
+		// but do set a condition if the only issue was
+		// during cleanup.
+		return cleanupErr
+	}
+	return syncErr
+}


Suggested change

func (a *Actuator) syncPassthrough(ctx context.Context, cr *minterv1.CredentialsRequest, cloudCredsSecret *corev1.Secret, logger log.FieldLogger) error {

syncErr := a.syncCredentialSecrets(ctx, cr, cloudCredsSecret, logger)

// Since we are live pivoting from Mint to Passthrough, try to clean up the old App Registration

cleanupErr := a.cleanupAfterPassthroughPivot(ctx, cr, cloudCredsSecret, logger)

if cleanupErr != nil {

logger.WithError(cleanupErr).Warn("unable to clean up previously minted App Regisration/Service Principal")

}

if syncErr == nil && cleanupErr != nil {

// A syncErr is more important to communicate about

// but do set a condition if the only issue was

// during cleanup.

return cleanupErr

}

return syncErr

}

func (a *Actuator) syncPassthrough(ctx context.Context, cr *minterv1.CredentialsRequest, cloudCredsSecret *corev1.Secret, logger log.FieldLogger) error {

syncErr := a.syncCredentialSecrets(ctx, cr, cloudCredsSecret, logger)

if syncErr != nil {

return syncErr

}

// Since we are live pivoting from Mint to Passthrough, try to clean up the old App Registration

cleanupErr := a.cleanupAfterPassthroughPivot(ctx, cr, cloudCredsSecret, logger)

if cleanupErr != nil {

logger.WithError(cleanupErr).Warn("unable to clean up previously minted App Regisration/Service Principal")

}

return cleanupErr

}

This actually addresses the potential that we would fail to sync the secret, but still attempted a cleanup. That could lead to bad things. Good catch!

akhil-rane · 2021-12-15T00:02:55Z

pkg/azure/actuator.go

-	// When a service principal is deleted, it's corresponding credentials becomes invalid.
-	// Pass-through credentials are not created through crafted service principal.
-	// When a request is deleted, there is no service principal to delete.
-	// Thus, corresponding secret still provides valid credentials.
-	// For that reason, existing secret object needs to be deleted as well to avoid
-	// credentials leaking.
-	//
-	// Also, there is no harm in deleting the secret in general. Every component consuming
-	// the secret will be forbidden to talk to Azure API once the service principal is destroyed.
-	existingSecret := &corev1.Secret{}
-	err = a.client.Get(ctx, client.ObjectKey{Namespace: cr.Spec.SecretRef.Namespace, Name: cr.Spec.SecretRef.Name}, existingSecret)
-	if err == nil {
-		logger.Infof("Deleting secret %v/%v", cr.Spec.SecretRef.Namespace, cr.Spec.SecretRef.Name)
-		if err := a.client.Delete(ctx, existingSecret); err != nil {
-			return fmt.Errorf("unable to delete secret %v/%v: %v", cr.Spec.SecretRef.Namespace, cr.Spec.SecretRef.Name, err)
-		}
-	} else if !kerrors.IsNotFound(err) {
-		return fmt.Errorf("unable to get secret %v/%v: %v", cr.Spec.SecretRef.Namespace, cr.Spec.SecretRef.Name, err)
-	}
-
-	if credentialsRootSecret.Annotations[constants.AnnotationKey] == constants.PassthroughAnnotation {
-		return nil
-	}


why do we want to remove this code? I believe if we remove this, the target secret would not be removed when the credentials request is deleted.

This code is already handled in the non-actuator-specific Delete() path https://github.com/openshift/cloud-credential-operator/blob/master/pkg/operator/credentialsrequest/credentialsrequest_controller.go#L436-L457
So this code was unnecessary to begin with.

akhil-rane · 2021-12-15T15:37:43Z

pkg/azure/actuator_test.go

 			validate: func(t *testing.T, c client.Client) {
 				cr := getCredRequest(t, c)
-				s := &corev1.Secret{}
-				// secret should be deleted
-				assert.Error(t, c.Get(context.TODO(),
-					types.NamespacedName{Name: cr.Spec.SecretRef.Name, Namespace: cr.Spec.SecretRef.Namespace},
-					s),
+
+				targetSecret := getCredRequestTargetSecret(t, c, cr)
+
+				rootSecret := getRootSecret(t, c)
+
+				// The targetSecret should be a copy of the root secret.
+				assertSecretEquality(t, rootSecret, targetSecret)
+			},


Do we need to check for OrphanedCloudResource condition here?

The condition is set by the credentials_request controller, so the best we can do with the test at the actuator-level is check for the right error returned by the actuator. (Although I should go ahead and add a test for the condition in the credentials_request controller. I'll work on that now).

joelddiaz · 2021-12-15T17:32:07Z

@akhil-rane i fixed up the code where you suggested and added a whole new set of test cases at the credentialsrequest_controller level to cover the new OrphanedCloudResource condition. PTAL

joelddiaz · 2021-12-15T22:31:12Z

/test e2e-azure

joelddiaz · 2021-12-15T22:31:25Z

/test verify

joelddiaz · 2021-12-16T13:30:55Z

/test verify

joelddiaz · 2021-12-16T13:52:21Z

/test e2e-azure-upgrade

openshift-ci · 2021-12-16T13:52:22Z

@joelddiaz: The specified target(s) for /test were not found.
The following commands are available to trigger required jobs:

/test coverage
/test e2e-aws
/test e2e-upgrade
/test images
/test unit
/test verify
/test verify-deps

The following commands are available to trigger optional jobs:

/test e2e-aws-manual-oidc
/test e2e-azure
/test e2e-gcp
/test e2e-gcp-manual-oidc
/test e2e-openstack
/test e2e-openstack-parallel

Use /test all to run the following jobs that were automatically triggered:

pull-ci-openshift-cloud-credential-operator-master-coverage
pull-ci-openshift-cloud-credential-operator-master-e2e-aws
pull-ci-openshift-cloud-credential-operator-master-e2e-upgrade
pull-ci-openshift-cloud-credential-operator-master-images
pull-ci-openshift-cloud-credential-operator-master-unit
pull-ci-openshift-cloud-credential-operator-master-verify
pull-ci-openshift-cloud-credential-operator-master-verify-deps

Details

In response to this:

/test e2e-azure-upgrade

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

joelddiaz · 2021-12-16T15:54:47Z

/test e2e-azure-upgrade

openshift-ci · 2021-12-16T16:04:23Z

@joelddiaz: The specified target(s) for /test were not found.
The following commands are available to trigger required jobs:

/test coverage
/test e2e-aws
/test e2e-upgrade
/test images
/test unit
/test verify
/test verify-deps

The following commands are available to trigger optional jobs:

/test e2e-aws-manual-oidc
/test e2e-azure
/test e2e-gcp
/test e2e-gcp-manual-oidc
/test e2e-openstack
/test e2e-openstack-parallel

Use /test all to run the following jobs that were automatically triggered:

pull-ci-openshift-cloud-credential-operator-master-coverage
pull-ci-openshift-cloud-credential-operator-master-e2e-aws
pull-ci-openshift-cloud-credential-operator-master-e2e-upgrade
pull-ci-openshift-cloud-credential-operator-master-images
pull-ci-openshift-cloud-credential-operator-master-unit
pull-ci-openshift-cloud-credential-operator-master-verify
pull-ci-openshift-cloud-credential-operator-master-verify-deps

Details

In response to this:

/test e2e-azure-upgrade

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

joelddiaz · 2021-12-16T16:08:26Z

/test verify

akhil-rane · 2021-12-16T16:53:42Z

this looks good

joelddiaz · 2021-12-16T17:01:09Z

@akhil-rane more changes to pull in an updated build-machinery-go package (that was failing the 'make verify' step).
Let me know when this looks okay, and I'll squash to make the PR look more reasonable.

joelddiaz · 2021-12-16T19:50:42Z

/test e2e-azure-upgrade

akhil-rane · 2021-12-16T19:51:21Z

yeah we can squash here, all good

failing on installing yaml-patch binary otherwise

Stop trying to detect whether the creds in kube-system/azure-credentials are good enough for Minting new credentials. We now will only support Manual mode (where the annotator does nothing) and Passthrough mode where we will blindly annotate the Secret as 'passthrough'.

Update the Azure actuator to only support passthrough mode. Attempt to clean up previously created App Registrations / Service Principals, but treat failures to clean up as non-critical. In the event that we fail to clean up, set a new "OrphanedCloudResource" condition to document that we were unable to clean up. When successfully cleaning up, clear out the old AzureStatus fields. Add test cases covering the new OrphanedCloudResources condition.

joelddiaz · 2021-12-16T20:30:04Z

/hold waiting for #426

lwan-wanglin · 2021-12-17T09:25:13Z

@joelddiaz Here are the test cases for the card https://polarion.engineering.redhat.com/polarion/#/project/OSE/workitems?query=trello%3ACCO%5C-173&sorting=id, PTAL! thanks. It's a little hard to reproduce the OrphanedCloudResource condition.

And here I have some questions:

If we need to remove azure mint support from validation part on installer code ?https://github.com/openshift/installer/blob/02c23e64691b499d9cae9d534892e0424469b1b2/pkg/types/validation/installconfig.go#L610
If we need to doc things somewhere like "make sure .spec.credentialsMode of cloudcredential CR is not "Mint" before upgrade your cluster to 4.10" , otherwise the upgrade will fail, cco logs will show the below error

time="2021-12-17T07:31:13Z" level=error msg="error checking whether credentials already exists: invalid mode" controller=credreq cr=openshift-cloud-credential-operator/openshift-cloud-network-config-controller secret=openshift-cloud-network-config-controller/cloud-credentials
time="2021-12-17T07:31:17Z" level=info msg="syncing credentials request" controller=credreq cr=openshift-cloud-credential-operator/azure-disk-csi-driver-operator

joelddiaz · 2021-12-17T12:02:30Z

/test e2e-azure

joelddiaz · 2021-12-17T13:17:56Z

@joelddiaz Here are the test cases for the card https://polarion.engineering.redhat.com/polarion/#/project/OSE/workitems?query=trello%3ACCO%5C-173&sorting=id, PTAL! thanks. It's a little hard to reproduce the OrphanedCloudResource condition.

Yes, this could be tricky. Perhaps you install an old version of OpenShift with Mint mode, disable CCO (either by marking it unmanaged in CVO and scaling down CVO, or move CCO to Manual mode), then swap the credentials in kube-system/azure-credentials with ones that can't do Mint mode, then upgrade to the new OpenShift/CCO, re-enable CCO, and it should migrate to Passthrough but fail to clean up the previously minted App Registrations.

And here I have some questions:

1. If we need to remove azure mint support from validation part on installer code ?https://github.com/openshift/installer/blob/02c23e64691b499d9cae9d534892e0424469b1b2/pkg/types/validation/installconfig.go#L610

Yes, good point. I'll add a Jira under the installer team.

2. If we need to doc things somewhere like "make sure .spec.credentialsMode of cloudcredential CR is not "Mint" before upgrade your cluster to 4.10" , otherwise the upgrade will fail, cco logs will show the below error
time="2021-12-17T07:31:13Z" level=error msg="error checking whether credentials already exists: invalid mode" controller=credreq cr=openshift-cloud-credential-operator/openshift-cloud-network-config-controller secret=openshift-cloud-network-config-controller/cloud-credentials
time="2021-12-17T07:31:17Z" level=info msg="syncing credentials request" controller=credreq cr=openshift-cloud-credential-operator/azure-disk-csi-driver-operator

Yes, the docs will need some careful updates. cc @jeana-redhat

joelddiaz · 2021-12-17T13:45:36Z

@joelddiaz Here are the test cases for the card https://polarion.engineering.redhat.com/polarion/#/project/OSE/workitems?query=trello%3ACCO%5C-173&sorting=id, PTAL! thanks. It's a little hard to reproduce the OrphanedCloudResource condition.

Oh, and the test cases look good!

joelddiaz · 2021-12-17T13:46:57Z

/test e2e-azure

sferich888 · 2021-12-17T15:13:54Z

/label px-approved

joelddiaz · 2021-12-17T15:42:29Z

@lwan-wanglin @jeana-redhat anything else needed before getting the qe/docs approve labels?
@akhil-rane anything else needed before an lgtm?

joelddiaz · 2021-12-17T18:55:29Z

/test e2e-azure

akhil-rane · 2021-12-17T19:32:01Z

/lgtm

openshift-ci · 2021-12-17T19:32:33Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: akhil-rane, joelddiaz

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details

Needs approval from an approver in each of these files:

~~OWNERS~~ [akhil-rane,joelddiaz]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

joelddiaz · 2021-12-17T20:11:08Z

/unhold
#246 has merged

lwan-wanglin · 2021-12-20T05:05:21Z

@joelddiaz Thanks for your given steps , I can reproduce deleting SP failure case, and I filed a new test case to cover it . I missed one check(verify SPs are deleted from Azure cloud) on previous test cases, now I update this step to the test cases, but I meet a situation, I find cco just remove role assignment, but not deleting applications, is it expected? if not , please let me know, I will file a bug to track.

For this PR, I think we can add qe-approved label.
/label qe-approved

joelddiaz · 2021-12-20T14:12:55Z

I find cco just remove role assignment, but not deleting applications, is it expected? if not , please let me know, I will file a bug to track.

If you are seeing that the App Registration are not being deleted, that is not expected. Do open a BZ.

jeana-redhat · 2021-12-20T20:18:55Z

Will def be able to doc this, just need to know when each piece is landing. I know y'all will keep me updated on that :)
/label docs-approved

openshift-ci · 2021-12-20T22:14:21Z

@joelddiaz: all tests passed!

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

akhil-rane · 2022-02-09T17:40:49Z

/cherry-pick release-4.9

openshift-cherrypick-robot · 2022-02-09T17:41:32Z

@akhil-rane: #433 failed to apply on top of branch "release-4.9":

Applying: update azure-sdk-for-go
.git/rebase-apply/patch:381: new blank line at EOF.
+
.git/rebase-apply/patch:1651: new blank line at EOF.
+
.git/rebase-apply/patch:4266: new blank line at EOF.
+
warning: 3 lines add whitespace errors.
Using index info to reconstruct a base tree...
M	go.mod
M	go.sum
M	vendor/modules.txt
Falling back to patching base and 3-way merge...
Auto-merging vendor/modules.txt
CONFLICT (content): Merge conflict in vendor/modules.txt
Removing vendor/github.com/Azure/azure-sdk-for-go/NOTICE
Removing vendor/github.com/Azure/azure-sdk-for-go/LICENSE
Auto-merging go.sum
CONFLICT (content): Merge conflict in go.sum
Auto-merging go.mod
CONFLICT (content): Merge conflict in go.mod
error: Failed to merge in the changes.
hint: Use 'git am --show-current-patch=diff' to see the failed patch
Patch failed at 0001 update azure-sdk-for-go
When you have resolved this problem, run "git am --continue".
If you prefer to skip this patch, run "git am --skip" instead.
To restore the original branch and stop patching, run "git am --abort".

Details

In response to this:

/cherry-pick release-4.9

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

Azure passthrough

Joel Diaz added 2 commits December 9, 2021 14:29

update azure-sdk-for-go

9af2273

go get github.com/Azure/azure-sdk-for-go go mod vendor ; go mod tidy commit

openshift-ci bot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Dec 14, 2021

openshift-ci bot requested review from akhil-rane and suhanime December 14, 2021 21:04

openshift-ci bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Dec 14, 2021

joelddiaz force-pushed the azure-passthrough branch from b990f9d to 83349f0 Compare December 15, 2021 13:09

akhil-rane reviewed Dec 15, 2021

View reviewed changes

Joel Diaz added 5 commits December 16, 2021 14:57

update build-machinery-go vendor

219c420

failing on installing yaml-patch binary otherwise

vendor updates after removing secret annotation on Azure

ca76de8

update vendoring after removing mint mode support

32b6237

joelddiaz force-pushed the azure-passthrough branch from bf76773 to 32b6237 Compare December 16, 2021 19:58

openshift-ci bot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Dec 16, 2021

joelddiaz mentioned this pull request Dec 17, 2021

stop considering Mint mode as supported on Azure openshift/installer#5496

Closed

openshift-ci bot added the px-approved Signifies that Product Support has signed off on this PR label Dec 17, 2021

openshift-ci bot assigned akhil-rane Dec 17, 2021

openshift-ci bot added the lgtm Indicates that a PR is ready to be merged. label Dec 17, 2021

openshift-ci bot removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Dec 17, 2021

openshift-ci bot added the qe-approved Signifies that QE has signed off on this PR label Dec 20, 2021

openshift-ci bot added the docs-approved Signifies that Docs has signed off on this PR label Dec 20, 2021

joelddiaz mentioned this pull request Dec 20, 2021

Azure passthrough docs #435

Merged

openshift-merge-robot merged commit 9906c6a into openshift:master Dec 20, 2021

abutcher mentioned this pull request Feb 17, 2022

Bug 2055821: Remove Azure mint mode support as Active Directory Graph API will be sunset #453

Merged

5 tasks

This was referenced Dec 9, 2022

CFE-543 : Apply user defined tags to all Azure resources created by OpenShift openshift/enhancements#1191

Merged

azure: move away from ADAL and AD Graph #502

Merged

ming1013 pushed a commit to ming1013/cloud-credential-operator that referenced this pull request Dec 15, 2025

Merge pull request openshift#433 from joelddiaz/azure-passthrough

aadccf7

Azure passthrough

Azure passthrough #433

Azure passthrough #433

Uh oh!

Conversation

joelddiaz commented Dec 14, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

codecov bot commented Dec 14, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Uh oh!

Uh oh!

akhil-rane Dec 14, 2021

Choose a reason for hiding this comment

Uh oh!

joelddiaz Dec 15, 2021

Choose a reason for hiding this comment

Uh oh!

akhil-rane Dec 15, 2021

Choose a reason for hiding this comment

Uh oh!

joelddiaz Dec 15, 2021

Choose a reason for hiding this comment

Uh oh!

akhil-rane Dec 15, 2021

Choose a reason for hiding this comment

Uh oh!

joelddiaz Dec 15, 2021

Choose a reason for hiding this comment

Uh oh!

joelddiaz commented Dec 15, 2021

Uh oh!

joelddiaz commented Dec 15, 2021

Uh oh!

joelddiaz commented Dec 15, 2021

Uh oh!

joelddiaz commented Dec 16, 2021

Uh oh!

joelddiaz commented Dec 16, 2021

Uh oh!

openshift-ci bot commented Dec 16, 2021

Uh oh!

joelddiaz commented Dec 16, 2021

Uh oh!

openshift-ci bot commented Dec 16, 2021

Uh oh!

joelddiaz commented Dec 16, 2021

Uh oh!

akhil-rane commented Dec 16, 2021

Uh oh!

joelddiaz commented Dec 16, 2021

Uh oh!

joelddiaz commented Dec 16, 2021

Uh oh!

akhil-rane commented Dec 16, 2021

Uh oh!

joelddiaz commented Dec 16, 2021

Uh oh!

lwan-wanglin commented Dec 17, 2021

Uh oh!

joelddiaz commented Dec 17, 2021

Uh oh!

joelddiaz commented Dec 17, 2021

Uh oh!

joelddiaz commented Dec 17, 2021

Uh oh!

joelddiaz commented Dec 17, 2021

Uh oh!

sferich888 commented Dec 17, 2021

Uh oh!

joelddiaz commented Dec 17, 2021

Uh oh!

joelddiaz commented Dec 17, 2021

Uh oh!

akhil-rane commented Dec 17, 2021

Uh oh!

openshift-ci bot commented Dec 17, 2021

Uh oh!

joelddiaz commented Dec 17, 2021

joelddiaz commented Dec 14, 2021 •

edited

Loading

codecov bot commented Dec 14, 2021 •

edited

Loading

lwan-wanglin commented Dec 20, 2021 •

edited

Loading