CLOUDP-350375: MongoDBMultiCluster: Add annotation to disable reconciliation

[m1kola]

Summary

The new mongodb.com/v1.disable-reconciliation annotation allows disabling reconciliation for MongoDBMultiCluster objects.

This will be used later when migrating multi-cluster replica sets from MongoDBMultiCluster to MongoDB.

Proof of Work

1. Add mongodb.com/v1.disableReconciliation to any MongoDBMultiCluster
2. Run kubectl wait mongodbmulticluster NAME -n NAMESPACE --for=jsonpath='{.status.phase}'=Disabled.
3. Check that the kubectl wait command prints mongodbmulticluster.mongodb.com/NAME condition met.

Checklist

• Have you linked a jira ticket and/or is the ticket in the title?
• Have you checked whether your jira ticket required DOCSP changes?
• Have you added changelog file?
  • use skip-changelog label if not needed
  • refer to Changelog files and Release Notes section in CONTRIBUTING.md for more details

[m1kola]

It will be something our users will be using during migration, but as part of this PR I"m not going to document the annotation/add a release note. It will be included in the migration guide once we release a version which is ready for migration. Until then - let's leave it undocumented.

[m1kola]

Had to define a constant because linter was not happy.

[viveksinghggits]

PR looks good to me. Just left a small comment/question.

[viveksinghggits]

q: It looks like disabled workflow was introduced to notify if a subresource is enabled or disabled. would it be a good idea to use the same to notify that a resource is deprecated?

I was also wondering if we should call it disabled or deprecated?

[m1kola]

@viveksinghggits I found this:

mongodb-kubernetes/api/v1/status/phase.go

Lines 20 to 21 in 219a3df

	// PhaseDisabled means that the resource is not enabled
	PhaseDisabled Phase = "Disabled"

I believe these statuses are generic. It just happened so that we only use it currently to report that a backup is disabled. But I think there is no reason not to use it for anything else. I'm happy to update this comment to reflect this:

mongodb-kubernetes/controllers/operator/workflow/disabled.go

Lines 7 to 10 in 219a3df

	// disabledStatus indicates that the subresource is not enabled
	type disabledStatus struct {
	*okStatus
	}

I was also wondering if we should call it disabled or deprecated?

We might be adding a deprecation message somewhere on the resource. But it will likely going to be a permanent message.

For the purpose of migration we need a status which can be used to programmatically check that the resource is not being reconciled.

[viveksinghggits]

I think just updating the comment should be ok. Just wanted to bring the attention to that comment, not a big issue.