KEP-1645: Add specification for multi-network scenario

[aattuluri]

• Add the necessary changes to support MCS spec for multi-network scenario.

Refer to this draft for more details.

[k8s-ci-robot]

Thanks for your pull request. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

📝 Please follow instructions at https://git.k8s.io/community/CLA.md#the-contributor-license-agreement to sign the CLA.

It may take a couple minutes for the CLA signature to be fully registered; after that, please reply here with a new comment and we'll verify. Thanks.

---

• If you've already signed a CLA, it's possible we don't have your GitHub username or you're using a different email address. Check your existing CLA data and verify that your email is set on your git commits.
• If you signed the CLA as a corporation, please sign in with your organization's credentials at https://identity.linuxfoundation.org/projects/cncf to be authorized.
• If you have done the above and are still having issues with the CLA being reported as unsigned, please log a ticket with the Linux Foundation Helpdesk: https://support.linuxfoundation.org/
• Should you encounter any issues with the Linux Foundation Helpdesk, send a message to the backup e-mail support address at: [email protected]

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

[k8s-ci-robot]

Welcome @aattuluri!

It looks like this is your first PR to kubernetes/enhancements 🎉. Please refer to our pull request process documentation to help your PR have a smooth ride to approval.

You will be prompted by a bot to use commands during the review process. Do not be afraid to follow the prompts! It is okay to experiment. Here is the bot commands documentation.

You can also check if kubernetes/enhancements has its own contribution guidelines.

You may want to refer to our testing guide if you run into trouble with your tests not passing.

If you are having difficulty getting your pull request seen, please follow the recommended escalation practices. Also, for tips and tricks in the contribution process you may want to read the Kubernetes contributor cheat sheet. We want to make sure your contribution gets all the attention it needs!

Thank you, and welcome to Kubernetes. 😃

[k8s-ci-robot]

Hi @aattuluri. Thanks for your PR.

I'm waiting for a kubernetes member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[aattuluri]

/assign @lauralorenz

[aattuluri]

/assign aattuluri

[palnabarun]

/ok-to-test
/uncc

[linux-foundation-easycla]

The committers are authorized under a signed CLA.

• ✅ aattuluri (a2fa9f4, 172dd80, 79db429)

[bowei]

This makes an implicit assumption that egress network connectivity is mapped via identifiers to figure if given traffic needs to go through a proxy or is a direct connection. Has it been considered whether or not to express that specifically (e.g. the API is focused on "this traffic is proxied" vs saying "we have labeled our networks and this labeling implicitly implies that there is some proxying". The second statement assumes more about the environment and imposes a specific naming, labeling mechanism.

[aattuluri]

Yes, will reword this to more generic after the right name is chosen in favor of network

[thockin]

Coming back to this - do we really need this to enable multi-net MCS? It seems like this may be prematurely standardizing. In some cases MCS implementations won't need this - they will have (or have access to) enough information via other channels. Are there actually implementations that are likely to need such metadata AND which can't get it some other way (e.g. interrogating the cloud provider)?

How will such implementations learn the addresses of the ingress and egress gateways?

[aattuluri]

Coming back to this - do we really need this to enable multi-net MCS? It seems like this may be prematurely standardizing. In some cases MCS implementations won't need this - they will have (or have access to) enough information via other channels. Are there actually implementations that are likely to need such metadata AND which can't get it some other way (e.g. interrogating the cloud provider)?

How will such implementations learn the addresses of the ingress and egress gateways?

Right now the implementations use different mechanisms (for example Istio uses `topology.istio.io/network annotation on a control plane namespace). This spec is an attempt to standardize how a network can be determined for MCS purposes (that can have other use cases as well) before there are several different implementations of the same and then later we standardize this with a spec that would require these implementations to be rewritten or render them non-compliant.

[steeling]

Note that the endpoints of the EndpointSlice for a cluster on discrete network may only be representative of the pods backing the multi-cluster service

How does the MCS controller know how to populate the endpoint slices with the appropriate endpoints? Does it use the ClusterProperty's network value? Or is that a network name that can signal to the MCS controller to use some mapping of endpoints that exists $somewhere?

Some common API here may be useful, ie: an annotation on either the ServiceImport, or the ServiceExport would allow for different services to leverage different services (may be for some QOS isolation reasons).

The annotation could target an Ingress endpoint, public hostname, IP address, or in-cluster service.

proxy.mcs.k8s.io: <prox-url>

I understand that the MCS controller implementation is not part of the multi cluster API, but it is at the very least useful to think of a full end to end solution to ensure the API is robust.

[steeling]

Hi all, just getting up to speed on this KEP so apologies for any misunderstandings, and thanks for all the work put in so far!

Is my understanding correct that the gist of this KEP is identifying whether or not specific clusters are part of the same network, but does not solve the issue of how to route to clusters on a different network?

[aattuluri]

Hi all, just getting up to speed on this KEP so apologies for any misunderstandings, and thanks for all the work put in so far!

Is my understanding correct that the gist of this KEP is identifying whether or not specific clusters are part of the same network, but does not solve the issue of how to route to clusters on a different network?

Sorry for the delay, yes, this one is to identify if a cluster is on a different network. The routing is left to the implementation.

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: aattuluri, thockin
Once this PR has been reviewed and has the lgtm label, please assign pmorie for approval. For more information see the Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

• keps/sig-multicluster/OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[k8s-ci-robot]

PR needs rebase.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[k8s-triage-robot]

The Kubernetes project currently lacks enough contributors to adequately respond to all PRs.

This bot triages PRs according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the PR is closed

You can:

• Mark this PR as fresh with /remove-lifecycle stale
• Close this PR with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

[k8s-triage-robot]

The Kubernetes project currently lacks enough active contributors to adequately respond to all PRs.

This bot triages PRs according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the PR is closed

You can:

• Mark this PR as fresh with /remove-lifecycle rotten
• Close this PR with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle rotten

[k8s-triage-robot]

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages PRs according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the PR is closed

You can:

• Reopen this PR with /reopen
• Mark this PR as fresh with /remove-lifecycle rotten
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/close

[k8s-ci-robot]

@k8s-triage-robot: Closed this PR.

In response to this:

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages PRs according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the PR is closed

You can:

• Reopen this PR with /reopen
• Mark this PR as fresh with /remove-lifecycle rotten
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.