Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow clusters to be linked without a gateway #11226

Merged
merged 8 commits into from
Aug 15, 2023
Merged

Conversation

mateiidavid
Copy link
Member

When a cluster has been installed without a gateway, it cannot be linked against, unless a load balancer service is used as an override. The service-mirror is tightly coupled with the notion of gateways. However, a gateway is not strictly necessary when clusters operate in a flat network.

As part of this change, linkerd multicluster link has been changed to allow clusters without gateways to be linked against. When a cluster does not have a gateway, all services must be exported in remote-discovery mode, otherwise routing wouldn't work.

In addition, when a cluster does not have a gateway, linking against it will not create a probe service (since there is nothing to probe). Lastly, a check has been modified to ignore checking replicated endpoints when a service is in remote-discovery mode (to avoid false positives).

When a cluster has been installed without a gateway, it cannot be linked
against, unless a load balancer service is used as an override. The
service-mirror is tightly coupled with the notion of gateways. However,
a gateway is not strictly necessary when clusters operate in a flat
network.

As part of this change, `linkerd multicluster link` has been changed to
allow clusters without gateways to be linked against. When a cluster
does not have a gateway, all services _must_ be exported in
`remote-discovery` mode, otherwise routing wouldn't work.

In addition, when a cluster does not have a gateway, linking against it
will not create a probe service (since there is nothing to probe).
Lastly, a check has been modified to ignore checking replicated
endpoints when a service is in remote-discovery mode (to avoid false
positives).

Signed-off-by: Matei David <[email protected]>
@mateiidavid mateiidavid requested a review from a team as a code owner August 9, 2023 15:04
Signed-off-by: Matei David <[email protected]>
Copy link
Member

@alpeb alpeb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

linkerd mc gateways is currently throwing an error currently for me, not sure if because I haven't totally set up things right locally. But most likely we'd like to filter out non-gateway remotes in there.

Also WDYT about having this flag be --gateway (defaults to true) instead to coincide with the linkerd mc install analog flag?

multicluster/cmd/service-mirror/main.go Show resolved Hide resolved
multicluster/cmd/link.go Outdated Show resolved Hide resolved
multicluster/cmd/link.go Outdated Show resolved Hide resolved
multicluster/cmd/link.go Outdated Show resolved Hide resolved
@mateiidavid
Copy link
Member Author

Tested this out end-to-end by branching out and merging the remote-disco changes:

# Overview of env
:; k get services --context=k3d-target
NAME         TYPE        CLUSTER-IP     EXTERNAL-IP   PORT(S)   AGE
kubernetes   ClusterIP   10.43.0.1      <none>        443/TCP   44h
podinfo      ClusterIP   10.43.24.197   <none>        80/TCP    44h

:; k get pods --context=k3d-target -n linkerd-multicluster
No resources found in linkerd-multicluster namespace.

:; k get services --context=k3d-source
NAME         TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)   AGE
kubernetes   ClusterIP   10.43.0.1       <none>        443/TCP   44h
nginx-svc    ClusterIP   10.43.174.249   <none>        80/TCP    44h
# Link & send traffic
# no gateway present so we link with --gateway=false
:; bin/linkerd --context=k3d-target mc link --cluster-name target --api-server-address="https://192.168.224.4:6443" --gateway=false| k apply -f -

:; k --context=k3d-source get pods -n linkerd-multicluster
NAME                                             READY   STATUS    RESTARTS   AGE
linkerd-gateway-658455dbbf-btfh2                 2/2     Running   0          22h
linkerd-service-mirror-target-7c6b789fb9-twqd8   2/2     Running   0          12s

:; k get services --context=k3d-source
NAME             TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)   AGE
kubernetes       ClusterIP   10.43.0.1       <none>        443/TCP   44h
nginx-svc        ClusterIP   10.43.174.249   <none>        80/TCP    44h
podinfo-target   ClusterIP   10.43.92.9      <none>        80/TCP    44s

:; k exec curl-c48c64d4-k2s8m -c curl -- curl -s podinfo-target
{
  "hostname": "podinfo-7c887c7bd5-tvkw9",
  "version": "5.0.1",
  "revision": "",
  "color": "#34577c",
  "logo": "https://raw.githubusercontent.com/stefanprodan/podinfo/gh-pages/cuddle_clap.gif",
  "message": "greetings from podinfo v5.0.1",
  "goos": "linux",
  "goarch": "amd64",
  "runtime": "go1.15.2",
  "num_goroutine": "6",
  "num_cpu": "16"
}                                                                                                                                                                                                                                                                                          
# Gateways & uninstall
:; bin/linkerd mc gateways
CLUSTER  ALIVE    NUM_SVC      LATENCY

:; bin/linkerd mc unlink --cluster-name target | k delete -f -
secret "cluster-credentials-target" deleted
link.multicluster.linkerd.io "target" deleted
clusterrole.rbac.authorization.k8s.io "linkerd-service-mirror-access-local-resources-target" deleted
clusterrolebinding.rbac.authorization.k8s.io "linkerd-service-mirror-access-local-resources-target" deleted
role.rbac.authorization.k8s.io "linkerd-service-mirror-read-remote-creds-target" deleted
rolebinding.rbac.authorization.k8s.io "linkerd-service-mirror-read-remote-creds-target" deleted
serviceaccount "linkerd-service-mirror-target" deleted
deployment.apps "linkerd-service-mirror-target" deleted
lease.coordination.k8s.io "service-mirror-write-target" deleted
service "podinfo-target" deleted
secret "cluster-credentials-target" deleted

# Service is gone
:; k --context=k3d-source get svc
NAME         TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)   AGE
kubernetes   ClusterIP   10.43.0.1       <none>        443/TCP   44h
nginx-svc    ClusterIP   10.43.174.249   <none>        80/TCP    44h

}

if opts.gatewayAddresses != "" && !opts.enableGateway {
return nil, fmt.Errorf("--gateway-addresses and --gateway=false are mutually exclusive, received selector: %s", opts.selector)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bad copy paste leaving the selector in the error message?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, my bad. On second thought, I left it at "are mutually exclusive", it should be easy to spot any overrides passed in without having to log them.

multicluster/cmd/link.go Outdated Show resolved Hide resolved
multicluster/cmd/gateways.go Outdated Show resolved Hide resolved
Copy link
Member

@alpeb alpeb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Delightful!

multicluster/service-mirror/probe_worker.go Outdated Show resolved Hide resolved
Signed-off-by: Matei David <[email protected]>
@adleong adleong merged commit 4f569ae into main Aug 15, 2023
37 checks passed
@adleong adleong deleted the matei/link-between-worlds branch August 15, 2023 19:12
alpeb added a commit that referenced this pull request Aug 16, 2023
This is a release candidate for stable-2.14.0; we encourage you to help trying
it out!

This edge release contains a number of improvements over the multi-cluster
features introduced in the last edge release supporting flat networks. It also
hardens the containers security stance by removing write access to the root
filesystem.

* Enhanced `linkerd multicluster link` to allow clusters to be linked without a
  gateway ([#11226])
* Added cluster store size gauge metric ([#11256])
* Disabled local traffic policy for remote discovery ([#11257])
* Fixed various innocuous multi-cluster warnings ([#11251], [#11246], [#11253])
* Set `readOnlyRootFilesystem: true` in all the containers, as they don't
  require write permissions ([#11221]; fixes [#11142]) (thanks @mikutas!)
@alpeb alpeb mentioned this pull request Aug 16, 2023
alpeb added a commit that referenced this pull request Aug 16, 2023
This is a release candidate for stable-2.14.0; we encourage you to help trying
it out!

This edge release contains a number of improvements over the multi-cluster
features introduced in the last edge release supporting flat networks. It also
hardens the containers security stance by removing write access to the root
filesystem.

* Enhanced `linkerd multicluster link` to allow clusters to be linked without a
  gateway ([#11226])
* Added cluster store size gauge metric ([#11256])
* Disabled local traffic policy for remote discovery ([#11257])
* Fixed various innocuous multi-cluster warnings ([#11251], [#11246], [#11253])
* Set `readOnlyRootFilesystem: true` in all the containers, as they don't
  require write permissions ([#11221]; fixes [#11142]) (thanks @mikutas!)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants