Skip to content

add deploy mode for klusterlet #114

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
openshift-merge-robot merged 1 commit into from
Dec 15, 2021
Merged

add deploy mode for klusterlet #114

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 4 additions & 4 deletions Makefile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -46,8 +46,8 @@ update-scripts:
.PHONY: update-scripts
update: update-scripts update-codegen-crds

update-with-container: update-scripts update-codegen-crds

generate-with-container: Dockerfile.build
build-runtime-image: Dockerfile.build
$(RUNTIME) build -t $(RUNTIME_IMAGE_NAME) -f Dockerfile.build .
$(RUNTIME) run -ti --rm -v $(PWD):/go/src/open-cluster-management.io/api:z -w /go/src/open-cluster-management.io/api $(RUNTIME_IMAGE_NAME) make update-with-container

update-with-container: build-runtime-image
$(RUNTIME) run -ti --rm -v $(PWD):/go/src/open-cluster-management.io/api:z -w /go/src/open-cluster-management.io/api $(RUNTIME_IMAGE_NAME) make update-scripts update-codegen-crds
21 changes: 19 additions & 2 deletions operator/v1/0000_00_operator.open-cluster-management.io_klusterlets.crd.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ spec:
- name: v1
schema:
openAPIV3Schema:
description: Klusterlet represents controllers on the managed cluster. When configured, the Klusterlet requires a secret named of bootstrap-hub-kubeconfig in the same namespace to allow API requests to the hub for the registration protocol.
description: Klusterlet represents controllers to install the resources for a managed cluster. When configured, the Klusterlet requires a secret named bootstrap-hub-kubeconfig in the agent namespace to allow API requests to the hub for the registration protocol. In Detached mode, the Klusterlet requires an additional secret named external-managed-kubeconfig in the agent namespace to allow API requests to the managed cluster for resources installation.
type: object
properties:
apiVersion:
Expand All @@ -33,6 +33,21 @@ spec:
clusterName:
description: ClusterName is the name of the managed cluster to be created on hub. The Klusterlet agent generates a random name if it is not set, or discovers the appropriate cluster name on OpenShift.
type: string
deployOption:
description: DeployOption contains the options of deploying a klusterlet
type: object
default:
mode: Default
required:
- mode
properties:
mode:
description: "Mode can be Default or Detached. For cluster-manager: - In Default mode, the Hub is installed as a whole and all parts of Hub are deployed in the same cluster. - In Detached mode, only crd and configurations are installed on one cluster(defined as hub-cluster). Controllers run in another cluster (defined as management-cluster) and connect to the hub with the kubeconfig in secret of \"external-hub-kubeconfig\"(a kubeconfig of hub-cluster with cluster-admin permission). For klusterlet: - In Default mode, all klusterlet related resources are deployed on the managed cluster. - In Detached mode, only crd and configurations are installed on the spoke/managed cluster. Controllers run in another cluster (defined as management-cluster) and connect to the mangaged cluster with the kubeconfig in secret of \"external-managed-kubeconfig\"(a kubeconfig of managed-cluster with cluster-admin permission). The purpose of Detached mode is to give it more flexibility, for example we can install a hub on a cluster with no worker nodes, meanwhile running all deployments on another more powerful cluster. And we can also register a managed cluster to the hub that has some firewall rules preventing access from the managed cluster. \n Note: Do not modify the Mode field once it's applied."
type: string
default: Default
enum:
- Default
- Detached
externalServerURLs:
description: ExternalServerURLs represents the a list of apiserver urls and ca bundles that is accessible externally If it is set empty, managed cluster has no externally accessible url that hub cluster can visit.
type: array
Expand All @@ -48,7 +63,7 @@ spec:
description: URL is the url of apiserver endpoint of the managed cluster.
type: string
namespace:
description: Namespace is the namespace to deploy the agent. The namespace must have a prefix of "open-cluster-management-", and if it is not set, the namespace of "open-cluster-management-agent" is used to deploy agent.
description: 'Namespace is the namespace to deploy the agent. The namespace must have a prefix of "open-cluster-management-", and if it is not set, the namespace of "open-cluster-management-agent" is used to deploy agent. Note: in Detach mode, this field will be **ignored**, the agent will be deployed to the namespace named <klusterlet''s name>-open-cluster-management-agent'
type: string
nodePlacement:
description: NodePlacement enables explicit control over the scheduling of the deployed pods.
Expand Down Expand Up @@ -85,9 +100,11 @@ spec:
registrationImagePullSpec:
description: RegistrationImagePullSpec represents the desired image configuration of registration agent.
type: string
default: quay.io/open-cluster-management/registration
workImagePullSpec:
description: WorkImagePullSpec represents the desired image configuration of work agent.
type: string
default: quay.io/open-cluster-management/work
status:
description: Status represents the current status of Klusterlet agent.
type: object
Expand Down
4 changes: 3 additions & 1 deletion operator/v1/0000_01_operator.open-cluster-management.io_clustermanagers.crd.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -33,11 +33,13 @@ spec:
deployOption:
description: DeployOption contains the options of deploying a cluster-manager Default mode is used if DeployOption is not set.
type: object
default:
mode: Default
required:
- mode
properties:
mode:
description: Mode can be Default or Detached. In Default mode, the Hub is installed as a whole and all parts of Hub are deployed in the same cluster. In Detached mode, only crd and configurations are installed on one cluster(defined as hub-cluster). Controllers run in another cluster (defined as management-cluster) and connect to the hub with the kubeconfig in secret of "external-hub-kubeconfig"(a kubeconfig of hub-cluster with cluster-admin permission). The purpose of Detached mode is to give it more flexibility, for example we can install a hub on a cluster with no worker nodes, meanwhile running all deployments on another more powerful cluster. Do not modify the Mode field once it's applied.
description: "Mode can be Default or Detached. For cluster-manager: - In Default mode, the Hub is installed as a whole and all parts of Hub are deployed in the same cluster. - In Detached mode, only crd and configurations are installed on one cluster(defined as hub-cluster). Controllers run in another cluster (defined as management-cluster) and connect to the hub with the kubeconfig in secret of \"external-hub-kubeconfig\"(a kubeconfig of hub-cluster with cluster-admin permission). For klusterlet: - In Default mode, all klusterlet related resources are deployed on the managed cluster. - In Detached mode, only crd and configurations are installed on the spoke/managed cluster. Controllers run in another cluster (defined as management-cluster) and connect to the mangaged cluster with the kubeconfig in secret of \"external-managed-kubeconfig\"(a kubeconfig of managed-cluster with cluster-admin permission). The purpose of Detached mode is to give it more flexibility, for example we can install a hub on a cluster with no worker nodes, meanwhile running all deployments on another more powerful cluster. And we can also register a managed cluster to the hub that has some firewall rules preventing access from the managed cluster. \n Note: Do not modify the Mode field once it's applied."
type: string
default: Default
enum:
Expand Down
21 changes: 19 additions & 2 deletions operator/v1/0001_00_operator.open-cluster-management.io_klusterlets.crd.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ spec:
status: {}
validation:
openAPIV3Schema:
description: Klusterlet represents controllers on the managed cluster. When configured, the Klusterlet requires a secret named of bootstrap-hub-kubeconfig in the same namespace to allow API requests to the hub for the registration protocol.
description: Klusterlet represents controllers to install the resources for a managed cluster. When configured, the Klusterlet requires a secret named bootstrap-hub-kubeconfig in the agent namespace to allow API requests to the hub for the registration protocol. In Detached mode, the Klusterlet requires an additional secret named external-managed-kubeconfig in the agent namespace to allow API requests to the managed cluster for resources installation.
type: object
properties:
apiVersion:
Expand All @@ -33,6 +33,21 @@ spec:
clusterName:
description: ClusterName is the name of the managed cluster to be created on hub. The Klusterlet agent generates a random name if it is not set, or discovers the appropriate cluster name on OpenShift.
type: string
deployOption:
description: DeployOption contains the options of deploying a klusterlet
type: object
default:
mode: Default
required:
- mode
properties:
mode:
description: "Mode can be Default or Detached. For cluster-manager: - In Default mode, the Hub is installed as a whole and all parts of Hub are deployed in the same cluster. - In Detached mode, only crd and configurations are installed on one cluster(defined as hub-cluster). Controllers run in another cluster (defined as management-cluster) and connect to the hub with the kubeconfig in secret of \"external-hub-kubeconfig\"(a kubeconfig of hub-cluster with cluster-admin permission). For klusterlet: - In Default mode, all klusterlet related resources are deployed on the managed cluster. - In Detached mode, only crd and configurations are installed on the spoke/managed cluster. Controllers run in another cluster (defined as management-cluster) and connect to the mangaged cluster with the kubeconfig in secret of \"external-managed-kubeconfig\"(a kubeconfig of managed-cluster with cluster-admin permission). The purpose of Detached mode is to give it more flexibility, for example we can install a hub on a cluster with no worker nodes, meanwhile running all deployments on another more powerful cluster. And we can also register a managed cluster to the hub that has some firewall rules preventing access from the managed cluster. \n Note: Do not modify the Mode field once it's applied."
type: string
default: Default
enum:
- Default
- Detached
externalServerURLs:
description: ExternalServerURLs represents the a list of apiserver urls and ca bundles that is accessible externally If it is set empty, managed cluster has no externally accessible url that hub cluster can visit.
type: array
Expand All @@ -48,7 +63,7 @@ spec:
description: URL is the url of apiserver endpoint of the managed cluster.
type: string
namespace:
description: Namespace is the namespace to deploy the agent. The namespace must have a prefix of "open-cluster-management-", and if it is not set, the namespace of "open-cluster-management-agent" is used to deploy agent.
description: 'Namespace is the namespace to deploy the agent. The namespace must have a prefix of "open-cluster-management-", and if it is not set, the namespace of "open-cluster-management-agent" is used to deploy agent. Note: in Detach mode, this field will be **ignored**, the agent will be deployed to the namespace named <klusterlet''s name>-open-cluster-management-agent'
type: string
nodePlacement:
description: NodePlacement enables explicit control over the scheduling of the deployed pods.
Expand Down Expand Up @@ -85,9 +100,11 @@ spec:
registrationImagePullSpec:
description: RegistrationImagePullSpec represents the desired image configuration of registration agent.
type: string
default: quay.io/open-cluster-management/registration
workImagePullSpec:
description: WorkImagePullSpec represents the desired image configuration of work agent.
type: string
default: quay.io/open-cluster-management/work
status:
description: Status represents the current status of Klusterlet agent.
type: object
Expand Down
55 changes: 41 additions & 14 deletions operator/v1/types.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -50,29 +50,45 @@ type ClusterManagerSpec struct {
// DeployOption contains the options of deploying a cluster-manager
// Default mode is used if DeployOption is not set.
// +optional
// +kubebuilder:default={mode: Default}
DeployOption DeployOption `json:"deployOption,omitempty"`
}

type InstallMode string

const (
InstallModeDefault InstallMode = "Default"
InstallModeDetached InstallMode = "Detached"
)

// DeployOption describes the deploy options for cluster-manager or klusterlet
type DeployOption struct {
// Mode can be Default or Detached.
// In Default mode, the Hub is installed as a whole and all parts of Hub are deployed in the same cluster.
// In Detached mode, only crd and configurations are installed on one cluster(defined as hub-cluster). Controllers run in another cluster (defined as management-cluster) and connect to the hub with the kubeconfig in secret of "external-hub-kubeconfig"(a kubeconfig of hub-cluster with cluster-admin permission).
// For cluster-manager:
// - In Default mode, the Hub is installed as a whole and all parts of Hub are deployed in the same cluster.
// - In Detached mode, only crd and configurations are installed on one cluster(defined as hub-cluster). Controllers run in another cluster (defined as management-cluster) and connect to the hub with the kubeconfig in secret of "external-hub-kubeconfig"(a kubeconfig of hub-cluster with cluster-admin permission).
// For klusterlet:
// - In Default mode, all klusterlet related resources are deployed on the managed cluster.
// - In Detached mode, only crd and configurations are installed on the spoke/managed cluster. Controllers run in another cluster (defined as management-cluster) and connect to the mangaged cluster with the kubeconfig in secret of "external-managed-kubeconfig"(a kubeconfig of managed-cluster with cluster-admin permission).
Copy link
Copy Markdown

@ianzhang366 ianzhang366 Dec 14, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@zhujian7 how do we specify the configuration of management-cluster to controllers?

Copy link
Copy Markdown
Member Author

@zhujian7 zhujian7 Dec 15, 2021 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In fact for registration-agent and work-agent, the configuration here specifically refers to the RBAC configuration(in the managed cluster). Currently, we don't need a configuration resource(cm, in the management cluster) for the controllers(registration-agent and work-agent), all configs are reflected by the flags.

// The purpose of Detached mode is to give it more flexibility, for example we can install a hub on a cluster with no worker nodes, meanwhile running all deployments on another more powerful cluster.
// Do not modify the Mode field once it's applied.
// +kubebuilder:validation:Required
// And we can also register a managed cluster to the hub that has some firewall rules preventing access from the managed cluster.
//
// Note: Do not modify the Mode field once it's applied.
//
// +required
// +default=Default
// +kubebuilder:validation:Required
// +kubebuilder:default=Default
// +kubebuilder:validation:Enum=Default;Detached
Mode InstallMode `json:"mode"`
}

// InstallMode represents the mode of deploy cluster-manager or klusterlet
type InstallMode string

const (
// InstallModeDefault is the default deploy mode.
// The cluster-manager will be deployed in the hub-cluster, the klusterlet will be deployed in the managed-cluster.
InstallModeDefault InstallMode = "Default"

// InstallModeDetached means deploying components outside.
// The cluster-manager will be deployed outside of the hub-cluster, the klusterlet will be deployed outside of the managed-cluster.
InstallModeDetached InstallMode = "Detached"
)

// ClusterManagerStatus represents the current status of the registration and work distribution controllers running on the hub.
type ClusterManagerStatus struct {
// ObservedGeneration is the last generation change you've dealt with
Expand Down Expand Up @@ -168,9 +184,11 @@ type ClusterManagerList struct {
// +kubebuilder:subresource:status
// +kubebuilder:resource:scope=Cluster

// Klusterlet represents controllers on the managed cluster. When configured,
// the Klusterlet requires a secret named of bootstrap-hub-kubeconfig in the
// same namespace to allow API requests to the hub for the registration protocol.
// Klusterlet represents controllers to install the resources for a managed cluster.
// When configured, the Klusterlet requires a secret named bootstrap-hub-kubeconfig in the
// agent namespace to allow API requests to the hub for the registration protocol.
// In Detached mode, the Klusterlet requires an additional secret named external-managed-kubeconfig
// in the agent namespace to allow API requests to the managed cluster for resources installation.
type Klusterlet struct {
metav1.TypeMeta `json:",inline"`
metav1.ObjectMeta `json:"metadata,omitempty"`
Expand All @@ -187,15 +205,19 @@ type KlusterletSpec struct {
// Namespace is the namespace to deploy the agent.
// The namespace must have a prefix of "open-cluster-management-", and if it is not set,
// the namespace of "open-cluster-management-agent" is used to deploy agent.
// Note: in Detach mode, this field will be **ignored**, the agent will be deployed to the
// namespace named <klusterlet's name>-open-cluster-management-agent
// +optional
Namespace string `json:"namespace,omitempty"`

// RegistrationImagePullSpec represents the desired image configuration of registration agent.
// +required
// +kubebuilder:default=quay.io/open-cluster-management/registration
RegistrationImagePullSpec string `json:"registrationImagePullSpec"`

// WorkImagePullSpec represents the desired image configuration of work agent.
// +required
// +kubebuilder:default=quay.io/open-cluster-management/work
WorkImagePullSpec string `json:"workImagePullSpec,omitempty"`

// ClusterName is the name of the managed cluster to be created on hub.
Expand All @@ -211,6 +233,11 @@ type KlusterletSpec struct {
// NodePlacement enables explicit control over the scheduling of the deployed pods.
// +optional
NodePlacement NodePlacement `json:"nodePlacement,omitempty"`

// DeployOption contains the options of deploying a klusterlet
// +optional
// +kubebuilder:default={mode: Default}
DeployOption DeployOption `json:"deployOption,omitempty"`
}

// ServerURL represents the apiserver url and ca bundle that is accessible externally
Expand Down
1 change: 1 addition & 0 deletions operator/v1/zz_generated.deepcopy.go
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading