Skip to content

Commit 01967d9

Browse files
jstueverjstuever
committed
Add doc for PrivateLink Controller
1 parent 2d25383 commit 01967d9

File tree

2 files changed

+277
-0
lines changed

2 files changed

+277
-0
lines changed

‎README.md

+1
Original file line numberDiff line numberDiff line change
@@ -25,6 +25,7 @@ In the future Hive will support more cloud providers.
2525
* [Using Hive](./docs/using-hive.md)
2626
* [Cluster Hibernation](./docs/hibernating-clusters.md)
2727
* [Cluster Pools](./docs/clusterpools.md)
28+
* [PrivateLink](./docs/privatelink.md)
2829
* [Hiveutil CLI](./docs/hiveutil.md)
2930
* [Scaling Hive](./docs/scaling-hive.md)
3031
* [Developing Hive](./docs/developing.md)

‎docs/privatelink.md

+276
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,276 @@
1+
# PrivateLink Controller
2+
3+
## Overview
4+
5+
Customers often do not want the services of their OpenShift cluster to be
6+
publicly available on the Internet. The OpenShift Installer allows creating
7+
clusters that have their services published only on the internal network by
8+
setting `publish: Internal` in the install-config.yaml.
9+
10+
Since Hive is usually running outside the network where the cluster is being
11+
deployed, special consideration must be taken to ensure it has access to the
12+
cluster's API server. While there are multiple ways to create publicly
13+
accessible endpoints that are limited such that only Hive can access them,
14+
these are generally not acceptable by security focused customers.
15+
16+
Cloud providers usually have a method that allows accessing private services
17+
from another network using the provider's internal networking instead of going
18+
over the public Internet. The PrivateLink controller leverages these features
19+
to enable Hive to access the APIs of these private clusters without exposing them
20+
to the Internet.
21+
22+
## Architecture
23+
24+
The PrivateLink controller has been designed to enable a hive cluster to exist
25+
on one platform while it deploys private clusters on any of the supported
26+
platforms. It accomplishes this by breaking the functionality into two
27+
segments (actuators):
28+
29+
### Hub Actuator
30+
31+
This actuator creates the necessary resources on the cloud provider that Hive is
32+
running on. Its job is to provide a DNS record, and any other necessary resources,
33+
to enable the Hive cluster to resolve the cluster's API to the Endpoint created
34+
by the Link Actuator.
35+
36+
Supported Platforms: [AWS](#aws-hub-actuator)
37+
38+
### Link Actuator
39+
40+
This actuator creates the necessary resources on the cloud provider that the
41+
private cluster is being provisioned into. Its job is to create an Endpoint,
42+
and any other necessary resources, to enable the Hive cluster to connect to the
43+
cluster's API without enabling public access.
44+
45+
Supported Platforms: [GCP](#gcp-link-actuator)
46+
47+
Note: Support for AWS private clusters is still managed by the original [AWS
48+
PrivateLink Controller](awsprivatelink.md). The goal is to eventually merge its
49+
functionality into this PrivateLink Controller.
50+
51+
## AWS Hub Actuator Configuration
52+
53+
This actuator is used when the Hive cluster exists in the AWS platform. It will
54+
ensure a new DNS zone exists for the private cluster and create a DNS record
55+
which resolves to the Endpoint created by the Link Actuator. This zone will
56+
then be attached to the associatedVPCs configured in the hiveconfig.
57+
58+
Note: The AWS Hub Actuator currently uses the same HiveConfig parameters as the
59+
original AWS PrivateLink controller. This is to avoid configuration drift
60+
between the two controllers until they can be merged together.
61+
62+
### Configure the primary AWS credentialsSecretRef
63+
64+
The Hub Actuator needs an AWS Service Account in order to manage the DNS
65+
Zone and records. The credentials for this service account should be stored in
66+
a secret in the Hive namespace. This secret can then be configured via the
67+
`credentialsSecretRef` parameter in the hiveconfig.
68+
69+
```yaml
70+
## hiveconfig
71+
spec:
72+
awsPrivateLink:
73+
## credentialsSecretRef points to a secret with permissions to create
74+
## resources in account where the inventory of VPCs exist.
75+
credentialsSecretRef:
76+
name: < hub-account-credentials-secret-name >
77+
```
78+
79+
### Configure the AWS associatedVPCs
80+
81+
The Hub Actuator needs to associate the DNS Zone with the Hive cluster's VPC in
82+
order for Hive to be able to resolve the API. This can be configured via the
83+
`associatedVPCs` parameter in the hiveconfig. Each VPC in this list will be
84+
associated with the DNS Zone. By default, they are expected to be in the same
85+
account as configured by the primary service account above. This can be
86+
overridden using the corresponding `credentialsSecretRef` fields.
87+
88+
When creating the DNS Zone, the Hub Actuator must attach it to at least one
89+
network that exists in the same account as the DNS Zone. For AWS private
90+
clusters, this is the network the endpoint is created in. However, this isn't
91+
possible for other platforms. The Actuator needs another way to choose which
92+
network to use. For this reason, this list should include at least one VPC
93+
where the `credentialsSecretRef` parameter is either empty or has a value equal
94+
to that of `spec.awsPrivateLink.credentialsSecretRef`.
95+
96+
```yaml
97+
## hiveconfig
98+
spec:
99+
awsPrivateLink:
100+
## this is a list of VPC where various Hive clusters exists.
101+
associatedVPCs:
102+
- region: region-hive1
103+
vpcID: vpc-hive1
104+
- region: region-hive2
105+
vpcID: vpc-hive2
106+
credentialsSecretRef:
107+
name: < optional credentials that have access to account where Hive2 VPC exists>
108+
```
109+
110+
## GCP Link Actuator Configuration
111+
112+
This actuator is used when the clusterDeployment specifies the GCP platform and
113+
has `spec.platform.gcp.privateServiceConnect.enabled` set to true. It will
114+
ensure an Endpoint and associated Service Attachment exist to enable access to
115+
the cluster's API.
116+
117+
### Configure the primary GCP credentialsSecretRef
118+
119+
The Link Actuator needs a GCP Service Account in order to manage the Endpoint.
120+
The credentials for this service account should be stored in a secret in the
121+
Hive namespace. This secret can then be configured via the
122+
`credentialsSecretRef` parameter in the hiveconfig.
123+
124+
```yaml
125+
## hiveconfig
126+
spec:
127+
privateLink:
128+
gcp:
129+
## credentialsSecretRef points to a secret with permissions to create
130+
## resources in account where the inventory of VPCs exist.
131+
credentialsSecretRef:
132+
name: < link-account-credentials-secret-name >
133+
```
134+
135+
This service account requires the following permissions:
136+
137+
- Endpoint Address
138+
- compute.addresses.create
139+
- compute.addresses.createInternal
140+
- compute.addresses.delete
141+
- compute.addresses.deleteInternal
142+
- compute.addresses.get
143+
- compute.addresses.list
144+
- compute.instances.update
145+
- compute.regionOperations.get
146+
- compute.subnetworks.get
147+
- Endpoint
148+
- compute.addresses.use
149+
- compute.forwardingRules.create
150+
- compute.forwardingRules.delete
151+
- compute.forwardingRules.get
152+
- compute.forwardingRules.pscCreate
153+
- compute.forwardingRules.pscDelete
154+
- compute.networks.use
155+
- compute.regionOperations.get
156+
- compute.subnetworks.use
157+
- servicedirectory.namespaces.create
158+
- servicedirectory.services.create
159+
- servicedirectory.services.delete
160+
161+
162+
### Configure the GCP endpointVPCInventory
163+
164+
The Link Actuator needs to know where to create the cluster's Endpoint. This
165+
can be configured in the `spec.privateLink.gcp.endpointVPCInventory` parameter
166+
in the hiveconfig. When creating the endpoint, the actuator will choose the
167+
subnet from this list that is in the same region and has the least number of
168+
addresses provisioned in the cloud provider.
169+
170+
1. Create a network for the PrivateLink Controller to use when creating
171+
endpoints.
172+
173+
1. Create one or more subnets in the network for each region that private
174+
clusters will be created in.
175+
176+
1. Make sure all the Hive environments (Hive VPCs) have network reachability to
177+
these subnets using peering, transit gateways, VPNs, etc.
178+
179+
1. Update the HiveConfig to enable these subnets.
180+
181+
```yaml
182+
## hiveconfig
183+
spec:
184+
privateLink:
185+
gcp:
186+
endpointVPCInventory:
187+
- network: network-1
188+
subnets:
189+
- region: us-east1
190+
subnet: subnet1
191+
- region: us-east2
192+
subnet: subnet2
193+
```
194+
195+
## Deploying a PrivateLink cluster on AWS
196+
197+
Support for AWS private clusters is still managed by the original [AWS
198+
PrivateLink Controller](awsprivatelink.md). The goal is to eventually merge its
199+
functionality into this PrivateLink Controller.
200+
201+
## Deploying a PrivateLink cluster on GCP
202+
203+
Once Hive has been configured to support PrivateLink GCP clusters, you can
204+
deploy a cluster by setting `privateServiceConnect.enabled` to true on the
205+
clusterDeployment. This is only supported in regions where Hive has been
206+
configured with at least one subnet in [endpointVPCInventory](#configure-the-gcp-endpointvpcinventory)
207+
with the same region. The Link Actuator uses the service account specified in
208+
the spec.platform.gcp.credentialsSecretRef parameter of the clusterDeployment
209+
when managing the Service Attachment, Subnet, and Firewall resources.
210+
211+
212+
```yaml
213+
## clusterDeployment
214+
spec:
215+
platform:
216+
gcp:
217+
privateServiceConnect:
218+
enabled: true
219+
```
220+
221+
By default, the Link Actuator will create a subnet and associated firewall in
222+
the clusters's VPC to be used when creating the Service Attachment. The default
223+
CIDR of the subnet can be overridden with the `serviceAttachment.subnet.cidr`
224+
parameter
225+
226+
```yaml
227+
## clusterDeployment
228+
spec:
229+
platform:
230+
gcp:
231+
privateServiceConnect:
232+
serviceAttachment:
233+
subnet:
234+
cidr: 192.168.1.0/30
235+
```
236+
237+
Alternatively, a pre-existing subnet can be specified and the Link Actuator
238+
will use it instead of creating a new one. This subnet must have a purpose of
239+
`PRIVATE_SERVICE_CONNECT` and have routing and firewall rule(s) enabling access
240+
to the api-internal load balancer. Specifying an existing subnet is required
241+
when using BYO VPC. The host project name must also be specified when using
242+
Shared VPC.
243+
244+
```yaml
245+
## clusterDeployment
246+
spec:
247+
platform:
248+
gcp:
249+
privateServiceConnect:
250+
serviceAttachment:
251+
subnet:
252+
existing:
253+
name: psc-subnet-name
254+
project: shared-vpc-host-project-name
255+
```
256+
The clusterDeployment service account requires the following permissions in addition to those required to install a cluster:
257+
258+
- Firewall
259+
- compute.firewalls.create
260+
- compute.firewalls.delete
261+
- compute.firewalls.get
262+
- compute.networks.updatePolicy
263+
- compute.regionOperations.get
264+
- Service Attachment
265+
- compute.forwardingRules.get
266+
- compute.regionOperations.get
267+
- compute.serviceAttachments.create
268+
- compute.serviceAttachments.delete
269+
- compute.serviceAttachments.get
270+
- compute.subnetworks.get
271+
- Subnet
272+
- compute.networks.get
273+
- compute.regionOperations.get
274+
- compute.subnetworks.create
275+
- compute.subnetworks.delete
276+
- compute.subnetworks.get

0 commit comments

Comments
 (0)