Skip to content

Commit

Permalink
Add OpenShift deployment support
Browse files Browse the repository at this point in the history
- Add new RBAC roles & clusterroles for all stages
- Add OCP specific artifacts
- Updated examples

Signed-off-by: Sebastian Jug <[email protected]>
  • Loading branch information
sjug committed Feb 4, 2021
1 parent 9dc1d06 commit 9bfcec8
Show file tree
Hide file tree
Showing 25 changed files with 639 additions and 17 deletions.
13 changes: 13 additions & 0 deletions deploy/role.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -96,6 +96,13 @@ rules:
- get
- list
- watch
- apiGroups:
- rbac.authorization.k8s.io
resources:
- roles
- rolebindings
verbs:
- '*'
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
Expand Down Expand Up @@ -270,3 +277,9 @@ rules:
- update
- patch
- delete
- apiGroups:
- security.openshift.io
resources:
- securitycontextconstraints
verbs:
- '*'
4 changes: 2 additions & 2 deletions example/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -71,12 +71,12 @@ tools to test RDMA and GPU-Direct RDMA traffic.
##### RDMA
__Pod1:__ Run `ib_write_bw` as server
```bash
# ib_write_bw -d <RDMA device e.g mlx5_0> -a -F --report_gbits -R -q 2
# ib_write_bw -d <RDMA device e.g mlx5_0> -a -F --report_gbits -R
```

__Pod2:__ Run `ib_write_bw` as client
```bash
# ib_write_bw -d <RDMA device e.g mlx5_0> -a -F --report_gbits -R -q 2 <Pod1 IP address>
# ib_write_bw -d <RDMA device e.g mlx5_0> -a -F --report_gbits -R <Pod1 IP address>
```

##### GPU-Direct RDMA
Expand Down
45 changes: 45 additions & 0 deletions example/crs/mellanox.com_v1alpha1_nicclusterpolicy_cr-ocp.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
# Copyright 2020 NVIDIA
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
apiVersion: mellanox.com/v1alpha1
kind: NicClusterPolicy
metadata:
name: nic-cluster-policy
spec:
ofedDriver:
image: mofed
repository: mellanox
version: 5.2-1.0.4.0
devicePlugin:
image: k8s-rdma-shared-dev-plugin
repository: mellanox
version: v1.1.0
# The config below directly propagates to k8s-rdma-shared-device-plugin configuration.
# Replace 'devices' with your (RDMA capable) netdevice name.
config: |
{
"configList": [
{
"resourceName": "hca_shared_devices_a",
"rdmaHcaMax": 1000,
"selectors": {
"ifNames": ["ens2f0"]
}
}
]
}
nvPeerDriver:
image: nv-peer-mem-driver
repository: mellanox
version: 1.0-9
gpuDriverSourcePath: /run/nvidia/driver
285 changes: 285 additions & 0 deletions example/deploy/role.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,285 @@
# Copyright 2020 NVIDIA
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
creationTimestamp: null
name: network-operator
namespace: mlnx-network-operator
rules:
- apiGroups:
- events.k8s.io
resources:
- events
verbs:
- create
- patch
- update
- apiGroups:
- ""
resources:
- pods
- pods/status
- services
- services/finalizers
- endpoints
- persistentvolumeclaims
- events
- configmaps
- secrets
verbs:
- create
- delete
- get
- list
- patch
- update
- watch
- apiGroups:
- apps
resources:
- deployments
- daemonsets
- replicasets
- statefulsets
verbs:
- create
- delete
- get
- list
- patch
- update
- watch
- apiGroups:
- monitoring.coreos.com
resources:
- servicemonitors
verbs:
- get
- create
- apiGroups:
- apps
resourceNames:
- network-operator
resources:
- deployments/finalizers
verbs:
- update
- apiGroups:
- mellanox.com
resources:
- '*'
verbs:
- create
- delete
- get
- list
- patch
- update
- watch
- apiGroups:
- ""
resources:
- nodes
verbs:
- get
- list
- watch
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
creationTimestamp: null
name: network-operator
namespace: mlnx-network-operator-resources
rules:
- apiGroups:
- ""
resources:
- pods
- services
- serviceaccounts
- services/finalizers
- endpoints
- persistentvolumeclaims
- events
- configmaps
- secrets
verbs:
- create
- delete
- get
- list
- patch
- update
- watch
- apiGroups:
- apps
resources:
- deployments
- daemonsets
- replicasets
- statefulsets
verbs:
- create
- delete
- get
- list
- patch
- update
- watch
- apiGroups:
- monitoring.coreos.com
resources:
- servicemonitors
verbs:
- get
- create
- apiGroups:
- apps
resourceNames:
- network-operator
resources:
- deployments/finalizers
verbs:
- update
- apiGroups:
- mellanox.com
resources:
- '*'
verbs:
- create
- delete
- get
- list
- patch
- update
- watch
- apiGroups:
- ""
resources:
- nodes
verbs:
- get
- list
- watch
- apiGroups:
- rbac.authorization.k8s.io
resources:
- roles
- rolebindings
verbs:
- '*'
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
creationTimestamp: null
name: network-operator
rules:
- apiGroups:
- mellanox.com
resources:
- '*'
verbs:
- create
- delete
- get
- list
- patch
- update
- watch
- apiGroups:
- ""
resources:
- nodes
verbs:
- get
- list
- watch
- apiGroups:
- apps
resources:
- daemonsets
verbs:
- get
- list
- watch
- apiGroups:
- apiextensions.k8s.io
resources:
- customresourcedefinitions
verbs:
- create
- delete
- get
- list
- patch
- update
- watch
- apiGroups:
- k8s.cni.cncf.io
resources:
- '*'
verbs:
- '*'
- apiGroups:
- rbac.authorization.k8s.io
resources:
- clusterrolebindings
- clusterroles
verbs:
- create
- delete
- get
- list
- patch
- update
- escalate
- watch
- apiGroups:
- ""
resources:
- pods
- pods/status
verbs:
- get
- update
- apiGroups:
- ""
- events.k8s.io
resources:
- events
verbs:
- create
- patch
- update
- apiGroups:
- whereabouts.cni.cncf.io
resources:
- ippools
verbs:
- get
- list
- watch
- create
- update
- patch
- delete
- apiGroups:
- security.openshift.io
resources:
- securitycontextconstraints
verbs:
- '*'
27 changes: 27 additions & 0 deletions example/networking/rdma-net-cr.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
apiVersion: k8s.cni.cncf.io/v1
kind: NetworkAttachmentDefinition
metadata:
annotations:
k8s.v1.cni.cncf.io/resourceName: rdma/hca_shared_devices_a
name: rdma-net
namespace: default
spec:
# Configuration below assumes 'ens2f0' as master netdevice for macvlan CNI,
# replace with (RDMA capable) netdevice of your choice.
# Note IPAM type is static, so IP configuration should be provided in pod spec of RDMA workloads.
config: |-
{
"cniVersion": "0.3.1",
"name": "rdma-net",
"plugins": [
{
"type": "macvlan",
"master": "ens2f0",
"mode": "bridge",
"mtu": 1500,
"ipam": {
"type": "static"
}
}
]
}
Loading

0 comments on commit 9bfcec8

Please sign in to comment.