Skip to content

Commit 3e8de86

Browse files
fntlnzfntlnz
authored
Merge pull request #40 from iovisor/32-pod-security-policy
feat(kubectl-trace): allow passing a service account to enable usage of pod security policies
2 parents 9e635f2 + b159809 commit 3e8de86

File tree

3 files changed

+145
-32
lines changed

3 files changed

+145
-32
lines changed

‎README.md

+116-7
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,12 @@
44

55
- [Installation](#installation)
66
- [Usage](#usage)
7+
* [Run a program from string literal](#run-a-program-from-string-literal)
8+
* [Run a program from file](#run-a-program-from-file)
9+
* [Run a program against a Pod](#run-a-program-against-a-pod)
710
* [Running against a Pod vs against a Node](#running-against-a-pod-vs-against-a-node)
11+
* [Using a custom service account](#using-a-custom-service-account)
12+
* [Executing in a cluster using Pod Security Policies](#executing-in-a-cluster-using-pod-security-policies)
813
* [More bpftrace programs](#more-bpftrace-programs)
914
- [Status of the project](#status-of-the-project)
1015
- [Contributing](#contributing)
@@ -29,7 +34,7 @@ This will download and compile `kubectl-trace` so that you can use it as a kubec
2934
You don't need to setup anything on your cluster before using it, please don't use it already
3035
on a production system, just because this isn't yet 100% ready.
3136

32-
**Run a program from string literal:**
37+
### Run a program from string literal
3338

3439
In this case we are running a program that probes a tracepoint
3540
on the node `ip-180-12-0-152.ec2.internal`.
@@ -39,15 +44,15 @@ kubectl trace run ip-180-12-0-152.ec2.internal -e "tracepoint:syscalls:sys_enter
3944
```
4045

4146

42-
**Run a program from file:**
47+
### Run a program from file
4348

4449
Here we run a program named `read.bt` against the node `ip-180-12-0-152.ec2.internal`
4550

4651
```
4752
kubectl trace run ip-180-12-0-152.ec2.internal -f read.bt
4853
```
4954

50-
**Run a program against a Pod**
55+
### Run a program against a Pod
5156

5257
![Screenshot showing the read.bt program for kubectl-trace](docs/img/pod.png)
5358

@@ -83,6 +88,114 @@ So, running against a pod **doesn't mean** that your bpftrace program will be co
8388
knowledge of the context of a container, in this case only the root process id is supported via the `$container_pid` variable.
8489

8590

91+
### Using a custom service account
92+
93+
By default `kubectl trace` will use the `default` service account in the target namespace (that is also `default`), to schedule the pods needed for your bpftrace program.
94+
95+
If you need to pass a service account you can use the `--serviceaccount` flag.
96+
97+
```bash
98+
kubectl trace run --serviceaccount=kubectltrace ip-180-12-0-152.ec2.internal -f read.bt
99+
```
100+
101+
### Executing in a cluster using Pod Security Policies
102+
103+
If your cluster has pod security policies you will need to make so that `kubectl trace` can
104+
use a service account that can run privileged containers.
105+
106+
That service account, then will need to be in a group that uses the proper privileged `PodSecurityPolicy`.
107+
108+
First, create the service account that you will use with `kubectl trace`,
109+
you can use a different namespace other than `default`, just remember to pass that namespace to the `run` command when you will use `kubectl trace`:
110+
111+
```yaml
112+
apiVersion: v1
113+
kind: ServiceAccount
114+
metadata:
115+
name: kubectltrace
116+
namespace: default
117+
```
118+
119+
Now that we have a `kubectltrace` service account let's create a Pod Security Policy:
120+
121+
```yaml
122+
apiVersion: policy/v1beta1
123+
kind: PodSecurityPolicy
124+
metadata:
125+
name: kubectltrace
126+
spec:
127+
fsGroup:
128+
rule: RunAsAny
129+
privileged: true
130+
runAsUser:
131+
rule: RunAsAny
132+
seLinux:
133+
rule: RunAsAny
134+
supplementalGroups:
135+
rule: RunAsAny
136+
volumes:
137+
- '*'
138+
allowedCapabilities:
139+
- '*'
140+
hostPID: true
141+
hostIPC: true
142+
hostNetwork: true
143+
hostPorts:
144+
- min: 1
145+
max: 65536
146+
```
147+
148+
Ok, this `PodSecurityPolicy` will allow users assigned to it to run privileged containers,
149+
`kubectl trace` needs that because of the extended privileges eBPF programs need to run with
150+
to trace your kernel and programs running in it.
151+
152+
Now with a `ClusterRoleBinding` you bind the `ClusterRole` with the `ServiceAccount`, so that
153+
they can work together with the `PodSecurityPolicy` we just created.
154+
155+
You can change the `namespace: default` here if you created the service account in a namespace other than `default`.
156+
157+
```yaml
158+
apiVersion: rbac.authorization.k8s.io/v1
159+
kind: ClusterRole
160+
metadata:
161+
name: kubectltrace-psp
162+
rules:
163+
- apiGroups:
164+
- policy
165+
resources:
166+
- podsecuritypolicies
167+
resourceNames:
168+
- kubectltrace
169+
verbs:
170+
- use
171+
---
172+
apiVersion: rbac.authorization.k8s.io/v1
173+
kind: ClusterRoleBinding
174+
metadata:
175+
name: kubectltrace-psp
176+
subjects:
177+
- kind: ServiceAccount
178+
name: kubectltrace
179+
namespace: default
180+
roleRef:
181+
apiGroup: rbac.authorization.k8s.io
182+
kind: ClusterRole
183+
name: kubectltrace-psp
184+
```
185+
186+
OK! Now that we are all set we can just run the program by specifying the service account
187+
we just created and it will use our pod security policy!
188+
189+
```bash
190+
kubectl trace run --serviceaccount=kubectltrace ip-180-12-0-152.ec2.internal -f read.bt
191+
```
192+
193+
If you used a different namespace other than default for your service account, you will want to specify the namespace too, like this:
194+
195+
```bash
196+
kubectl trace run --namespace=mynamespace --serviceaccount=kubectltrace ip-180-12-0-152.ec2.internal -f read.bt
197+
```
198+
86199
### More bpftrace programs
87200

88201
Need more programs? Look [here](https://github.com/iovisor/bpftrace/tree/master/tools).
@@ -119,10 +232,6 @@ kubectl trace run pod/<pod-name> -c <container> f read.bt
119232
120233
So I would say, the next thing is to run bpftrace programs at a pod scope other than at node scope.</strike>
121234
122-
**bpftrace work**
123-
124-
I also plan to contribute some IO functions to bpftrace to send data to a backend database like InfluxDB instead of only stdout
125-
because that would enable having things like graphs showing
126235
127236
## Contributing
128237

‎pkg/cmd/run.go

+18-15
Original file line numberDiff line numberDiff line change
@@ -54,13 +54,14 @@ type RunOptions struct {
5454
explicitNamespace bool
5555

5656
// Local to this command
57-
container string
58-
eval string
59-
program string
60-
resourceArg string
61-
attach bool
62-
isPod bool
63-
podUID string
57+
container string
58+
eval string
59+
program string
60+
resourceArg string
61+
attach bool
62+
isPod bool
63+
podUID string
64+
serviceAccount string
6465

6566
nodeName string
6667

@@ -103,6 +104,7 @@ func NewRunCommand(factory factory.Factory, streams genericclioptions.IOStreams)
103104
cmd.Flags().BoolVarP(&o.attach, "attach", "a", o.attach, "Wheter or not to attach to the trace program once it is created")
104105
cmd.Flags().StringVarP(&o.eval, "eval", "e", "", "Literal string to be evaluated as a bpftrace program")
105106
cmd.Flags().StringVarP(&o.program, "filename", "f", "", "File containing a bpftrace program")
107+
cmd.Flags().StringVar(&o.serviceAccount, "serviceaccount", "default", "Service account to use to set in the pod spec of the kubectl-trace job")
106108

107109
return cmd
108110
}
@@ -265,14 +267,15 @@ func (o *RunOptions) Run() error {
265267
}
266268

267269
tj := tracejob.TraceJob{
268-
Name: fmt.Sprintf("%s%s", meta.ObjectNamePrefix, string(juid)),
269-
Namespace: o.namespace,
270-
ID: juid,
271-
Hostname: o.nodeName,
272-
Program: o.program,
273-
PodUID: o.podUID,
274-
ContainerName: o.container,
275-
IsPod: o.isPod,
270+
Name: fmt.Sprintf("%s%s", meta.ObjectNamePrefix, string(juid)),
271+
Namespace: o.namespace,
272+
ServiceAccount: o.serviceAccount,
273+
ID: juid,
274+
Hostname: o.nodeName,
275+
Program: o.program,
276+
PodUID: o.podUID,
277+
ContainerName: o.container,
278+
IsPod: o.isPod,
276279
}
277280

278281
job, err := tc.CreateJob(tj)

‎pkg/tracejob/job.go

+11-10
Original file line numberDiff line numberDiff line change
@@ -24,14 +24,15 @@ type TraceJobClient struct {
2424
}
2525

2626
type TraceJob struct {
27-
Name string
28-
ID types.UID
29-
Namespace string
30-
Hostname string
31-
Program string
32-
PodUID string
33-
ContainerName string
34-
IsPod bool
27+
Name string
28+
ID types.UID
29+
Namespace string
30+
ServiceAccount string
31+
Hostname string
32+
Program string
33+
PodUID string
34+
ContainerName string
35+
IsPod bool
3536
}
3637

3738
// WithOutStream setup a file stream to output trace job operation information
@@ -101,7 +102,6 @@ func (t *TraceJobClient) findConfigMapsWithFilter(nf TraceJobFilter) ([]apiv1.Co
101102
}
102103

103104
func (t *TraceJobClient) GetJob(nf TraceJobFilter) ([]TraceJob, error) {
104-
105105
jl, err := t.findJobsWithFilter(nf)
106106
if err != nil {
107107
return nil, err
@@ -218,7 +218,8 @@ func (t *TraceJobClient) CreateJob(nj TraceJob) (*batchv1.Job, error) {
218218
Template: apiv1.PodTemplateSpec{
219219
ObjectMeta: commonMeta,
220220
Spec: apiv1.PodSpec{
221-
HostPID: true,
221+
HostPID: true,
222+
ServiceAccountName: nj.ServiceAccount,
222223
Volumes: []apiv1.Volume{
223224
apiv1.Volume{
224225
Name: "program",

0 commit comments

Comments
 (0)