Kubernetes: add instance-wide default and enforced Security Context

docs/docs/30-administration/10-configuration/11-backends/20-kubernetes.md

@@ -359,6 +359,26 @@ backend_options:
 The feature requires Kubernetes v1.30 or above.
 :::
 
+You can set `allowPrivilegeEscalation` to `false` to prevent a container from gaining more privileges than its parent process.
+
+```yaml
+backend_options:
+  kubernetes:
+    securityContext:
+      allowPrivilegeEscalation: false
+```
+
+You can also drop [Linux capabilities](https://man7.org/linux/man-pages/man7/capabilities.7.html) from a container. Adding capabilities is not allowed.
+
+```yaml
+backend_options:
+  kubernetes:
+    securityContext:
+      capabilities:
+        drop:
+          - ALL
+```
+
 ### Annotations and labels
 
 You can specify arbitrary [annotations](https://kubernetes.io/docs/concepts/overview/working-with-objects/annotations/) and [labels](https://kubernetes.io/docs/concepts/overview/working-with-objects/labels/) to be set on the Pod definition for a given workflow step using the following configuration:
@@ -428,6 +448,82 @@ steps:
 
 If ports are defined on a service, then woodpecker will create a normal service for the pod, which use hosts override using the services cluster IP.
 
+## Running in an unprivileged namespace
+
+Woodpecker by default requires the namespace where workflow pods run to be privileged.
+
+However, it's possible to configure the agent in such a way that allows workflow pods to run in an unprivileged namespace.
+This comes with some drawbacks and it's the reason why its disabled by default.
+
+Major drawbacks are:
+
+- You won't be able to use commands like `apk add` or `apt install` in most images.
+  The easiest way to workaround this is by building your own image with the tools you require already prebundled in it.
+  This also have the advantage that workflows will run faster, since it won't need to fetch packages during each run.
+- If you need to build Docker/OCI images, you'll need to use a rootless builder like Buildah or BuildKit in rootless mode.
+- The default clone step currently doesn't work in unprivileged namespaces, however its possible to define your own clone step that can run unprivileged. More details below.
+
+Please note, this guide assumes you already have a working woodpecker instance running in your kubernetes cluster in a privileged namespace.
+
+### Setting security context environment variables
+
+Depending on how you installed the woodpecker server and agent, this step may be different.
+To make this guide as generic as possible, we will only list the environment variables that need to be updated.
+
+On your woodpecker-agent Deployment/StatefulSet, set this environment variables:
+
+```sh
+WOODPECKER_BACKEND_K8S_DEFAULT_SECCTX='{"runAsUser":1000,"runAsGroup":1000,"fsGroup":1000,"fsGroupChangePolicy": "OnRootMismatch"}'
+WOODPECKER_BACKEND_K8S_ENFORCED_SECCTX='{"privileged":false,"runAsNonRoot":true,"allowPrivilegeEscalation":false,"seccompProfile": {"type": "RuntimeDefault"}, "capabilities": {"drop": ["ALL"]}}'
+```
+
+Wait until the update rolls out.
+
+### Setting up the namespace
+
+Make the namespace where woodpecker worker pods run restricted, if you haven't done it yet:
+
+```sh
+kubectl label namespace woodpecker \
+  pod-security.kubernetes.io/enforce=restricted \
+  pod-security.kubernetes.io/audit=restricted \
+  pod-security.kubernetes.io/warn=restricted \
+  --overwrite
+```
+
+Please note here we use the namespace name `woodpecker`, but you should replace it with the actual namespace name you're using for woodpecker worker pods. If you have set `WOODPECKER_BACKEND_K8S_NAMESPACE`, then this is the namespace you should update. If you haven't, worker pods will run by default in the same namespace as the `woodpecker-agent`.
+
+### Unprivileged clone step
+
+Currently, the default git clone step depends on the kubernetes container runtime to create its working directory.
+Most container runtimes will create it owned by root by default, which will make the plugin fail with `Permission denied` errors if we dont precreate it, since the container will run unprivileged.
+
+Also, the default git clone plugin will use /app as its home, which is owned by root and writable only by root in the image, so we'll need to change that too.
+
+This is how our workflow should look like:
+
+```yaml
+# skip the default clone step since we're replacing it with our own.
+skip_clone: true
+
+steps:
+  # precreate the `plugin-git` working directory, so it won't fail with `Permission denied` errors later.
+  - name: prepare
+    image: alpine
+    commands:
+      - mkdir -p $CI_WORKSPACE
+
+  - name: clone
+    image: quay.io/woodpeckerci/plugin-git
+    settings:
+      # set home to /tmp, which is writable by everybody in the `plugin-git` image.
+      home: /tmp
+```
+
+### Final notes about unprivileged namespaces
+
+Please note this setup is experimental, and you may encounter permission issues with other plugins.
+
 ## Environment variables
 
 These env vars can be set in the `env:` sections of the agent.
@@ -551,6 +647,30 @@ Determines if containers must be required to run as non-root users.
 
 ---
 
+### BACKEND_K8S_DEFAULT_SECCTX <!-- cspell:ignore SECCTX NONROOT -->
+
+- Name: `WOODPECKER_BACKEND_K8S_DEFAULT_SECCTX`
+- Default: none
+
+The default security context that will be applied to all step pods.
+
+Must be a YAML object, e.g. `{"runAsUser":1000,"runAsGroup":1000,"fsGroup":1000,"fsGroupChangePolicy": "OnRootMismatch"}`
+
+The security context defined here can be overriden by workflow steps. If you want to define a security context that cannot be overriden, check the next option.
+
+---
+
+### BACKEND_K8S_ENFORCED_SECCTX <!-- cspell:ignore SECCTX NONROOT -->
+
+- Name: `WOODPECKER_BACKEND_K8S_ENFORCED_SECCTX`
+- Default: none
+
+The security context that will be applied to all step pods. Cannot be overriden by workflow steps.
+
+Must be a YAML object, e.g. `{"privileged":false,"runAsNonRoot":true,"allowPrivilegeEscalation":false,"seccompProfile": {"type": "RuntimeDefault"}, "capabilities": {"drop": ["ALL"]}}`
+
+---
+
 ### BACKEND_K8S_PULL_SECRET_NAMES
 
 - Name: `WOODPECKER_BACKEND_K8S_PULL_SECRET_NAMES`

pipeline/backend/kubernetes/backend_options.go

@@ -66,14 +66,16 @@ const (
 )
 
 type SecurityContext struct {
-	Privileged          *bool                                `mapstructure:"privileged"`
-	RunAsNonRoot        *bool                                `mapstructure:"runAsNonRoot"`
-	RunAsUser           *int64                               `mapstructure:"runAsUser"`
-	RunAsGroup          *int64                               `mapstructure:"runAsGroup"`
-	FSGroup             *int64                               `mapstructure:"fsGroup"`
-	FsGroupChangePolicy *kube_core_v1.PodFSGroupChangePolicy `mapstructure:"fsGroupChangePolicy"`
-	SeccompProfile      *SecProfile                          `mapstructure:"seccompProfile"`
-	ApparmorProfile     *SecProfile                          `mapstructure:"apparmorProfile"`
+	Privileged               *bool                                `mapstructure:"privileged"`
+	RunAsNonRoot             *bool                                `mapstructure:"runAsNonRoot"`
+	RunAsUser                *int64                               `mapstructure:"runAsUser"`
+	RunAsGroup               *int64                               `mapstructure:"runAsGroup"`
+	FSGroup                  *int64                               `mapstructure:"fsGroup"`
+	FsGroupChangePolicy      *kube_core_v1.PodFSGroupChangePolicy `mapstructure:"fsGroupChangePolicy"`
+	SeccompProfile           *SecProfile                          `mapstructure:"seccompProfile"`
+	ApparmorProfile          *SecProfile                          `mapstructure:"apparmorProfile"`
+	AllowPrivilegeEscalation *bool                                `mapstructure:"allowPrivilegeEscalation"`
+	Capabilities             *Capabilities                        `mapstructure:"capabilities"`
 }
 
 type SecProfile struct {
@@ -83,6 +85,10 @@ type SecProfile struct {
 
 type SecProfileType string
 
+type Capabilities struct {
+	Drop []string `mapstructure:"drop"`
+}
+
 // SecretRef defines Kubernetes secret reference.
 type SecretRef struct {
 	Name   string       `mapstructure:"name"`

pipeline/backend/kubernetes/flags.go

@@ -108,6 +108,18 @@ var Flags = []cli.Flag{
 		Name:    "backend-k8s-secctx-nonroot",
 		Usage:   "`run as non root` Kubernetes security context option",
 	},
+	&cli.StringFlag{
+		Sources: cli.EnvVars("WOODPECKER_BACKEND_K8S_DEFAULT_SECCTX"), // cspell:words secctx
+		Name:    "backend-k8s-default-secctx",
+		Usage:   "default Kubernetes security context option",
+		Value:   "",
+	},
+	&cli.StringFlag{
+		Sources: cli.EnvVars("WOODPECKER_BACKEND_K8S_ENFORCED_SECCTX"), // cspell:words secctx
+		Name:    "backend-k8s-enforced-secctx",
+		Usage:   "enforced Kubernetes security context option, cannot be overridden by step options",
+		Value:   "",
+	},
 	&cli.StringSliceFlag{
 		Sources: cli.EnvVars("WOODPECKER_BACKEND_K8S_PULL_SECRET_NAMES"),
 		Name:    "backend-k8s-pod-image-pull-secret-names",

pipeline/backend/kubernetes/kubernetes.go

@@ -75,7 +75,8 @@ type config struct {
 	PodAffinity                 *kube_core_v1.Affinity
 	PodAffinityAllowFromStep    bool
 	ImagePullSecretNames        []string
-	SecurityContext             SecurityContextConfig
+	DefaultSecurityContext      SecurityContext
+	EnforcedSecurityContext     SecurityContext
 	NativeSecretsAllowFromStep  bool
 	PriorityClassName           string
 }
@@ -87,11 +88,6 @@ func (c *config) GetNamespace(orgID int64) string {
 	return c.Namespace
 }
 
-type SecurityContextConfig struct {
-	RunAsNonRoot bool
-	FSGroup      *int64
-}
-
 func newDefaultDeleteOptions() kube_meta_v1.DeleteOptions {
 	gracePeriodSeconds := int64(0) // immediately
 	propagationPolicy := kube_meta_v1.DeletePropagationBackground
@@ -120,10 +116,10 @@ func configFromCliContext(ctx context.Context) (*config, error) {
 				PodNodeSelector:             make(map[string]string), // just init empty map to prevent nil panic
 				PodAffinityAllowFromStep:    c.Bool("backend-k8s-pod-affinity-allow-from-step"),
 				ImagePullSecretNames:        c.StringSlice("backend-k8s-pod-image-pull-secret-names"),
-				SecurityContext: SecurityContextConfig{
-					RunAsNonRoot: c.Bool("backend-k8s-secctx-nonroot"), // cspell:words secctx nonroot
-					FSGroup:      newInt64(defaultFSGroup),
+				DefaultSecurityContext: SecurityContext{
+					FSGroup: newInt64(defaultFSGroup),
 				},
+				EnforcedSecurityContext:    SecurityContext{},
 				NativeSecretsAllowFromStep: c.Bool("backend-k8s-allow-native-secrets"),
 			}
 			// Unmarshal label and annotation settings here to ensure they're valid on startup
@@ -157,6 +153,25 @@ func configFromCliContext(ctx context.Context) (*config, error) {
 					return nil, err
 				}
 			}
+			if defaultSecCtx := c.String("backend-k8s-default-secctx"); defaultSecCtx != "" {
+				var sc SecurityContext
+				if err := yaml.Unmarshal([]byte(defaultSecCtx), &sc); err != nil {
+					log.Error().Err(err).Msgf("could not unmarshal default security context '%s'", defaultSecCtx)
+					return nil, err
+				}
+				config.DefaultSecurityContext = sc
+			}
+			if enforcedSecCtx := c.String("backend-k8s-enforced-secctx"); enforcedSecCtx != "" {
+				var sc SecurityContext
+				if err := yaml.Unmarshal([]byte(enforcedSecCtx), &sc); err != nil {
+					log.Error().Err(err).Msgf("could not unmarshal enforced security context '%s'", enforcedSecCtx)
+					return nil, err
+				}
+				config.EnforcedSecurityContext = sc
+			}
+			if c.Bool("backend-k8s-secctx-nonroot") {
+				config.EnforcedSecurityContext.RunAsNonRoot = newBool(true)
+			}
 
 			return &config, nil
 		}

pipeline/backend/kubernetes/kubernetes_test.go

@@ -35,14 +35,14 @@ import (
 func TestGettingConfig(t *testing.T) {
 	engine := kube{
 		config: &config{
-			Namespace:            "default",
-			StorageClass:         "hdd",
-			VolumeSize:           "1G",
-			StorageRwx:           false,
-			PodLabels:            map[string]string{"l1": "v1"},
-			PodAnnotations:       map[string]string{"a1": "v1"},
-			ImagePullSecretNames: []string{"regcred"},
-			SecurityContext:      SecurityContextConfig{RunAsNonRoot: false},
+			Namespace:              "default",
+			StorageClass:           "hdd",
+			VolumeSize:             "1G",
+			StorageRwx:             false,
+			PodLabels:              map[string]string{"l1": "v1"},
+			PodAnnotations:         map[string]string{"a1": "v1"},
+			ImagePullSecretNames:   []string{"regcred"},
+			DefaultSecurityContext: SecurityContext{RunAsNonRoot: newBool(false)},
 		},
 	}
 	config := engine.getConfig()
@@ -52,7 +52,7 @@ func TestGettingConfig(t *testing.T) {
 	config.PodLabels = nil
 	config.PodAnnotations["a2"] = "v2"
 	config.ImagePullSecretNames = append(config.ImagePullSecretNames, "docker.io")
-	config.SecurityContext.RunAsNonRoot = true
+	config.DefaultSecurityContext.RunAsNonRoot = newBool(true)
 
 	assert.Equal(t, "default", engine.config.Namespace)
 	assert.Equal(t, "hdd", engine.config.StorageClass)
@@ -61,7 +61,7 @@ func TestGettingConfig(t *testing.T) {
 	assert.Len(t, engine.config.PodLabels, 1)
 	assert.Len(t, engine.config.PodAnnotations, 1)
 	assert.Len(t, engine.config.ImagePullSecretNames, 1)
-	assert.False(t, engine.config.SecurityContext.RunAsNonRoot)
+	assert.False(t, *engine.config.DefaultSecurityContext.RunAsNonRoot)
 }
 
 func TestSetupWorkflow(t *testing.T) {
@@ -73,14 +73,14 @@ func TestSetupWorkflow(t *testing.T) {
 
 	engine := kube{
 		config: &config{
-			Namespace:            namespace,
-			StorageClass:         "hdd",
-			VolumeSize:           "1G",
-			StorageRwx:           false,
-			PodLabels:            map[string]string{"l1": "v1"},
-			PodAnnotations:       map[string]string{"a1": "v1"},
-			ImagePullSecretNames: []string{"regcred"},
-			SecurityContext:      SecurityContextConfig{RunAsNonRoot: false},
+			Namespace:              namespace,
+			StorageClass:           "hdd",
+			VolumeSize:             "1G",
+			StorageRwx:             false,
+			PodLabels:              map[string]string{"l1": "v1"},
+			PodAnnotations:         map[string]string{"a1": "v1"},
+			ImagePullSecretNames:   []string{"regcred"},
+			DefaultSecurityContext: SecurityContext{RunAsNonRoot: newBool(false)},
 		},
 		client: fake.NewClientset(),
 	}
@@ -181,6 +181,74 @@ func TestAffinityFromCliContext(t *testing.T) {
 	require.NoError(t, err)
 }
 
+func TestSecctxNonrootFromCliContext(t *testing.T) {
+	t.Setenv("WOODPECKER_BACKEND_K8S_SECCTX_NONROOT", "true")
+
+	cmd := &cli.Command{
+		Flags: Flags,
+		Action: func(ctx context.Context, c *cli.Command) error {
+			ctx = context.WithValue(ctx, types.CliCommand, c)
+			config, err := configFromCliContext(ctx)
+
+			require.NoError(t, err)
+			require.NotNil(t, config)
+
+			// Verify security context was parsed
+			require.NotNil(t, config.EnforcedSecurityContext)
+			assert.True(t, *config.EnforcedSecurityContext.RunAsNonRoot)
+			return nil
+		},
+	}
+	err := cmd.Run(context.Background(), []string{"test"})
+	require.NoError(t, err)
+}
+
+func TestSecurityContextFromCliContext(t *testing.T) {
+	t.Setenv("WOODPECKER_BACKEND_K8S_DEFAULT_SECCTX", `{
+		"runAsUser":1000,
+		"runAsGroup":1000,
+		"fsGroup":1000,
+		"fsGroupChangePolicy": "OnRootMismatch"
+	}`)
+	t.Setenv("WOODPECKER_BACKEND_K8S_ENFORCED_SECCTX", `{
+		"privileged":false,
+		"runAsNonRoot":true,
+		"allowPrivilegeEscalation":false,
+		"seccompProfile": {"type": "RuntimeDefault"},
+		"capabilities": {"drop": ["ALL"]}
+	}`)
+
+	cmd := &cli.Command{
+		Flags: Flags,
+		Action: func(ctx context.Context, c *cli.Command) error {
+			ctx = context.WithValue(ctx, types.CliCommand, c)
+			config, err := configFromCliContext(ctx)
+
+			require.NoError(t, err)
+			require.NotNil(t, config)
+
+			// Verify security context was parsed
+			require.NotNil(t, config.DefaultSecurityContext)
+			require.NotNil(t, config.EnforcedSecurityContext)
+
+			assert.Equal(t, (int64)(1000), *config.DefaultSecurityContext.RunAsUser)
+			assert.Equal(t, (int64)(1000), *config.DefaultSecurityContext.RunAsGroup)
+			assert.Equal(t, (int64)(1000), *config.DefaultSecurityContext.FSGroup)
+			assert.Equal(t, kube_core_v1.PodFSGroupChangePolicy("OnRootMismatch"), *config.DefaultSecurityContext.FsGroupChangePolicy)
+
+			assert.False(t, *config.EnforcedSecurityContext.Privileged)
+			assert.True(t, *config.EnforcedSecurityContext.RunAsNonRoot)
+			assert.False(t, *config.EnforcedSecurityContext.AllowPrivilegeEscalation)
+			assert.Equal(t, SecProfileType("RuntimeDefault"), config.EnforcedSecurityContext.SeccompProfile.Type)
+			assert.Equal(t, []string{"ALL"}, config.EnforcedSecurityContext.Capabilities.Drop)
+
+			return nil
+		},
+	}
+	err := cmd.Run(context.Background(), []string{"test"})
+	require.NoError(t, err)
+}
+
 func makeStep(uuid string) *types.Step {
 	return &types.Step{
 		UUID:  uuid,