feat(kube-system): add cilium network segmentation policies

[Tanguille]

First network policies on a previously flat pod network (zero policies existed cluster-wide; Cilium enable-policy: default, so unselected pods stay default-allow). Two static, maintenance-free CiliumNetworkPolicies wired via kube-system/network-policies.

Policy 1 — database-ingress-allowlist (namespace: database)

endpointSelector: {} puts every database pod (postgres, pgbouncer poolers, dragonfly, cnpg + dragonfly operators) into default-deny ingress. Allowed ingress:

• From namespaces ai, default, media, observability, security, database — the only postgres/pgbouncer/dragonfly consumers (derived from the cnpg databases/ CRs + dragonfly host refs). observability covers vmagent scraping the exporters; database covers intra-ns operator/pooler/replica traffic.
• From entity kube-apiserver — cnpg serves validating/mutating webhooks the apiserver must reach.
• Kubelet health probes keep working via Cilium allow-localhost: auto (host-firewall is off); egress from DB pods is untouched (only ingress rules present), so backups to NFS/S3 and operator→apiserver are unaffected.

Denies: ingress from any other namespace (kube-system, network, kopiur-system, actions-runner, cert-manager, rook-ceph, web3, ai-non-consumers, etc.).

Policy 2 — deny-kube-apiserver-egress (namespaces: media, default, web3)

An egressDeny to entity kube-apiserver on the internet-exposed app namespaces, with enableDefaultDeny.egress: false so it is a pure deny overlay — DNS, internet, and all other cluster egress remain default-allow. endpointSelector uses matchExpressions: app.kubernetes.io/managed-by NotIn [kopiur], which also matches label-absent app pods (so they ARE denied) while excluding kopiur mover pods, which legitimately need the apiserver (kopiur-mover SA confirmed present in media/default; web3 has none). Audited running pods in all three namespaces: every workload is an application pod with no apiserver need.

Denies: kube-apiserver egress from all app pods in media/default/web3. Allows: everything else, plus full egress for kopiur movers.

Verification

• kustomize build on the app dir and parent kube-system both succeed; namespaces render correctly (no namespace-transformer clobber — ks.yaml intentionally omits targetNamespace).
• kubectl apply --dry-run=server accepts all 4 objects against the deployed Cilium CRD (schema 1.32.7): enableDefaultDeny, egressDeny.toEntities: kube-apiserver, and NotIn all valid.
• Confirmed cilium-config: enable-policy: default, enable-host-firewall: false.

Post-merge checks: after Flux applies, watch for connection resets on DB consumers (hubble observe --namespace database --verdict DROPPED) and confirm kopiur backup/restore jobs in media/default still reach the apiserver. If a consumer namespace was missed, add one matchLabels: io.kubernetes.pod.namespace entry to policy 1.

Rollback

Delete the two policy files (network-policies/app/database-ingress.yaml, deny-apiserver-egress.yaml) — or remove ./network-policies/ks.yaml from kube-system/kustomization.yaml to drop the whole app; Flux prunes both.

[tanguille-cluster]

@@ (root level) @@
# kustomize.toolkit.fluxcd.io/v1/Kustomization/kube-system/network-policies
! + one document added:
+ apiVersion: kustomize.toolkit.fluxcd.io/v1
+ kind: Kustomization
+ metadata:
+   name: network-policies
+   namespace: kube-system
+   labels:
+     kustomize.toolkit.fluxcd.io/name: cluster-apps
+     kustomize.toolkit.fluxcd.io/namespace: flux-system
+ spec:
+   decryption:
+     provider: sops
+     secretRef:
+       name: sops-age
+   deletionPolicy: WaitForTermination
+   interval: 1h
+   patches:
+   - patch: |
+       apiVersion: helm.toolkit.fluxcd.io/v2
+       kind: HelmRelease
+       metadata:
+         name: _
+       spec:
+         install:
+           crds: CreateReplace
+         rollback:
+           cleanupOnFail: true
+         upgrade:
+           cleanupOnFail: true
+           crds: CreateReplace
+           strategy:
+             name: RemediateOnFailure
+           remediation:
+             remediateLastFailure: true
+             retries: 2
+     target:
+       kind: HelmRelease
+       group: helm.toolkit.fluxcd.io
+   path: ./kubernetes/apps/kube-system/network-policies/app
+   postBuild:
+     substituteFrom:
+     - name: cluster-settings
+       kind: ConfigMap
+       optional: false
+     - name: cluster-secrets
+       kind: Secret
+       optional: false
+   prune: true
+   retryInterval: 2m
+   sourceRef:
+     name: flux-system
+     kind: GitRepository
+     namespace: flux-system
+   timeout: 5m
+   wait: false

@@ (root level) @@
# cilium.io/v2/CiliumNetworkPolicy/database/database-ingress-allowlist
! + one document added:
+ apiVersion: cilium.io/v2
+ kind: CiliumNetworkPolicy
+ metadata:
+   name: database-ingress-allowlist
+   namespace: database
+   labels:
+     kustomize.toolkit.fluxcd.io/name: network-policies
+     kustomize.toolkit.fluxcd.io/namespace: kube-system
+ spec:
+   endpointSelector: {}
+   ingress:
+   - fromEndpoints:
+     - matchLabels:
+         io.kubernetes.pod.namespace: ai
+     - matchLabels:
+         io.kubernetes.pod.namespace: default
+     - matchLabels:
+         io.kubernetes.pod.namespace: media
+     - matchLabels:
+         io.kubernetes.pod.namespace: observability
+     - matchLabels:
+         io.kubernetes.pod.namespace: security
+     - matchLabels:
+         io.kubernetes.pod.namespace: database
+   - fromEntities:
+     - kube-apiserver

@@ (root level) @@
# cilium.io/v2/CiliumNetworkPolicy/media/deny-kube-apiserver-egress
! + one document added:
+ apiVersion: cilium.io/v2
+ kind: CiliumNetworkPolicy
+ metadata:
+   name: deny-kube-apiserver-egress
+   namespace: media
+   labels:
+     kustomize.toolkit.fluxcd.io/name: network-policies
+     kustomize.toolkit.fluxcd.io/namespace: kube-system
+ spec:
+   egressDeny:
+   - toEntities:
+     - kube-apiserver
+   enableDefaultDeny:
+     egress: false
+   endpointSelector:
+     matchExpressions:
+     - key: app.kubernetes.io/managed-by
+       operator: NotIn
+       values:
+       - kopiur

@@ (root level) @@
# cilium.io/v2/CiliumNetworkPolicy/default/deny-kube-apiserver-egress
! + one document added:
+ apiVersion: cilium.io/v2
+ kind: CiliumNetworkPolicy
+ metadata:
+   name: deny-kube-apiserver-egress
+   namespace: default
+   labels:
+     kustomize.toolkit.fluxcd.io/name: network-policies
+     kustomize.toolkit.fluxcd.io/namespace: kube-system
+ spec:
+   egressDeny:
+   - toEntities:
+     - kube-apiserver
+   enableDefaultDeny:
+     egress: false
+   endpointSelector:
+     matchExpressions:
+     - key: app.kubernetes.io/managed-by
+       operator: NotIn
+       values:
+       - kopiur

@@ (root level) @@
# cilium.io/v2/CiliumNetworkPolicy/web3/deny-kube-apiserver-egress
! + one document added:
+ apiVersion: cilium.io/v2
+ kind: CiliumNetworkPolicy
+ metadata:
+   name: deny-kube-apiserver-egress
+   namespace: web3
+   labels:
+     kustomize.toolkit.fluxcd.io/name: network-policies
+     kustomize.toolkit.fluxcd.io/namespace: kube-system
+ spec:
+   egressDeny:
+   - toEntities:
+     - kube-apiserver
+   enableDefaultDeny:
+     egress: false
+   endpointSelector:
+     matchExpressions:
+     - key: app.kubernetes.io/managed-by
+       operator: NotIn
+       values:
+       - kopiur

[github-actions]

AI Automated Review

Analysis engine: qwen-3.6-fast@http://litellm.ai.svc.cluster.local/v1 (openai)

Recommendation

Approve — The PR introduces well-scoped CiliumNetworkPolicies that establish baseline network segmentation for database ingress and apiserver egress. The manifests follow repository conventions, are internally consistent, and address the stated security goals without introducing breaking changes to existing workloads.

Change-by-Change Findings

kubernetes/apps/kube-system/kustomization.yaml

• Adds ./network-policies/ks.yaml to the kube-system Kustomization resources list. This follows the established pattern of other kube-system components (cilium, coredns, metrics-server, etc.).

kubernetes/apps/kube-system/network-policies/app/database-ingress.yaml

• Introduces a default-deny ingress policy for the database namespace with explicit allowlists for consuming namespaces (ai, default, media, observability, security, database) and kube-apiserver webhook access.
• Uses endpointSelector: {} which applies to all pods in the namespace, as documented in comments.
• Correctly uses fromEndpoints with namespace labels and fromEntities for kube-apiserver.

kubernetes/apps/kube-system/network-policies/app/deny-apiserver-egress.yaml

• Three identical policies targeting media, default, and web3 namespaces to deny egress to kube-apiserver.
• Uses matchExpressions with NotIn operator to exclude kopiur-managed pods, which legitimately need apiserver access.
• Sets enableDefaultDeny.egress: false to ensure this is a pure deny overlay without affecting other egress traffic.

kubernetes/apps/kube-system/network-policies/app/kustomization.yaml

• Properly structured Kustomization with explicit comment about no namespace transformer needed since each policy declares its own namespace.
• Lists both policy files as resources.

kubernetes/apps/kube-system/network-policies/ks.yaml

• Flux Kustomization resource following repository conventions (interval, prune, sourceRef).
• Correctly omits targetNamespace since policies fan out to multiple namespaces via their own metadata.namespace.

Standards Compliance

• Conventional Commits: Title follows feat(scope): description pattern ✓
• Kustomize structure: Follows existing patterns with separate app directory and ks.yaml ✓
• Schema references: Uses appropriate JSON schemas for CiliumNetworkPolicy and Kustomization ✓
• No secrets committed: No sensitive data in manifests ✓
• Small, focused change: Adds new functionality without modifying existing resources ✓

Must Check Items

1. Validate manifest against target cluster version: The PR author confirms dry-run validation against deployed Cilium CRD schema 1.32.7. The manifests use standard CiliumNetworkPolicy v2 API which is stable across recent versions.

2. Check for resource quota / limit changes: Not applicable - this PR only adds network policies, not workload resources with quotas or limits.

3. Verify file path sanitization: Not applicable - these are Kubernetes manifest files, not user-input processing code.

4. Check for directory traversal vulnerabilities: Not applicable - network policies don't process file paths from user input.

Unknowns / Needs Verification

• Cilium configuration compatibility: The PR states enable-policy: default and enable-host-firewall: false are confirmed in cilium-config. While this should be verified before merge, it's outside the scope of manifest review.
• Namespace existence: Assumes database, media, default, and web3 namespaces exist and contain the expected workloads. This should be verified against the live cluster state.
• Consumer namespace completeness: The allowed ingress namespaces for database access were derived from CNPG Database CRs and dragonfly host refs. Any future apps needing database access will require policy updates.

Risk Assessment

• Low risk: Network policies are additive and can be easily rolled back by removing the kustomization reference.
• Recovery path: Clear rollback instructions provided in PR body.
• Impact scope: Limited to specific namespaces and traffic patterns; does not affect cluster-wide networking.