feat(helm): per-role ServiceAccounts — least-privilege IAM seam (IRSA)

deploy/helm/ourios/Chart.yaml

@@ -10,7 +10,7 @@ description: |
   audit store on object storage.
 type: application
 # Chart version — bumped on chart changes, independent of the app version.
-version: 0.3.2
+version: 0.4.0
 # Tracks the ourios-server release the chart publishes alongside.
 # Surfaces as the app.kubernetes.io/version label. The deployed image
 # tag is set via image.tag (defaults to `latest`), not this field.

deploy/helm/ourios/README.md

@@ -160,6 +160,108 @@ For a **non-AWS provider**, also set `storage.s3.endpoint` to its S3 endpoint UR
 `storage.s3.region` to the provider's region (or a placeholder like `us-east-1`
 if it has none).
 
+## Per-role IAM (least privilege)
+
+A single credential shared by all three workloads holds the union of their
+privileges — a compromised querier could then delete data it never needed to
+touch. The roles' object-store needs are strictly narrower, and they split
+cleanly:
+
+| Role      | S3 actions                                            | Holds delete? |
+| --------- | ----------------------------------------------------- | ------------- |
+| querier   | `GetObject`, `ListBucket` (see cache note)            | no            |
+| receiver  | `PutObject` only                                      | no            |
+| compactor | `GetObject`, `PutObject`, `DeleteObject`, `ListBucket`| **only one**  |
+
+The receiver only *writes* data/audit objects — its production path never
+issues a read, list, or delete against the store. Manifest swaps (conditional
+`PutObject`) belong to the compaction path, and reclaiming compacted inputs is
+the compactor's job alone; the querier only reads. With this split, a
+compromised receiver can pollute but neither read nor destroy history, a
+compromised querier can read but not destroy, and *nothing* except the
+singleton compactor can delete data.
+
+**Querier cache note (RFC 0033):** after a cache miss the querier attempts a
+*best-effort* write-through of its template-map cache artifact (and cleanup of
+the stale v1 key). Under the read-only policy above this publish simply fails —
+**by contract that never fails a query** (it is a telemetry-only outcome), but
+the cache never populates, so every query re-pays the audit fold. To keep the
+cache warm without widening the read path, grant the querier `PutObject` +
+`DeleteObject` scoped to the one cache key it writes:
+`arn:aws:s3:::<bucket>/audit/tenant_id=*/template_map*` (with a
+`storage.s3.prefix`, `arn:aws:s3:::<bucket>/<prefix>/audit/tenant_id=*/template_map*`)
+— the querier still cannot touch data objects.
+
+Each role opts into its own ServiceAccount (falling back to the shared
+`serviceAccount` otherwise), carrying its own IRSA role:
+
+```sh
+helm install ourios ./ourios \
+  --set storage.backend=s3 --set storage.s3.bucket=<bucket> \
+  --set receiver.serviceAccount.create=true \
+  --set receiver.serviceAccount.annotations."eks\.amazonaws\.com/role-arn"=arn:aws:iam::<acct>:role/ourios-receiver \
+  --set querier.serviceAccount.create=true \
+  --set querier.serviceAccount.annotations."eks\.amazonaws\.com/role-arn"=arn:aws:iam::<acct>:role/ourios-querier \
+  --set compactor.serviceAccount.create=true \
+  --set compactor.serviceAccount.annotations."eks\.amazonaws\.com/role-arn"=arn:aws:iam::<acct>:role/ourios-compactor \
+  --set serviceAccount.create=false
+```
+
+(The last flag skips the shared ServiceAccount — with all three roles on
+their own accounts it would be rendered but unused.)
+
+Because IRSA credentials come from STS, they are **short-lived and rotated
+automatically** — no static key exists to leak or forget to rotate. Each IAM
+role's trust policy federates to its ServiceAccount
+(`system:serviceaccount:<namespace>:<fullname>-<role>` — check the
+rendered name with `helm template`, since the chart's fullname collapses
+to the release name when it already contains "ourios"); `eksctl create
+iamserviceaccount` or the Terraform `iam-role-for-service-accounts` module wires
+this in one step. The permission policy per role (swap the `Action` list per the
+table; the example shows the `storage.s3.prefix`-scoped form — with no
+prefix, drop the `Condition` and use `<bucket>/*` on the object arn):
+
+```json
+{
+  "Version": "2012-10-17",
+  "Statement": [
+    {
+      "Effect": "Allow",
+      "Action": ["s3:GetObject", "s3:PutObject", "s3:DeleteObject"],
+      "Resource": "arn:aws:s3:::<bucket>/<prefix>/*"
+    },
+    {
+      "Effect": "Allow",
+      "Action": "s3:ListBucket",
+      "Resource": "arn:aws:s3:::<bucket>",
+      "Condition": { "StringLike": { "s3:prefix": "<prefix>/*" } }
+    }
+  ]
+}
+```
+
+(Without the `s3:prefix` condition, `ListBucket` on the bucket arn allows
+listing **every** key in the bucket — object access would be scoped but
+listing would not.)
+
+Two hardening notes:
+
+- **Second control on deletes**: S3 versioning (or object lock) on the bucket
+  makes the compactor's `DeleteObject` reversible — destructive action then
+  requires defeating two independent mechanisms, not one.
+- **Non-AWS providers**: the same split works anywhere the provider offers
+  scoped credentials (MinIO policies, Ceph users, R2 API tokens) — issue three
+  static credentials with the table's permissions and give each role its own
+  `Secret` out-of-band. The chart's `storage.s3.existingSecret` currently wires
+  one shared Secret; per-role static Secrets need per-role values files or
+  out-of-band env injection.
+
+For mTLS identity rotation (SPIRE, cert-manager): the binary hot-reloads its TLS
+listener certificates (RFC 0030), so a sidecar like `spiffe-helper` writing
+rotating certs to a shared volume integrates without restarts. TLS listener
+config is delivered via the RFC 0020 config file out-of-band today — chart-level
+TLS values are not yet exposed.
+
 ## Compactor topology
 
 The `ourios-server` binary runs the compaction role by default. To avoid every
@@ -198,7 +300,10 @@ is intentionally out of scope. Tune the cadence via `compactor.intervalSecs`.
 | `querier.defaultWindowSecs` | `3600` | Default look-back for a query with no `range(...)`. |
 | `compactor.enabled` | `true` | Dedicated singleton compactor Deployment. Setting `false` **fails render** (the only sweeper — hazard #4). |
 | `compactor.intervalSecs` | `300` | Compaction sweep cadence (used only by the dedicated compactor). |
-| `serviceAccount.annotations` | `{}` | AWS EKS IRSA `eks.amazonaws.com/role-arn` goes here (alternative to `storage.s3.existingSecret`). |
+| `serviceAccount.annotations` | `{}` | AWS EKS IRSA `eks.amazonaws.com/role-arn` goes here (alternative to `storage.s3.existingSecret`). One identity for all roles — prefer the per-role split. |
+| `<role>.serviceAccount.create` | `false` | Role-scoped ServiceAccount for `receiver`/`querier`/`compactor` — the least-privilege IAM seam ("Per-role IAM" above). `false` falls back to the shared `serviceAccount`. |
+| `<role>.serviceAccount.annotations` | `{}` | The role's own IRSA `role-arn` (querier read-only, receiver no-delete, compactor sole delete-holder). |
+| `<role>.serviceAccount.name` | `""` | With `create=true`: overrides the rendered `<fullname>-<role>` name. With `create=false`: binds an **existing** ServiceAccount of that name (managed out-of-band). |
 | `otel.exporterEndpoint` | `""` | OTLP endpoint for Ourios's own self-telemetry. |
 | `extraEnv` | `[]` | Extra env vars (e.g. `OTEL_*`). No plaintext creds. |
 

deploy/helm/ourios/templates/_helpers.tpl

@@ -50,6 +50,12 @@ app.kubernetes.io/name: {{ include "ourios.name" . }}
 app.kubernetes.io/instance: {{ .Release.Name }}
 {{- end }}
 
+{{/*
+The three workload roles. The single source for every template that
+ranges over roles — add/remove roles here only.
+*/}}
+{{- define "ourios.roles" -}}receiver querier compactor{{- end }}
+
 {{/*
 Create the name of the service account to use
 */}}
@@ -61,6 +67,29 @@ Create the name of the service account to use
 {{- end }}
 {{- end }}
 
+{{/*
+The ServiceAccount a role's pods run as. Pass (dict "root" $ "role"
+"receiver"|"querier"|"compactor"): the role's own serviceAccount when
+configured (create=true renders it; name alone binds an existing one),
+falling back to the shared serviceAccount otherwise. Role-scoped accounts
+are the least-privilege seam — with IRSA, each carries its own
+eks.amazonaws.com/role-arn (README "Per-role IAM").
+*/}}
+{{- define "ourios.roleServiceAccountName" -}}
+{{- $ := .root -}}
+{{- $sa := (index $.Values .role).serviceAccount | default dict -}}
+{{- if $sa.create -}}
+{{- /* Truncate the base, not the joined name — a 63-char fullname must
+never swallow the role suffix, or all three roles collide on one SA. */ -}}
+{{- $base := include "ourios.fullname" $ | trunc (int (sub 62 (len .role))) | trimSuffix "-" -}}
+{{- default (printf "%s-%s" $base .role) $sa.name }}
+{{- else if $sa.name -}}
+{{- $sa.name }}
+{{- else -}}
+{{- include "ourios.serviceAccountName" $ }}
+{{- end }}
+{{- end }}
+
 {{/*
 The image reference (repository:tag). The tag defaults to `latest` (a
 publishable floating tag) rather than the chart appVersion — appVersion tracks
@@ -206,12 +235,20 @@ web-identity credentials — so configuring both is rejected.
 */}}
 {{- define "ourios.s3CredentialsEnvFrom" -}}
 {{- if eq .Values.storage.backend "s3" }}
-{{- $roleArn := index (.Values.serviceAccount.annotations | default dict) "eks.amazonaws.com/role-arn" }}
-{{- if and $roleArn (not .Values.serviceAccount.create) }}
+{{- $anyArn := index (.Values.serviceAccount.annotations | default dict) "eks.amazonaws.com/role-arn" }}
+{{- if and $anyArn (not .Values.serviceAccount.create) }}
 {{- fail "serviceAccount.annotations \"eks.amazonaws.com/role-arn\" (IRSA) requires serviceAccount.create=true so the chart applies it; with create=false the chart renders no ServiceAccount and the annotation has no effect. Either set serviceAccount.create=true, or annotate your existing ServiceAccount out-of-band and remove it here." }}
 {{- end }}
-{{- if and .Values.storage.s3.existingSecret $roleArn }}
-{{- fail "storage.s3.existingSecret and IRSA (serviceAccount.annotations \"eks.amazonaws.com/role-arn\") are mutually exclusive: static keys would shadow the web-identity credentials. Set exactly one credential mode." }}
+{{- range $role := splitList " " (include "ourios.roles" $) }}
+{{- $sa := (index $.Values $role).serviceAccount | default dict }}
+{{- $roleArn := index ($sa.annotations | default dict) "eks.amazonaws.com/role-arn" }}
+{{- if and $roleArn (not $sa.create) }}
+{{- fail (printf "%s.serviceAccount.annotations \"eks.amazonaws.com/role-arn\" (IRSA) requires %s.serviceAccount.create=true so the chart applies it; with create=false the annotation has no effect. Either set create=true, or annotate the existing ServiceAccount out-of-band and remove it here." $role $role) }}
+{{- end }}
+{{- $anyArn = or $anyArn $roleArn }}
+{{- end }}
+{{- if and .Values.storage.s3.existingSecret $anyArn }}
+{{- fail "storage.s3.existingSecret and IRSA (an \"eks.amazonaws.com/role-arn\" annotation on the shared or a per-role serviceAccount) are mutually exclusive: static keys would shadow the web-identity credentials. Set exactly one credential mode." }}
 {{- end }}
 {{- with .Values.storage.s3.existingSecret }}
 envFrom:

deploy/helm/ourios/templates/compactor-deployment.yaml

@@ -41,7 +41,7 @@ spec:
       imagePullSecrets:
         {{- toYaml . | nindent 8 }}
       {{- end }}
-      serviceAccountName: {{ include "ourios.serviceAccountName" . }}
+      serviceAccountName: {{ include "ourios.roleServiceAccountName" (dict "root" $ "role" "compactor") }}
       automountServiceAccountToken: {{ .Values.serviceAccount.automount }}
       {{- with .Values.podSecurityContext }}
       securityContext:

deploy/helm/ourios/templates/querier-deployment.yaml

@@ -40,7 +40,7 @@ spec:
       imagePullSecrets:
         {{- toYaml . | nindent 8 }}
       {{- end }}
-      serviceAccountName: {{ include "ourios.serviceAccountName" . }}
+      serviceAccountName: {{ include "ourios.roleServiceAccountName" (dict "root" $ "role" "querier") }}
       automountServiceAccountToken: {{ .Values.serviceAccount.automount }}
       {{- with .Values.podSecurityContext }}
       securityContext:

deploy/helm/ourios/templates/receiver-statefulset.yaml

@@ -43,7 +43,7 @@ spec:
       imagePullSecrets:
         {{- toYaml . | nindent 8 }}
       {{- end }}
-      serviceAccountName: {{ include "ourios.serviceAccountName" . }}
+      serviceAccountName: {{ include "ourios.roleServiceAccountName" (dict "root" $ "role" "receiver") }}
       automountServiceAccountToken: {{ .Values.serviceAccount.automount }}
       {{- with .Values.podSecurityContext }}
       securityContext:

deploy/helm/ourios/templates/serviceaccount.yaml

@@ -11,3 +11,26 @@ metadata:
   {{- end }}
 automountServiceAccountToken: {{ .Values.serviceAccount.automount }}
 {{- end }}
+{{- /*
+Role-scoped ServiceAccounts (least-privilege object-store access, README
+"Per-role IAM"): one per role that opts in via .serviceAccount.create.
+automount follows the shared serviceAccount setting.
+*/ -}}
+{{- range $role := splitList " " (include "ourios.roles" $) }}
+{{- $sa := (index $.Values $role).serviceAccount | default dict }}
+{{- if $sa.create }}
+---
+apiVersion: v1
+kind: ServiceAccount
+metadata:
+  name: {{ include "ourios.roleServiceAccountName" (dict "root" $ "role" $role) }}
+  labels:
+    {{- include "ourios.labels" $ | nindent 4 }}
+    app.kubernetes.io/component: {{ $role }}
+  {{- with $sa.annotations }}
+  annotations:
+    {{- toYaml . | nindent 4 }}
+  {{- end }}
+automountServiceAccountToken: {{ $.Values.serviceAccount.automount }}
+{{- end }}
+{{- end }}

deploy/helm/ourios/values.yaml

@@ -103,6 +103,19 @@ receiver:
     storageClassName: ""
     accessModes:
       - ReadWriteOnce
+  # Optional role-scoped ServiceAccount (least-privilege object-store access:
+  # give each role its own IAM identity — see "Per-role IAM" in the README).
+  # create=false (default) uses the shared serviceAccount below. With IRSA,
+  # annotate with this role's eks.amazonaws.com/role-arn — the receiver's
+  # role needs s3:PutObject ONLY (its production path never reads, lists,
+  # or deletes). `name`:
+  # create=true + empty name renders `<fullname>-receiver`;
+  # create=true + name overrides that; create=false + name binds an
+  # EXISTING ServiceAccount of that name (managed out-of-band).
+  serviceAccount:
+    create: false
+    annotations: {}
+    name: ""
   resources: {}
   podAnnotations: {}
   podLabels: {}
@@ -117,6 +130,12 @@ querier:
   # Default look-back for a query with no range(...) stage, in seconds
   # (OURIOS_QUERIER_DEFAULT_WINDOW_SECS; RFC 0016 §7). Must be non-zero.
   defaultWindowSecs: 3600
+  # Optional role-scoped ServiceAccount (see receiver.serviceAccount). The
+  # querier's IAM role is read-only: Get/List, no writes, no deletes.
+  serviceAccount:
+    create: false
+    annotations: {}
+    name: ""
   resources: {}
   podAnnotations: {}
   podLabels: {}
@@ -129,6 +148,12 @@ compactor:
   enabled: true
   # Sweep cadence (OURIOS_COMPACTION_INTERVAL_SECS), used only by this compactor.
   intervalSecs: 300
+  # Optional role-scoped ServiceAccount (see receiver.serviceAccount). The
+  # compactor is the only role whose IAM role holds s3:DeleteObject.
+  serviceAccount:
+    create: false
+    annotations: {}
+    name: ""
   resources: {}
   podAnnotations: {}
   podLabels: {}
@@ -147,12 +172,16 @@ extraEnv: []
   # - name: OTEL_EXPORTER_OTLP_PROTOCOL
   #   value: http/protobuf
 
+# The shared ServiceAccount, used by every role that does not define its own
+# role-scoped one (receiver/querier/compactor .serviceAccount, above).
 serviceAccount:
   create: true
   automount: true
   # IRSA (AWS EKS-specific credential mode): set
   #   eks.amazonaws.com/role-arn: arn:aws:iam::<account>:role/<role>
-  # as an alternative to storage.s3.existingSecret (mutually exclusive).
+  # as an alternative to storage.s3.existingSecret (mutually exclusive). A
+  # role-arn here grants ONE identity to all three roles; prefer the per-role
+  # ServiceAccounts for the least-privilege split (README "Per-role IAM").
   annotations: {}
   name: ""