diff --git a/deploy/helm/ourios/Chart.yaml b/deploy/helm/ourios/Chart.yaml index fea0b0ef4..bf65239c1 100644 --- a/deploy/helm/ourios/Chart.yaml +++ b/deploy/helm/ourios/Chart.yaml @@ -10,7 +10,7 @@ description: | audit store on object storage. type: application # Chart version — bumped on chart changes, independent of the app version. -version: 0.3.2 +version: 0.4.0 # Tracks the ourios-server release the chart publishes alongside. # Surfaces as the app.kubernetes.io/version label. The deployed image # tag is set via image.tag (defaults to `latest`), not this field. diff --git a/deploy/helm/ourios/README.md b/deploy/helm/ourios/README.md index c865c692d..5340816d0 100644 --- a/deploy/helm/ourios/README.md +++ b/deploy/helm/ourios/README.md @@ -160,6 +160,108 @@ For a **non-AWS provider**, also set `storage.s3.endpoint` to its S3 endpoint UR `storage.s3.region` to the provider's region (or a placeholder like `us-east-1` if it has none). +## Per-role IAM (least privilege) + +A single credential shared by all three workloads holds the union of their +privileges — a compromised querier could then delete data it never needed to +touch. The roles' object-store needs are strictly narrower, and they split +cleanly: + +| Role | S3 actions | Holds delete? | +| --------- | ----------------------------------------------------- | ------------- | +| querier | `GetObject`, `ListBucket` (see cache note) | no | +| receiver | `PutObject` only | no | +| compactor | `GetObject`, `PutObject`, `DeleteObject`, `ListBucket`| **only one** | + +The receiver only *writes* data/audit objects — its production path never +issues a read, list, or delete against the store. Manifest swaps (conditional +`PutObject`) belong to the compaction path, and reclaiming compacted inputs is +the compactor's job alone; the querier only reads. With this split, a +compromised receiver can pollute but neither read nor destroy history, a +compromised querier can read but not destroy, and *nothing* except the +singleton compactor can delete data. + +**Querier cache note (RFC 0033):** after a cache miss the querier attempts a +*best-effort* write-through of its template-map cache artifact (and cleanup of +the stale v1 key). Under the read-only policy above this publish simply fails — +**by contract that never fails a query** (it is a telemetry-only outcome), but +the cache never populates, so every query re-pays the audit fold. To keep the +cache warm without widening the read path, grant the querier `PutObject` + +`DeleteObject` scoped to the one cache key it writes: +`arn:aws:s3:::/audit/tenant_id=*/template_map*` (with a +`storage.s3.prefix`, `arn:aws:s3::://audit/tenant_id=*/template_map*`) +— the querier still cannot touch data objects. + +Each role opts into its own ServiceAccount (falling back to the shared +`serviceAccount` otherwise), carrying its own IRSA role: + +```sh +helm install ourios ./ourios \ + --set storage.backend=s3 --set storage.s3.bucket= \ + --set receiver.serviceAccount.create=true \ + --set receiver.serviceAccount.annotations."eks\.amazonaws\.com/role-arn"=arn:aws:iam:::role/ourios-receiver \ + --set querier.serviceAccount.create=true \ + --set querier.serviceAccount.annotations."eks\.amazonaws\.com/role-arn"=arn:aws:iam:::role/ourios-querier \ + --set compactor.serviceAccount.create=true \ + --set compactor.serviceAccount.annotations."eks\.amazonaws\.com/role-arn"=arn:aws:iam:::role/ourios-compactor \ + --set serviceAccount.create=false +``` + +(The last flag skips the shared ServiceAccount — with all three roles on +their own accounts it would be rendered but unused.) + +Because IRSA credentials come from STS, they are **short-lived and rotated +automatically** — no static key exists to leak or forget to rotate. Each IAM +role's trust policy federates to its ServiceAccount +(`system:serviceaccount::-` — check the +rendered name with `helm template`, since the chart's fullname collapses +to the release name when it already contains "ourios"); `eksctl create +iamserviceaccount` or the Terraform `iam-role-for-service-accounts` module wires +this in one step. The permission policy per role (swap the `Action` list per the +table; the example shows the `storage.s3.prefix`-scoped form — with no +prefix, drop the `Condition` and use `/*` on the object arn): + +```json +{ + "Version": "2012-10-17", + "Statement": [ + { + "Effect": "Allow", + "Action": ["s3:GetObject", "s3:PutObject", "s3:DeleteObject"], + "Resource": "arn:aws:s3::://*" + }, + { + "Effect": "Allow", + "Action": "s3:ListBucket", + "Resource": "arn:aws:s3:::", + "Condition": { "StringLike": { "s3:prefix": "/*" } } + } + ] +} +``` + +(Without the `s3:prefix` condition, `ListBucket` on the bucket arn allows +listing **every** key in the bucket — object access would be scoped but +listing would not.) + +Two hardening notes: + +- **Second control on deletes**: S3 versioning (or object lock) on the bucket + makes the compactor's `DeleteObject` reversible — destructive action then + requires defeating two independent mechanisms, not one. +- **Non-AWS providers**: the same split works anywhere the provider offers + scoped credentials (MinIO policies, Ceph users, R2 API tokens) — issue three + static credentials with the table's permissions and give each role its own + `Secret` out-of-band. The chart's `storage.s3.existingSecret` currently wires + one shared Secret; per-role static Secrets need per-role values files or + out-of-band env injection. + +For mTLS identity rotation (SPIRE, cert-manager): the binary hot-reloads its TLS +listener certificates (RFC 0030), so a sidecar like `spiffe-helper` writing +rotating certs to a shared volume integrates without restarts. TLS listener +config is delivered via the RFC 0020 config file out-of-band today — chart-level +TLS values are not yet exposed. + ## Compactor topology The `ourios-server` binary runs the compaction role by default. To avoid every @@ -198,7 +300,10 @@ is intentionally out of scope. Tune the cadence via `compactor.intervalSecs`. | `querier.defaultWindowSecs` | `3600` | Default look-back for a query with no `range(...)`. | | `compactor.enabled` | `true` | Dedicated singleton compactor Deployment. Setting `false` **fails render** (the only sweeper — hazard #4). | | `compactor.intervalSecs` | `300` | Compaction sweep cadence (used only by the dedicated compactor). | -| `serviceAccount.annotations` | `{}` | AWS EKS IRSA `eks.amazonaws.com/role-arn` goes here (alternative to `storage.s3.existingSecret`). | +| `serviceAccount.annotations` | `{}` | AWS EKS IRSA `eks.amazonaws.com/role-arn` goes here (alternative to `storage.s3.existingSecret`). One identity for all roles — prefer the per-role split. | +| `.serviceAccount.create` | `false` | Role-scoped ServiceAccount for `receiver`/`querier`/`compactor` — the least-privilege IAM seam ("Per-role IAM" above). `false` falls back to the shared `serviceAccount`. | +| `.serviceAccount.annotations` | `{}` | The role's own IRSA `role-arn` (querier read-only, receiver no-delete, compactor sole delete-holder). | +| `.serviceAccount.name` | `""` | With `create=true`: overrides the rendered `-` name. With `create=false`: binds an **existing** ServiceAccount of that name (managed out-of-band). | | `otel.exporterEndpoint` | `""` | OTLP endpoint for Ourios's own self-telemetry. | | `extraEnv` | `[]` | Extra env vars (e.g. `OTEL_*`). No plaintext creds. | diff --git a/deploy/helm/ourios/templates/_helpers.tpl b/deploy/helm/ourios/templates/_helpers.tpl index 0c86825b0..a3da034c6 100644 --- a/deploy/helm/ourios/templates/_helpers.tpl +++ b/deploy/helm/ourios/templates/_helpers.tpl @@ -50,6 +50,12 @@ app.kubernetes.io/name: {{ include "ourios.name" . }} app.kubernetes.io/instance: {{ .Release.Name }} {{- end }} +{{/* +The three workload roles. The single source for every template that +ranges over roles — add/remove roles here only. +*/}} +{{- define "ourios.roles" -}}receiver querier compactor{{- end }} + {{/* Create the name of the service account to use */}} @@ -61,6 +67,29 @@ Create the name of the service account to use {{- end }} {{- end }} +{{/* +The ServiceAccount a role's pods run as. Pass (dict "root" $ "role" +"receiver"|"querier"|"compactor"): the role's own serviceAccount when +configured (create=true renders it; name alone binds an existing one), +falling back to the shared serviceAccount otherwise. Role-scoped accounts +are the least-privilege seam — with IRSA, each carries its own +eks.amazonaws.com/role-arn (README "Per-role IAM"). +*/}} +{{- define "ourios.roleServiceAccountName" -}} +{{- $ := .root -}} +{{- $sa := (index $.Values .role).serviceAccount | default dict -}} +{{- if $sa.create -}} +{{- /* Truncate the base, not the joined name — a 63-char fullname must +never swallow the role suffix, or all three roles collide on one SA. */ -}} +{{- $base := include "ourios.fullname" $ | trunc (int (sub 62 (len .role))) | trimSuffix "-" -}} +{{- default (printf "%s-%s" $base .role) $sa.name }} +{{- else if $sa.name -}} +{{- $sa.name }} +{{- else -}} +{{- include "ourios.serviceAccountName" $ }} +{{- end }} +{{- end }} + {{/* The image reference (repository:tag). The tag defaults to `latest` (a publishable floating tag) rather than the chart appVersion — appVersion tracks @@ -206,12 +235,20 @@ web-identity credentials — so configuring both is rejected. */}} {{- define "ourios.s3CredentialsEnvFrom" -}} {{- if eq .Values.storage.backend "s3" }} -{{- $roleArn := index (.Values.serviceAccount.annotations | default dict) "eks.amazonaws.com/role-arn" }} -{{- if and $roleArn (not .Values.serviceAccount.create) }} +{{- $anyArn := index (.Values.serviceAccount.annotations | default dict) "eks.amazonaws.com/role-arn" }} +{{- if and $anyArn (not .Values.serviceAccount.create) }} {{- fail "serviceAccount.annotations \"eks.amazonaws.com/role-arn\" (IRSA) requires serviceAccount.create=true so the chart applies it; with create=false the chart renders no ServiceAccount and the annotation has no effect. Either set serviceAccount.create=true, or annotate your existing ServiceAccount out-of-band and remove it here." }} {{- end }} -{{- if and .Values.storage.s3.existingSecret $roleArn }} -{{- fail "storage.s3.existingSecret and IRSA (serviceAccount.annotations \"eks.amazonaws.com/role-arn\") are mutually exclusive: static keys would shadow the web-identity credentials. Set exactly one credential mode." }} +{{- range $role := splitList " " (include "ourios.roles" $) }} +{{- $sa := (index $.Values $role).serviceAccount | default dict }} +{{- $roleArn := index ($sa.annotations | default dict) "eks.amazonaws.com/role-arn" }} +{{- if and $roleArn (not $sa.create) }} +{{- fail (printf "%s.serviceAccount.annotations \"eks.amazonaws.com/role-arn\" (IRSA) requires %s.serviceAccount.create=true so the chart applies it; with create=false the annotation has no effect. Either set create=true, or annotate the existing ServiceAccount out-of-band and remove it here." $role $role) }} +{{- end }} +{{- $anyArn = or $anyArn $roleArn }} +{{- end }} +{{- if and .Values.storage.s3.existingSecret $anyArn }} +{{- fail "storage.s3.existingSecret and IRSA (an \"eks.amazonaws.com/role-arn\" annotation on the shared or a per-role serviceAccount) are mutually exclusive: static keys would shadow the web-identity credentials. Set exactly one credential mode." }} {{- end }} {{- with .Values.storage.s3.existingSecret }} envFrom: diff --git a/deploy/helm/ourios/templates/compactor-deployment.yaml b/deploy/helm/ourios/templates/compactor-deployment.yaml index 36b6f8fbc..c79983f4e 100644 --- a/deploy/helm/ourios/templates/compactor-deployment.yaml +++ b/deploy/helm/ourios/templates/compactor-deployment.yaml @@ -41,7 +41,7 @@ spec: imagePullSecrets: {{- toYaml . | nindent 8 }} {{- end }} - serviceAccountName: {{ include "ourios.serviceAccountName" . }} + serviceAccountName: {{ include "ourios.roleServiceAccountName" (dict "root" $ "role" "compactor") }} automountServiceAccountToken: {{ .Values.serviceAccount.automount }} {{- with .Values.podSecurityContext }} securityContext: diff --git a/deploy/helm/ourios/templates/querier-deployment.yaml b/deploy/helm/ourios/templates/querier-deployment.yaml index 63a3169c9..6503c575a 100644 --- a/deploy/helm/ourios/templates/querier-deployment.yaml +++ b/deploy/helm/ourios/templates/querier-deployment.yaml @@ -40,7 +40,7 @@ spec: imagePullSecrets: {{- toYaml . | nindent 8 }} {{- end }} - serviceAccountName: {{ include "ourios.serviceAccountName" . }} + serviceAccountName: {{ include "ourios.roleServiceAccountName" (dict "root" $ "role" "querier") }} automountServiceAccountToken: {{ .Values.serviceAccount.automount }} {{- with .Values.podSecurityContext }} securityContext: diff --git a/deploy/helm/ourios/templates/receiver-statefulset.yaml b/deploy/helm/ourios/templates/receiver-statefulset.yaml index 93f2d748f..b663ef349 100644 --- a/deploy/helm/ourios/templates/receiver-statefulset.yaml +++ b/deploy/helm/ourios/templates/receiver-statefulset.yaml @@ -43,7 +43,7 @@ spec: imagePullSecrets: {{- toYaml . | nindent 8 }} {{- end }} - serviceAccountName: {{ include "ourios.serviceAccountName" . }} + serviceAccountName: {{ include "ourios.roleServiceAccountName" (dict "root" $ "role" "receiver") }} automountServiceAccountToken: {{ .Values.serviceAccount.automount }} {{- with .Values.podSecurityContext }} securityContext: diff --git a/deploy/helm/ourios/templates/serviceaccount.yaml b/deploy/helm/ourios/templates/serviceaccount.yaml index 00492908d..2cc2b59dc 100644 --- a/deploy/helm/ourios/templates/serviceaccount.yaml +++ b/deploy/helm/ourios/templates/serviceaccount.yaml @@ -11,3 +11,26 @@ metadata: {{- end }} automountServiceAccountToken: {{ .Values.serviceAccount.automount }} {{- end }} +{{- /* +Role-scoped ServiceAccounts (least-privilege object-store access, README +"Per-role IAM"): one per role that opts in via .serviceAccount.create. +automount follows the shared serviceAccount setting. +*/ -}} +{{- range $role := splitList " " (include "ourios.roles" $) }} +{{- $sa := (index $.Values $role).serviceAccount | default dict }} +{{- if $sa.create }} +--- +apiVersion: v1 +kind: ServiceAccount +metadata: + name: {{ include "ourios.roleServiceAccountName" (dict "root" $ "role" $role) }} + labels: + {{- include "ourios.labels" $ | nindent 4 }} + app.kubernetes.io/component: {{ $role }} + {{- with $sa.annotations }} + annotations: + {{- toYaml . | nindent 4 }} + {{- end }} +automountServiceAccountToken: {{ $.Values.serviceAccount.automount }} +{{- end }} +{{- end }} diff --git a/deploy/helm/ourios/values.yaml b/deploy/helm/ourios/values.yaml index f677bc0cb..ac561250f 100644 --- a/deploy/helm/ourios/values.yaml +++ b/deploy/helm/ourios/values.yaml @@ -103,6 +103,19 @@ receiver: storageClassName: "" accessModes: - ReadWriteOnce + # Optional role-scoped ServiceAccount (least-privilege object-store access: + # give each role its own IAM identity — see "Per-role IAM" in the README). + # create=false (default) uses the shared serviceAccount below. With IRSA, + # annotate with this role's eks.amazonaws.com/role-arn — the receiver's + # role needs s3:PutObject ONLY (its production path never reads, lists, + # or deletes). `name`: + # create=true + empty name renders `-receiver`; + # create=true + name overrides that; create=false + name binds an + # EXISTING ServiceAccount of that name (managed out-of-band). + serviceAccount: + create: false + annotations: {} + name: "" resources: {} podAnnotations: {} podLabels: {} @@ -117,6 +130,12 @@ querier: # Default look-back for a query with no range(...) stage, in seconds # (OURIOS_QUERIER_DEFAULT_WINDOW_SECS; RFC 0016 §7). Must be non-zero. defaultWindowSecs: 3600 + # Optional role-scoped ServiceAccount (see receiver.serviceAccount). The + # querier's IAM role is read-only: Get/List, no writes, no deletes. + serviceAccount: + create: false + annotations: {} + name: "" resources: {} podAnnotations: {} podLabels: {} @@ -129,6 +148,12 @@ compactor: enabled: true # Sweep cadence (OURIOS_COMPACTION_INTERVAL_SECS), used only by this compactor. intervalSecs: 300 + # Optional role-scoped ServiceAccount (see receiver.serviceAccount). The + # compactor is the only role whose IAM role holds s3:DeleteObject. + serviceAccount: + create: false + annotations: {} + name: "" resources: {} podAnnotations: {} podLabels: {} @@ -147,12 +172,16 @@ extraEnv: [] # - name: OTEL_EXPORTER_OTLP_PROTOCOL # value: http/protobuf +# The shared ServiceAccount, used by every role that does not define its own +# role-scoped one (receiver/querier/compactor .serviceAccount, above). serviceAccount: create: true automount: true # IRSA (AWS EKS-specific credential mode): set # eks.amazonaws.com/role-arn: arn:aws:iam:::role/ - # as an alternative to storage.s3.existingSecret (mutually exclusive). + # as an alternative to storage.s3.existingSecret (mutually exclusive). A + # role-arn here grants ONE identity to all three roles; prefer the per-role + # ServiceAccounts for the least-privilege split (README "Per-role IAM"). annotations: {} name: ""