feat(web3): drop hugepages, fold the xmrig guard into monero

.claude/gitops-entropy/phase-1-architecture.md

@@ -0,0 +1,38 @@
+# Phase 1: GitOps Architecture Review
+**Completed:** 2026-08-06
+
+## Findings
+
+**Layout.** `kubernetes/apps/<namespace>/<app>/{ks.yaml,app/}` is applied consistently across all 16 namespace directories and ~90 `ks.yaml` files. Multi-component apps (nextcloud, cloudnative-pg, toolhive, immich) correctly split into multiple Kustomizations per subdirectory (`app/`, `cluster/`, `databases/`, `crds/`, `config/`) rather than cramming everything into one. No dev/staging/prod chain, as expected for a single-cluster homelab.
+
+**Kustomization shape.** Individual `ks.yaml` files look minimal (no `retryInterval`, `timeout`, or `postBuild.substituteFrom`) — this is *not* drift. The top-level `kubernetes/flux/cluster/ks.yaml` (`cluster-apps` Kustomization) injects these into every child via a `patches:` block targeting `group: kustomize.toolkit.fluxcd.io, kind: Kustomization`, with an explicit escape-hatch label (`flux.toolkit.fluxcd.io/substitute-from: skip`) documented inline. This is a clean, deliberate DRY pattern, not an anti-pattern — flag as a positive, not an issue.
+
+**Single source of truth.** No `kubectl apply` scripts, no committed `suspend: true` resources found in `kubernetes/`. `scripts/` and `.taskfiles/` contain only read-only diagnostics (`debug-toolhive-mcp.sh` is pure `kubectl get/logs/describe`, no mutating calls) and a Talos schematic-update helper. One `prune: false`: `kubernetes/apps/ai/toolhive/ks.yaml` on `toolhive-operator-crds` — standard, deliberate CRD-protection pattern (avoids Flux deleting CRDs on Kustomization removal), not an imperative escape hatch.
+
+**God directories/manifests.** None. Largest directory is 13 files with existing sub-grouping (`cloudnative-pg/databases`, `toolhive/config`). Largest manifest is 480 lines (`default/nextcloud/app/helmrelease.yaml`), a single HelmRelease with legitimately large `values:` — under the 300-line manifest threshold in relative terms once you account for it being one logical unit, but still the biggest file in the repo (see Issues).
+
+**Naming.** Directory name, Kustomization `metadata.name`, and `app-template` release name align across all spot-checked apps (wizarr, changedetection, memini, omniroute, etc.). Style is split between `name: &app <x>` (anchor reused later) and plain `name: <x>` — cosmetic inconsistency only, not worth fixing.
+
+**Dependency graph.** `dependsOn` is used correctly where it matters: nextcloud → `cloudnative-pg-cluster` + `dragonfly-cluster`, litellm → `litellm-operator` + `cloudnative-pg-databases`, toolhive-operator → toolhive-operator-crds → toolhive config (3-stage chain). No circular dependencies found. No missing dependsOn spotted on the apps checked.
+
+**Secrets.** One Secret manifest without an inline `sops:` block: `kubernetes/apps/media/qbittorrent/tools/qbitrr/secret.yaml`. Verified — it uses `stringData: SONARR_API_KEY: ${SONARR_API_KEY}` etc., resolved via Flux `postBuild.substituteFrom: cluster-secrets` (the actual secret values live encrypted in `cluster-secrets.sops.yaml`). Not a plaintext leak — correct use of substitution instead of SOPS-encrypting the same value twice.
+
+**Cluster-settings usage.** `kubernetes/components/common/cluster-settings.yaml` centralizes all cluster IPs (OPNsense, control nodes, TrueNAS, IPMI, etc.) as a ConfigMap consumed via `postBuild.substituteFrom`. The only raw IPs found outside it are generic RFC1918 CIDR ranges in nextcloud/smtp-relay (`192.168.0.0/16`, `10.0.0.0/8`) used for trusted-proxy config — appropriately generic, not cluster-specific singles.
+
+**Native sidecar pattern.** All 5 files using `initContainers` (fileflows, qbitrr, immich-server, moonraker-obico, llmkube qwen36-27b-sglang) use `restartPolicy: Always` — the modern native-sidecar API is used consistently; no legacy sidecar hacks found.
+
+## Issues Found
+
+| Severity | Location | Issue | Recommendation |
+|----------|----------|-------|-----------------|
+| High | All 152 `OCIRepository` sources (e.g. `kubernetes/apps/kube-system/cilium/app/ocirepository.yaml:11` `ref: tag: 1.20.0`) | 0 of 152 OCI sources pin by `ref.digest` or use `spec.verify` (cosign). Tags are mutable; Renovate bumps tags but a registry-side retag/compromise is silently pulled. | Not urgent to fix wholesale, but at least add `spec.verify` (keyless cosign) for security-sensitive images (cilium, cert-manager, rook-ceph) where upstream publishes signatures. |
+| Low | `kubernetes/apps/default/nextcloud/app/helmrelease.yaml` (480 lines) | Largest single manifest in repo; still one logical HelmRelease `values:` block, not urgent, but a candidate to split `values` into a separate `values.yaml`/ConfigMap if it grows further. | Watch, no action needed now. |
+| Info (not a defect) | `kubernetes/apps/ai/toolhive/ks.yaml:14` `prune: false` | Deliberate CRD-protection, correctly scoped to the `crds` Kustomization only. | No action — documented for completeness per audit scope item 2. |
+
+## Action Items
+- [ ] Evaluate adding `spec.verify` (cosign) to OCIRepository sources for security-critical charts (cilium, cert-manager, rook-ceph, envoy-gateway) — lowest-effort, highest-value subset rather than all 152.
+- [ ] No structural or single-source-of-truth violations require action.
+
+## Summary Stats
+- Total issues: 2 (plus 1 informational, not counted as a defect)
+- Critical: 0 | High: 1 | Medium: 0 | Low: 1

.claude/gitops-entropy/phase-2-manifest-quality.md

@@ -0,0 +1,99 @@
+# Phase 2: Kubernetes Manifest Quality
+**Completed:** 2026-08-06
+
+## Findings
+
+The repo is close to clean on this pass. Deprecated APIs, `:latest`/mutable image
+tags, and legacy `Ingress` objects are all absent. Critical-path infra (DNS,
+gateway, database, cache) is correctly redundant. The few real issues are
+namespace-hygiene and label nits, not correctness bugs.
+
+- **Removed APIs**: zero hits for `extensions/v1beta1`, `networking.k8s.io/v1beta1`
+  Ingress, `policy/v1beta1` PSP, `autoscaling/v2beta2`, `apiextensions.k8s.io/v1beta1`
+  across `kubernetes/`, `bootstrap/`, `talos/`.
+- **Ingress vs Gateway API**: zero `kind: Ingress` objects and zero `ingress-nginx`
+  references anywhere in scope. All external/internal traffic runs through
+  `kubernetes/apps/network/envoy-gateway/app/envoy.yaml` (Gateway API, 4 `Gateway`
+  objects) with 17 `HTTPRoute` resources cluster-wide. Migration is complete —
+  no EOL ingress-nginx risk.
+- **Image hygiene**: no `:latest`, no bare (untagged) images, no mutable tags
+  (`:main`/`:stable`/`:edge`/`:nightly`/`:rolling`) found. Every `OCIRepository`
+  and app-template `image.tag` is semver or digest pinned. Note: `hermes-agent`
+  (`kubernetes/apps/ai/hermes/app/helmrelease.yaml:32`) is now pinned to
+  `v2026.8.3@sha256:...`, not the `:main` rolling tag recorded in prior memory —
+  that risk appears already resolved.
+- **Resources**: every `HelmRelease` with a `controllers:` block sets `resources:`
+  — no container found with zero requests. Two files
+  (`kubernetes/apps/flux-system/flux-instance/app/helmrelease.yaml:50,66` and
+  `kubernetes/apps/ai/litellm/app/helmrelease.yaml:18`) set `limits.memory` with
+  no explicit `requests.memory`. Not flagged as a QoS bug: the Kubernetes API
+  server defaults `requests` to the `limits` value per-resource when requests
+  are omitted, so these containers are effectively Guaranteed for memory, not
+  degraded. Cosmetic only — explicit requests would just be clearer to a reader.
+- **Replicas / SPOF (critical-path infra only)**: CoreDNS = 2
+  (`kubernetes/apps/kube-system/coredns/app/helmrelease.yaml:16`), k8s-gateway = 2
+  (`kubernetes/apps/network/k8s-gateway/app/helmrelease.yaml:14`), envoy-gateway
+  control plane = 2 (`kubernetes/apps/network/envoy-gateway/app/helmrelease.yaml:17`)
+  with the actual proxy running as a DaemonSet on every node
+  (`kubernetes/apps/network/envoy-gateway/app/envoy.yaml:14`), CloudNativePG
+  cluster = 3 instances + redundant poolers
+  (`kubernetes/apps/database/cloudnative-pg/cluster/cluster.yaml:11`), Dragonfly =
+  3 replicas + PDB (`kubernetes/apps/database/dragonfly/cluster/cluster.yaml:6`).
+  No SPOF found on genuinely critical-path infra.
+- **Probes**: no `probes.*.enabled: false` found anywhere. Slow-starting
+  workloads already carry deliberate startup/liveness tuning: Jellyfin
+  (`kubernetes/apps/media/jellyfin/app/helmrelease.yaml:53-56`, startup
+  failureThreshold 30), Immich server + machine-learning (both have startup
+  probes), Hermes (custom startup probe, failureThreshold 30), and the SGLang
+  `InferenceService` (`kubernetes/apps/ai/llmkube/models/qwen36-27b-sglang.yaml:263-285`)
+  which deliberately makes liveness inert (`exec: ["true"]`, periodSeconds 3600)
+  and relies on a 120-failure/15s startup probe so long model loads can't trigger
+  a restart — a well-reasoned pattern, not a gap.
+- **Graceful shutdown**: only 5 manifests set `terminationGracePeriodSeconds`
+  explicitly (qbitrr 30s, Hermes 90s, xmrig 15s, xmrig-guard 15s) — everything
+  else relies on the 30s default. Nothing in scope looks clearly undersized:
+  the database/cache tier (CNPG, Dragonfly) is operator-managed and the
+  remaining workloads are stateless HTTP services with fast shutdown paths
+  (e.g. Jellyfin adds a 5s `preStop` sleep on top of the default, which covers
+  its case). No action item here.
+- **Raw (non-app-template) manifests**: only one raw Deployment-style workload
+  exists in scope — `kubernetes/apps/kube-system/amdgpu-undervolt.yaml` (a
+  `DaemonSet`). It has `app.kubernetes.io/name` but no
+  `app.kubernetes.io/component` label, and no liveness/readiness probes. Low
+  severity: it's a one-shot sysfs-tuning daemon that `sleep infinity`s after
+  applying settings, so a probe would add little value.
+- **Namespace hygiene**: the literal Kubernetes `default` namespace hosts real
+  production apps — nextcloud, immich, searxng, karakeep, homepage,
+  changedetection, dumbassets, picoshare, obico, spoolman, it-tools
+  (`kubernetes/apps/default/kustomization.yaml:4`). This is against the general
+  best practice of never using `default` for real workloads, but it's clearly
+  deliberate and consistently governed here — it has its own `CiliumNetworkPolicy`
+  (`kubernetes/apps/kube-system/network-policies/app/deny-apiserver-egress.yaml`,
+  the `namespace: default` block) and database ingress rules
+  (`kubernetes/apps/database/cloudnative-pg/cluster` ingress policy) explicitly
+  reference it as a first-class namespace. Flagging as a naming/convention nit
+  only, not a functional bug — renaming it now would be a wide, low-value churn
+  for a working setup.
+- Immich (`kubernetes/apps/default/immich/ks.yaml`) is commented out of
+  `kubernetes/apps/default/kustomization.yaml:9` — dead reference left in the
+  tree. Not scored as an issue (deliberate disable), but worth a cleanup pass
+  if it's staying off long-term.
+
+## Issues Found
+| Severity | Resource | Location | Issue | Recommendation |
+|----------|----------|----------|-------|----------------|
+| Low | amdgpu-undervolt DaemonSet | `kubernetes/apps/kube-system/amdgpu-undervolt.yaml:6-16` | Raw manifest missing `app.kubernetes.io/component` label | Add `app.kubernetes.io/component: gpu-tuning` (or similar) to metadata and pod template labels |
+| Low | default namespace | `kubernetes/apps/default/kustomization.yaml:4` | Literal Kubernetes `default` namespace used for ~11 production apps instead of a dedicated namespace | Cosmetic/convention only; leave as-is unless doing a larger namespace reorg — not worth churn on its own |
+| Low | flux-instance controllers | `kubernetes/apps/flux-system/flux-instance/app/helmrelease.yaml:50,66` | `limits.memory` set with no explicit `requests.memory` | Optional clarity fix: add matching `requests.memory` (k8s already defaults it to the limit value, so this is style-only) |
+| Low | litellm | `kubernetes/apps/ai/litellm/app/helmrelease.yaml:17-19` | Same pattern: `limits.memory` without explicit `requests.memory` | Same as above, style-only |
+| Info | hermes-agent image | `kubernetes/apps/ai/hermes/app/helmrelease.yaml:31-32` | Previously-flagged `:main` rolling tag risk (accepted exception per prior review) now appears resolved — image is digest-pinned | No action; confirms the prior accepted risk is gone |
+| Info | immich | `kubernetes/apps/default/kustomization.yaml:9` | `# - ./immich/ks.yaml` left commented out | Remove if immich is staying disabled long-term, to avoid stale references |
+
+## Action Items
+- None blocking. Optional: add `app.kubernetes.io/component` to `amdgpu-undervolt.yaml`.
+- Optional: add explicit `requests.memory` to flux-instance and litellm for readability (no functional effect).
+- Optional cleanup: drop the commented-out immich `ks.yaml` reference if it's not coming back soon.
+
+## Summary Stats
+- Total issues: 6
+- Critical: 0 | High: 0 | Medium: 0 | Low: 4 | Info: 2