chore(spark): add Barrett runner token secret - #87
Merged
barrettruth merged 1 commit intoApr 30, 2026
Conversation
Move the Barrett Forgejo runner token into the Spark SOPS flow so the user-scoped runner pool can read a declarative `/run/secrets` path instead of depending only on the live host-local env file.
harivansh-afk
added a commit
that referenced
this pull request
May 16, 2026
## Summary Re-enable the llama-cpp service on spark but with auto-sleep so it stops pinning ~76 GiB of VRAM and NVIDIA driver host pages 24/7. That was the original reason the service was disabled in #74. ## Changes - `enable = true` so the systemd unit has a real `ExecStart` again - `-c 32768` (was `131072`): shrinks KV cache from ~6.5 GiB to ~1.6 GiB on GPU - `--sleep-idle-seconds 600`: weights + KV cache unload from VRAM after 10 min idle; next request reloads transparently (flag is present in the current nixpkgs llama-cpp build, verified via `llama-server --help`) - `OOMScoreAdjust = 1000` on the unit: llama-server is the first kill target if memory pressure ever spikes, protecting forgejo ## How the auto-sleep works `--sleep-idle-seconds` (PR ggml-org/llama.cpp#18228) keeps the server process alive at near-zero footprint after the timeout, then lazy-reloads on the next request. Different from `--standby-timeout`, which kills the whole server process. Sleep is the better fit here because pi keeps a long-lived OpenAI-compatible client against `127.0.0.1:8080`. ## Test plan - [ ] `nixos-rebuild switch --flake .#spark` on spark - [ ] `systemctl status llama-cpp` shows `active (running)` - [ ] `curl -s http://127.0.0.1:8080/health` returns `{"status":"ok"}` - [ ] First completion request loads the model; `nvidia-smi` shows VRAM usage climb to ~70 GiB - [ ] Wait ~11 minutes idle; `nvidia-smi` shows VRAM drop back to near-zero, process still alive - [ ] Next request reloads model and serves a completion - [ ] Pi (`dots/pi/models.json`) routes through unchanged - [ ] `systemctl status forgejo` shows no degradation while idle Reviewed-on: https://git.harivan.sh/harivansh-afk/nix/pulls/87 Co-authored-by: Harivansh Rathi <rathiharivansh@gmail.com> Co-committed-by: Harivansh Rathi <rathiharivansh@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
The Barrett Spark runner split still depends on a host-local
/etc/forgejo-runner.envfallback because the runner token is not yet managed through the Spark SOPS secret set.Solution
Add a dedicated
secrets/spark/barrett-forgejo-runner-tokenfile and a small Barrett system module that exposes it at/run/secrets/barrett-forgejo-runner-token. This lets the existing user-scoped runner helper read the declarative secret path as intended.