Skip to content

fix(secrets): the Claude coding-plan token had no slot to arrive through - #2700

Merged
POWERFULMOVES merged 4 commits into
mainfrom
fix/claude-coding-plan-slot
Aug 23, 2026
Merged

POWERFULMOVES merged 4 commits into
mainfrom
fix/claude-coding-plan-slot

Conversation

@POWERFULMOVES

Copy link
Copy Markdown
Owner

The bug

archon boots healthy, passes its health check, and dies on the first workflow:

errorCode: "billing_error"  apiErrorStatus: 400
"Credit balance is too low"

The cause is not a missing value. It is a missing slot.

Layer State on main
env.tier-agent.example:79-83 declares CLAUDE_CODE_OAUTH_TOKEN (since #2359)
docker-compose.yml / .agents.yml wire ${CLAUDE_CODE_OAUTH_TOKEN:-} into archon
CHIT registry no entry — the funnel has no route for it
live env.tier-agent carries none of the five Claude auth vars the example promises

Meanwhile ANTHROPIC_API_KEY is a registered slot, is set fleet-wide from env.shared, and the Claude CLI prefers it over the OAuth token. Archon says so in its own source (packages/providers/src/claude/provider.ts:144):

// the CLI prefers ANTHROPIC_API_KEY over the OAuth token, so injecting the install key alongside it would silently rebill the run

So the metered key wins and the coding plan never gets a chance. MODEL_FABRIC_CONTRACT.md:44 already lists Claude Code Max in the approved coding-plan inventory — this is a missing slot, not a new policy decision.

The change

Registered via the sanctioned path only: registry entry, then chit-manifest-register, then chit-manifest-sync. No manifest was hand-edited.

- id: claude_code_oauth_token
  source: {type: cgp, label: CLAUDE_CODE_OAUTH_TOKEN}
  targets:
    - {file: .env.generated,  key: CLAUDE_CODE_OAUTH_TOKEN}
    - {file: env.tier-agent,  key: CLAUDE_CODE_OAUTH_TOKEN}
  required: false

Necessary but not sufficient — read this before expecting archon to work

This routes the label. It does not conjure a value.

secrets_sync.py skips an entry entirely when no source value exists:

source_key = _first_usable(secrets, entry)
if source_key is None:
    if entry.required:
        missing.append(entry.label)
    continue        # not written at all

So until a token is provisioned into the CGP source, the key is emitted not at all — not emitted empty. That is the same blank-is-not-absent rule #2688 established: an empty secret is not a secret, so absent stays absent.

archon therefore behaves exactly as it does today until the value lands. No regression, and no fix yet either.

Sequencing after merge: mint the token (claude setup-token, in a real terminal), then export PMOVES_ROTATE_VALUE=... + make -C pmoves secrets-rotate KEY=CLAUDE_CODE_OAUTH_TOKEN (the env-var form keeps a bearer credential out of shell history and ps), then restart archon and re-run a workflow. Only after that is confirmed should CLAUDE_USE_GLOBAL_AUTH flip to false — flipping it first trips archon's exit(1) boot guard at index.ts:271-292.

Second fix, unavoidable and deliberate

The same sync emits min_length: 64 onto secret_key_base in the v1 manifest.

PR #2688 added that constraint to the registry, to v2, and to the sync logic — but never re-ran the sync. And secrets_sync.py:336 reads v1.

So the min-length gate shipped in #2688 has been inert on main: the constraint lived in v2 and in the tooling while the file the enforcer actually reads never carried it. A gate that could not fire.

It cannot be excluded here without hand-editing a machine-emitted file, which the pipeline forbids. So it ships with this change, explicitly rather than silently.

Gates

chit-manifest-register --check:  manifest complete: all registry labels present, no constraint drift
secrets-funnel (full chain):     0 errors, 12 warnings, env.tier-agent regenerated (46 entries)

Negative-tested, not trusted: injecting an unregistered label makes the check fail with Error 1 and would add: ZZ_NEGATIVE_TEST_KEY -> tier llm. The label was removed and the file restored byte-exact before commit.

Noted while verifying, not fixed here

The funnel audit already reports this whole bug class as a non-blocking WARN across seven tier files — for example:

[WARN] env.tier-agent: 24 keys in .example but missing from runtime:
       AGENT_ZERO_API_BASE, AGENT_ZERO_HEALTH_PATH, ..., AGENT_ZERO_MESSAGE_PATH....

AGENT_ZERO_MESSAGE_PATH in that list is the same var whose absence makes the supervisor proxy POST /tasks to /api_message, a route the runtime does not serve (it registers only /api/<path:path>, so the real route is /api/api_message). Separate lane; recorded here so it is not rediscovered.

🤖 Generated with Claude Code

https://claude.ai/code/session_01X9mpK1ZFjexjthZbvcb94f

archon boots healthy, passes its health check, and dies on the first
workflow with `billing_error: Credit balance is too low`. The cause is not
a missing value, it is a missing slot.

env.tier-agent.example has declared CLAUDE_CODE_OAUTH_TOKEN since PR #2359,
and both compose files wire `${CLAUDE_CODE_OAUTH_TOKEN:-}` into the archon
service. But the label was never in the CHIT registry, so the funnel had no
route for it and the live env.tier-agent carries none of the five Claude
auth vars the example promises.

Meanwhile ANTHROPIC_API_KEY is a registered slot, is set fleet-wide from
env.shared, and the Claude CLI PREFERS it over the OAuth token. archon says
so in its own source: "the CLI prefers ANTHROPIC_API_KEY over the OAuth
token, so injecting the install key alongside it would silently rebill the
run". So the metered key wins and the coding plan never gets a chance.

MODEL_FABRIC_CONTRACT.md:44 already lists `Claude Code Max` in the approved
coding-plan inventory. This is a missing slot, not a new policy decision.

Registered via the sanctioned path only -- registry entry, then
chit-manifest-register, then chit-manifest-sync. No manifest was hand-edited.

NECESSARY BUT NOT SUFFICIENT. This routes the label; it does not conjure a
value. secrets_sync.py skips an entry entirely when no source value exists
(`if source_key is None: continue`), so until a token is provisioned into
the CGP source the key is emitted NOT AT ALL -- not emitted empty. That is
the same blank-is-not-absent rule #2688 established: an empty secret is not
a secret, so absent stays absent. archon therefore behaves exactly as it
does today until the value lands. No regression, and no fix yet either.

SECOND FIX, unavoidable and deliberate: the same sync emits `min_length: 64`
onto secret_key_base in the v1 manifest. PR #2688 added that constraint to
the registry, to v2, and to the sync logic, but never re-ran the sync -- and
secrets_sync.py reads v1 (tools/secrets_sync.py:336). So the min-length gate
shipped in #2688 has been INERT on main: the constraint lived in v2 and in
the tooling while the file the enforcer reads never carried it. It could not
be excluded here without hand-editing a machine-emitted file, which the
pipeline forbids, so it ships with this change rather than silently.

Gates:
- chit-manifest-register --check: manifest complete, no constraint drift
- negative-tested: an unregistered label fails the check with Error 1
- secrets-funnel run end to end: 0 errors, env.tier-agent regenerated

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01X9mpK1ZFjexjthZbvcb94f
@coderabbitai

coderabbitai Bot commented Aug 23, 2026

Copy link
Copy Markdown
Contributor

Important

  • 🔍 Trigger review

This repository does not receive automatic reviews because it has fewer than 10 stars.

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: 6bbbf41f-e31f-4e9a-bd51-d8e18d6b663d


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@github-actions github-actions Bot added chit-aware Touches a CHIT-aware service — control-body signature gate applies secrets-pipeline Secrets funnel / CHIT manifest tooling labels Aug 23, 2026
@github-actions

Copy link
Copy Markdown
Contributor

🔏 CHIT-aware change — control-body routing

This PR touches a CHIT-aware service (ports 8086/8087/8103/8106/8113/9224 surface). Before a Control-Body ACK:

  • the diff (or PR body) should reference a CHIT signature trail — chit.signed.v1, a CHIT: trailer, or a make -C pmoves sign-trail entry;
  • the chit-pr-audit-agent (.claude/agents/chit-pr-audit-agent.md) is the gate that grants/refuses ACK eligibility;
  • signing quickstart: pmoves/docs/AGENTS/CHIT_SIGNING_TUTORIAL.md.

Advisory routing only — the blocking contract check is CHIT Contract.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: fb3db3f2b6

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread pmoves/chit/secrets_manifest_v2.yaml
Review on #2700 caught that the manifest slot alone does not reach production
or self-hosted nodes. sync-secrets-local.yml builds the CGP bundle solely from
its explicit env: map, and the v1 sync discards github_secret targets -- so a
registered label with no row in that map funnels nothing on any node hydrated
from the bundle. Archon would keep the metered-key behaviour this change set
exists to fix.

The workflow says so in its own comment, two lines above where the key was
missing: "Absent from this map = never delivered."

Adds CLAUDE_CODE_OAUTH_TOKEN beside the other prod-environment-scoped
coding-plan keys (ALIBABA_PRO_CODING_PLAN, KIMI_CODING_API, KILOCODE_API_KEY),
which is where it belongs -- it is the same class of credential and resolves
the same way under environment: prod.

This is the third routing layer the token has to cross: the registry entry so
the manifest emits it, this map so the bundle carries it, and the GitHub prod
secret so there is a value at all. Declaring a value and routing a value are
separate acts, and only the last one is load-bearing.

YAML re-parsed clean after the edit.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01X9mpK1ZFjexjthZbvcb94f
@github-actions github-actions Bot added the workflows GitHub Actions workflows label Aug 23, 2026
@POWERFULMOVES
POWERFULMOVES merged commit 19d8072 into main Aug 23, 2026
26 checks passed
@POWERFULMOVES
POWERFULMOVES deleted the fix/claude-coding-plan-slot branch August 23, 2026 23:52
POWERFULMOVES added a commit that referenced this pull request Aug 24, 2026
…ds it

Codex, correctly: registering the manifest slot does not deliver the value.
sync-secrets-local.yml builds its CGP bundle SOLELY from the workflow's
explicit env map, and the v1 sync discards github_secret targets -- so a
registered label with no row in that map funnels nothing on any node
hydrated from the bundle.

#2700 hit this exact trap and wrote the warning inline five lines above
where this row now goes. I read that PR and still shipped the half that
does not reach anyone.

It matters more here than it did there, because the nodes hydrated from the
bundle are precisely the ones this slot exists for: the 5090 has no Actions
runner and hydrates via `secrets-funnel-from-prod`. Without this row the
operator could create the Prod secret, the funnel would report success, and
env.tier-data would still have no password -- an optional entry skipped in
silence.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FkwiW3VY1xWmahTAtVioxz
POWERFULMOVES added a commit that referenced this pull request Aug 24, 2026
…#2705)

* feat(secrets): juicefs_meta's password had no route to any other node

B850 mounts pmoves-media as the scoped `juicefs_meta` role. The role exists,
the cutover happened, the mount is healthy. The credential, however, never
entered the pipeline:

    /home/pmoves/.pmoves-secrets/juicefs_meta_pw
      -> bind -> /run/secrets/jfs_meta_pw

A hand-placed file, under a different user's home, referenced NOWHERE in this
repo and holding no CHIT registry entry. So B850 works and no second node can
mount without someone hand-copying a secret -- the exact failure the funnel
exists to prevent, and the blocker the 5090 is currently parked on.

This is load-bearing now rather than tidy. PR #2702 made pg_hba admit ONLY
juicefs_meta from the tailnet and reject every other role there, so a remote
mount has no fallback credential: it authenticates as this role or not at all.

Registered through the sanctioned path only -- REGISTRY entry, then
chit-manifest-register, then chit-manifest-sync. No manifest was hand-edited.

    "JUICEFS_META_PASSWORD": {"tier": "data", "required": False}

tier=data because juicefs-mount and juicefs-gateway both read env.tier-data.
required=False unlike its neighbours: only nodes that actually mount
pmoves-media need it, and under `--merge` (strict) a required slot fails the
whole funnel on every node that legitimately lacks it. The usual argument
against required=False -- that it buys silence -- does not apply here: an
absent value fails loudly at mount time with an auth error.

v2 also emits `docker_secret: pmoves_juicefs_meta_password`, which is the
shape B850's hand-bound secret already has, so the pipeline can materialize
what is currently placed by hand.

Necessary but not sufficient, in #2700's sense: this routes the label. The
value still has to be put into CHIT, and the compose defaults for
juicefs-format/juicefs-gateway still name supabase_admin -- both tracked
separately.

Verified: chit-manifest-check reports in sync; chit-manifest-register --check
reports complete with no constraint drift; re-running sync is byte-stable;
both manifests changed by additions only (+23, 0 deletions).

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FkwiW3VY1xWmahTAtVioxz

* fix(secrets): the route terminated one hop short of the node that needs it

Codex, correctly: registering the manifest slot does not deliver the value.
sync-secrets-local.yml builds its CGP bundle SOLELY from the workflow's
explicit env map, and the v1 sync discards github_secret targets -- so a
registered label with no row in that map funnels nothing on any node
hydrated from the bundle.

#2700 hit this exact trap and wrote the warning inline five lines above
where this row now goes. I read that PR and still shipped the half that
does not reach anyone.

It matters more here than it did there, because the nodes hydrated from the
bundle are precisely the ones this slot exists for: the 5090 has no Actions
runner and hydrates via `secrets-funnel-from-prod`. Without this row the
operator could create the Prod secret, the funnel would report success, and
env.tier-data would still have no password -- an optional entry skipped in
silence.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FkwiW3VY1xWmahTAtVioxz

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
POWERFULMOVES added a commit that referenced this pull request Aug 24, 2026
#2718)

* fix(ci): re-enable the four Claude review workflows on the coding plan

All four were down, and two of them invisibly.

  claude-code-review.yml      if: false          + empty ANTHROPIC_API_KEY
  review-comment-monitor.yml  if: false          + empty ANTHROPIC_API_KEY
  review-autofix.yml          active             + empty ANTHROPIC_API_KEY
  claude.yml                  active             + expired OAuth token

The failure chain is the same one #2700 fixed at the fleet level, replayed in
CI: the OAuth token expired, so the workflows were repointed at
ANTHROPIC_API_KEY -- which was also empty -- so two were switched off with
`if: false` to stop them producing failing checks. Each step was locally
reasonable. The result was a repo that appears to have Claude review configured
and has none, with no failing check to say so, because a disabled job does not
appear in the checks list at all.

CLAUDE_CODE_OAUTH_TOKEN was re-minted 2026-08-24, so the premise recorded in
those comments -- "expired and won't be rotated" -- no longer holds.

Changes:
- restore the original bot-filter guard in review-comment-monitor.yml, which was
  preserved in comments rather than deleted (that foresight is why this is a
  small change)
- drop `if: false` from claude-code-review.yml
- point all four at claude_code_oauth_token, replacing anthropic_api_key

On the credential choice: the old comment argued the API key was "the persistent
path" because the subscription token expires. True, and it still expires -- but
ANTHROPIC_API_KEY was empty, so persistent meant persistently unauthenticated.
An expiring credential that works beats a permanent one that is not set. The
coding plan also means these runs draw plan quota rather than per-token spend.

This does not displace the fleet review model. Codex bot, CodeRabbit and the CLI
agents on self-hosted runners keep doing what they do; this restores the
pre-merge lane they cannot cover -- a diff-scoped review on PR open/update, and
triage of what the other bots produce.

Verified: all four parse, no job is disabled, and no anthropic_api_key reference
remains in any of them.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01X9mpK1ZFjexjthZbvcb94f

* fix(ci): gate the pull_request reviewer to same-repo; leave triage ungated

Review raised two fork-secret P1s. One is right and is fixed; the other rests
on a premise that does not hold, and acting on it would have removed working
behaviour.

claude-code-review.yml fires on `pull_request`. GitHub withholds Actions secrets
from fork-originated runs of that event, so CLAUDE_CODE_OAUTH_TOKEN would be
empty and the action would fail rather than review -- recreating the exact
failing-check noise `if: false` was added to stop. Added the same-repo guard.

review-comment-monitor.yml fires on `pull_request_review`, which runs in the
BASE repository context and DOES receive secrets. The fork restriction is
specific to `pull_request`. Left ungated, with the reasoning recorded in the
file so the next reader does not re-derive it.

The evidence is in this repo. review-autofix.yml shares the pull_request_review
trigger, uses the same secret, and carries a same-repo guard -- but its own
comment says why, and it is not about credentials: it holds contents:write and
pushes to the PR head branch, so it needs head==base to avoid pushing to the
wrong repo. If secrets were unavailable on fork pull_request_review runs, that
comment would have said so instead.

The triage job holds only pull-requests:write and issues:write. It posts one
comment and never pushes, so it works on fork PRs today. Gating it would have
turned a working lane off in the name of hardening, which is the failure mode
worth avoiding here.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01X9mpK1ZFjexjthZbvcb94f

---------

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

chit-aware Touches a CHIT-aware service — control-body signature gate applies secrets-pipeline Secrets funnel / CHIT manifest tooling workflows GitHub Actions workflows

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant