docs(anthropic): correct OAuth billing — plan allowance, not extra-usage credits - #75147
docs(anthropic): correct OAuth billing — plan allowance, not extra-usage credits#75147griffinwork40 wants to merge 3 commits into
Conversation
…age credits
The docs stated that Anthropic OAuth "only works if you're on a Claude Max plan
and have purchased extra usage credits", that "the base Max plan allowance is
not consumed by Hermes — only the extra/overage credits", and that "Claude Pro
subscribers cannot use this path."
Measured against Anthropic's own rate-limit response headers, that is not how
the OAuth path bills. On a Claude **Team** subscription with the overage lane
explicitly disabled, requests were served normally and attributed to the
subscription's included plan window:
anthropic-ratelimit-unified-status: allowed
anthropic-ratelimit-unified-representative-claim: five_hour
anthropic-ratelimit-unified-overage-status: rejected
anthropic-ratelimit-unified-overage-disabled-reason: out_of_credits
`overage-status: rejected` with `out_of_credits` means no extra-usage credits
were available to draw on, so an allowed request necessarily came out of the
plan allowance. Reproduced across a range of system-prompt sizes (200 to 40,000
chars, 65 to 9,178 input tokens); every request landed in the same lane.
The practical consequence is the opposite of what the docs implied, and worth
stating plainly: OAuth *does* spend the same budget as Claude Code and
claude.ai, so heavy Hermes use competes with your own interactive usage.
This corrects the claim in all five places it appeared (providers.md caveat and
summary table, quickstart table, environment-variables prose,
credential-pools example) and documents how a reader can confirm the lane on
their own account from the `anthropic-ratelimit-unified-*` headers.
Scope: verified on Team. Max is expected to match; Pro is untested, and the note
says so rather than replacing one unverified claim with another.
teknium1
left a comment
There was a problem hiding this comment.
Thanks for bringing concrete account-level evidence to a claim that current documentation states too strongly. Current main still says OAuth requires Max plus extra credits (website/docs/integrations/providers.md:113-116), while the native-route resolver describes direct Messages OAuth as subscription-plan billed (hermes_cli/runtime_provider.py:106-133).
Problems
- The same requirement remains visible in desktop onboarding (
apps/desktop/src/components/onboarding/providers.tsx:12-15) and the Chinese docs (website/i18n/zh-Hans/docusaurus-plugin-content-docs/current/integrations/providers.md:21,99-102); neither surface is changed here. - The changed summaries say a generic Claude subscription works, but the new note limits measurement to Team and identifies Pro as unverified. Please keep concise summaries within that verified scope.
Suggested changes
- Update the desktop and localized documentation claims alongside the four English pages.
- Qualify summary-table wording to match the Team-only evidence until other tiers are verified.
Automated hermes-sweeper review.
| | **GitHub Copilot** | `hermes model` (OAuth device code flow, `COPILOT_GITHUB_TOKEN`, `GH_TOKEN`, or `gh auth token`) | | ||
| | **GitHub Copilot ACP** | `hermes model` (spawns local `copilot --acp --stdio`) | | ||
| | **Anthropic** | `hermes model` (Claude Max + extra usage credits via OAuth; also supports Anthropic API key or manual setup-token — see note below) | | ||
| | **Anthropic** | `hermes model` (Claude subscription via OAuth — spends your included plan allowance; also supports Anthropic API key or manual setup-token — see note below) | |
There was a problem hiding this comment.
This summary says any Claude subscription can use OAuth, but the new note below says only Team was measured and Pro remains unverified. Please qualify this compact summary to the verified scope rather than making a broader tier guarantee.
…esolution fails Correcting the billing claim is not sufficient on its own: there is a real path where the plan allowance genuinely is not consumed, and it looks identical to the behaviour the old caveat described as policy. resolve_anthropic_token ranks the Claude Code credential above ANTHROPIC_API_KEY, so a working OAuth login always wins. But if OAuth resolution returns nothing and ANTHROPIC_API_KEY is set, resolution falls through to the API key and bills pay-per-token. Every log call on that path is logger.debug, so nothing surfaces at default verbosity — the plan allowance is just quietly untouched. Documents how to tell the lanes apart from the response headers, and names one known trigger: on macOS, Claude Code stores MCP-server OAuth state under the same Keychain service as the login credential, so an unscoped lookup could return the MCP item and conclude the user was not signed in.
|
Cross-reference: #75146 may explain why the original caveat was written, which I think matters for reviewing this one. That PR fixes a macOS Keychain lookup that could return Timeline: the Keychain read landed 2026-04-24 ( The measurement backing this PR is independent of that bug: it used a verified OAuth token and reads Anthropic's own I've also pushed a |
Retracting the central claim — converting this to draftNew evidence from the same account contradicts the measurement this PR is built on. I am not comfortable leaving a confident correction to your docs standing while I cannot reproduce its premise, so I have moved this to draft. Please do not merge it as written. What I originally measured (unchanged, still real)Nine OAuth inference requests earlier today were served with Anthropic's own attribution headers reading: That is: served, attributed to the plan window, with the overage lane explicitly closed for lack of credits. What contradicts itHours later, same token, same account, the production path now fails reproducibly (2/2): And this is not plan exhaustion —
No window is anywhere near 100%, One more datum: the usage payload exposes a top-level Where that leaves itI can no longer support "purchased extra usage credits are not required." Your original caveat may well be substantially right, and my correction over-generalized from a window in which requests happened to be served. I would rather say that plainly than have you merge my text. What I still believe is independently sound, and would be happy to split into a separate PR:
Apologies for the noise. #75146 (the Keychain fix) and #75235 (the test isolation fix) are unaffected — neither depends on this claim. |
…e runs out The corrected text said credits are not required, which is what the headers show for served requests, but it left the reader unprepared for the "You're out of extra usage" message that appears once an allowance is exhausted. Anthropic's own wording on the usage endpoint is "usage credits cover you when you hit your plan limits" — credits are the spillover, not the entry ticket. States that distinction explicitly so the page does not overpromise.
Withdrawing my retraction — the original measurement standsMy previous comment was an overcorrection, and I want to be precise about the error rather than just flip back. What I did wrong: I treated "requests are now being refused" as evidence against "successful requests bill to the plan allowance." Those are different propositions. A refusal tells you a limit was reached. It says nothing about which lane a served request is attributed to. I conflated the two and retracted on a reasoning error, not on new counter-evidence. The lane evidence is unchanged and still directNine served requests, Anthropic's own attribution headers: plus Requests were served, attributed to the plan window, with the overage lane closed for lack of credits. That is exactly the claim in this PR, and nothing since has contradicted it. The important part: this PR depends on #75146The reason the original caveat was written is almost certainly the bug #75146 fixes, and the fix is what makes this corrected text true.
So the old caveat was an accurate description of buggy behaviour, and this PR is only true once #75146 lands. I should have said that from the start instead of framing it as "the docs are wrong." Please treat this as stacked on #75146, and I am happy to fold these commits into that PR if you would rather review one change. Honest residual uncertaintyI could not re-run the measurement today: the same token now returns One lead for anyone who wants to dig: I have added a sentence to the page noting that hitting the allowance surfaces as an extra-usage prompt, so the corrected text does not overpromise. |
|
Closing in favour of #75146, which now carries these three documentation commits alongside the code fix. Consolidating because the two are not independent: the corrected billing text is only true after the Keychain lookup is fixed. Before that fix, on macOS with Nothing is lost — all three commits ( Apologies for the churn on this one: I opened it separately, then retracted and un-retracted the central claim before landing on the right framing. #75146 has the settled version. |
What does this PR do?
Corrects the documented billing model for Anthropic OAuth. The docs currently state three things that measurement contradicts:
Measured against Anthropic's own rate-limit response headers, OAuth requests bill against the subscription's included plan allowance, and purchased extra-usage credits are not required.
Evidence
Tested on a Claude Team subscription with the overage lane explicitly disabled. Anthropic returned:
overage-status: rejectedwithout_of_creditsmeans there were no extra-usage credits available to draw on. The request was neverthelessallowedand attributed torepresentative-claim: five_hour— the plan window. An allowed request on an account with no overage credits necessarily came out of the plan allowance.Reproduced across a range of system-prompt sizes (200 → 40,000 chars, 65 → 9,178 input tokens); every request landed in the same lane. Also reproduced with and without the
x-anthropic-billing-headermarker proposed in #69844 / #72173 — on this account the lane was identical either way, which is a separate data point for those PRs and not something this one depends on.Why this matters beyond accuracy
The practical consequence is the opposite of what the docs implied, and worth stating plainly: because OAuth spends the same budget as Claude Code and claude.ai, heavy Hermes use competes with your own interactive usage. A reader following the current text would expect Hermes to be isolated from their Claude Code budget. It isn't.
The current text also actively deters valid setups — a Team or credit-free Max subscriber reads "only works if... purchased extra usage credits" and reaches for an API key they don't need.
Related Issue
No existing issue or PR — I searched and found nothing covering this caveat.
Type of Change
Changes Made
The claim appeared in five places; all are corrected consistently:
website/docs/integrations/providers.md— the:::cautionblock (rewritten as:::info, with the header evidence and a self-check recipe) and the auth-methods summary tablewebsite/docs/getting-started/quickstart.md— provider tablewebsite/docs/reference/environment-variables.md— native-Anthropic auth prose, now cross-linked to the providers pagewebsite/docs/user-guide/features/credential-pools.md—hermes auth add anthropic --type oauthexample commentAlso adds a short
:::note Untested tiersadmonition, so the correction doesn't silently overreach (see below).How to Test
Anyone with a Claude subscription can confirm the lane on their own account:
Or run
/usagein an interactive session to see remaining plan windows before and after a request. Both routes are now documented in the providers page so readers can verify rather than trust.Docs-only change: no code paths touched.
ruff check .andpython scripts/check-windows-footguns.py --allboth clean. Admonition fences verified balanced (delta +1 open / +1 close; note the file already carries a pre-existing off-by-one onmain, untouched here).Checklist
Code
fix(scope):,feat(scope):, etc.)pytest tests/ -qand all tests pass — N/A for a docs-only change; lint gates run and cleanDocumentation & Housekeeping
docs/, docstrings)cli-config.yaml.exampleif I added/changed config keys — N/ACONTRIBUTING.mdorAGENTS.mdif I changed architecture or workflows — N/AOn scope, stated honestly: this was verified on a Team plan. Max is expected to behave identically but I could not test it, and Pro is unverified. Rather than replace one unverified claim with another, the new text says exactly what was measured and adds a note inviting a report from Pro/Max users, with the header names they'd need. If maintainers have Pro data showing OAuth genuinely fails there, I'm glad to fold it back in — the specific correction that stands regardless is that purchased extra-usage credits are not a requirement, since a credit-free account demonstrably works.