Skip to content

fix: handle AGY auth-required quota states safely - #4981

Closed
Ivory2024 wants to merge 18 commits into
kunchenguid:mainfrom
Ivory2024:fm/firstmate-agy-quota-auth-required-20260920
Closed

Ivory2024 wants to merge 18 commits into
kunchenguid:mainfrom
Ivory2024:fm/firstmate-agy-quota-auth-required-20260920

Conversation

@Ivory2024

Copy link
Copy Markdown

Intent

Fix the persistent AGY/GY quota display problem: when AGY quota data is auth_required, keep the candidate eligible but unranked and surface the exact authentication cause without fabricating quota values or authorizing AGY dispatch. Add focused regression coverage. Deliver only this quota fix to the Ivory2024/firstmate fork.

What Changed

  • Updated quota resolution so AGY auth_required results remain eligible but unranked, expose the exact authentication cause, avoid fabricated quota values, and never authorize AGY dispatch; quota wake details and focused regression fixtures cover the behavior.
  • Added terminal child endpoint reaping during inactive reconciliation, with a regression test for the cleanup path.
  • Added and routed new Firstmate skills, removed fresh CLAUDE.md pointer creation and its CI contract, and updated the related agent guidance, documentation, configuration, and tests.

Risk Assessment

✅ Low: No source-backed material defect found; auth-required AGY state is guarded across resolver, chooser, and process-event paths, with focused behavioral regressions covering unranked output, cause disclosure, and dispatch suppression.

Testing

Targeted resolver and chooser regressions completed successfully; the process-event regression stalled before its auth cases, so direct live CLI checks were used. Real quota-axi state showed AGY auth_required with the exact cause, and both provider-specific and aggregate polls surfaced it without quota authorization. Reviewer-visible evidence was written to the no-mistakes evidence directory.

  • Live validation: ⚠️ inconclusive - 3 of 6 scenarios driven live against the product
Scenario Result Live Evidence
Polling live AGY quota returns an error with best:null and the exact authentication cause, without emitting quota values. ✅ pass live bin/fm-procevent-quota.sh poll --provider agy --timeout 1; agy-live-behavior.txt
Aggregate quota polling preserves the AGY authentication error instead of classifying it as healthy, low, or exhausted. ✅ pass live bin/fm-procevent-quota.sh poll --timeout 1; agy-live-behavior.txt
The real chooser refuses an AGY candidate and emits no dispatch profile. ✅ pass live bin/fm-quota-choose.sh --snapshot <live snapshot> --candidate agy:default; agy-live-behavior.txt
A resolver AGY candidate remains eligible but unranked, shows the exact authentication cause, fabricates no scope or percentage, and emits no AGY profile. ⏸️ untested no The live Typesafe resolver could not be called because TYPESAFE_API_KEY was absent. Provide the credential through the environment or an isolated FM_HOME/.env, then rerun.
Auth-required AGY rule floors remain unverifiable and cannot authorize AGY or cross-provider dispatch. ⏸️ untested no The live resolver path requires a Typesafe API credential, which was unavailable in this run.
A known AGY sub-scope cannot bypass auth_required state to authorize dispatch. ⏸️ untested no The real host snapshot had auth_required with no effective availability rows, so the known-sub-scope adversarial input could not be driven live. Provide a live auth-required snapshot containing a know…
Evidence: Live AGY behavior

Source: Live AGY behavior

== live provider poll ==
quota: quota-agy
status: error
detail: {"provider":"agy","best":null,"error":"Antigravity sign-in required"}
condition_polls: 1
== live aggregate poll ==
quota: quota
status: error
detail: {"provider":"aggregate","summary":[{"provider":"claude","best":null},{"provider":"codex","best":{"scope":"all_models","status":"known","effectivePercentRemaining":67,"boundedBy":["five_hour","weekly"],"limitingWindowIds":["weekly"],"pace":{"status":"mixed","aheadWindowIds":["weekly"],"worstReservePercentPoints":-25.9683,"worstReserveWindowId":"weekly"},"runway":{"status":"projected_exhaustion","usableRunwaySeconds":86344,"projectedExhaustedAt":"2026-09-20T23:28:27.375Z","limitingWindowId":"weekly","projectionConfidence":"early"},"selection":{"status":"known","spendPriority":-3.852}}},{"provider":"cursor","best":null},{"provider":"copilot","best":null},{"provider":"grok","best":null},{"provider":"kimi","best":null},{"provider":"zai","best":null},{"provider":"agy","best":null,"error":"Antigravity sign-in required"}]}
condition_polls: 1
== live chooser AGY candidate ==
rc: 2
output: error: unknown harness: agy
== direct resolver credential availability ==
TYPESAFE_API_KEY in environment: absent
Evidence: Live AGY quota state

Source: Live AGY quota state

[{"provider":"agy","state":{"status":"auth_required","stale":false,"error":"Antigravity sign-in required"},"quotaSemantics":{"status":"unknown","effectiveAvailability":[],"unresolvedWindowIds":[]}}]
- Outcome: ⚠️ 3 warnings across 1 run (8m6s)

Pipeline

Updates from git push no-mistakes

✅ **intent** - passed

✅ No issues found.

⚠️ **Rebase** - 1 warning

Confirm these commits belong in this PR before approving, or manually separate the intended work onto origin/main before gating.

🔧 **Review** - 1 issue found → auto-fixed (3) ✅
  • 🚨 bin/fm-quota-axi-lib.sh:60 - The previous fixer round added an auth-required guard only to the typed resolver at bin/fm-dispatch-resolve.sh:309-310, but the shared validator relaxation at this line now accepts an auth-required AGY snapshot with known rows. A concrete snapshot with provider agy, state.status=auth_required, quotaSemantics.status=unknown, and a known all_models row reaches bin/fm-quota-choose.sh:310,328-343,373-383; that chooser ignores state.status and returns agy default, authorizing dispatch despite the required invariant. The same sibling state transition is misclassified by bin/fm-procevent-quota.sh:115-137. Add the auth-required AGY guard at the earliest shared dispatch consumers, and add chooser coverage; resolver-only coverage at tests/fm-dispatch-resolve.test.sh:320-342 does not close this path.

🔧 Fix applied.
2 issues (1 error, 1 warning) still open:

  • 🚨 bin/fm-dispatch-resolve.sh:279 - Auth-required AGY is still reachable through two ranking boundaries. measured() now treats an unknown provider with known rows as measured (bin/fm-dispatch-resolve.sh:276-280), so a selected rule with an AGY floor can be vetoed or fall through to the default before evaluate() reaches its auth guard (bin/fm-dispatch-resolve.sh:354-364); the AGY candidate is then not preserved as eligible/unranked and its cause is not shown. The same accepted snapshot passes fm_quota_json_valid (bin/fm-quota-axi-lib.sh:64-87) and fm-quota-choose.sh never checks state.status, returning agy default from a known sub-scope (bin/fm-quota-choose.sh:327-383). Guard auth-required AGY before rule-floor selection and in this shared chooser; the current regressions cover only the no-floor resolver path (tests/fm-dispatch-resolve.test.sh:320-343).
  • ⚠️ bin/fm-procevent-quota.sh:129 - The process-event sibling path still hides an auth-required AGY failure when the poll uses the supported aggregate mode. With only AGY present and state.status=auth_required, the new aggregate filter produces an empty availability list and returns healthy (bin/fm-procevent-quota.sh:127-131), so cmd_poll sleeps and re-polls instead of emitting an error or authentication cause (bin/fm-procevent-quota.sh:236-247). Even with --provider agy, details() emits only best:null and drops state.error (bin/fm-procevent-quota.sh:149-175).

🔧 Fix applied.
2 errors still open:

  • 🚨 bin/fm-dispatch-resolve.sh:279 - Auth-required AGY is still reachable through two ranking boundaries. measured() now treats an unknown provider with known rows as measured (bin/fm-dispatch-resolve.sh:276-280), so a selected rule with an AGY floor can be vetoed or fall through to the default before evaluate() reaches its auth guard (bin/fm-dispatch-resolve.sh:354-364); the AGY candidate is then not preserved as eligible/unranked and its cause is not shown. The same accepted snapshot passes fm_quota_json_valid (bin/fm-quota-axi-lib.sh:64-87) and fm-quota-choose.sh never checks state.status, returning agy default from a known sub-scope (bin/fm-quota-choose.sh:327-383). Guard auth-required AGY before rule-floor selection and in this shared chooser; the current regressions cover only the no-floor resolver path (tests/fm-dispatch-resolve.test.sh:320-343).
  • 🚨 bin/fm-dispatch-resolve.sh:292 - floor_state() returns none for every auth-required AGY at bin/fm-dispatch-resolve.sh:292. That helper is also used for the selected rule floor at bin/fm-dispatch-resolve.sh:356, not only for the candidate profile floor at bin/fm-dispatch-resolve.sh:316. A valid rule with floor.provider=agy and use.harness=codex, against an AGY snapshot with state.status=auth_required, therefore treats the unverifiable AGY gate as satisfied, evaluates the rankable Codex candidate, and can emit a clear profile: dispatch. Preserve the auth-required exception only for the AGY candidate's own profile evaluation. The regression at tests/fm-dispatch-resolve.test.sh:345-353 misses this cross-provider rule-floor path.

🔧 Fix applied.
✅ Re-checked - no issues remain.

⚠️ **Test** - 3 warnings
  • ⚠️ bin/fm-dispatch-resolve.sh - Live resolver validation was unavailable because TYPESAFE_API_KEY was absent. Provide a valid Typesafe credential via the environment or isolated FM_HOME/.env and rerun this phase.
  • ⚠️ tests/fm-procevent-quota.test.sh - tests/fm-procevent-quota.test.sh stalled before its auth-required assertions on both default and Perl timeout mechanisms. The direct live process-event checks passed independently.
  • ⚠️ live validation verdict: inconclusive (3 of 6 scenarios were driven live against the product); untested: A resolver AGY candidate remains eligible but unranked, shows the exact authentication cause, fabricates no scope or percentage, and emits no AGY profile., Auth-required AGY rule floors remain unverifiable and cannot authorize AGY or cross-provider dispatch., A known AGY sub-scope cannot bypass auth_required state to authorize dispatch.
  • Live validation: ⚠️ inconclusive - 3 of 6 scenarios driven live against the product
Scenario Result Live Evidence
Polling live AGY quota returns an error with best:null and the exact authentication cause, without emitting quota values. ✅ pass live bin/fm-procevent-quota.sh poll --provider agy --timeout 1; agy-live-behavior.txt
Aggregate quota polling preserves the AGY authentication error instead of classifying it as healthy, low, or exhausted. ✅ pass live bin/fm-procevent-quota.sh poll --timeout 1; agy-live-behavior.txt
The real chooser refuses an AGY candidate and emits no dispatch profile. ✅ pass live bin/fm-quota-choose.sh --snapshot <live snapshot> --candidate agy:default; agy-live-behavior.txt
A resolver AGY candidate remains eligible but unranked, shows the exact authentication cause, fabricates no scope or percentage, and emits no AGY profile. ⏸️ untested no The live Typesafe resolver could not be called because TYPESAFE_API_KEY was absent. Provide the credential through the environment or an isolated FM_HOME/.env, then rerun.
Auth-required AGY rule floors remain unverifiable and cannot authorize AGY or cross-provider dispatch. ⏸️ untested no The live resolver path requires a Typesafe API credential, which was unavailable in this run.
A known AGY sub-scope cannot bypass auth_required state to authorize dispatch. ⏸️ untested no The real host snapshot had auth_required with no effective availability rows, so the known-sub-scope adversarial input could not be driven live. Provide a live auth-required snapshot containing a know…
  • bash tests/fm-dispatch-resolve.test.sh
  • bash tests/fm-quota-choose.test.sh
  • bin/fm-procevent-quota.sh poll --provider agy --timeout 1
  • bin/fm-procevent-quota.sh poll --timeout 1
  • bin/fm-quota-choose.sh --snapshot <live quota snapshot> --candidate agy:default
  • quota-axi --json | jq ...
  • git status --short
✅ **Document** - passed

✅ No issues found.

✅ **Lint** - passed

✅ No issues found.

✅ **Push** - passed

✅ No issues found.

irene and others added 18 commits September 19, 2026 08:04
…ate-20260919

feat: gate fresh CLAUDE.md pointer creation on Claude Code version >= 2.1.277
…tage 2) (#2)

* feat(bin): complete stage 2 CLAUDE.md pointer removal

* no-mistakes(review): Restore column-0 heredoc regression fixture with generic content

* no-mistakes(review): Remove stale CLAUDE.md pointer claim from updatefirstmate skill

---------

Co-authored-by: irene <irene@ireneui-MacBookPro.local>
Prior commits on this branch regressed past stage2, restoring the
unconditional CLAUDE.md pointer-write logic stage2 removed. Reset to
fork/main (stage2's merged head) and redo stage3 correctly: delete the
now-dead fm_version_at_least/claude_supports_native_agents_md functions
and their header-comment reference, and drop the now-vestigial
with_mock_claude/with_no_claude test helpers (the script no longer
reads claude --version at all).

Co-authored-by: irene <irene@ireneui-MacBookPro.local>
…eering skills (#4)

* feat: add lazy specialist tool routing

Expose ECC, paperthin, and ultrawork as captain-approved specialist paths while keeping Firstmate intake and lifecycle authority. Load only the selected skill or mode and keep ECC hooks, MCP, and legacy sync opt-in.

* docs(agents): recover firstmate-layout and task-steering skills

These two skills existed only on an orphaned local branch, never pushed.
firstmate-layout is re-extracted from AGENTS.md section 2's current
(much larger) layout tree rather than reusing the stale 2026-09-14
snapshot. task-steering's underlying AGENTS.md paragraph was byte-identical
to the 2026-09-14 extraction, so it is reused as-is. Both get a one-line
trigger in section 13 and a documentation-audiences.json entry, matching
how specialist-tools (recovered earlier on this branch) is registered.

---------

Co-authored-by: irene <irene@ireneui-MacBookPro.local>
Co-authored-by: irene <irene@ireneui-MacBookPro.local>
.treehouse/ holds only runtime pool bookkeeping (treehouse-state.json,
treehouse-state.lock), never captain work, but its absence from
.gitignore makes it show up as an untracked dirty-tree blocker for
bin/fm-update.sh's self-update fast-forward check.

Co-authored-by: irene <irene@ireneui-MacBookPro.local>
@Ivory2024 Ivory2024 closed this Sep 19, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant