fix(auth): serialize concurrent xAI OAuth write-through to global root - #47460
fix(auth): serialize concurrent xAI OAuth write-through to global root#47460sprmn24 wants to merge 1 commit into
Conversation
teknium1
left a comment
There was a problem hiding this comment.
Thanks for targeting a real current-main race in the xAI root write-through path.
Problems
- The new lock covers only
hermes_cli/auth.py:4041-4054. The credential-pool companion path still updates the same global file without a lock atagent/credential_pool.py:529-537, and calls it forxai-oauthatagent/credential_pool.py:975-995. A pool refresh can still race this path. - The added assertion at
tests/hermes_cli/test_xai_oauth_writethrough.py:229accepts either complete final token set. Since_save_auth_storealready atomically replaces a complete JSON payload (hermes_cli/auth.py:1137-1153), the unlocked last-writer-wins implementation also satisfies that assertion.
Suggested changes
- Share one global-root auth-store locking helper between both write-through paths.
- Add a deterministic cross-path concurrency test that proves independent root-store updates are preserved, or directly instruments critical-section exclusion.
Automated hermes-sweeper review.
| return | ||
| _store_provider_state(global_store, "xai-oauth", dict(state), set_active=False) | ||
| _save_auth_store(global_store, global_path) | ||
| with _file_lock( |
There was a problem hiding this comment.
This lock protects only the non-pool helper. agent/credential_pool.py:529-537 performs the same global-root read-modify-write for xai-oauth without taking this sidecar lock, so a pool refresh can still race this code. Please share the global-root locking mechanism with that helper.
|
|
||
| token_a = {"access_token": "token-A-access", "refresh_token": "token-A-refresh"} | ||
| token_b = {"access_token": "token-B-access", "refresh_token": "token-B-refresh"} | ||
| assert final_tokens == token_a or final_tokens == token_b, ( |
There was a problem hiding this comment.
This accepts the pre-fix last-writer-wins result as well: _save_auth_store already atomically replaces a complete JSON payload. Please make the test observe mutual exclusion or assert preservation of independent concurrent root-store updates.
|
Closing as superseded: PR #65656 (salvage of #65264 by @dschnurbusch, merged today) generalized this fix — global-root write-through is now serialized under a lock keyed to the target store's canonical path via |
What does this PR do?
_write_through_xai_oauth_to_global_root()syncs a freshly-refreshed xAI OAuth token set from a profile store back to the global rootauth.json. The function held the profile-scoped file lock while doing so, but performed the read-modify-write on the globalauth.jsonwithout acquiring any lock on that file.When two profiles refresh their xAI tokens concurrently:
auth.json(both seerefresh_token_old)refresh_token_A→ global filerefresh_token_B→ global file (overwrites A's token)refresh_token_Abut the file now hasrefresh_token_Binvalid_grant→ forced re-authenticationRelated Issue
Adjacent to the already-fixed concurrent refresh race in #10147 / #15120 (Nous OAuth). Same class of bug, new code path.
Type of Change
Changes Made
hermes_cli/auth.py: Added_global_root_lock_holder = threading.local()(separate from_auth_lock_holderto prevent_file_lock's reentrancy depth counter from silently skipping the global lock when called from within a profile lock). Wrapped the entire read-modify-write in_write_through_xai_oauth_to_global_root()inside_file_lock(global_path.with_suffix(".lock"), _global_root_lock_holder, AUTH_LOCK_TIMEOUT_SECONDS, ...), matching the.locksidecar convention used by_auth_lock_path().tests/hermes_cli/test_xai_oauth_writethrough.py: Addedtest_concurrent_write_through_does_not_lose_second_token— fires two threads simultaneously viathreading.Barrier(2), asserts the final globalauth.jsonholds exactly one complete token set (not a corrupted mix ofaccess_tokenfrom A andrefresh_tokenfrom B).How to Test
All 5 tests pass.
Checklist
pytest tests/ -qand all tests pass