Conversation
…teway route Fixes NousResearch#92265 (proposed fix NousResearch#2; #1 and NousResearch#4 are separate follow-ups, see below). ensureGatewayForAgent() and ensureGatewayForProfile() both decided whether a secondary activation "succeeded" by checking Boolean(entry.connection) alone. entry.connection is set in openSecondary() BEFORE the WebSocket dial completes (`entry.connection = conn` happens ahead of `await entry.gateway.connect(wsUrl)`), so a transient first-dial failure -- caught by the surrounding try/catch and left for scheduleReconnect's backoff retry -- still left entry.connection truthy. Both functions then treated this as a successful activation: applyActive() switched g.activeKey and published $gateway to the closed socket, and publishActiveConnection() pushed the connection descriptor to the UI. The next chat RPC then failed with "Hermes gateway is not connected" against a route the user/desktop believed was live. Added an isOpen(entry.gateway) check alongside the existing Boolean(entry.connection) check in both functions' activation/publish conditions, gating BOTH applyActive() (which switches g.activeKey and publishes $gateway) and publishActiveConnection() (which pushes the connection descriptor) on the socket having actually reached 'open'. A failed first dial now correctly returns false / leaves the previous active route untouched, matching option 3 from the issue's own proposed fix ("if both bounded attempts fail, keep the existing active route") -- the existing scheduleReconnect backoff still owns recovery for that entry going forward. Not implemented in this PR (separate, lower-priority follow-ups): - Proposed fix #1 (one immediate bounded reconnect attempt before returning activation status) -- a larger behavioral change with its own retry/timing tradeoffs; left to a separate PR. - Proposed fix NousResearch#4 (Bot Mode's own connection-ID-only guard in plugins/hermes-bots/plugin.js) -- host.ensureAgent() calls into the now-fixed gateway.ts functions, so this class of bug is already closed at the root; Bot Mode's own additional profile/state verification may still be worth adding but is a separate, narrower hardening pass on top of this fix. Found and fixed a genuine test-suite inconsistency while verifying: the existing "refreshes the active connection after a pooled profile reconnect succeeds" test in gateway-shared-remote.test.ts asserted setConnection was called once after a SINGLE ensureGatewayForProfile() call whose first dial failed -- i.e. it encoded the exact bug this issue reports as the EXPECTED, correct behavior. Rewrote it to assert the corrected contract: the failed first attempt does not call setConnection at all, and a realistic retry (calling ensureGatewayForProfile() again, since g.activeKey correctly never left the primary after the failed attempt -- ensureActiveGatewayOpen() is for reconnecting an already-active gateway that went stale, not retrying an activation that never succeeded) succeeds and publishes once the second dial goes through. Added a new test file (gateway-secondary-open-check.test.ts) following the established mocking pattern from gateway-agent-scope.test.ts, covering both ensureGatewayForAgent and ensureGatewayForProfile: a transient first-dial failure does not activate/publish (the exact reported symptom), and a successful dial still activates/publishes normally (sanity, no regression to the happy path). Verified as genuine regressions by reverting both isOpen() checks and confirming 2 of 4 new tests fail with exactly the reported symptom (activated resolves true / the primary gets replaced despite the failed dial). 44/44 pass across all 9 gateway-related test files (no regression).
Contributor
Reviewed What's good
Suggestions
Precise, well-tested race fix. |
Collaborator
|
Merged via #95082 at b52b05c — your commit was cherry-picked into the consolidated salvage PR with authorship preserved, so this work carries your name in git history. Thanks for requiring an open socket before publishing a secondary gateway route — the tightest-scoped fix of the #92265 swarm, which is why it was the one we took. Closing this original PR now that the consolidated branch has landed on main. Thanks for contributing to the desktop multi-gateway campaign (tracker: #94724). |
This was referenced Aug 26, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #92265 (proposed fix #2; #1 and #4 are separate follow-ups, see below).
Root cause
ensureGatewayForAgent()/ensureGatewayForProfile()decided activation success by checkingBoolean(entry.connection)alone.entry.connectionis set inopenSecondary()before the WebSocket dial completes, so a transient first-dial failure -- caught and left forscheduleReconnect's backoff -- still leftentry.connectiontruthy. Both functions then published the closed socket as the active route, and the next chat RPC failed with "Hermes gateway is not connected".Fix
Added an
isOpen(entry.gateway)check alongside the existing check in both functions, gating bothapplyActive()(switchesg.activeKey, publishes$gateway) andpublishActiveConnection()on the socket having actually reachedopen. A failed first dial now correctly returnsfalse/ leaves the previous active route untouched, matching option 3 from the issue's proposed fix.Not implemented (separate follow-ups): proposed fix #1 (bounded reconnect before returning status) is a larger behavioral change left to a separate PR; proposed fix #4 (Bot Mode's own connection-ID-only guard) is unnecessary at the root now since
host.ensureAgent()calls into these now-fixed functions, though additional hardening there may still be worthwhile separately.Found while verifying
An existing test in
gateway-shared-remote.test.tsencoded the exact bug as EXPECTED behavior (assertingsetConnectionwas called once after a single call whose first dial failed). Rewrote it to assert the corrected contract: the failed attempt doesn't publish, and a realistic retry succeeds once the second dial goes through.Verification
Added a new test file covering both functions. Verified as genuine regressions by reverting both
isOpen()checks and confirming 2/4 new tests fail with exactly the reported symptom.44/44 pass across all 9 gateway-related test files (no regression).