fix(simplex): escalate non-retryable WebSocket handshake rejections instead of looping forever - #35557
Conversation
The SimpleX adapter's reconnect loop caught a rejected WebSocket handshake (websockets InvalidStatus, a WebSocketException subclass) in the generic reconnect arm and busy-looped forever with no gateway-visible fatal state — so a permanently misconfigured daemon (bad path/auth/4xx) never surfaced an error. Add a specific arm in _ws_listener that catches the handshake-rejection exception and reads the HTTP status: - 4xx: permanent, call _set_fatal_error(..., retryable=False) and break. - 5xx/unknown: left to the existing reconnect path (possibly transient). A _handshake_status_code helper reads the status from the newer InvalidStatus (.response.status_code) or the older InvalidStatusCode (.status_code); the import is guarded for cross-version support. Transient errors (ConnectionClosed etc.) keep retrying as before. Adds tests asserting a 4xx escalates to a non-retryable fatal and exits, while a 5xx is retried.
tonydwb
left a comment
There was a problem hiding this comment.
Code Review Summary
Verdict: Approved ✅
Review Findings
This PR escalates non-retryable 4xx WebSocket handshake rejections to a fatal error state instead of busy-looping forever. Previously, a permanently misconfigured daemon (wrong path, auth failure) would cause infinite reconnection attempts with no operator signal.
✅ Looks Good
- Design: Clear distinction between 4xx (permanent — fatal escalation) and 5xx/unknown (transient — keep retrying). This matches the adapter pattern used by IRC and Mattermost.
_handshake_status_codehelper: Handles both websockets >= 14 (InvalidStatus.response.status_code) and olderInvalidStatusCode.status_code— good cross-version compatibility.- Import guard: The try/except import for
InvalidStatus/InvalidStatusCodeis clean and maintainable. - Error message:
"SimpleX daemon rejected WebSocket handshake (HTTP {status})"— informative and actionable. - Tests:
test_ws_listener_escalates_4xx_handshake_rejection— patcheswebsockets.connectwith a realInvalidStatus(403), verifies_set_fatal_errorcalled exactly once withretryable=False, and usesasyncio.wait_for(timeout=5)to catch the old busy-loop behavior.test_ws_listener_retries_5xx_handshake_rejection— verifies 503 keeps retrying without escalation, guarding against over-broad escalation.
- Diff: 157 additions, 0 deletions — all new code, no risk of regression.
No Issues Found
Reviewed by Hermes Agent
tonydwb
left a comment
There was a problem hiding this comment.
Code Review Summary
Verdict: Approved ✅
Review
Fixes SimpleX WebSocket handshake rejection causing an infinite reconnect loop. 4xx handshake rejections (permanent misconfiguration) now escalate to fatal error instead of retrying forever.
✅ Looks Good
- Correct escalation: 4xx → fatal, non-retryable. 5xx → still retried as potentially transient.
- Version-safe: Handles both newer
InvalidStatusand olderInvalidStatusCodefromwebsocketslibrary. - Good test coverage: Two focused async tests — 4xx escalates, 5xx retries.
- Prevents operator-blind failures: Gateway-visible fatal state means operators get a signal instead of silent busy-loop.
Reviewed by Hermes Agent (cron job)
24b2cc5 to
47a3f3f
Compare
teknium1
left a comment
There was a problem hiding this comment.
Thanks for isolating a real reconnect-loop gap. Current main still catches a rejected InvalidStatus in the generic retry arm at plugins/platforms/simplex/adapter.py:332-344; a controlled 403 repro logged a reconnect and left fatal_error unset.
Problems
plugins/platforms/simplex/adapter.py:284treats every 4xx as permanently fatal. The PR proves 403, but the condition covers the entire 400–499 range. Existing auth-retry coverage is explicit about terminal 401/403 (tests/gateway/test_ws_auth_retry.py:21-77); please define the SimpleX-specific terminal set and preserve retry for transient or ambiguous 4xx cases.- The new 4xx test mocks
_set_fatal_error(tests/gateway/test_simplex_plugin.py:365-375), so it does not verify the gateway-visible state produced byBasePlatformAdapter._set_fatal_error()(gateway/platforms/base.py:2705-2710).
Suggested changes
- Add a retry regression test for a non-terminal 4xx status alongside 403 and 503.
- Exercise the real fatal setter while mocking its status writer, then assert the fatal fields and
_runningstate.
Automated hermes-sweeper review.
| # busy-looping forever. 5xx may be transient, so keep | ||
| # retrying those via the generic path below. | ||
| status = _handshake_status_code(e) | ||
| if status is not None and 400 <= status < 500: |
There was a problem hiding this comment.
This makes every 4xx terminal, although the demonstrated permanent case is 403. Please use an explicit SimpleX-specific terminal-status set and add coverage for a transient or ambiguous 4xx response so this does not turn a recoverable rejection into a permanently disabled adapter.
| fatal_calls = [] | ||
| monkeypatch.setattr( | ||
| adapter, | ||
| "_set_fatal_error", |
There was a problem hiding this comment.
Mocking _set_fatal_error verifies the call but bypasses the gateway-visible state this PR is intended to establish. Prefer mocking _write_runtime_status_safe and assert the real setter leaves _running false and records the expected fatal fields.
Summary
When the simplex-chat daemon rejects the WebSocket handshake with an HTTP status (e.g. a permanently misconfigured endpoint, wrong path, or auth failure), the
websocketsclient raisesInvalidStatus. BecauseInvalidStatusis a subclass ofWebSocketException, the SimpleX adapter's reconnect loop (_ws_listener) caught it in the generic reconnect arm and busy-looped forever with exponential backoff — never surfacing a gateway-visible fatal state, so operators got no signal that the connection could never succeed.This adds a specific arm that catches the handshake-rejection exception and inspects the HTTP status:
_set_fatal_error("handshake_rejected", ..., retryable=False)and break out of the reconnect loop, matching the fatal-error convention used by the base adapter and siblings (e.g. IRC, Mattermost's permanent-auth-failure stop).Transient errors (
ConnectionClosed, otherWebSocketExceptions) keep retrying exactly as before.A small helper,
_handshake_status_code, reads the status from either the newerInvalidStatus(.response.status_code) or the olderInvalidStatusCode(.status_code), and the import is guarded so the adapter works acrosswebsocketsversions.Tests
Added two focused async tests to
tests/gateway/test_simplex_plugin.py:test_ws_listener_escalates_4xx_handshake_rejection— patcheswebsockets.connectto raise a realInvalidStatus(403), runs_ws_listener, and asserts_set_fatal_erroris called exactly once withretryable=Falseand the loop exits. It wraps the listener inasyncio.wait_for(..., timeout=5)so the old busy-loop behavior fails fast instead of hanging.test_ws_listener_retries_5xx_handshake_rejection— a 503 rejection is retried (loop does not escalate), guarding against over-broad escalation.The 4xx test is red on current main (the handshake rejection falls into the generic reconnect arm and the loop never exits, tripping the 5s timeout) and green with this change. The 5xx test passes both before and after, confirming transient behavior is preserved.
(One unrelated test in the file,
test_standalone_send_missing_url, is environment-dependent and is not affected by this change; where it fails, it fails identically on cleanmain.)Notes
websockets15.0.1 the rejection class isInvalidStatus(exposing.response.status_code); the code and tests also handle the olderInvalidStatusCodealias for portability across versions._ws_listenerregion (narrowing the generic catch fromWebSocketExceptiontoConnectionClosedplus a bareexcept Exception), but it adds no handshake-rejection / 4xx fatal handling — so even under feat(simplex): groups, native attachments, text batching, auto-accept #27978 a 4xxInvalidStatuswould still fall through and loop forever. This fix is not a duplicate; it stands on its own against currentmain, and if feat(simplex): groups, native attachments, text batching, auto-accept #27978 lands first it slots into the restructured loop as a small mechanical rebase. No other open SimpleX PR (fix(simplex): make websocket handling reliable #26480, fix(simplex): avoid reconnecting healthy idle WebSocket #27120, fix(simplex): make polling recovery visible and bounded #27415, Fix(gateway): 3 bugs blocking bidirectional simpleX messaging #26433, fix(simplex): propagate WebSocket send failures to SendResult #27628) touches handshake/fatal escalation.