fix(desktop): raise remote liveness probe timeout from 2.5s to 10s - #49841
Open
GratefulDave wants to merge 1 commit into
Open
fix(desktop): raise remote liveness probe timeout from 2.5s to 10s#49841GratefulDave wants to merge 1 commit into
GratefulDave wants to merge 1 commit into
Conversation
…ousResearch#49787) The hermes:connection:revalidate IPC handler probes /api/status with a hardcoded 2500ms timeout. This is too tight for remote backends reached via high-latency paths (e.g. Tailscale relay): the probe races to timeout even while the backend is healthy, causing: Remote Hermes backend is ready → Cached remote Hermes backend failed liveness probe; dropping stale connection. → startHermes() rebuilds the connection from scratch → immediately revalidates on the next wake/focus event → loop The probe fires on every sleep/wake (onPowerResume), network online event, and window visibilitychange — so on a Tailscale-connected remote the loop runs continuously, showing the boot progress overlay repeatedly and never settling. Fix: raise the revalidate timeout to 10_000ms, matching the timeout already used for the connection test at line 4437. A genuinely dead backend still fails within 10s; a live backend over a high-latency relay no longer gets false-evicted. Verified: Tailscale round-trip on the affected setup is ~223ms average with occasional spikes. 10s gives ample headroom without delaying detection of a truly unreachable backend. Fixes NousResearch#49787
teknium1
reviewed
Jul 14, 2026
teknium1
left a comment
Contributor
There was a problem hiding this comment.
Thanks for isolating the remote liveness failure mode. The current implementation still has the reported behavior: apps/desktop/electron/main.ts:7405 probes /api/status with timeoutMs: 2_500, and its catch resets the remote connection at apps/desktop/electron/main.ts:7408-7415.
Problems
- The diff modifies
apps/desktop/electron/main.cjs, but current main renamed that file toapps/desktop/electron/main.tsin39d09453f95e8aefc0c97e5d9b30ff341cae9ed8. The live handler is therefore unchanged by this PR as submitted. - There is no focused regression test for
hermes:connection:revalidate; repository searches find the production handler and stale-probe log, but no corresponding test.
Suggested changes
- Apply the
10_000timeout atapps/desktop/electron/main.ts:7405. - Add a focused test for a delayed-but-successful remote status probe and for a genuinely failed probe resetting the cached remote connection.
Automated hermes-sweeper review.
| const base = conn.baseUrl.replace(/\/+$/, '') | ||
| try { | ||
| await fetchPublicJson(`${base}/api/status`, { timeoutMs: 2_500 }) | ||
| await fetchPublicJson(`${base}/api/status`, { timeoutMs: 10_000 }) |
Contributor
There was a problem hiding this comment.
Current main renamed this source file to apps/desktop/electron/main.ts in 39d09453f95e8aefc0c97e5d9b30ff341cae9ed8; the live handler is now at main.ts:7405 and still uses 2_500. Port this change to that handler so the fix reaches current main.
19 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes the connect → immediately-stale → reconnect loop reported in #49787 for remote backends reached via high-latency paths (Tailscale relay, cross-region VPN).
Root cause
hermes:connection:revalidateprobes/api/statuswith a hardcoded2_500ms timeout:This fires on every sleep/wake (
onPowerResume), network online event, andvisibilitychange. On a Tailscale-connected remote, relay latency can intermittently exceed 2.5s — causing the probe to time out and drop the cached connection as stale even when the backend is perfectly healthy. The result is an infinite loop:This is distinct from the bootstrap marker issue also reported in #49787 (setup screen on every launch), which has a separate fix.
Fix
Raise the revalidate timeout to
10_000ms, matching the timeout already used for the connection test (fetchPublicJsonat line 4437):A genuinely dead backend still fails within 10s. A live backend over a high-latency relay no longer gets false-evicted.
Verification
GET /api/statusreturns HTTP 200 with valid JSON when probed directly; Let's Encrypt cert, no TLS issuestimeoutMs: 2_500: desktop.log shows repeatedCached remote Hermes backend failed liveness probewithin seconds of eachRemote Hermes backend is ready— confirmed via 8–10 loop iterations per sessiontimeoutMs: 10_000: probe succeeds reliably; loop stopsRelated