Skip to content

fix(dashboard): send periodic WS heartbeat pings to stop live-dashboard reconnect churn - #10452

Merged
diegosouzapw merged 3 commits into
release/v3.8.50from
fix/10319-live-ws-heartbeat-ping
Aug 18, 2026
Merged

diegosouzapw merged 3 commits into
release/v3.8.50from
fix/10319-live-ws-heartbeat-ping

Conversation

@diegosouzapw

Copy link
Copy Markdown
Owner

Closes #10319

Root cause

Confirmed in two files (matches the analyzed plan _tasks/pipeline/bugs/2-implementing/10319-live-ws-client-never-sends-heartbeat-ping.plan.md):

  • Client (src/hooks/useLiveDashboard.ts): ws.onopen sent only { type: "subscribe", channels }. No setInterval/keepalive timer existed anywhere in the hook, so the client never emitted the protocol's declared { type: "ping" } heartbeat frame.
  • Server (src/server/ws/liveServer.ts): client.lastActivity was refreshed only from inbound client messages. startHeartbeat terminates any client whose inbound gap exceeds HEARTBEAT_TIMEOUT_MS (35s, hardcoded). The server's own outbound { type: "pong" } did not reset lastActivity (inbound-only), so it could not prevent the terminate.

Net effect: a healthy, connected-but-idle dashboard client (the normal case — the dashboard just watches, sends nothing) was force-terminated (ws.terminate(), abnormal close) roughly every 35-45s, and the browser auto-reconnected in a loop — exactly the reporter's symptom (wss://…/live-ws failed: The network connection was lost every ~35-45s).

Fix (both directions, per the plan's recommendation)

  1. Client — useLiveDashboard.ts now starts a 15s setInterval on open that sends { type: "ping" }, cleared on every exit path (close, unmount, disable, reconnect) to avoid leaked timers. This hook is the shared base for useLiveRequests, useLiveComboStatus, useLiveConnectionStatus.
  2. Server (defense in depth) — startHeartbeat now also bumps client.lastActivity when sending the outbound pong, so even a third-party/bespoke client that never pings is not dropped for being idle.

Regression tests

  • tests/unit/useLiveDashboard-heartbeat.test.tsx (vitest, fake timers): asserts the hook emits periodic { type: "ping" } frames after open, and that the interval is cleared on close and on unmount (no leaked timers).
  • tests/integration/live-ws-heartbeat-keepalive.test.ts (node:test, real WS server via scripts/start-ws-server.mjs): a subscribed-but-silent client must stay connected past the 35s HEARTBEAT_TIMEOUT_MS (~50s window) — the permanent version of the plan file's TDD RED repro, which failed before this fix (AssertionError: true !== false, connection was terminated at ~44s).

TDD RED→GREEN evidence is documented in the analyzed plan file (verdict: fix-confirmed, confidence: CONFIRMED). Pure protocol defect, not environment-specific — no VPS live validation required per the plan.

Gates run locally

  • npm run typecheck:core — GREEN
  • npx eslint --suppressions-location config/quality/eslint-suppressions.json <changed files> — GREEN, 0 errors
  • node scripts/check/check-file-size.mjs — no new violations on touched files
  • node scripts/check/check-complexity.mjs — GREEN (2456 vs baseline 2774)
  • node scripts/check/check-cognitive-complexity.mjs — GREEN (1104 vs baseline 1223)
  • node scripts/check/check-test-discovery.mjs — GREEN, new test files discovered by the runner

Note: npm run test:unit and the targeted npx vitest run tests/unit/useLiveDashboard-heartbeat.test.tsx could not be completed locally in this session — this devbox is running a 13-way parallel campaign fan-out and sustained a load average of ~150-240 on 16 cores for 30+ minutes; the local test:unit process measured 0.0% CPU over 26 minutes of wall time (confirmed via ps), and vitest's worker-pool spawn repeatedly hit its internal startup timeout under the same contention ([vitest-pool-runner]: Timeout waiting for worker to respond — an infra symptom, not a test assertion failure). Both new tests were manually verified line-by-line for correctness against the existing tests/integration/live-ws-startup.test.ts harness pattern and the hook's actual timer/cleanup logic. CI's test-unit and test-vitest jobs will run the authoritative suite on this PR.

⚠️ base-red inherited: #9985 — ESLint errors (2) from #10250 (i18n PT-PT translation), unrelated to this change.

…rd reconnect churn

The live-dashboard WS client (src/hooks/useLiveDashboard.ts) only sent a
subscribe frame on open and never emitted the protocol's { type: "ping" }
heartbeat. The server (src/server/ws/liveServer.ts) refreshes client
liveness only from inbound messages and terminates any client idle past
HEARTBEAT_TIMEOUT_MS (35s), so a healthy, connected-but-idle dashboard
client was force-terminated roughly every 35-45s, causing constant
reconnect churn (#10319).

Fix (both directions, per the analyzed plan):
- Client: start a 15s ping interval on open, cleared on close/unmount/
  reconnect, so the connection stays inside the server's liveness window.
- Server (defense in depth): the outbound heartbeat pong now also bumps
  client.lastActivity, so even a third-party client that never pings is
  not dropped for being idle.

Regression coverage:
- tests/unit/useLiveDashboard-heartbeat.test.tsx: fast fake-timer check
  that the hook emits periodic ping frames and cleans up the interval on
  close/unmount (no leaked timers).
- tests/integration/live-ws-heartbeat-keepalive.test.ts: real WS-server
  integration test asserting a silent-but-subscribed client stays
  connected past the 35s heartbeat timeout (~50s window), converted from
  the plan file's TDD RED repro.

Closes #10319
@diegosouzapw

Copy link
Copy Markdown
Owner Author

Code review

Found 1 important issue.

The new client.lastActivity = now at src/server/ws/liveServer.ts:435 runs for every socket whose local readyState is OPEN, immediately before the server sends its application-level { type: "pong" }. A half-open or black-holed TCP peer can remain locally OPEN without receiving or acknowledging that message, so the timeout at lines 425-429 is never reached. Such stale entries can accumulate until MAX_CLIENTS is exhausted and healthy dashboard connections receive SERVER_FULL. Refresh liveness only on inbound client activity, or after an actual WebSocket-level pong acknowledgement, and add a stale-open-socket regression test. This conclusion is static review evidence; no runtime test was run on the PR head.

diegosouzapw and others added 2 commits August 17, 2026 11:19
Keep application-level heartbeat responses from refreshing server liveness, and add a regression covering silent stale sockets alongside clients that answer protocol heartbeats.

Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>
@diegosouzapw
diegosouzapw merged commit a55daac into release/v3.8.50 Aug 18, 2026
7 checks passed
@diegosouzapw
diegosouzapw deleted the fix/10319-live-ws-heartbeat-ping branch August 19, 2026 18:47
maxmad64bis pushed a commit to maxmad64bis/OmniRoute that referenced this pull request Aug 19, 2026
startHeartbeat() only refreshed a client's liveness on inbound application
messages (diegosouzapw#10452), evicting a read-only subscriber that never sends anything.
Send a protocol-level ping (RFC 6455 5.5.2) on each tick; conformant clients
answer with a pong control frame automatically, refreshed in a new
ws.on("pong") handler. A socket that stopped reading still goes unanswered and
is reaped as before, so diegosouzapw#10452's protection is unchanged. The application-level
pong is kept for clients that still rely on it.

Extended the integration test with a third connection mode covering both
properties.
HouMinXi pushed a commit to HouMinXi/OmniRoute that referenced this pull request Aug 20, 2026
…rd reconnect churn (diegosouzapw#10452)

* fix(dashboard): send periodic WS heartbeat pings to stop live-dashboard reconnect churn

The live-dashboard WS client (src/hooks/useLiveDashboard.ts) only sent a
subscribe frame on open and never emitted the protocol's { type: "ping" }
heartbeat. The server (src/server/ws/liveServer.ts) refreshes client
liveness only from inbound messages and terminates any client idle past
HEARTBEAT_TIMEOUT_MS (35s), so a healthy, connected-but-idle dashboard
client was force-terminated roughly every 35-45s, causing constant
reconnect churn (diegosouzapw#10319).

Fix (both directions, per the analyzed plan):
- Client: start a 15s ping interval on open, cleared on close/unmount/
  reconnect, so the connection stays inside the server's liveness window.
- Server (defense in depth): the outbound heartbeat pong now also bumps
  client.lastActivity, so even a third-party client that never pings is
  not dropped for being idle.

Regression coverage:
- tests/unit/useLiveDashboard-heartbeat.test.tsx: fast fake-timer check
  that the hook emits periodic ping frames and cleans up the interval on
  close/unmount (no leaked timers).
- tests/integration/live-ws-heartbeat-keepalive.test.ts: real WS-server
  integration test asserting a silent-but-subscribed client stays
  connected past the 35s heartbeat timeout (~50s window), converted from
  the plan file's TDD RED repro.

Closes diegosouzapw#10319

* fix(dashboard): stop renewing stale LiveWS sockets

Keep application-level heartbeat responses from refreshing server liveness, and add a regression covering silent stale sockets alongside clients that answer protocol heartbeats.

Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>

---------

Co-authored-by: adevwithpurpose <adevwithpurpose@users.noreply.github.com>
giauphan pushed a commit to giauphan/OmniRoute that referenced this pull request Aug 20, 2026
…rd reconnect churn (diegosouzapw#10452)

* fix(dashboard): send periodic WS heartbeat pings to stop live-dashboard reconnect churn

The live-dashboard WS client (src/hooks/useLiveDashboard.ts) only sent a
subscribe frame on open and never emitted the protocol's { type: "ping" }
heartbeat. The server (src/server/ws/liveServer.ts) refreshes client
liveness only from inbound messages and terminates any client idle past
HEARTBEAT_TIMEOUT_MS (35s), so a healthy, connected-but-idle dashboard
client was force-terminated roughly every 35-45s, causing constant
reconnect churn (diegosouzapw#10319).

Fix (both directions, per the analyzed plan):
- Client: start a 15s ping interval on open, cleared on close/unmount/
  reconnect, so the connection stays inside the server's liveness window.
- Server (defense in depth): the outbound heartbeat pong now also bumps
  client.lastActivity, so even a third-party client that never pings is
  not dropped for being idle.

Regression coverage:
- tests/unit/useLiveDashboard-heartbeat.test.tsx: fast fake-timer check
  that the hook emits periodic ping frames and cleans up the interval on
  close/unmount (no leaked timers).
- tests/integration/live-ws-heartbeat-keepalive.test.ts: real WS-server
  integration test asserting a silent-but-subscribed client stays
  connected past the 35s heartbeat timeout (~50s window), converted from
  the plan file's TDD RED repro.

Closes diegosouzapw#10319

* fix(dashboard): stop renewing stale LiveWS sockets

Keep application-level heartbeat responses from refreshing server liveness, and add a regression covering silent stale sockets alongside clients that answer protocol heartbeats.

Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>

---------

Co-authored-by: adevwithpurpose <adevwithpurpose@users.noreply.github.com>
muhamadgalihsaputra pushed a commit to niyatna/NiyatnaRoute that referenced this pull request Sep 27, 2026
…rd reconnect churn (diegosouzapw#10452)

* fix(dashboard): send periodic WS heartbeat pings to stop live-dashboard reconnect churn

The live-dashboard WS client (src/hooks/useLiveDashboard.ts) only sent a
subscribe frame on open and never emitted the protocol's { type: "ping" }
heartbeat. The server (src/server/ws/liveServer.ts) refreshes client
liveness only from inbound messages and terminates any client idle past
HEARTBEAT_TIMEOUT_MS (35s), so a healthy, connected-but-idle dashboard
client was force-terminated roughly every 35-45s, causing constant
reconnect churn (diegosouzapw#10319).

Fix (both directions, per the analyzed plan):
- Client: start a 15s ping interval on open, cleared on close/unmount/
  reconnect, so the connection stays inside the server's liveness window.
- Server (defense in depth): the outbound heartbeat pong now also bumps
  client.lastActivity, so even a third-party client that never pings is
  not dropped for being idle.

Regression coverage:
- tests/unit/useLiveDashboard-heartbeat.test.tsx: fast fake-timer check
  that the hook emits periodic ping frames and cleans up the interval on
  close/unmount (no leaked timers).
- tests/integration/live-ws-heartbeat-keepalive.test.ts: real WS-server
  integration test asserting a silent-but-subscribed client stays
  connected past the 35s heartbeat timeout (~50s window), converted from
  the plan file's TDD RED repro.

Closes diegosouzapw#10319

* fix(dashboard): stop renewing stale LiveWS sockets

Keep application-level heartbeat responses from refreshing server liveness, and add a regression covering silent stale sockets alongside clients that answer protocol heartbeats.

Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>

---------

Co-authored-by: adevwithpurpose <adevwithpurpose@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[BUG] Live dashboard WS client never sends heartbeat ping — server terminates every ~35s

2 participants