Skip to content

test(web-server): deflake pub event broadcast coverage - #37359

Closed
hanzckernel wants to merge 2 commits into
NousResearch:mainfrom
hanzckernel:fix/webserver-events-broadcast-test-flake
Closed

test(web-server): deflake pub event broadcast coverage#37359
hanzckernel wants to merge 2 commits into
NousResearch:mainfrom
hanzckernel:fix/webserver-events-broadcast-test-flake

Conversation

@hanzckernel

Copy link
Copy Markdown
Contributor

Summary

  • deflake the dashboard pub/events websocket broadcast regression test
  • split the previous two-websocket TestClient scenario into deterministic contract checks:
    • /api/pub forwards publisher frames to _broadcast_event
    • _broadcast_event fans out a frame to all registered channel subscribers

Why

The old test opened /api/events and /api/pub websockets at the same time through Starlette's TestClient, then waited for the subscriber client to receive the forwarded frame. Under load this can time out even though the production split points are simple: /api/pub receives a frame and calls _broadcast_event, while _broadcast_event sends the payload to each registered subscriber.

The replacement keeps those two behavior contracts covered without depending on a racy multi-websocket TestClient receive path.

Scope

Changed file:

  • tests/hermes_cli/test_web_server.py

No production code changes.

Verification

  • original test reproduced locally as flaky: failed once during repeated runs
  • new /api/pub forwarding test: passed 30 repeated runs
  • new _broadcast_event fanout test: passed 30 repeated runs
  • python -m pytest tests/hermes_cli/test_web_server.py::TestPtyWebSocket -q -o 'addopts='
    • 13 passed
  • python -m py_compile tests/hermes_cli/test_web_server.py
  • git diff --check
  • independent review: approved with no blockers

Related

Related to the failing test (2) CI shard observed on #37350.

@alt-glitch alt-glitch added type/test Test coverage or test infrastructure P3 Low — cosmetic, nice to have comp/cli CLI entry point, hermes_cli/, setup wizard labels Jun 2, 2026
…st-main

# Conflicts:
#	tests/hermes_cli/test_web_server.py
@hanzckernel

Copy link
Copy Markdown
Contributor Author

Closing this as obsolete after re-checking current main.

The intent here was to deflake the pub-event broadcast coverage by separating the websocket fanout contract from the /api/pub forwarding path. Current main already has the durable version of that coverage: the test now exercises _broadcast_event fanout directly, and the production /api/pub path is just a thin forwarder into that helper.

So this PR no longer carries enough incremental value to keep open. Thanks — this was the right direction, but the underlying test shape has since been absorbed on main.

@hanzckernel
hanzckernel deleted the fix/webserver-events-broadcast-test-flake branch July 23, 2026 10:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/cli CLI entry point, hermes_cli/, setup wizard P3 Low — cosmetic, nice to have type/test Test coverage or test infrastructure

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants