ci(client-auto): assert the node actually joined a mesh - #598
Merged
Conversation
The client-auto CI step previously only verified that the local management API binds. It treated 'no peers / no mesh joined' as inconclusive and exited 0, which meant a silent regression in Nostr discovery, gossip, or the join handshake would not fail CI as long as the console port came up. Now poll /api/status for up to 60s after the API is reachable and require mesh_id set AND (peers non-empty OR first_joined_mesh_ts set) before passing. Fail loudly with the last /api/status body if that signal never appears, and fail (not pass) when the binary exits with 'no meshes found'. This couples the CI step to public-mesh availability on purpose: if the public mesh is down or discovery is broken, that's something we want to know about, not silently pass over.
Contributor
There was a problem hiding this comment.
Pull request overview
Updates the CI “Client-auto boot test” to assert that mesh-llm client --auto not only brings up the console API, but also successfully joins a public mesh (via /api/status signals), so discovery/gossip/join regressions fail CI instead of being treated as inconclusive.
Changes:
- Removes the prior “no-mesh grace / inconclusive” early-success path when discovery finds no meshes.
- Treats “No meshes found after …” process exit as a hard CI failure.
- After the console API is reachable, polls
/api/statusfor up to 60s and requires evidence of a mesh join (mesh_id+ (peersnon-empty ORfirst_joined_mesh_tsset)).
Comments suppressed due to low confidence (1)
scripts/ci-client-auto-test.sh:133
- The comment says the join poll "falls back to the existing 'inconclusive' path" when the public mesh has no reachable peers, but the current logic always fails after JOIN_WAIT if no join signal appears. Please update the comment and the polling log message to match the actual requirement (mesh_id set AND (peers non-empty OR first_joined_mesh_ts set)) so future readers aren’t misled.
# We poll for up to JOIN_WAIT seconds. If the public mesh has no reachable
# peers in this CI run, we fall back to the existing "inconclusive" path.
JOIN_WAIT=60
echo "Polling /api/status for a join signal (mesh_id + peers, up to ${JOIN_WAIT}s)..."
JOINED=false
The earlier draft of this script treated 'no public mesh discovered' as inconclusive and exited 0. When that fallback was removed, the comment above the join-polling loop and the polling log message were not updated and still referred to the old behavior. Update the comment to state the real predicate (mesh_id set AND (peers non-empty OR first_joined_mesh_ts set)) and make explicit that there is no inconclusive fallback \u2014 if the predicate is never satisfied, the step fails. Also update the polling log line so it matches the actual signal being checked. Spotted by copilot-pull-request-reviewer on #598.
ndizazzo
approved these changes
May 20, 2026
ndizazzo
left a comment
ndizazzo
left a comment
Collaborator
There was a problem hiding this comment.
Looks fine, but a small edge case is that it might give a false positive... If no public mesh is found, the client might fall back to creating/using its own standalone mesh
* main: docs(AGENTS): add confidence-testing recipe for routing/MoA/gossip changes (#613) ci(sdk-smoke): install lld in macOS swift smoke job (#610) MoA: mesh mode and many inference critical fixes, and quic keep alive (#566) fix(ci): small update for lint rule (#608) mockup: Reserves high-fidelity UI mockup (#560) Add advisory capacity evaluation for model targets (#579) Harden Skippy layer package materialization cache (#583) fix(release): pin Windows CUDA to sccache-compatible version (#606) fix(build-windows): tolerate dead sccache server in CUDA retry path (#604) chore(version): synchronize version bump everywhere (#562) fix(mesh): skip filtered peers in gossip dial loop to unwedge `--auto` (#602) docs(agents): clarify just build vs release-build for serious testing (#599) build: ozempic — slim binary -42 MB / -47 MB (#592)
Collaborator
Author
|
yeah @ndizazzo good point - I am actually not sure what I am really testing here... so will try one more time and close it if not sure. |
…d_mesh_ts The previous predicate accepted mesh_id set AND (peers non-empty OR first_joined_mesh_ts set). That second disjunct is satisfied by the standalone-fallback path: when client --auto finds no candidates via Nostr discovery, run_auto_start_new_mesh generates a fresh local mesh_id and sets first_joined_mesh_ts to now with zero peers. The test would pass green while the node is talking to nobody — exactly the regression this step is meant to catch. Tighten the predicate to mesh_id set AND peers non-empty. The poll loop already waits up to 60s and fails closed, so this becomes "at least one peer within 60s or fail". Updates the comment block and failure message to spell out why first_joined_mesh_ts is no longer accepted.
michaelneale
added a commit
that referenced
this pull request
May 22, 2026
…-grace * origin/main: ci(client-auto): assert the node actually joined a mesh (#598)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changed
The
Client-auto boot testCI step now requires the node to actually join the public mesh, not just bind its console port.Before this change, the step passed as long as
http://localhost:3132/api/statusreturned 2xx. It explicitly treated "no public mesh discovered" and "node exited with 'no meshes found'" as inconclusive and exited 0. That meant a silent regression in Nostr discovery, gossip, or the join handshake would not fail CI \u2014 the script was effectively a console-bind smoke, not a public-mesh smoke.After this change, once the management API is reachable, the script polls
/api/statusfor up to 60s and requires:mesh_idset, andpeersnon-empty orfirst_joined_mesh_tsset.If that signal never appears, the step fails with the last
/api/statusbody for diagnosis. A "No meshes found after" process exit is now also a hard failure rather than an early pass.Why
The earlier behavior masked exactly the kind of break it was supposed to catch. If we want a CI signal that
client --autoworks end-to-end against the public mesh, the only honest version is one that fails when the join never happens.Tradeoff
This couples the PR/main CI lane to public-mesh availability. If meshllm.cloud / Nostr discovery is genuinely down, this step will fail until it recovers. That's intentional \u2014 we want to know about that, not silently pass over it.
Diagnostics on failure
On failure, the step prints:
No meshes found yet(discovery returned nothing)/api/statusbody, indented for readabilityFiles
scripts/ci-client-auto-test.shonly. No workflow YAML changes needed \u2014.github/workflows/ci.yml:330and.github/workflows/pr_builds.yml:395already invoke this script.