fix(mesh): skip filtered peers in gossip dial loop to unwedge --auto

[michaelneale]

What

mesh-llm serve --auto model load wedged intermittently on the live public mesh: process up, mesh chatter visible, but no LaunchPlan and no model load. After this PR, --auto is reliable AND we keep PR #576's anti-spam (ghost peers stay out of the local table, routing, UI).

This supersedes PR #601 (which worked around the wedge by disabling PR #576's filters entirely — that left ~640 ghost peers in the table).

Why --auto wedges

attempt_run_auto_join → join_with_retry → connect_to_peer(invite_peer)
  → initiate_gossip (30s timeout wraps the WHOLE continuation including
                     the dial loop below)
    → apply_gossip_announcements
      → apply_announced_peers (PR #576 filters remove ghosts here)
      → for each ann in their_announcements:
          maybe_connect_discovered_peer(addr)
            → connect_to_peer(addr)
              → if state.peers.contains_key(&peer_id) { return Ok(()) }
              → else: 30s connect timeout per unreachable host

connect_to_peer's fast-path keys on state.peers.contains_key. PR #576's filter removes filtered peers from state.peers before the dial loop, so the fast-path no longer fires — and each unreachable ghost gets a real 30s connect timeout, sequentially. With ~445 v0.57.x ghosts in a payload, initiate_gossip is pinned for hours, attempt_run_auto_join never returns, run_auto never reaches LaunchPlan.

Fix

Apply the same filter predicates inside maybe_connect_discovered_peer before the dial. 18-line skip, single call site, no change to the ingest filters.

Validation

Local 4-run reliability test (serve --auto --model Qwen/Qwen2.5-3B-Instruct-GGUF:qwen2.5-3b-instruct-q4_k_m):

Run	loaded	ready	peers	inference
1	✅	✅	37	Surething
2	✅	✅	43	Sure thing.
3	✅	✅	44	Sure thing.
4	✅	✅	43	Surething.

Peer table stays clean (~40 peers, all v0.60+, no v0.57.x ghosts). Full startup sequence completes (CKPT 9: after setup_run_auto_console_state reached in every run).

cargo test -p mesh-llm-host-runtime --lib — 1424/1424 pass, including:

• PR mesh: drop peers below v0.60.0 from gossip ingest and re-broadcast #576's existing 6 filter tests still green.
• New regression: maybe_connect_discovered_peer_skips_filtered_announcements exercises the skip path — a below-floor announcement and an idle-transitive announcement both return well under the 30s connect timeout, with no entries created in state.connections or state.peers.

Architecture

The bug was a hidden coupling between two independent gossip code paths:

1. update_transitive_peer (ingest) — knew about the filter and used state.peers.remove to enforce it.
2. apply_gossip_announcements (dial loop) — keyed its short-circuit on state.peers.contains_key, which the ingest had just invalidated for filtered peers.

The fix decouples them: the dial loop now applies the filter independently of state.peers. Both paths use the same predicates (version_allowed_for_rebroadcast, peer_is_idle_transitive_client), so adding a new filter dimension in the future is a single-place change.

No protocol change. No mesh-compatibility change. Behaviour against older peers is unchanged — they were filtered before, they are still filtered, we just don't burn 30s timeouts dialing them now.

Supersedes

PR #601 (micn/disable-gossip-ingest-filters).

[Copilot]

Pull request overview

This PR fixes an intermittent mesh-llm serve --auto startup wedge on the public mesh by preventing the gossip discovery dial loop from attempting to connect to peers that are intentionally filtered out of the local peer table.

Changes:

• Add a pre-dial filter in maybe_connect_discovered_peer to skip peers below the version floor and idle transitive clients (avoids sequential 30s dial timeouts on “ghost” peers).
• Add a regression test ensuring filtered announcements are skipped quickly and don’t create connections/peers entries.

[ndizazzo]

Interesting - so whoever/whatever is doing this likely recognized this aspect of mesh and added the flood of nodes knowing it would potentially starve starting up

[michaelneale]

@ndizazzo maybe, but I don't think so - it would always have, just was luck it sometimes worked I think, I don't think anything really changed.