feat: run mesh as a library, and join gated relays

[michaelneale]

Lets you run a full mesh node from the SDK, and also join a gated relay if you need to.

Adds a per-relay bearer token to the iroh relay map so mesh-llm can register with a gated iroh-relay (one running AccessConfig::Restricted) while public relays in the same map continue to register without auth.

[Copilot]

Pull request overview

Adds support for per-relay bearer authentication so the runtime can register with restricted (gated) iroh relays while continuing to use unauthenticated public relays.

Changes:

• Introduces --relay-auth URL=TOKEN CLI parsing and wiring through runtime startup.
• Extends mesh endpoint construction to attach auth tokens to specific relay configs when building an iroh::RelayMap.
• Updates mesh tests and adds focused unit tests for relay-auth parsing and relay-map token attachment behavior.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 1 comment.

File	Description
crates/mesh-llm-host-runtime/src/runtime/mod.rs	Builds a relay-auth map from CLI args and passes it into mesh::Node::start.
crates/mesh-llm-host-runtime/src/mesh/mod.rs	Adds per-relay auth token support in relay map construction and threads it into endpoint/control listener setup; adds unit tests.
crates/mesh-llm-host-runtime/src/mesh/tests.rs	Updates control-listener test call sites for the new relay_auths parameter.
crates/mesh-llm-host-runtime/src/cli/mod.rs	Adds --relay-auth flag with a URL=TOKEN parser plus parser unit tests.

Comments suppressed due to low confidence (1)

crates/mesh-llm-host-runtime/src/mesh/mod.rs:277

• Rustdoc link [RelayMap] likely won't resolve here because RelayMap isn't in scope (the function returns iroh::RelayMap via a fully-qualified path). Consider changing the link target to [iroh::RelayMap] (or plain backticks) to avoid broken intra-doc links/warnings.

fn effective_relay_urls(relay_urls: &[String]) -> Vec<String> {
    if relay_urls.is_empty() {
        vec![
            "https://usw1-2.relay.michaelneale.mesh-llm.iroh.link./".into(),
            "https://aps1-1.relay.michaelneale.mesh-llm.iroh.link./".into(),

[ndizazzo]

Found one more thing:

Medium: --relay-auth fails before serve / client in split-arg form
In crates/mesh-llm-host-runtime/src/cli/mod.rs, normalize_runtime_surface_args() skips known value-taking flags while searching for the legacy pseudo-subcommand, but the new --relay-auth flag is not included.

Example: mesh-llm --relay-auth https://gated.example/=token serve --relay https://gated.example/ --auto stops scanning at the token value, never normalizes serve, and Clap can reject/misparse the command.

Suggest: Add "--relay-auth" to value_taking_flags and add parser tests for --relay-auth URL=TOKEN serve ... and --relay-auth URL=TOKEN client ...

[michaelneale]

Thanks both — pushed f76042cc:

@ndizazzo — fixed. Added --relay-auth to normalize_runtime_surface_args' value-taking list, plus two new parser tests:

• normalize_runtime_surface_args_treats_relay_auth_as_value_taking_before_serve — verifies mesh-llm --relay-auth URL=TOKEN serve --relay … --auto discovers the serve surface and cli.relay_auth gets populated.
• normalize_runtime_surface_args_relay_auth_before_client_invocation — same for client, with a base64-padded NIP-98-style token (eyJ…payload==) so we exercise the =-in-token edge.

@copilot — dropped the broken [RelayMap] intra-doc link for [iroh::RelayMap].

While here, added a real defence for the feature too — an in-process iroh-relay with AccessConfig::Restricted and 4 e2e tests:

• matching token admits the endpoint (online() resolves),
• wrong token surfaces not authorized on home_relay_status and never reaches online(),
• missing token also rejected,
• mixed gated + public relay map authenticates only the gated relay.

These exercise the actual wire path through relay_map_from_urls → iroh::RelayConfig::with_auth_token → relay WebSocket upgrade. Build cost: iroh-relay with server,test-utils features in [dev-dependencies] only — shipped binary tree unchanged.

[michaelneale]

Pushed 20d6cb82: the Rust SDK that motivated this PR in the first place.

mesh_llm::sdk::RuntimeBuilder mirrors the binary's CLI 1:1 — every mesh-llm flag has a builder method, including --relay-auth:

mesh_llm::sdk::RuntimeBuilder::new()
    .client(true)
    .auto(true)
    .relay("https://gated.example/")
    .relay_auth("https://gated.example/", "<nip98-bearer>")
    .run()
    .await?;

How it works

The builder collects fields, serialises them to argv, and feeds that argv to the same runtime::run entry point the binary uses. To enable this, runtime::run() is split into a thin entry that reads std::env::args_os() plus a new run_with_args(args) that takes a caller-supplied argv. Binary behaviour is unchanged.

One code path, two surfaces. The CLI parser is the same parser the SDK targets.

What it covers

The realistic CLI surface: client/serve, auto, publish, mesh_name, region, join, discover, model/gguf/mmproj, port, console_port, headless, blackboard, name, max_vram_gb, no_enumerate_host, relay, relay_auth, nostr_relay, bind_port/bind_ip, listen_all, config, owner_key, owner_required, node_label, trust_owner. Plus a .arg(...) escape hatch.

Known limitations (documented in the module)

• No spawn() method: the runtime future is not currently Send, so tokio::spawn would be unsound. Embedders use .run().await or wrap in tokio::task::LocalSet themselves. We can revisit once the runtime is Send-clean.
• Introspecting a running runtime (invite token, peers, served models) goes through the management API on --console (default :3131). That's what the CLI's TUI and web console do too. A rich in-process handle would need real refactoring of run() to return early with a handle; out of scope here.

Tests

7 unit tests pin each setter's argv. The decisive one is argv_is_parseable_by_the_real_cli_parser: it round-trips builder argv through normalize_runtime_surface_args + Cli::try_parse_from and asserts the fields land. If a future refactor renames a flag, this test fails immediately and points at the drifted builder method.

Stability statement

mesh_llm::sdk::* is documented as the only stable Rust surface in the mesh-llm crate. Everything else still reachable via the existing wildcard re-export of mesh-llm-host-runtime is implementation detail and may change in any release. The docstring on mesh-llm's lib.rs says so explicitly.

[Copilot]

Pull request overview

Copilot reviewed 8 out of 9 changed files in this pull request and generated 5 comments.

[michaelneale]

Follow-up work to plan

PR is in draft. Notes here on the next steps that this PR clears the runway for. Whether any of these land in this PR or a follow-up is open \u2014 captured here so we don't lose context either way.

1. Make cargo build of mesh-llm work without just

Today cargo build -p mesh-llm from a clean tree fails: skippy-ffi/build.rs only links pre-built llama.cpp static libs, and those libs are produced by scripts/prepare-llama.sh + scripts/build-llama.sh orchestrated by just build before cargo runs.

This is the core blocker for any external consumer (mesh-llm = { git = "...", rev = "..." }) and for any future crates.io publish.

Feasibility: confirmed locally. Adding ~74 lines to skippy-ffi/build.rs (a thin auto-prepare/auto-build path that invokes the existing scripts when the canary static lib is missing) gets cargo build -p mesh-llm working end-to-end from a clean checkout, ~55s on an M-series Mac. Subsequent builds are fast (canary present, scripts skipped). just build still works unchanged (canary present from cargo's previous run, scripts skipped).

Critical caveat to get right (the real bug we hit during the feasibility test):

build-llama.sh defaults to LLAMA_BACKEND=cpu. skippy-ffi/build.rs (the link path) defaults to metal on apple-darwin and cpu elsewhere via default_build_dir. These two must agree, always, or the build silently produces the wrong artefacts and then fails to link.

The autobuild path therefore MUST:

• Compute the backend exactly once, in one place.
• Honour overrides in priority order: LLAMA_STAGE_BACKEND \u2192 SKIPPY_LLAMA_BACKEND \u2192 LLAMA_BACKEND \u2192 target-based default (metal on apple-darwin, else cpu). This is the same order the scripts already use.
• Pass that backend into the script invocation via LLAMA_BACKEND, LLAMA_STAGE_BACKEND, and SKIPPY_LLAMA_BACKEND env vars (the scripts inspect all three).
• Use the same backend value when computing the link path's build_dir so the autobuild and the link agree by construction.

It should also:

• Be opt-out: SKIPPY_FFI_NO_AUTOBUILD=1 skips the autobuild (for CI lanes that pre-stage a different llama tree).
• No-op when the scripts aren't present (i.e. the crate is being consumed outside the workspace) \u2014 fall through to the regular link path so the existing linker error tells the user what to do.
• Surface a cargo:warning=\u2026 line before the first autobuild so a developer knows why their first cargo build is suddenly compiling C++ for ~30-50s.

GPU backends (CUDA / ROCm / Vulkan) add more required env (LLAMA_STAGE_CUDA_ARCHITECTURES, LLAMA_STAGE_AMDGPU_TARGETS, etc.). The autobuild should pass-through anything the user has set, and only override what's missing.

Pre-commit on that change: just build && cargo build -p mesh-llm from .deps cleared, on at least macOS arm64 and Linux CPU.

2. Publishing mesh-llm to crates.io

The autobuild above unlocks git-dep consumption (mesh-llm = { git = "...", rev = "..." }). It does not unlock cargo add mesh-llm because:

• scripts/prepare-llama.sh / scripts/build-llama.sh live in the workspace, not the published crate.
• third_party/llama.cpp/patches/ and upstream.txt similarly aren't shipped with a cargo publish of mesh-llm/skippy-ffi.

The standard pattern (see llama-cpp-sys-2 on crates.io for the existence proof) is to vendor the post-patch llama.cpp source inside the published crate via include = [...] in Cargo.toml. Concretely:

• A pre-publish step materialises the patched llama.cpp source into crates/skippy-ffi/llama.cpp/ (the same source prepare-llama.sh produces in .deps/ today).
• Cargo.toml include = [...] lists the source files to ship.
• The autobuild path from (1) detects "vendored source present" \u2192 skips the prepare step and goes straight to cmake-against-vendored-source.

Workspace dev flow is unchanged: scripts still produce .deps/llama.cpp/ for fast iteration; the vendored copy is a publish-time artefact only.

Also requires publishing the unpublished workspace deps mesh-llm pulls in (mesh-llm-host-runtime, skippy-runtime, skippy-server, mesh-llm-ui, mesh-llm-system, mesh-llm-plugin, mesh-mixture-of-agents, openai-frontend, model-resolver, model-package, skippy-protocol, skippy-coordinator, skippy-topology). The existing scripts/publish-crates.sh chain (model-ref \u2192 \u2026 \u2192 mesh-api) is the template \u2014 extend it.

3. spawn() on RuntimeBuilder

Currently absent because the runtime future is not Send. To add it cleanly:

• Audit non-Send types held across .await points in runtime::run_with_args. Known offenders include Box<dyn SearchFormatter> and Box<dyn ModelsFormatter> in the models CLI commands (the build error we hit when I first tried to tokio::spawn the future surfaced dyn SearchFormatter cannot be shared between threads safely).
• Add + Send + Sync to the trait objects (and any others surfaced by clippy's -D warnings once enabled on the spawn path).
• Add a RuntimeBuilder::spawn(self) -> tokio::task::JoinHandle<Result<()>> method once the future compiles as Send.

Until then run().await is the only entry. Documented in the module-level doc comment.

4. Rich in-process introspection

The runtime API on --console (default :3131) is what the CLI's TUI and web console use, and it's how SDK consumers should pull invite-token / peers / served-models from a running runtime today. That's deliberate in this PR \u2014 runtime::run_with_args blocks until shutdown, so there's no "started, here's a handle, your turn" point.

If we want a real RuntimeHandle with invite_token(), peers(), shutdown() exposed in-process (no HTTP roundtrip), runtime::run_with_args needs to be split into a startup phase that returns a handle plus a background driver task. Non-trivial \u2014 touches the full startup sequence (auto-discovery, model load, proxy bind, console bind). Worth doing once a real consumer asks for it.

5. Embed-only feature flag

For consumers who want mesh-llm's mesh fabric but route inference elsewhere (no in-process llama.cpp), a --no-default-features build of mesh-llm that excludes skippy-ffi entirely would be valuable. Removes the llama.cpp build dependency end-to-end. Requires feature-gating mesh-llm-host-runtime's inference module and a few other points where the runtime currently assumes local serving is available.

---

These are the bookmarks. Any of them may land in this PR or a follow-up.

[michaelneale]

Copilot review pass — addressed

Reviewers added: @ndizazzo, @i386.

Walked Copilot's two review passes:

Live finding (fixed in 89c4900a): Token leakage in parse_relay_auth_pair error messages. The original format!("... got {s:?}") echoed the full URL=TOKEN input back, which would land bearer tokens in terminal output, logs, and bug reports if a user mistyped --relay-auth. Now:

• Missing = separator → redact whole input (can't tell URL from token).
• Empty URL (=token) → redact (the value after = is the secret).
• Empty token (URL=) → URL is safe to name; no token to leak.

Added parser_errors_never_leak_token_portion test that injects a known token, drives each error path, and asserts it never appears in the error message. Good catch from Copilot.

Stale findings (ignore): Three comments in Copilot's second pass refer to crates/mesh-llm-host-runtime/src/sdk.rs and rustdoc links into it. That file was reverted (4cfae45a) once we saw #634 had already shipped the proper SDK story (MeshNode / MeshNodeBuilder in mesh-llm-api-server + Swift/Kotlin/Node bindings + native runtime artifact crates). Re-doing it here would have been duplication.

First-pass intra-doc link [RelayMap] → [iroh::RelayMap] was already fixed in f76042cc.

[Copilot]

Pull request overview

Copilot reviewed 5 out of 6 changed files in this pull request and generated 7 comments.

[michaelneale]

Copilot review pass 3 \u2014 addressed

Seven comments, all the same shape: &HashMap::new() passed into maybe_start_control_listener(...).await, flagged as "temporary borrowed across await".

Verdict: harmless today (compiles, tests pass on every CI lane), because the callee only borrows the reference synchronously before its internal .await points. Rust's temporary-lifetime-extension rules cover the actual call boundary.

But the pattern is fragile \u2014 if the callee's signature ever shifts the borrow across an await, every test would break at once. Cheap future-proofing, applied in 6fd108b5: introduced an empty_relay_auths() helper and bound each call's map to a local before passing &auths. 7 sites, no behaviour change, all 119 mesh tests still pass.

Net diff is slightly smaller (+31 -50) because the helper consolidates what was 7 inline temporary-construction sites.

[Copilot]

Pull request overview

Copilot reviewed 11 out of 12 changed files in this pull request and generated 4 comments.

[i386]

Code review pass: two host-runtime SDK lifecycle issues to address before merge.

[Copilot]

Pull request overview

Copilot reviewed 11 out of 12 changed files in this pull request and generated 3 comments.

Comments suppressed due to low confidence (1)

crates/mesh-llm-api-server/Cargo.toml:43

• base64 and serde_json are added as dev-dependencies here, but there are no references to either in crates/mesh-llm-api-server (including the new host-runtime tests). Consider removing them to avoid unnecessary dependency bloat/compile time.

[dev-dependencies]
tokio = { version = "1", features = ["macros", "rt", "rt-multi-thread", "time"] }
# In-process iroh-relay for testing the host-runtime feature: real
# AccessConfig::Restricted relay, real endpoint bind, real WebSocket
# upgrade. Same dev-dep iroh feature combo `mesh-llm-host-runtime`
# already uses for its own gated-relay tests.
iroh = { version = "1.0.0-rc.0", features = ["test-utils"] }
iroh-relay = { version = "1.0.0-rc.0", features = ["server", "test-utils"] }
futures-util = "0.3"
base64 = "0.22"
serde_json = "1"

[Copilot]

Pull request overview

Copilot reviewed 14 out of 15 changed files in this pull request and generated 3 comments.

[ndizazzo]

Mine found the same:

---

Medium — host node leaks when OpenAI proxy bind fails
crates/mesh-llm-api-server/src/node.rs:502-513
If .openai_port(...) is set to a port that is already in use or cannot be bound, start_host_node(), join(), and node.start_accepting() have already succeeded before start_openai_proxy(...) returns an error. Because self.inner.host_node is not stored yet, a later stop() has no handle to shut the node down, leaving QUIC/background tasks alive. Add cleanup on this error path.

[michaelneale]

not sure if I like this approach yet, need to think about how rust crate is pre-built and shared

[michaelneale]

Superseded by two unrelated PRs split out of this one:

• feat(cli): --relay-auth URL=TOKEN for gated iroh-relays #698 carries the gated-relay half (--relay-auth URL=TOKEN).
• Rust SDK: in-process mesh node from cargo (design + trial) #690 carries the Rust SDK / cargo-distribution half (MeshNodeBuilder running a real iroh node under the host-runtime feature, run_serve(MeshServeSpec), plus all the workspace + publish + release-pipeline plumbing needed for external consumers to depend on mesh-llm-api-server via cargo).

Closing this one in favour of the cleaner split.