Add serve/client commands and local GPU inspection for #184

[i386]

Implements the runtime-surface and GPU-inspection portion of #184.

Summary

This PR makes the local runtime entrypoints explicit and adds a local GPU inspection command.

Users can now run:

mesh-llm serve ...
mesh-llm client ...
mesh-llm gpus

This keeps the local startup surface clearer without changing the later config work in #184.

mesh-llm serve is also necessary groundwork for the next config change. Once local config is introduced, mesh-llm serve will be the command that loads startup model configuration from ~/.mesh-llm/config.toml. Splitting the runtime surface now gives that later config work a clear owner instead of continuing to hang model startup behavior off ambiguous top-level flags.

New Commands

The new canonical runtime commands are:

mesh-llm serve --auto --model Qwen3-8B-Q4_K_M
mesh-llm serve --join <token>
mesh-llm client --auto
mesh-llm client --join <token>
mesh-llm gpus

mesh-llm gpus also has a gpu alias:

mesh-llm gpu

Deprecated Top-Level Forms

These older top-level runtime forms still work in this PR, but they are now migration shims:

mesh-llm --auto --model Qwen3-8B-Q4_K_M
mesh-llm --gguf /path/to/model.gguf
mesh-llm --auto --client
mesh-llm --client --join <token>

They now warn and route to the new command surface.

Example:

⚠️ top-level serving flags now map to `mesh-llm serve`.
  Please use: mesh-llm serve --auto --model Qwen3-8B-Q4_K_M

⚠️ top-level `--client` now maps to `mesh-llm client`.
  Please use: mesh-llm client --auto

Mixed legacy usage such as top-level --client plus serving flags is rejected.

Example Output

Client startup now uses the new client-oriented ready output:

📡 Client ready:
  API: http://localhost:9337
  Console: http://localhost:3131

GPU inspection prints local GPU identity, backend device, VRAM, and cached bandwidth when available:

🖥️ GPU 0
  Name: NVIDIA RTX 4090
  Stable ID: pci:0000:65:00.0
  Backend device: CUDA0
  VRAM: 24.0 GiB
  Bandwidth: 1008.0 GB/s
  Unified memory: no

If no GPUs are detected:

⚠️ No GPUs detected on this node.

Included In This PR

• serve and client as explicit local runtime entrypoints
• legacy top-level runtime rewrites with ⚠️ migration warnings
• mesh-llm gpus and mesh-llm gpu
• additive GpuFacts support in local hardware survey
• cached benchmark bandwidth attached to mesh-llm gpus output when available
• script and test updates to the new runtime surface
• README and docs updates showing the new commands

Not Included In This PR

• unified local ~/.mesh-llm/config.toml serve config
• pinned gpu_id assignment
• mesh-wide config reconciliation/editor work

Validation

cargo test -p mesh-llm

[Copilot]

Pull request overview

This PR introduces explicit local runtime entrypoints (mesh-llm serve, mesh-llm client) plus a new mesh-llm gpus inspection command, while keeping the legacy top-level flags working via argument normalization + migration warnings. It also expands hardware detection to surface stable-ish per-GPU identity and updates CI/scripts/docs to the new CLI surface.

Changes:

• Add gpus CLI subcommand and expand HardwareSurvey with per-GPU GpuFacts (stable ID, backend device, cached bandwidth).
• Normalize serve/client runtime surfaces at process start, emit migration warnings for legacy top-level usage, and update passive-mode readiness logs.
• Update CI scripts + documentation/examples to use mesh-llm serve ..., mesh-llm client ..., and mesh-llm gpus.

Reviewed changes

Copilot reviewed 22 out of 22 changed files in this pull request and generated 5 comments.

Show a summary per file

File	Description
scripts/ci-split-test.sh	Update multi-node CI split test to invoke serve/client surfaces.
scripts/ci-smoke-test.sh	Update smoke test args to use serve.
scripts/ci-compat-smoke.sh	Update compat smoke test args to use serve.
scripts/ci-client-auto-test.sh	Update client auto boot test to use client and new readiness log text.
scripts/benchmark-prefix-affinity.sh	Update benchmark orchestration to use serve/client surfaces.
README.md	Refresh top-level usage to serve/client and add gpus mention/examples.
mesh-llm/tests/test_blackboard.sh	Update blackboard test harness to use client surface.
mesh-llm/src/system/hardware.rs	Add GpuFacts, GPU name expansion + NVIDIA identity parsing, and hydrate per-GPU facts into surveys.
mesh-llm/src/runtime/mod.rs	Parse normalized runtime args, emit legacy warnings, and update passive readiness logging.
mesh-llm/src/runtime/discovery.rs	Update auto-spawned launcher behavior to start mesh-llm client --auto.
mesh-llm/src/cli/mod.rs	Add gpus command, update help text, and implement runtime-surface normalization + migration warning helpers.
mesh-llm/src/cli/commands/mod.rs	Wire gpus command into CLI dispatch.
mesh-llm/src/cli/commands/gpus.rs	New command implementation to print GPU facts and cached bandwidth.
mesh-llm/src/cli/commands/blackboard.rs	Update user guidance to reference mesh-llm client ....
mesh-llm/src/api/mod.rs	Update model “run” command hints to mesh-llm serve --model ....
mesh-llm/README.md	Update examples to serve/client and mention gpus.
mesh-llm/docs/TESTING.md	Update testing docs to serve/client and add gpus inspection section.
mesh-llm/docs/MoE_DEPLOY_DESIGN.md	Update command examples to mesh-llm serve ....
mesh-llm/docs/DESIGN.md	Update join example to mesh-llm serve --join.
docs/USAGE.md	Update usage docs to serve/client and add gpus section.
docs/index.html	Update website examples to mesh-llm client ....
docs/AGENTS.md	Update agent docs to mesh-llm client ....

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

mesh-llm serve with no additional args gets normalized into a no-arg invocation because unwrap_or(true) makes next_is_flag true when arg[2] is missing. That bypasses the early args_os().len() == 1 help/exit path in runtime::run() and can lead to unexpected behavior (starting a node instead of showing help). Consider treating bare serve as a usage error or rewriting it to --help rather than removing the token when no further args are present.

Suggested change

	Some("serve") => {
	let next_is_flag = original
	.get(2)
	.and_then(|arg| arg.to_str())
	.map(|arg| arg.starts_with('-'))
	.unwrap_or(true);
	if next_is_flag {
	normalized.remove(1);
	explicit_surface = Some(RuntimeSurface::Serve);
	}
	}
	Some("serve") => match original.get(2).and_then(|arg| arg.to_str()) {
	Some(arg) if arg.starts_with('-') => {
	normalized.remove(1);
	explicit_surface = Some(RuntimeSurface::Serve);
	}
	None => {
	normalized[1] = OsString::from("--help");
	explicit_surface = Some(RuntimeSurface::Serve);
	}
	_ => {}
	},

[i386]

@copilot apply changes based on the comments in this thread

[Copilot]

@copilot apply changes based on the comments in this thread

Applied all 5 suggestions from the review thread in cb8076d:

1. Bare serve → --help: mesh-llm serve with no args now rewrites to --help instead of silently stripping the token and potentially starting a node. Added a test covering this case.

2. inferred_gpu_name_count(): Added this helper to parse N× / Nx / NX prefixes from summarized GPU names, so expected_count correctly derives 2 from "2× NVIDIA A100" instead of falling back to 1.

3. Gate detect_nvidia_identities(): Now only called when metrics.contains(&Metric::GpuName), avoiding unnecessary nvidia-smi spawning for lightweight queries.

4. Metric::GpuFacts + gated hydrate_gpu_facts: Added Metric::GpuFacts to the enum. query() now only calls hydrate_gpu_facts when that metric is requested. survey() includes it so existing callers are unaffected.

5. VRAM in decimal GB: mesh-llm gpus now formats VRAM as "24.0 GB" (bytes/1e9) to match the rest of the CLI output. Test updated accordingly.