fix(sglang): relay multinode DP sidecar KV events

[jain-ria]

Multinode SGLang attention-DP deployments previously launched a Dynamo sidecar only on the leader, leaving follower KV events unavailable to the router. Launch native gRPC and a coupled sidecar on each follower that owns a KV publisher, while retaining one serving registration per logical worker. Dynamo selects serving or telemetry mode from engine metadata; srtctl no longer passes --telemetry-only. Followers without publishers remain engine-only.

Reserve the full global-DP publisher-port range to prevent collisions between colocated workers, keep prefill bootstrap addresses leader-only, and omit an explicit NCCL port for ordinary-DP sidecar engines so SGLang can allocate separate ports for its independent TP groups. Includes agg/disagg examples, docs, launch snapshots and regression coverage. No frontend-readiness changes.

Requires compatible Dynamo PR14908 and SGLang PR39659, including their native libraries. Python-only source overlays are insufficient.

Validation after merging main

Merge commit c285a29 incorporates upstream main 848f72d. Ruff, formatting, type checks and generated schema checks pass; focused sidecar/port/design checks: 95 passed. Full suite: 4,146 passed, 2 skipped, 6 deselected, 3 failed. Two failures pass isolated reruns (console wrapping and status-server timeout); the remaining MLPerf snapshot mount-order difference also fails on untouched upstream main under a non-default temporary directory. No patch-specific CPU test failure remains.

Fresh GPU runs use actual srtctl apply, Qwen/Qwen3-0.6B BF16, A16 GPUs, Dynamo 4da4ccbe938d and SGLang 047eb45dfb3b, with native libraries freshly rebuilt and hashes verified. Runtime dependencies were aligned to Torch 2.14.1+cu130, Transformers 5.17.0 and FlashInfer 0.7.0.post1, including matching cubins/JIT cache. SA-Bench: ISL 128, OSL 64, concurrency 4; custom post-evaluation checks cover rank routing, cache reuse, KV events and disaggregated transfers.

Deployment	Fresh result
Ordinary DP aggregate: two TP1/DP2 engines on two nodes	PASS: 40/40 benchmark requests; 36/36 targeted requests, including 28 unpinned cache checks and KV Stored events from all four worker/rank targets.
Attention DP aggregate: one TP2/attention-DP2 engine across two nodes	FAIL: follower sidecar rejects KV source rank 1 is outside dp_size 1. Current SGLang reports dp_size=1, attn_dp_size=2; Dynamo still validates/registers against dp_size.
Ordinary DP disaggregate: two prefill and two decode TP1/DP2 engines across four nodes	FAIL: 16/40 benchmark requests succeeded; 24 failed. Targeted checks passed two DP0 requests, then failed the first prefill-DP1 request with HTTP500. SGLang checks its attention rank0 against the ordinary-DP bootstrap rank1.
Attention DP disaggregate: TP2/attention-DP2 prefill and decode across four nodes	FAIL before serving: prefill follower independently reproduces KV source rank 1 is outside dp_size 1, with attn_dp_size=2.

Validation is not fully green with the current dependency heads. SGLang needs to use the system-DP rank for ordinary-DP bootstrap registration/checks; Dynamo needs to interpret SGLang's separate system-DP and attention-DP widths consistently during metadata validation and worker registration. These failures remain in the dependency PRs; no SRT readiness gate or workaround was added.

Native dev builds use RUST_MIN_STACK=33554432 (32 MiB). This is functional smoke validation, not release-performance or default-stack qualification. No dependency source patches were applied for these fresh runs.

Earlier attention-DP agg/disagg runs passed 40/40 benchmark and 27/27 or 31/31 targeted requests using Dynamo 9ab50011f8bf, SGLang 8eaffcf88dac and srt-slurm 978b35b. Those results do not establish compatibility of the current dependency heads.

[github-actions]

src/srtctl diff: +81 -9