[guapisolo]

Depends on: #3908
Stacked on the parent PR's branch. Review / merge the parent first.

Summary

Run SGLang's Mooncake output store under the Kubernetes backend.

Motivation

Under --cluster-backend kubernetes startup rejects --sglang-output-store-backend: Kubernetes renders one command per pool and binds every engine to 0.0.0.0, so no engine can name the address its Mooncake peers reach it at, and the rendered engine and session-server commands still carry the master address from before the launcher's in-cluster rewrite. Kubernetes runs therefore keep returning replay outputs inline.

Usage

python scripts/run_qwen3_4b.py train --cluster-backend kubernetes ... \
  --object-store-backend mooncake --sglang-output-store-backend mooncake \
  --sglang-output-store-client-config '{"local_buffer_size": "1gb"}'
# Each engine registers its pod IP as local_hostname; the readers fetch bundles from there.

Design Notes

• Mental model: the launcher builds one launch context per pool, whose non-master ports carry host 0.0.0.0 and peer_host $(MILES_POD_IP); inference.py hands host_for_peers to compute_engine_launch_cmd, which writes it into the engine's extra config as local_hostname; kubelet expands the variable from the env the chart fills from status.podIP. Ray leaves peer_host unset, so its engines keep their own host.
• peer_host sits beside external_host on HostAndPort, so sglang_engine.py stays backend-agnostic, as test_platform_contract.py requires.
• _compute_train_argv rewrites args.mooncake_store_init_kwargs to the in-cluster master before compute_specs, as it already did for the pods' argv.
• MILES_POD_IP is platform-owned, so a run's own env cannot override it.

Verification

• tests/fast at this head on a Mac under CPU CI's requirements plus [Feat] RL: Add radix transfer plugins sgl-project/sglang#42647 (backends/sglang_utils, utils/external_utils, utils/test_arguments.py, ray/specs, utils/workers/real_ray, charts/miles_run/test_colocate.py, tools/kubernetes, test_platform_contract.py, session and router): 3476 passed; only test_session_server_benchmark.py fails, locally, for want of Linux prctl.
• test_pool_entry.py: a rendered engine command binds 0.0.0.0 and names $(MILES_POD_IP) as its peer host; test_compute_server_args.py: a bind-all engine takes its peer host as local_hostname.

Review Focus

• pool_entry._launch_context: every non-master port gets the pod IP as its peer host.
• Not run on a real cluster: whether a pod IP works as Mooncake's local_hostname over RDMA in the pod network.

[claude]

Claude Code Review

This repository is configured for manual code reviews. Comment @claude review for a one-time review, or @claude review always to subscribe this PR to a review on every future push.

_{Tip: disable this comment in your organization's Code Review settings.}