[TRTLLM-15264][doc] Kimi K3 disagg: stop recommending a UCX_TLS pin by default

examples/kimi_k3/disagg/README.md

@@ -53,14 +53,12 @@ for constraints.
 
 ## Launch sequence (manual, single ctx + single gen)
 
-Each K3 worker spans 16 GPUs (4 NVL72 nodes at 4 GPUs/node). Environment
-prerequisites for every worker shell (see caveats below for why):
-
-```bash
-export UCX_TLS=tcp,self,sm,cuda_copy,cuda_ipc   # on clusters where verbs cannot
-                                                # initialize; a container-default
-                                                # UCX_TLS=tcp breaks V2 NIXL
-```
+Each K3 worker spans 16 GPUs (4 NVL72 nodes at 4 GPUs/node). Leave
+`UCX_TLS` unset in every worker shell so UCX selects transports itself,
+RDMA/verbs included; in particular, unset any container-default
+`UCX_TLS=tcp`, which breaks V2 NIXL VRAM registration. Override the
+transport list only on clusters whose verbs transports cannot
+initialize (see caveats 1 and 4 below for the symptom and the override).
 
 1. Start the context server (16-rank MPI world across its 4 nodes):
 
@@ -103,9 +101,11 @@ python3 examples/disaggregated/slurm/benchmark/submit.py \
 - **Gen-only baseline**: set `benchmark.mode: gen_only_no_context`
   (submit.py exports `TRTLLM_DISAGG_BENCHMARK_GEN_ONLY=1` to the
   workers) to measure the decode-side ceiling without KV transfer.
-- The harness's `start_worker.sh` clears `UCX_TLS`; the config carries
-  the transport pin via `TRTLLM_WORKER_UCX_TLS`, which `start_worker.sh`
-  re-exports as `UCX_TLS` after the clear.
+- The harness's `start_worker.sh` clears any container-provided
+  `UCX_TLS`, so workers run with UCX's own transport selection by
+  default. Clusters that need a transport override (caveat 4) carry it
+  via `TRTLLM_WORKER_UCX_TLS` in `worker_env_var`, which
+  `start_worker.sh` re-exports as `UCX_TLS` after the clear.
 - pyxis/enroot resets image-defined variables (notably `PATH`) at
   container start, so the config injects the in-place TRT-LLM venv via
   `TRTLLM_PATH_PREPEND` / `TRTLLM_PYTHONPATH_PREPEND`, applied inside
@@ -131,11 +131,14 @@ python3 examples/disaggregated/slurm/benchmark/submit.py \
 3. **Matched-DP only.** Keep ctx and gen at identical DEP16 with
    attention-DP on both sides; heterogeneous parallelism with
    attention-DP off is rejected (see constraints above).
-4. **Cluster environment** (NVL72 nodes): on clusters where verbs
-   transports cannot initialize, pin
-   `UCX_TLS=tcp,self,sm,cuda_copy,cuda_ipc` (`UCX_TLS=all` hangs setup,
-   see caveat 1) and never run V2 NIXL with a container-default
-   `UCX_TLS=tcp` (breaks V2 NIXL VRAM registration) — unset/override it.
+4. **Cluster environment** (NVL72 nodes): the default is to leave
+   `UCX_TLS` unset and let UCX pick transports (RDMA/verbs included).
+   Two exceptions: never run V2 NIXL with a container-default
+   `UCX_TLS=tcp` (breaks V2 NIXL VRAM registration; unset it), and on
+   clusters whose verbs transports cannot initialize, pin
+   `UCX_TLS=tcp,self,sm,cuda_copy,cuda_ipc` (an effective `UCX_TLS=all`
+   hangs setup there, see caveat 1). The pin excludes RDMA/verbs, so do
+   not carry it to clusters where verbs works.
    No bounce env override is needed: the byte gate
    (`TRTLLM_KV_CACHE_BOUNCE_MIN_BYTES`, default 2 MiB) is always cleared
    by K3 payloads (constraints section above).

examples/kimi_k3/disagg/benchmark_kimi_k3_dep16.yaml

@@ -57,13 +57,16 @@ environment:
   trtllm_wheel_path: ""
   work_dir: ""
   # Cluster notes:
-  # - On clusters where verbs UCX transports cannot initialize on the
-  #   compute nodes, UCX_TLS must be pinned — UCX_TLS=all can wedge
-  #   native NIXL init asymmetrically, leaving the surviving ranks hung
-  #   in the V2 setup MPI collectives. start_worker.sh clears
-  #   UCX_TLS (a container-provided UCX_TLS=tcp also breaks V2 NIXL VRAM
-  #   registration), so the pin is carried via TRTLLM_WORKER_UCX_TLS and
-  #   re-exported by start_worker.sh after the clear.
+  # - UCX transports: the default is no pin. start_worker.sh clears any
+  #   container-provided UCX_TLS (UCX_TLS=tcp breaks V2 NIXL VRAM
+  #   registration) and UCX then selects transports itself, RDMA/verbs
+  #   included. Only on clusters where verbs transports cannot
+  #   initialize on the compute nodes (symptom: UCX_TLS=all wedges
+  #   native NIXL init asymmetrically, surviving ranks hang in the V2
+  #   setup MPI collectives), add
+  #   TRTLLM_WORKER_UCX_TLS=tcp,self,sm,cuda_copy,cuda_ipc to
+  #   worker_env_var below; start_worker.sh re-exports it as UCX_TLS
+  #   after the clear. Note that list excludes RDMA/verbs.
   # - pairs with the workers'
   #   kv_cache_bounce_size_mb (fabric-VMM bounce; measured
   #   ~455 GB/s/GPU). The default gate of 96 blocks would silently skip
@@ -73,7 +76,7 @@ environment:
   #   in-place venv is injected via TRTLLM_PATH_PREPEND /
   #   TRTLLM_PYTHONPATH_PREPEND, which the harness scripts apply inside
   #   the container (setting PATH= here would be silently dropped).
-  worker_env_var: "TLLM_LOG_LEVEL=INFO TRTLLM_SERVER_DISABLE_GC=1 TRTLLM_WORKER_DISABLE_GC=1 ENROOT_ALLOW_DEV=yes TRTLLM_WORKER_UCX_TLS=tcp,self,sm,cuda_copy,cuda_ipc TRTLLM_PATH_PREPEND=<trtllm_venv_bin_dir> TRTLLM_PYTHONPATH_PREPEND=<trtllm_repo>"
+  worker_env_var: "TLLM_LOG_LEVEL=INFO TRTLLM_SERVER_DISABLE_GC=1 TRTLLM_WORKER_DISABLE_GC=1 ENROOT_ALLOW_DEV=yes TRTLLM_PATH_PREPEND=<trtllm_venv_bin_dir> TRTLLM_PYTHONPATH_PREPEND=<trtllm_repo>"
   server_env_var: "TRTLLM_SERVER_DISABLE_GC=1 TRTLLM_PATH_PREPEND=<trtllm_venv_bin_dir> TRTLLM_PYTHONPATH_PREPEND=<trtllm_repo>"
 
 worker_config: