Default fully-async recipes to retract weight-update pauses

[fzyzcjy]

TODO: show cache weight version, fix the unbounded issue

Summary

--pause-generation-mode in_place leaves the SGLang radix (prefix) cache poisoned with pre-update KV after every weight update, so later rollouts silently generate on a mixture of new weights and old-policy KV. Every fully-async recipe on main currently defaults to or hardcodes in_place. This PR flips them to retract, which recomputes retracted requests' KV under the new weights and flushes the prefix cache on every update.

Mechanism

• pause_generation(mode="in_place") freezes in-flight requests so their KV survives the update — that is the mode's purpose. Because those frozen requests hold KV, the engine-side cache flush cannot run: scheduler.flush_cache refuses while requests hold pages, and the engine's update paths assert on flush success. sglang's own io_struct.py documents it: "In 'inplace' mode, flush_cache will fail if there are any requests".
• The miles update paths therefore skip the flush entirely under in_place (update_weight_from_distributed/mixin.py:315-316, same guard in delta.py and update_weight_from_tensor.py).
• Skipping the flush is forced, but its fallout is unhandled: nothing ever invalidates the radix tree. Prefixes cached under pre-update weights keep serving every later request that shares them, across arbitrarily many updates. A radix hit reuses the cached KV without recompute, and LRU keeps hot prefixes (shared system prompt, the per-prompt prefix under --n-samples-per-prompt, earlier turns of a multi-turn conversation) alive indefinitely — so the poisoning is unbounded in time and covers all subsequent traffic, not just the requests frozen across one update.

Symptom

• Rollouts become a hybrid behavior policy: fresh weights on the un-cached suffix, stale KV on the cached prefix. The recorded rollout_log_probs describe that hybrid, while the trainer recomputes log probs under the current weights, so train/train_rollout_logprob_abs_diff grows with every update instead of sitting at cross-implementation numeric noise, and the effective behavior policy lags the trainer roughly in proportion to the cached-prefix token share.
• The existing rollout/weight_version/mixed_version_ratio metric stays near zero throughout: per-request weight-version events only track requests alive across an update, and radix reuse by new requests is invisible to them. The corruption is therefore silent on the dashboard.
• Severity scales with radix hit rate × per-update policy movement — worst for exactly the standard RL shape: shared system prompt, many samples per prompt, --update-weights-interval 1.

Why retract is the right default now

• retract returns in-flight requests to the waiting queue, recomputes their KV under the new weights, and the update path flushes the prefix cache — both halves of the staleness problem disappear.
• Its historical blocker is gone: the flush-refused-under-load failure was fixed by [Fix] retract-mode flush_cache no-op crash #1750 plus [sglang-miles] Fix flush_cache() no-op after pause_generation in retract sgl-project/sglang#31962, so retract works under high concurrency.
• in_place stays available for setups that need it, but until the engine invalidates version-stale prefixes on its own, it should be an explicit opt-in with known cost, not the default.

Changes

file	change
examples/infra_features/fully_async/run_qwen3_30b_a3b_fully_async.py	default in_place → retract
examples/infra_features/fully_async/run_qwen3_5_4b_fully_async_eval.py	hardcoded → retract
examples/infra_features/random_async/run_random_async_3node.py	default → retract
examples/swe-agent-harbor-docker/run-glm47-flash-agentic-async.py	default → retract
examples/experimental/openenv/glm52_tbench2/run_glm5_2_744b_a40b_daytona.py	hardcoded → retract
scripts/run_inkling.py	hardcoded → retract
docs/user-guide/fully-async.md, docs/examples/infra-features/random-async.md, examples/infra_features/random_async/README.md	document the default and the reason

Deliberate exception: examples/multi_lora/run_multi_lora.py keeps in_place — its adapter upsert relies on it (an unload would deadlock behind paused in-flight requests), and its --use-tis corrects the training-side bias since the recorded log probs faithfully describe the (lagged) behavior policy.

Follow-up

• Engine-side long-term fix, tracked separately: evict the evictable radix nodes when the weight version bumps. Frozen requests' nodes are lock-protected, so that would make in_place safe again (including for multi-turn workloads, which lose cross-turn prefix reuse under any flush-based scheme).

Tests

• pre-commit run --all-files clean. The change is recipe defaults plus docs; behavior is covered by the existing fully-async paths that already run retract.

[claude]

Claude Code Review

This repository is configured for manual code reviews. Comment @claude review for a one-time review, or @claude review always to subscribe this PR to a review on every future push.

_{Tip: disable this comment in your organization's Code Review settings.}