Use shared rollout updates and replay Ouro execution depths

[0z5a]

Incremental review: Files changed.

Direct dependency and ancestor review: #4 → #3 → #1. Baseline: cleaned #4 head 2d04ab6a66e7b6f223d935bff3e368d16a97cdda. This fork PR shows only this layer.

Adapt the Ouro recipe to the public rollout/update contract in vllm-rlt#78, add physical HF↔Megatron parameter conversion and score training tokens at their actual decode depth after full-depth prefill. CUDA graphs use the engine's graph settings; checkpoints preserve the schedule and data contract.

Minhao proposed aligning the wrapper with his engine contract. The Thor replay checks established full-four-loop prefill and actual-depth decoding for K=2/3/4, early stops and delayed stops. Seed handling was corrected. The public implementation maps log_probs to logprobs and uses engine-level probability processing. His exact upstream PR1 was not available; formal compatibility acceptance remains pending.

Full Ouro-1.4B, five alternating warmed rollout pairs:

Workload	Eager (s)	Graph (s)	Speed ratio	Latency reduction
K=2	1.0689	0.9939	1.076×	7.02%
K=3	1.5190	1.4207	1.069×	6.47%
K=4	1.9414	1.8234	1.065×	6.08%
K=4 target / K=2 draft	1.4952	1.3741	1.088×	8.11%
Thor complete GRPO step	12.8710	13.5940	0.947×	No improvement
A100 prefill/decode complete step	62.1185	62.2978	0.997×	No improvement

These workload-specific rollout gains do not establish complete-step acceleration. A100 measurements had shared-resource contention and host-staged transfers (UCX_TLS=tcp,cuda_copy).

Consistency / recovery check	Result
Seed regression	7 CPU and 6 CUDA checks passed
Full-model depth/stop replay	Five cases passed
Thor 5 updates vs 4 + resumed fifth	All 272 model tensors and Adam state exact; all 269 HF tensors exact
A100 3 updates vs 2 + resumed third	Same exact checkpoint and HF comparisons
Early-stop actor score MAE	3.622689 before → 0.011142 after; fixed K=4 residual 0.026607

This historical fork layer is consolidated in upstream #459. Added tests, logs, reports and coordination files were removed; measurements remain in this description. Completed-task model files were cleaned up. Full-model convergence and broader topology coverage remain outstanding.

[0z5a]

Superseded by vllm-project#459, now merged into 0z5a-branch. This historical Ouro layer is included in the consolidated upstream integration; closing the duplicate PR.