translation: mechanical slime->vime / sglang->vllm cleanup (#107 follow-up)

.github/workflows/pr-test.yml

@@ -556,10 +556,10 @@ jobs:
     env:
       GITHUB_COMMIT_NAME: ${{ github.sha }}_${{ github.event.pull_request.number || 'non-pr' }}
       WANDB_API_KEY: ${{ secrets.WANDB_API_KEY }}
-      SLIME_TEST_ENABLE_INFINITE_RUN: ${{ (github.event_name == 'workflow_dispatch' && github.event.inputs.infinite_run) || 'false' }}
-      SLIME_TEST_USE_DEEPEP: ${{ matrix.info.use_deepep || '0' }}
-      SLIME_TEST_USE_FP8_ROLLOUT: ${{ matrix.info.use_fp8_rollout || '0' }}
-      SLIME_TEST_ENABLE_EVAL: ${{ matrix.info.enable_eval || '1' }}
+      VIME_TEST_ENABLE_INFINITE_RUN: ${{ (github.event_name == 'workflow_dispatch' && github.event.inputs.infinite_run) || 'false' }}
+      VIME_TEST_USE_DEEPEP: ${{ matrix.info.use_deepep || '0' }}
+      VIME_TEST_USE_FP8_ROLLOUT: ${{ matrix.info.use_fp8_rollout || '0' }}
+      VIME_TEST_ENABLE_EVAL: ${{ matrix.info.enable_eval || '1' }}
 
     steps:
       - name: Checkout repository

examples/fully_async/README.md

@@ -1,46 +1,77 @@
-# Fully Asynchronous Rollout Example
+# Fully-Async Rollout Example
 
-This example shows a simple way to make rollout generation **fully asynchronous**: a single global worker is created once and then keeps running in the background, continuously pulling prompts and launching generation tasks. Training only needs to fetch already finished results. This removes the per‑step wait that happens in the normal synchronous style.
+End-to-end demo of vime's fully-async rollout path. A background asyncio
+worker keeps a fixed pool of in-flight generations across rollout boundaries,
+so the next training step doesn't wait for the slowest in-flight sample.
+The worker itself lives in `vime.rollout.fully_async_rollout`; this
+directory is just the launch script.
 
 ## Files
-The fully-async worker has been **promoted from this example into the core package** — it now lives in
-`vime/rollout/fully_async_rollout.py`. This directory keeps only the launch script:
-* `run-qwen3-4b-fully_async.sh`: example launch script with Qwen3‑4B.
 
-## Prerequisite
-First set up model & environment following the Qwen3-4B example.
+* `run-qwen3-4b-fully_async.sh` — fully-async demo with Qwen3-4B on
+  dapo-math-17k.
+
+## Prerequisites
+
+```
+/root/Qwen3-4B/            # HF checkpoint
+/root/Qwen3-4B_torch_dist/ # tools/convert_hf_to_torch_dist.py
+/path/to/dapo-math-17k.jsonl  # set PROMPT_SET in the script
+```
+
+## Run
 
-## Quick Start
 ```bash
 cd vime
 bash examples/fully_async/run-qwen3-4b-fully_async.sh
 ```
-You should see log lines like:
+
+You should see:
+
 ```
-Creating new global async worker...
-Continuous async rollout worker started
+fully-async rollout 0: target=8 queue_warm=0
+fully-async rollout 0: done in ...s, queue_left=...
 ```
 
-## How It Works (Very Short)
-* First call: create `AsyncRolloutWorker` (thread + asyncio loop).
-* Loop keeps up to `--rollout-batch-size` tasks in flight using `generate_and_rm_group`.
-* Completed groups are pushed into a queue; caller drains until it has enough samples.
-* Worker is stopped automatically at process exit.
-
-## Limitations
-* No evaluation mode.
-* Ordering is best effort (sorted at the end by index).
-* Minimal error handling.
+## How To Plug Your Own Generate Into This
 
-## Config Differences (2 Key Points)
-To enable the fully async pattern there are only two changes compared to a normal run:
+Two pieces flip the standard pipeline into fully-async:
 
-1. Use the async training driver: `train_async.py` (not `train.py`).
+1. Use the async training driver: `python3 train_async.py` (not `train.py`).
 2. Set the rollout function path:
-	```bash
-	--rollout-function-path vime.rollout.fully_async_rollout.generate_rollout_fully_async
-	```
+   ```
+   --rollout-function-path vime.rollout.fully_async_rollout.generate_rollout_fully_async
+   ```
 
-Why is it still "fully" async although `train_async.py` itself schedules rollouts step‑by‑step?
+For custom per-sample logic, use vime's standard plug-in points — they
+work unchanged under fully-async:
+
+```
+--custom-generate-function-path your.module.generate     # (args, sample, sampling_params) -> Sample | list[Sample]
+--custom-rm-path                your.module.reward      # (args, sample | list[Sample]) -> float | list[float]
+```
+
+See `examples/coding_agent_rl/` for a non-trivial example that plugs in a
+multi-turn agent this way.
+
+## Worker Internals (Very Short)
+
+* First call: create a process-wide `AsyncRolloutWorker` (thread + asyncio
+  loop). The worker is shared across all subsequent `generate_rollout`
+  calls so its queue stays warm.
+* Loop keeps up to `args.vllm_server_concurrency` tasks in flight using
+  `generate_and_rm_group`.
+* Completed groups land on an output queue; each `generate_rollout` call
+  drains until it has `rollout_batch_size` groups and returns them sorted
+  by `sample.index`.
+* Groups containing an `ABORTED` sample are pushed back into
+  `data_buffer.add_samples` instead of being shipped to training.
+* Worker is stopped automatically at process exit via `atexit`.
+
+## Limitations
 
-Because the real generation work is done by a **persistent background worker** created in `generate_rollout_fully_async`. Each call from `train_async.py` only drains already completed samples from the worker's output queue; the worker has been continuously generating since the first call. Thus rollout production (model inference) and training consume happen in parallel with minimal waiting.
+* No evaluation mode (would conflict with the continuous-running model).
+* Ordering across rollouts is best-effort — within a rollout, groups are
+  sorted by index before being handed to training.
+* TODO: partial-rollout-style resume for `ABORTED` trajectories is not
+  yet wired; for now the trajectory is re-queued and starts over.

examples/geo3k_vlm_multi_turn/env_geo3k.py

@@ -32,7 +32,7 @@ class Geo3kEnv(BaseInteractionEnv):
     an `answer` argument. We run the math reward checker against the ground truth and
     return feedback for wrong answers. The episode ends when the answer is correct,
     when max_turns is reached, or when the response has no valid tool call; in the
-    last case we still try to score a boxed/text answer, matching SkyRL's env.
+    last case we still try to score a boxed/text answer.
     """
 
     def __init__(self, *, ground_truth: str | None = None, max_turns: int | None = None):

scripts/run-minimax-m2.sh

@@ -34,8 +34,8 @@ BASE_DIR=${BASE_DIR:-"/root"}
 CKPT_ARGS=(
    --hf-checkpoint ${BASE_DIR}/MiniMax-M2.5
    --ref-load ${BASE_DIR}/MiniMax-M2.5_torch_dist
-   --load ${BASE_DIR}/MiniMax-M2.5_slime/
-   --save ${BASE_DIR}/MiniMax-M2.5_slime/
+   --load ${BASE_DIR}/MiniMax-M2.5_vime/
+   --save ${BASE_DIR}/MiniMax-M2.5_vime/
    --save-interval 20
    --megatron-to-hf-mode raw
    --model-name minimax_m2

scripts/run-qwen2.5-0.5B-reproducibility.sh

@@ -1,6 +1,8 @@
 #!/bin/bash
 
 # for rerun the task
+pkill -9 -f '[v]llm serve|VLL[M]::'
+sleep 3
 ray stop --force
 pkill -9 ray
 pkill -9 python
@@ -13,12 +15,7 @@ set -ex
 # will prevent ray from buffering stdout/stderr
 export PYTHONUNBUFFERED=1
 
-# Bitwise reproduction depends on a fixed parallel/reduction layout, so the GPU
-# count is pinned here (matching the upstream recipe) rather than auto-detected.
-NUM_GPUS=8
-
 SCRIPT_DIR="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" &>/dev/null && pwd)"
-VIME_ROOT="$(cd -- "${SCRIPT_DIR}/.." &>/dev/null && pwd)"
 source "${SCRIPT_DIR}/models/qwen2.5-0.5B.sh"
 
 CKPT_ARGS=(
@@ -112,27 +109,21 @@ MISC_ARGS=(
 )
 
 # launch the master node of ray in container
-export MASTER_ADDR=${MASTER_ADDR:-"127.0.0.1"}
-ray start --head --node-ip-address ${MASTER_ADDR} --num-gpus ${NUM_GPUS} --disable-usage-stats --dashboard-host=0.0.0.0 --dashboard-port=8265
-
-# Build the runtime environment JSON with proper variable substitution.
-# The NCCL_ALGO / NVTE / CUBLAS settings below are required for bitwise determinism.
-RUNTIME_ENV_JSON="{
-  \"env_vars\": {
-    \"PYTHONPATH\": \"${VIME_ROOT}:/root/Megatron-LM/\",
-    \"CUDA_DEVICE_MAX_CONNECTIONS\": \"1\",
-    \"NCCL_ALGO\": \"Ring\",
-    \"NVTE_ALLOW_NONDETERMINISTIC_ALGO\": \"0\",
-    \"CUBLAS_WORKSPACE_CONFIG\": \":4096:8\"
-  }
-}"
+ray start --head --node-ip-address 127.0.0.1 --num-gpus 8 --disable-usage-stats
 
 ray job submit --address="http://127.0.0.1:8265" \
-   --runtime-env-json="${RUNTIME_ENV_JSON}" \
+   --runtime-env-json='{
+     "env_vars": {
+        "PYTHONPATH": "/root/Megatron-LM",
+        "CUDA_DEVICE_MAX_CONNECTIONS": "1",
+        "NCCL_ALGO": "Ring",
+        "NVTE_ALLOW_NONDETERMINISTIC_ALGO": "0",
+        "CUBLAS_WORKSPACE_CONFIG": ":4096:8"
+     }
+   }' \
    -- python3 train.py \
-   --train-backend megatron \
    --actor-num-nodes 1 \
-   --actor-num-gpus-per-node ${NUM_GPUS} \
+   --actor-num-gpus-per-node 8 \
    --colocate \
    --calculate-per-token-loss \
    ${MODEL_ARGS[@]} \

tests/test_agent_adapters.py

@@ -109,16 +109,16 @@ def flush() -> None:
 def test_session_id_comes_from_protocol_fields_not_custom_header():
     assert (
         openai._request_session_id(
-            FakeRequest({"X-Slime-Session-Id": "custom"}),
+            FakeRequest({"X-Vime-Session-Id": "custom"}),
             {"metadata": {"session_id": "meta-session"}, "user": "body-user"},
         )
         == "meta-session"
     )
     assert (
-        openai._request_session_id(FakeRequest({"X-Slime-Session-Id": "custom"}), {"user": "body-user"}) == "body-user"
+        openai._request_session_id(FakeRequest({"X-Vime-Session-Id": "custom"}), {"user": "body-user"}) == "body-user"
     )
     assert (
-        anthropic._request_session_id(FakeRequest({"X-Slime-Session-Id": "custom", "X-Api-Key": "anthropic-key"}))
+        anthropic._request_session_id(FakeRequest({"X-Vime-Session-Id": "custom", "X-Api-Key": "anthropic-key"}))
         == "anthropic-key"
     )
     assert (
@@ -138,7 +138,7 @@ def test_anthropic_translation_keeps_tool_results_and_tool_schema():
             "content": [
                 {"type": "thinking", "thinking": "plan"},
                 {"type": "text", "text": "ok"},
-                {"type": "tool_use", "name": "lookup", "input": {"q": "slime"}},
+                {"type": "tool_use", "name": "lookup", "input": {"q": "vime"}},
             ],
         },
         {"role": "user", "content": [{"type": "tool_result", "tool_use_id": "u1", "content": "result"}]},
@@ -156,7 +156,7 @@ def test_anthropic_translation_keeps_tool_results_and_tool_schema():
             "role": "assistant",
             "content": "ok",
             "reasoning_content": "plan",
-            "tool_calls": [{"function": {"name": "lookup", "arguments": {"q": "slime"}}}],
+            "tool_calls": [{"function": {"name": "lookup", "arguments": {"q": "vime"}}}],
         },
         {"role": "tool", "content": "result"},
     ]
@@ -185,7 +185,7 @@ def test_openai_translation_and_responses_input_shapes():
                     {
                         "id": "call_1",
                         "type": "function",
-                        "function": {"name": "lookup", "arguments": {"q": "slime"}},
+                        "function": {"name": "lookup", "arguments": {"q": "vime"}},
                     }
                 ],
             },
@@ -210,7 +210,7 @@ def test_openai_translation_and_responses_input_shapes():
                 {
                     "id": "call_1",
                     "type": "function",
-                    "function": {"name": "lookup", "arguments": '{"q": "slime"}'},
+                    "function": {"name": "lookup", "arguments": '{"q": "vime"}'},
                 }
             ],
         },
@@ -313,7 +313,7 @@ async def fake_generate(prompt_ids, session, body, app, **kwargs):
 
     async def run_case():
         monkeypatch.setattr(openai, "_generate", fake_generate)
-        raw = "use it <tool_call><function=lookup><parameter=query>slime</parameter></function></tool_call>"
+        raw = "use it <tool_call><function=lookup><parameter=query>vime</parameter></function></tool_call>"
         tokenizer = ToyTokenizer({(451,): raw})
         adapter = openai.OpenAIAdapter(tokenizer=tokenizer, vllm_url="http://unused")
         adapter.open_session("sid-chat-tool-stream", sampling_defaults={"max_new_tokens": 8})
@@ -350,7 +350,7 @@ async def run_case():
         assert any(c["choices"][0]["delta"] == {"content": "use it"} for c in chunks)
         assert tool_delta["tool_calls"][0]["index"] == 0
         assert tool_delta["tool_calls"][0]["function"]["name"] == "lookup"
-        assert tool_delta["tool_calls"][0]["function"]["arguments"] == '{"query": "slime"}'
+        assert tool_delta["tool_calls"][0]["function"]["arguments"] == '{"query": "vime"}'
         assert chunks[-1]["choices"][0]["finish_reason"] == "tool_calls"
         assert segments[0].response_ids == [451]
 
@@ -364,7 +364,7 @@ async def fake_generate(prompt_ids, session, body, app, **kwargs):
 
     async def run_case():
         monkeypatch.setattr(openai, "_generate", fake_generate)
-        raw = "look <tool_call><function=lookup><parameter=query>slime</parameter></function></tool_call>"
+        raw = "look <tool_call><function=lookup><parameter=query>vime</parameter></function></tool_call>"
         tokenizer = ToyTokenizer({(301,): raw})
         adapter = openai.OpenAIAdapter(tokenizer=tokenizer, vllm_url="http://unused")
         adapter.open_session("sid-responses", sampling_defaults={"max_new_tokens": 8})
@@ -399,7 +399,7 @@ async def run_case():
         assert output_types == ["message", "function_call"]
         assert data["output"][0]["content"][0]["text"] == "look"
         assert function_call["name"] == "lookup"
-        assert function_call["arguments"] == '{"query": "slime"}'
+        assert function_call["arguments"] == '{"query": "vime"}'
         assert segments[0].response_ids == [301]
 
     asyncio.run(run_case())
@@ -414,7 +414,7 @@ async def fake_generate(prompt_ids, session, body, app, **kwargs):
 
     async def run_case():
         monkeypatch.setattr(openai, "_generate", fake_generate)
-        raw = "<tool_call><function=lookup><parameter=query>slime</parameter></function></tool_call>"
+        raw = "<tool_call><function=lookup><parameter=query>vime</parameter></function></tool_call>"
         tokenizer = ToyTokenizer({(551,): raw})
         adapter = openai.OpenAIAdapter(tokenizer=tokenizer, vllm_url="http://unused")
         adapter.open_session("sid-responses-tool-stream", sampling_defaults={"max_new_tokens": 8})
@@ -450,7 +450,7 @@ async def run_case():
         assert created["type"] == "response.created"
         assert [item["type"] for item in created["response"]["output"]] == ["function_call"]
         assert completed_call["name"] == "lookup"
-        assert completed_call["arguments"] == '{"query": "slime"}'
+        assert completed_call["arguments"] == '{"query": "vime"}'
         assert segments[0].response_ids == [551]
 
     asyncio.run(run_case())
@@ -616,7 +616,7 @@ async def run_case():
         upstream_server = TestServer(upstream_app)
         await upstream_server.start_server()
 
-        tool_raw = "<tool_call><function=lookup><parameter=query>slime</parameter></function></tool_call>"
+        tool_raw = "<tool_call><function=lookup><parameter=query>vime</parameter></function></tool_call>"
         tokenizer = ScriptedTokenizer(
             prompts=[
                 [10, 11],
@@ -637,7 +637,7 @@ async def run_case():
                 headers={"Authorization": "Bearer sid-openai-token"},
                 json={
                     "model": "actor",
-                    "input": "find slime",
+                    "input": "find vime",
                     "max_output_tokens": 5,
                     "tools": [
                         {
@@ -657,12 +657,12 @@ async def run_case():
                 json={
                     "model": "actor",
                     "input": [
-                        {"role": "user", "content": "find slime"},
+                        {"role": "user", "content": "find vime"},
                         function_call,
                         {
                             "type": "function_call_output",
                             "call_id": function_call["call_id"],
-                            "output": "found slime",
+                            "output": "found vime",
                         },
                     ],
                     "max_output_tokens": 7,
@@ -684,7 +684,7 @@ async def run_case():
         assert first.status == 200
         assert second.status == 200
         assert function_call["name"] == "lookup"
-        assert function_call["arguments"] == '{"query": "slime"}'
+        assert function_call["arguments"] == '{"query": "vime"}'
         assert second_data["output"][0]["content"][0]["text"] == "done"
         assert [req["token_ids"] for req in upstream.requests] == [[10, 11], [10, 11, 20, 21, 30, 31]]
         assert upstream.routing_keys == ["sid-openai-token", "sid-openai-token"]
@@ -712,7 +712,7 @@ async def run_case():
         upstream_server = TestServer(upstream_app)
         await upstream_server.start_server()
 
-        tool_raw = "<tool_call><function=lookup><parameter=query>slime</parameter></function></tool_call>"
+        tool_raw = "<tool_call><function=lookup><parameter=query>vime</parameter></function></tool_call>"
         tokenizer = ScriptedTokenizer(
             prompts=[
                 [110, 111],
@@ -737,7 +737,7 @@ async def run_case():
                 json={
                     "model": "actor",
                     "max_tokens": 5,
-                    "messages": [{"role": "user", "content": [{"type": "text", "text": "find slime"}]}],
+                    "messages": [{"role": "user", "content": [{"type": "text", "text": "find vime"}]}],
                     "tools": [
                         {
                             "name": "lookup",
@@ -756,15 +756,15 @@ async def run_case():
                     "model": "actor",
                     "max_tokens": 7,
                     "messages": [
-                        {"role": "user", "content": [{"type": "text", "text": "find slime"}]},
+                        {"role": "user", "content": [{"type": "text", "text": "find vime"}]},
                         {"role": "assistant", "content": first_data["content"]},
                         {
                             "role": "user",
                             "content": [
                                 {
                                     "type": "tool_result",
                                     "tool_use_id": tool_use["id"],
-                                    "content": "found slime",
+                                    "content": "found vime",
                                 }
                             ],
                         },
@@ -786,7 +786,7 @@ async def run_case():
         assert first.status == 200
         assert second.status == 200
         assert tool_use["name"] == "lookup"
-        assert tool_use["input"] == {"query": "slime"}
+        assert tool_use["input"] == {"query": "vime"}
         assert second_data["content"] == [{"type": "text", "text": "anthropic done"}]
         assert [req["token_ids"] for req in upstream.requests] == [[110, 111], [110, 111, 120, 121, 130]]
         assert upstream.routing_keys == ["sid-anthropic-token", "sid-anthropic-token"]