diff --git a/3rdparty/Gym-workspace/Gym b/3rdparty/Gym-workspace/Gym index bb9b26de3da..d9ad9d2e693 160000 --- a/3rdparty/Gym-workspace/Gym +++ b/3rdparty/Gym-workspace/Gym @@ -1 +1 @@ -Subproject commit bb9b26de3dac1294faccccb5ed70f3386c16de10 +Subproject commit d9ad9d2e693cc1378b89e14147ed41e67cf9f1e0 diff --git a/docs/design-docs/rollout-verification-boundary.md b/docs/design-docs/rollout-verification-boundary.md new file mode 100644 index 00000000000..c733ec69400 --- /dev/null +++ b/docs/design-docs/rollout-verification-boundary.md @@ -0,0 +1,652 @@ +# Freeze the Training Trajectory Before Verification + +Status: Proposal + +## Summary + +Token-capture rollouts need an explicit boundary between model generation and +reward verification. The boundary must identify the exact committed model call +whose ancestry becomes the training trajectory, and it must be durable before +the verifier is allowed to run. + +Add a `freeze_training_trajectory` transition to the Gym capture gate. The +trusted agent harness invokes it after the main agent stops and before the +verifier can consume the generated answer or patch. The gate atomically: + +1. verifies that every admitted model call has reached a terminal capture + state; +2. resolves the harness-provided terminal logical request to a committed model + call; +3. validates the terminal call's parent chain; +4. records an immutable `GenerationBoundary`; and +5. rejects any later training-capture admission for the rollout. + +Verification runs only after the gate acknowledges that boundary. After the +verifier returns a reward, the existing seal operation produces a token-free +`RolloutReceipt` using the already-frozen terminal model call. The finalizer +continues to fetch deltas from TransferQueue (TQ), verify them, and linearize +only the terminal call's ancestry. + +This makes the training trajectory deterministic without relying on TQ row +order, wall-clock timestamps, completion-file modification times, or whichever +model call happened to finish last globally. + +## Terminology + +- **Model call**: one request admitted by the capture gate and assigned a + `model_call_id`. +- **Logical request**: the harness-visible identity for a model request. It is + mapped one-to-one to a model call within a rollout. +- **Main trajectory**: the agent session whose result is submitted to the + verifier and whose ancestry is eligible for training. +- **Sibling call**: a committed call in the same rollout that is not an + ancestor of the selected terminal call, such as a subagent or abandoned + branch. +- **Generation boundary**: the durable gate record that freezes the terminal + model call before verification. +- **Seal**: the post-verification transition that binds the reward to the + frozen trajectory and returns a `RolloutReceipt`. + +"Terminal" does not mean the last call to finish across all concurrent agent +sessions. It means the last call in the selected main trajectory. Concurrent +subagent calls may finish later, but they must not replace the main terminal or +enter its parent chain. + +## Current behavior + +The current branch has strong per-call attribution: + +- NeMo RL assigns a unique `rollout_id` to each generation. +- Gym assigns a `model_call_id` to every correlated model request. +- The worker stages each delta under the deterministic TQ key + `/`. +- Staged rows and receipt records carry `rollout_id`, `model_call_id`, + `parent_call_id`, lengths, weight version, and integrity digests. +- The finalizer starts at `RolloutReceipt.terminal_model_call_id` and follows + only its parent ancestry to build the training row. + +The SWE harness currently derives `terminal_logical_request_id` from the +`response.id` in the newest copied completion artifact for the main session. +NeMo RL passes that value to the gate when it seals the rollout after Gym has +already run the verifier. + +This selects a terminal chain deterministically in the normal SWE path, but it +does not prove a generation/verification boundary: + +- TQ rows have identity but no authoritative execution order. +- The gate does not record when generation ended or verification began. +- The terminal is selected from a persisted artifact rather than handed off + directly at the agent-to-verifier transition. +- The gate remains open to new training-capture calls until the post-verifier + seal request arrives. +- `seal_rollout` can verify that no calls are currently in flight, but it + cannot prove that the selected call was frozen before verification started. + +Manifest order must not be used as a substitute. Calls can execute +concurrently, and insertion or commit order does not identify the main +trajectory. + +## Required invariants + +The implementation must enforce the following invariants. + +### Identity + +1. A logical request maps to at most one model call within a rollout. +2. The frozen terminal logical request resolves to exactly one committed model + call. +3. Every record in the selected ancestry belongs to the same rollout. +4. Every non-root record names a committed parent whose cumulative length + equals the child's `prev_len`. + +### Boundary ordering + +1. The main agent has stopped issuing calls before the freeze request. +2. The gate acknowledges the freeze before verification begins. +3. No training-capture call is admitted after the freeze. +4. A freeze fails while any admitted call is still committing or otherwise + lacks a known terminal capture outcome. + +### Training selection + +1. The terminal model call is immutable after the freeze. +2. The reward is attached only to that frozen terminal and its ancestry. +3. Sibling calls never enter the training row. +4. Missing or ambiguous terminal attribution fails closed; it never falls back + to manifest order or a timestamp. + +### Retry safety + +1. An identical freeze retry returns the same boundary. +2. Reusing the freeze operation ID with a different terminal is rejected. +3. An identical seal retry returns the same receipt. +4. A seal request cannot change the frozen terminal. + +## Proposed protocol + +### Lifecycle + +```mermaid +stateDiagram-v2 + [*] --> Open: register rollout + Open --> Open: admit and commit calls + Open --> Frozen: freeze_training_trajectory + Open --> Failed: capture or rollout failure + Frozen --> Frozen: idempotent freeze retry + Frozen --> Sealed: seal with verifier reward + Frozen --> Failed: explicit verifier/rollout abort + Sealed --> Sealed: idempotent seal retry + Failed --> [*] + Sealed --> [*] +``` + +The verifier is outside the gate state machine, but its ordering is strict: + +```text +agent calls complete + -> freeze_training_trajectory (durable ACK) + -> verifier starts + -> seal(reward, boundary_id) + -> finalizer verifies TQ rows and publishes the training row +``` + +### Structured terminal handoff + +The trusted harness must produce a structured generation result directly from +the completed main session: + +```python +GenerationOutcome( + rollout_id=..., + trajectory_id=..., + terminal_logical_request_id=..., + terminal_turn_index=..., +) +``` + +`terminal_logical_request_id` must come from the live model response or from a +logical request ID generated before dispatch and echoed by the response. It +must not be inferred by scanning files after verification. + +`trajectory_id` identifies the main session when the harness supports +subagents. `terminal_turn_index` is a monotonically increasing main-session +turn number used for validation and diagnostics. It is not a global ordering +across sibling sessions. + +Persisted completion files can remain observability artifacts. During +migration, an adapter may recover a terminal ID from them, but strict mode must +require the direct structured handoff. + +### Call admission metadata + +For the gate to validate which terminal is eligible, each admitted call should +carry optional trajectory metadata: + +```python +CallAttribution( + trajectory_id=..., + trajectory_role="main" | "subagent" | "other", + turn_index=..., +) +``` + +The gate enforces uniqueness of `(trajectory_id, turn_index)` within a rollout. +For the main trajectory, the frozen terminal must: + +- have `trajectory_role == "main"`; +- match the `trajectory_id` in `GenerationOutcome`; and +- have the highest committed `turn_index` in that main trajectory. + +The parent graph remains the token authority. Turn indices help reject an +incorrect terminal selection but are not used to concatenate tokens. + +Harnesses that cannot yet provide trajectory metadata may use a compatibility +mode that validates only the logical request and parent chain. Such rollouts +should expose a metric showing that the stronger main-session check was not +available. + +### Freeze request + +Add an authenticated control operation: + +```http +POST /training-token-capture/control/rollouts/{rollout_id}/freeze +``` + +```json +{ + "owner_id": "...", + "operation_id": "...", + "trajectory_id": "...", + "terminal_logical_request_id": "...", + "terminal_turn_index": 17 +} +``` + +The request is valid only while the rollout is `Open`. Under the rollout's +atomic state transaction, the gate: + +1. authenticates the boundary caller; +2. rejects any calls in `admitted` or `committing` state; +3. rejects a rollout with a failed or poisoned call; +4. resolves `terminal_logical_request_id` through the rollout's logical-request + index; +5. requires the resolved call to be committed and staged; +6. validates trajectory metadata when present; +7. validates the terminal call's complete, acyclic parent chain; +8. computes a digest over the selected call IDs and their immutable manifest + records; +9. persists a `GenerationBoundary`; and +10. changes the rollout state to `Frozen` before returning success. + +The response is token-free: + +```python +GenerationBoundary( + schema_version=1, + boundary_id=..., + rollout_id=..., + trajectory_id=..., + terminal_logical_request_id=..., + terminal_model_call_id=..., + terminal_turn_index=..., + terminal_commit_sequence=..., + selected_call_ids=(...), + selected_manifest_digest=..., +) +``` + +`terminal_commit_sequence` is a gate-assigned monotonic sequence useful for +audit and metrics. It does not define the trajectory; the explicit terminal +and parent links do. + +`boundary_id` should be a deterministic digest of the immutable request and +selected manifest, so an identical retry returns the same value. + +The digest input is canonical and domain-separated: + +```text +SHA-256( + "nemo-gym-generation-boundary-v1" + || rollout_id + || trajectory_id + || terminal_logical_request_id + || terminal_model_call_id + || terminal_turn_index + || each selected CallRecord in root-to-terminal order +) +``` + +Strings and records use the same length-delimited canonical encoding on every +producer and verifier. JSON serialization, dictionary insertion order, and +timestamps are not digest inputs. + +### Admission after freeze + +Once a rollout is `Frozen`, the training-capture route rejects new model calls +before inference dispatch. This is the enforcement that the current +post-verifier seal cannot provide. + +If verification itself needs an LLM, it must use one of the following: + +- an uncorrelated evaluation endpoint; +- a separately registered verification rollout; or +- a capability explicitly scoped to non-training capture. + +Verifier calls must never share the frozen rollout's training data capability. + +### Seal request + +After verification, NeMo RL seals using the boundary identity: + +```http +POST /training-token-capture/control/rollouts/{rollout_id}/seal +``` + +```json +{ + "owner_id": "...", + "operation_id": "...", + "boundary_id": "...", + "reward": 1.0 +} +``` + +The gate verifies that: + +- the rollout is `Frozen`; +- `boundary_id` matches the stored boundary; +- the selected manifest still matches its frozen digest; and +- no capture failure was recorded. + +The receipt's `terminal_model_call_id` is copied from the boundary. The seal +request cannot supply or replace it. + +For one compatibility release, `seal` may continue accepting +`terminal_logical_request_id`, but it must exactly match the stored boundary. +Calling `seal` on an unfrozen rollout should be allowed only behind an explicit +legacy compatibility flag and should increment a metric. + +### Receipt extension + +The sealed receipt embeds the complete token-free boundary because the +finalizer does not have access to live gate state: + +```python +RolloutReceipt( + rollout_id=..., + reward=..., + terminal_model_call_id=..., + manifest=[...], + generation_boundary=GenerationBoundary(...), + capture_poisoned=False, + failure_reason=None, +) +``` + +Receipt validation requires: + +- `generation_boundary.rollout_id == receipt.rollout_id`; +- `generation_boundary.terminal_model_call_id == + receipt.terminal_model_call_id`; +- every `selected_call_id` is present exactly once in `manifest`; and +- recomputing the boundary digest from the receipt manifest yields the embedded + `boundary_id`. + +This lets any finalizer validate terminal selection without trusting the +component that transported the receipt. + +## TransferQueue and finalization + +The minimal protocol does not require ordering fields in TQ. Each staged row +already has the identity and parent link required to verify a selected chain. +The boundary belongs in gate or ledger state because it describes rollout +lifecycle, not token payload storage. + +The finalizer should additionally validate that: + +1. the receipt contains a supported generation-boundary schema version; +2. `terminal_model_call_id` matches the boundary terminal; +3. the recomputed selected call-ID sequence and manifest digest match the + boundary; and +4. the linearized row contains exactly the frozen terminal ancestry. + +The initial implementation can keep the current receipt manifest containing +all committed calls. The finalizer will verify all rows but train only the +terminal chain. A later wire revision may split this into: + +- a selected-chain manifest used for training; and +- a cleanup manifest for unselected sibling staging keys. + +That split would prevent an unselected sibling's missing row from invalidating +an otherwise sound main trajectory, but it is not required to establish the +verification boundary. + +## Authorization and secret custody + +The model-calling agent sandbox currently receives a rollout-local data +capability. It must not receive authority to freeze or seal the rollout. +Otherwise untrusted agent code could end generation early or choose its own +training terminal. + +The freeze operation therefore needs a boundary capability held only by the +trusted Gym orchestration process. Two implementation options are acceptable: + +1. issue a rollout-scoped boundary capability at registration and pass it only + to the trusted agent wrapper; or +2. give the trusted Gym server a server-to-server credential scoped to freeze + operations. + +The preferred option is a rollout-scoped boundary capability because it limits +the effect of credential disclosure. Its plaintext value must not be written +to completion files, `params.json`, logs, the agent container, receipts, or TQ +tags. Gate state stores only its digest. + +The existing NeMo RL owner remains responsible for register, fail, and seal. + +## Integration with the capture ledger proposal + +This protocol works with both the current `GateStateStore` and the proposed +per-rollout [capture ledger](token-capture-ledger.md). + +With the current store, add the boundary and lifecycle state to +`GateRolloutState` and retain it in the seal tombstone. + +With `CaptureLedger`, add one event: + +```python +GenerationFrozen( + operation_id=..., + boundary=GenerationBoundary(...), +) +``` + +The event is appended and fsynced under the rollout lock. Later `CallStarted` +events are invalid. `RolloutSealed` references the boundary ID and reward. The +ledger fold can reproduce both the boundary and receipt for idempotent retries. + +The generation-boundary contract should be implemented independently of the +gate-storage migration so the correctness guarantee does not depend on which +state backend lands first. + +## Failure handling + +| Failure | Required outcome | +| --- | --- | +| Main agent exits without a terminal logical request | Freeze fails; rollout is non-trainable and staged rows are cleaned up | +| Terminal logical request is unknown | Freeze fails closed; never select another manifest row | +| Terminal call was admitted but did not commit | Freeze fails while the call is in flight; after timeout, mark the rollout failed | +| Any call is still committing | Return a retryable conflict; do not start verification | +| Terminal trajectory metadata does not match | Freeze fails closed | +| Parent is missing, cyclic, or length-inconsistent | Freeze fails closed | +| Freeze response is lost | Retry with the same operation ID and receive the identical boundary | +| A call arrives after freeze | Reject before inference dispatch | +| Verifier crashes | The boundary remains frozen; policy decides whether to seal with a failure reward or fail the rollout | +| Seal response is lost | Retry with the same operation ID and boundary ID | +| Gate process crashes after freeze | Recover the durable boundary and continue verification/seal retry | +| Boundary capability leaks to the agent | Treat as a security incident; scoped capability limits exposure to one rollout | + +Ambiguous capture outcomes remain non-trainable. The boundary must not turn a +missing acknowledgement into evidence that a call did not execute. + +## Metrics and observability + +Add the following gate metrics: + +- `generation_freeze_succeeded` +- `generation_freeze_retries` +- `generation_freeze_conflicts` +- `generation_freeze_missing_terminal` +- `generation_freeze_invalid_terminal` +- `generation_freeze_pending_calls` +- `post_freeze_calls_rejected` +- `legacy_unfrozen_seals` +- `trajectory_metadata_compatibility_mode` + +Logs should include rollout ID, boundary ID, terminal model-call ID, selected +call count, and rejection category. They must not include capability values or +token arrays. + +For end-to-end diagnosis, expose these non-authoritative timestamps: + +- main agent stopped; +- freeze requested; +- freeze committed; +- verifier started; +- verifier completed; and +- seal committed. + +Correctness uses state transitions, not timestamp comparisons. + +## Compatibility and configuration + +Use one centralized user-facing setting: + +```yaml +token_capture: + generation_boundary_mode: compat # off | compat | required +``` + +- `off`: retain the current post-verifier terminal selection. Intended only as + an emergency rollback mode. +- `compat`: use the boundary when the harness supplies one, otherwise use the + legacy seal path and increment `legacy_unfrozen_seals`. +- `required`: do not start verification without a durable boundary, and reject + receipts that do not embed one. + +The first release can default existing configurations to `compat` while the +SWE capture recipes explicitly set `required`. After all supported capture +harnesses implement the boundary, change the centralized schema default to +`required` and remove the legacy terminal argument in a later wire revision. + +No call site may silently substitute a mode. The default belongs in the +user-facing config schema and exemplar YAML, consistent with NeMo RL config +conventions. + +## Expected code changes + +| Repository path | Change | +| --- | --- | +| Gym `nemo_gym/token_id_capture/staging/records.py` | Add versioned `CallAttribution` and `GenerationBoundary` wire models; extend `RolloutReceipt` | +| Gym `nemo_gym/token_id_capture/gate_store.py` | Add rollout lifecycle, boundary-capability digest, frozen boundary, and retry tombstone state | +| Gym `nemo_gym/token_id_capture/gate.py` | Implement atomic freeze, graph validation, boundary digest, post-freeze admission rejection, and boundary-bound seal | +| Gym `nemo_gym/token_id_capture/control_routes.py` | Add the authenticated freeze route and typed request/response models | +| Gym capture middleware and model adapter | Carry trajectory attribution and reject frozen-rollout calls before engine dispatch | +| Gym `responses_api_agents/swe_agents/app.py` | Split agent completion from verification, retain the direct terminal logical ID, freeze, then unblock evaluation | +| NeMo RL `nemo_rl/environments/nemo_gym.py` | Configure boundary mode, distribute scoped credentials, and seal with `boundary_id` | +| NeMo RL `nemo_rl/experience/blackbox_finalizer.py` | Validate the embedded boundary and exact selected ancestry before publishing | +| NeMo RL token-capture config and exemplar YAMLs | Define the centralized boundary mode and strict SWE recipe posture | + +The SWE runner currently starts the evaluation container early and lets it wait +for the model patch. That optimization can remain, but the patch or other +verifier input must stay unavailable until the freeze acknowledgement is +durable. A prestarted process does not violate the boundary as long as it +cannot begin verification work. + +## Implementation plan + +### Phase 1: Gate boundary + +1. Add `Open`, `Frozen`, and terminal rollout state to the current Gym gate. +2. Add `GenerationBoundary` and the authenticated freeze route. +3. Reject training-capture admission after freeze. +4. Change seal to consume a boundary ID and copy the frozen terminal. +5. Preserve boundary information in sealed tombstones for retry recovery. + +### Phase 2: Trusted harness handoff + +1. Have the model-client adapter retain the main session's terminal logical + request ID directly. +2. Add main-trajectory identity and turn indices where agent frameworks expose + them. +3. Freeze after the agent process exits and before making its patch or answer + available to the verifier. +4. Keep completion-file extraction only as an explicitly measured compatibility + path. +5. Ensure subagent sessions cannot overwrite the main terminal. + +### Phase 3: NeMo RL integration + +1. Register or distribute the rollout-scoped boundary capability without + exposing it to the sandbox. +2. Carry `boundary_id` in the token-free Gym result. +3. Seal with `boundary_id` and reward. +4. Reject receipt-mode results that lack a boundary when strict mode is + enabled. +5. Validate boundary identity during black-box finalization. + +### Phase 4: Ledger backend + +1. Add `GenerationFrozen` to `CaptureLedger`. +2. Reproduce boundary and seal retry behavior by folding ledger events. +3. Remove duplicate boundary state when the gate-store migration completes. + +## Required tests + +### Gate unit tests + +- Freeze resolves a logical request to the expected model call. +- Freeze stores the terminal chain and rejects a sibling as terminal when it is + not part of the main trajectory. +- Concurrent main and subagent branches freeze the main terminal regardless of + completion order. +- Freeze fails with admitted or committing calls. +- Freeze fails for failed, poisoned, unknown, or uncommitted calls. +- Freeze validates an acyclic, length-consistent parent graph. +- Identical freeze retries return an identical boundary. +- Conflicting freeze retries are rejected. +- New call admission is rejected after freeze. +- Seal cannot change the terminal and requires the correct boundary ID. +- State-store restart preserves freeze and seal retry behavior. + +### Harness tests + +- The main session's direct response ID is selected without an mtime scan. +- The freeze acknowledgement occurs before the verifier is unblocked. +- Subagent completion files and later sibling completions cannot change the + terminal. +- The boundary capability is absent from sandbox mounts, persisted parameters, + completion files, logs, and returned metadata. +- Missing direct terminal attribution fails closed in strict mode. + +### NeMo RL tests + +- Receipt-mode postprocessing requires a boundary ID in strict mode. +- Seal forwards the boundary ID and reward without choosing a terminal. +- The finalizer rejects a receipt/boundary terminal mismatch. +- The finalizer linearizes exactly the frozen ancestry. +- A rejected or missing boundary produces the existing masked placeholder and + preserves GRPO group shape. + +### End-to-end tests + +- A SWE rollout stages multiple turns, freezes, verifies, seals, and trains the + exact main terminal chain. +- A concurrent subagent finishes after the main terminal but is excluded from + training. +- A verifier that attempts to reuse the training capture route is rejected. +- Gate-worker restart between freeze and seal preserves the selected + trajectory. +- Lost freeze and seal responses are recovered through idempotent retries. + +## Alternatives considered + +### Select the final manifest row + +Rejected. Manifest order reflects admission or storage behavior, not the main +trajectory, and concurrent calls make it ambiguous. + +### Select the greatest timestamp or sequence number + +Rejected as the trajectory definition. A later subagent or verifier call could +win. Gate sequence numbers remain useful only for audit after the main +trajectory is explicitly identified. + +### Keep selecting the newest completion file + +Rejected as the strict contract. File modification time is not an atomic +generation boundary and cannot stop later capture admissions. It remains a +temporary compatibility adapter. + +### Seal before verification + +Rejected because the current seal includes the reward and terminates rollout +state. Splitting terminal freeze from reward seal preserves the useful +post-verification receipt contract while establishing the boundary earlier. + +### Let the agent sandbox freeze its own trajectory + +Rejected. The sandbox is part of the evaluated workload and must not control +which of its calls becomes training data. + +## Recommendation + +Implement `freeze_training_trajectory` as a distinct, durable transition and +make verifier startup depend on its acknowledgement. Use an explicit main +trajectory terminal from the live model-call path, reject later +training-capture admissions, and allow the post-verification seal to attach +only the reward. + +This is the smallest protocol change that turns the current trusted terminal +hint into an enforceable guarantee that the exact pre-verification trajectory +is the one used for training. diff --git a/docs/design-docs/token-capture-ledger.md b/docs/design-docs/token-capture-ledger.md new file mode 100644 index 00000000000..f6cebb59d3c --- /dev/null +++ b/docs/design-docs/token-capture-ledger.md @@ -0,0 +1,135 @@ +# Token Capture Lineage Ledger + +Exact-token capture for blackbox agentic rollouts is coordinated by a single +per-rollout **capture ledger**: NeMo Gym's `LineageStore`, extended so that its +append-only JSONL rows are simultaneously the request-time lineage index and +the token-free record of rollout capture state. There is no separate gate +state machine; serving workers coordinate only through the ledger, and NeMo RL +(the rollout owner) assembles the `RolloutReceipt` itself at rollout end. + +The external staging contract (`StagingSink` / `StagingSource`), the vLLM +worker capture path, and the `verify_and_linearize()` trust boundary are +unchanged from the worker-custody design. The recipe exercising this path end +to end is [Nano SWE with Token Capture](../guides/nano-swe-token-capture.md); +the verification trust boundary is described in +[Rollout Verification Boundary](rollout-verification-boundary.md). + +## Why a ledger and not a gate + +An earlier iteration paired the lineage store with a `RolloutCaptureGate` and +a cross-process `GateStateStore`. The gate did not provide a second lineage +algorithm — parent resolution ran upstream through `LineageStore.resolve()`, +and the gate cross-checked that result against its own copy of the call state, +storing each call's cumulative token IDs **twice** (gate state + lineage +JSONL). Its file-backed state store also serialized the entire global gate +state — every live rollout's cumulative token arrays — under one exclusive +lock, three transactions per model call. + +Everything the gate legitimately provided — admission, rollout completeness, +terminal selection, cleanup — is either a pure function of the lineage result +or belongs to the framework that already owns the rollout. So each +responsibility moved to its natural owner and the redundant state machine was +deleted. + +## The ledger + +`FileLineageStore` writes one locked, fsynced JSONL row per committed call. +In external-staging mode (`token_id_capture.external_staging: true`) each row +additionally carries the token-free `CallRecord` custody columns — +`parent_call_id`, `staging_key`, `weight_version`, `prev_len` / `delta_len` / +`cum_len`, the staged record's `digest` and `extras_digest`, `mode`, and +`logical_request_id` (the client header when present, else the vLLM response +id). Three surfaces make it the single record of capture state (the +`CaptureLedger` protocol): + +- `record(...)` — the extended commit row, written by the model server's + commit hook after the worker's `CommitCoords` arrive. +- `record_failure(rollout_id, model_call_id, reason)` — a poison row for a + call whose capture did not commit. Failure rows carry no fingerprint, so + `resolve()` can never return them as parents. +- `manifest(rollout_id)` — the token-free read-back (committed rows + + failures), exposed over one bearer-protected control route: + `GET /training-token-capture/rollouts/{rollout_id}/manifest`. + +`InMemoryLineageStore` cannot serve the ledger role: its resolution index +evicts rollouts under memory bounds, which is fine for a cache but not for a +completeness record. External staging requires a non-evicting store and +rejects the in-memory store at startup. + +## Admission is a pure function + +When external staging is enabled, `resolve_parent()` builds the +`CaptureAdmission` directly from the lineage result — a strict tri-state: + +| Lineage outcome | Admission | +| --- | --- | +| `ROOT` — empty assistant fingerprint, or unmatched fingerprint on a rollout with no ledger rows (seeded assistant history) | `text` mode, no parent | +| `MATCH` — unique fingerprint match with verified context digest | `token_in` mode, `required_prefix_token_ids` = parent's cumulative tokens | +| `UNRESOLVED` — non-empty fingerprint with no match, ambiguity, or digest mismatch | no admission; `record_failure()` poisons the call | + +`UNRESOLVED` is never silently converted into a new root: doing so would turn +earlier policy-generated tokens into mask-zero prompt tokens and corrupt the +training row. The completion still serves the agent; only training capture is +poisoned. + +## Commit ordering + +The invariant the external sink requires — *a call must not become a lineage +parent until its staged record is durable* — holds structurally: the worker +stages through `StagingSink.stage()` before acknowledging, coordinates exist +only after the bytes are durable, and the ledger row (which is what makes a +call resolvable as a parent) is written only after the coordinates arrive. +On `disposition == "staged"` the commit hook reconstructs +`cumulative = parent_tokens + token_ids_delta` and appends the extended row; +on `capture_failed`, missing coordinates, or any acknowledgement error it +appends a failure row instead. A request that dies after admission is poisoned +from the capture middleware's `finally` hook. + +## Framework-owned receipt and cleanup + +NeMo RL fetches the manifest at rollout end and assembles the receipt locally: + +- `manifest` = the fetched `CallRecord` list, deduped by `model_call_id`; +- `terminal_model_call_id` = the row whose `logical_request_id` matches the + rollout's reported terminal logical request (a response id); +- `capture_poisoned` = any failure row present, or no row for the terminal + request. + +Terminal selection has a strict precedence: **declared > heuristic > mask**. A +harness-declared terminal is authoritative — a declared id that matches no +committed row masks the rollout and never falls back. When the harness reports +no terminal at all, Gym's `select_terminal_call` infers one from the +manifest's explicit parent links (earliest-admitted root by `admitted_at`, an +extended sibling beating an abandoned childless retry); any ambiguous shape — +a retry of the final call, divergent extended branches — masks with the +selection reason. The heuristic only chooses *among* digest-verified rows: +`verify_and_linearize` still verifies the chosen chain. The receipt records +the path in `terminal_selection` and the finalizer emits +`finalize/heuristic_terminal_fraction` per group. + +`verify_and_linearize(receipt, snapshots)` runs unchanged. Retry duplicates +appear as dead-branch sibling rows in the manifest: their staged rows are +fetched, verified, and cleaned like any other, but they never join the +terminal chain (`_validate_manifest_graph` tolerates rows unreferenced by the +terminal chain). Cleanup is manifest-enumerated in the finalizer; an abandoned +dispatch's staged rows are swept with the staging partition at run end (there +is no prefix-clear primitive in the data plane yet). + +## Failure semantics (all fail-closed) + +- **Capture fails mid-rollout:** the model call still succeeds for the agent; + a failure row is written. Later calls miss resolution → `UNRESOLVED` → + more failure rows. Finalization sees failure rows → poisoned → masked + placeholder row (the group still publishes exactly N rows). +- **Terminal response lost, harness retries:** the retry is a sibling row + (per-request `uuid4` identity). The harness reports the retry's response id, + so receipt assembly selects the retry's row; the lost attempt is a dead + branch. An ambiguous mid-rollout sibling (identical regenerated text) + poisons via `UNRESOLVED` instead of silently becoming a root. +- **Crash after staging, before the ledger append:** descendants resolve + `UNRESOLVED` and poison; a terminal orphan poisons via the missing terminal + row. + +Retry *idempotency* (harness-minted logical request ids + deterministic +`model_call_id`, collapsing identical retries into the same row instead of +poisoning) is an explicit follow-up; no retry outcome is silently wrong today. diff --git a/docs/guides/nano-swe-token-capture.md b/docs/guides/nano-swe-token-capture.md new file mode 100644 index 00000000000..c165f1c0df6 --- /dev/null +++ b/docs/guides/nano-swe-token-capture.md @@ -0,0 +1,325 @@ +# Nano SWE RL with Ledger-Authoritative Token Capture + +A reproducible 6-node recipe that runs agentic SWE RL on +Nemotron-3-Nano-30B-A3B with **exact-token capture** enabled: the vLLM worker +stages each model call's token delta durably into the TransferQueue data +plane, the Gym capture ledger serves verified prefix token ids back on every follow-up +call (token-in), and the trainer consumes rows rebuilt from the staged deltas. +No token echo over HTTP, no re-tokenization of agent history — the tokens the +engine sampled are byte-for-byte the tokens the trainer sees. + +It builds directly on the [Nano SWE TransferQueue +recipe](nano-swe-transferqueue.md); read that first for the cluster shape, +`swe_nano.env` setup, and the SingleController constraints. This guide covers +only what token capture adds. + +## Verified result + +Pool-only smoke run on 6 GB200 NVL72 nodes (Slurm job `6294776`, +2026-08-18): + +| | | +|---|---| +| Entrypoint | `examples/run_grpo_single_controller.py` | +| Config | `examples/configs/ultra/nano_swe_teacher_sc.yaml` | +| Model | `nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16` | +| Shape | train 4 nodes (TP2·PP2·CP4, EP1) / gen 2 nodes (vLLM TP2 → 4 engines) | +| Smoke sizing | 2 prompts × 4 generations = GBS 8, one training step | +| Finalization | fixed pool of 2 CPU Ray actors; routed-expert assembly deferred to policy workers | +| Outcome | exit 0, step 1/1, `global_valid_seqs=8`, `invalid_row_rate=0`, routed-expert coverage 1.0, zero capture failures | +| Timing | 33m29s wall clock; 765.4s setup on warm shared caches | +| Launched via | `swe_nano_sc_capture.sh` (batch) | + +## How it works + +Legacy SWE runs recover training tokens by echoing token ids in every model +response and re-tokenizing the agent's rendered history each turn. Both are +lossy for a reasoning model — the chat template strips `` content from +history, and re-tokenizing an assistant turn can split differently than the +tokens the model actually sampled. Token capture replaces that path. + +### Anatomy of one rollout + +The call flow for a single SWE rollout, end to end: + +1. **Dispatch.** The `SingleControllerActor`'s rollout pump pulls a prompt + group from the dataset and hands it to the NemoGym environment actor, + which POSTs `/run` to the `swe_agents_train` Gym server. The run body + carries a fresh `ng_rollout_id`. There is no registration step: the + rollout's ledger file is created lazily by its first committed call. +2. **Sandbox + agent.** The SWE harness materializes the SWE-bench instance + in a sandbox and starts the OpenHands agent. The rollout id is snapshotted + into the agent's config so every LLM request uses the rollout-prefixed + token-capture route. +3. **Agent turn loop.** Each turn, OpenHands POSTs `/v1/chat/completions` + with the *full rendered history* to the policy model server, which runs + in **ledger mode**: + - the server fingerprints the incoming history and resolves which + committed ledger row this call continues (its *parent*); admission is a + pure function of that lineage result; + - on a match it attaches the parent's **exact cumulative token ids** + (`required_prefix_token_ids`) plus a capture context + (`rollout_id`, `model_call_id`, `parent_call_id`, `prev_len`), and + forwards the request to a vLLM engine; + - no assistant-authored history is a true text root; assistant history + without one verified committed parent is `UNRESOLVED` and fails closed + instead of silently starting a new chain. + Lineage and capture custody are stored together in a per-rollout + append-only JSONL file under the shared capture directory, so consecutive + calls may land on different serving workers without losing ancestry or + serializing unrelated rollouts. +4. **Generate + stage.** The vLLM worker splices the supplied prefix + verbatim, renders only the new tail through the chat template, and + generates. Before acknowledging the call it **stages the call's token + delta** — `rendered_prompt[prev_len:] + generated` ids, a loss mask + (0.0 on carried prompt, 1.0 on generated), and per-token logprobs — to + the TransferQueue staging partition (synchronous `tq_put`: bytes are + durable before the response leaves the worker). The response back to the + ledger carries only text plus token-light `CommitCoords` (~4 B/token). +5. **Commit.** Gym reconstructs cumulative tokens from the coordinates and + appends one ledger row carrying both lineage and staging custody + (including the call's `logical_request_id`). This is the authoritative + commit the next turn resolves against; Gym strips the coordinates and + returns a plain OpenAI-shaped completion to the agent. Steps 3-5 repeat for every tool call the agent + makes (tool execution happens agent-side between turns). +6. **Verify + assemble.** When the agent finishes, the harness runs the + SWE-bench verifier to score the patch (reward 0/1) and reports the + terminal response id. RL fetches the rollout's **token-free manifest** + (`GET /training-token-capture/rollouts/{id}/manifest`) and assembles the + `RolloutReceipt` locally: the manifest of committed calls (model-call ids, + staging keys, digests, weight versions), a `terminal_model_call_id` + selected by the terminal logical request id, and fail-closed poisoning + (any failure row, or a missing terminal row, masks the rollout). + Harnesses that report no terminal id (declared > heuristic > mask + precedence) fall back to Gym's `select_terminal_call`, which infers the + terminal from the manifest's parent links and masks on any ambiguity; the + per-group `finalize/heuristic_terminal_fraction` metric meters that + fallback and should stay 0 on the SWE recipe, whose harness declares. +7. **Finalize.** The controller constructs a metadata-only + `FinalizationRequest`, releases the rollout concurrency permit, and submits + it to a fixed pool of CPU Ray actors. Each actor owns a connect-only TQ + client and a `BlackboxFinalizer`; it fetches the staged deltas named by the + manifest, re-verifies them (digest, lengths, mask shape, weight-version + tags), linearizes the terminal chain into one exact token row, and publishes + it to the training partition. A rejected rollout becomes a masked + placeholder so the GRPO group keeps its shape. The pool is the only + token-capture finalization path; there is no inline controller finalizer. + In the deferred-route posture below, staged rows remain until policy workers + assemble routes and the training step consumes them; direct mode clears them + immediately after publication. +8. **Train.** Once a global batch of rows is buffered, the SC takes an + optimizer step and syncs weights to the engines; the bumped + `weight_version` is stamped on subsequent calls so refit boundaries are + visible in the data. + +The token path in that flow, compressed: + +``` +agent (nv-OpenHands) ledger (vllm_model, external staging) + rollout-prefixed capture path ───► fingerprints incoming history → + resolves the parent call → sends the + parent's exact prefix token ids + │ required_prefix_token_ids + ▼ +vLLM worker: splices the prefix verbatim, renders only the new tail, + generates, then STAGES the call's token delta (ids + mask + logprobs) + to TransferQueue — a synchronous tq_put, durable before the call is + acked — and returns token-light CommitCoords on the response + │ coords (≈4 B/token) + ▼ +ledger atomically publishes coords + lineage; when the rollout ends RL +fetches the token-free manifest (model_call_ids and staging keys) from the +control route and assembles the RolloutReceipt itself + ▼ +fixed CPU Ray finalizer pool: accepts metadata only, fetches staged deltas +by key, verifies digests, rebuilds the exact row, and publishes it to TQ +``` + +The heavy bytes (token arrays, logprobs) move exactly once, worker→TQ, +node-locally. The ledger hop and the `/run` response stay token-light. + +The pieces, by repo: + +| Component | Where | +|---|---| +| Ledger mode, prefix serving, lineage | Gym `responses_api_models/vllm_model/app.py` + `nemo_gym/token_id_capture/lineage.py` | +| Rollout attribution | Gym capture middleware + `swe_agents` rollout-prefixed routing | +| Wire schema (deltas, coords, receipts) | Gym `nemo_gym/token_id_capture/staging/records.py` | +| Worker-side capture + prefix splice | `nemo_rl/models/generation/vllm/vllm_worker_async.py` | +| Staging sink/source over TransferQueue | `nemo_rl/data_plane/tq_token_sink.py` | +| Receipt → training row | `nemo_rl/experience/blackbox_finalizer.py` | +| Metadata-only finalizer actor pool | `nemo_rl/experience/finalizer_actor.py` | + +## Quick start + +### Prepare the checkout + +Run from a networked shell at this repository's root. Before submitting, +change every per-user write setting in `swe_nano.env`: + +| Variable | Required value | +|---|---| +| `CODE_DIR` | Absolute path to **this checkout**; the launcher mounts it into the container | +| `WORKSPACE_DIR` | Writable results, Ray-log, and checkpoint root | +| `HF_HOME` | Writable Hugging Face cache (~60 GB for the model) | +| `PERSISTENT_CACHE` | Writable vLLM, Triton, and Inductor cache | +| `NRL_MEGATRON_CHECKPOINT_DIR` | Writable Megatron conversion cache, normally below `PERSISTENT_CACHE` | +| `SLURM_ACCOUNT` | Slurm account you can charge | + +The container, SWE data, sandbox SIFs, and model name in the shared read-only +block can be reused. Keep `USE_SNAPSHOT=0` to execute the live files in +`CODE_DIR`; set a unique `SC_EXP_NAME` and staging partition for every active +run. Export `HF_TOKEN` if the model is not already cached. + +### Reproduce the one-step, six-node smoke + +The following is the pool-only posture validated by job `6294776`. The batch +wrapper itself supplies `token_capture.enabled=true` and pins Gym rollout +attempts to one: + +```bash +CAPTURE_OVERRIDES=( + grpo.max_num_steps=1 + grpo.num_prompts_per_step=2 + policy.train_global_batch_size=8 + token_capture.num_finalizer_workers=2 + token_capture.defer_routed_experts_to_policy=true + token_capture.staging_partition=rollout_staging_my_capture_smoke + +env.nemo_gym.policy_model.responses_api_models.vllm_model.num_workers=2 + +policy.router_replay.enabled=true + async_rl.sampler.name=windowed + +async_rl.sampler.max_staleness_versions=1 + +env.nemo_gym.model_endpoint_readiness_timeout_seconds=1800 + policy.generation.vllm_cfg.reasoning_parser_plugin=/opt/nemo-rl/nemo_rl/models/generation/vllm/reasoning_parsers/nano_v3_reasoning_parser.py +) + +# Resolve and inspect the six-node driver command without submitting. +DRY_RUN=1 SC_EXP_NAME=my-capture-smoke NG_TIC_FP_CANONICAL=1 \ + WALLTIME=1:49:00 bash swe_nano_sc_capture.sh "${CAPTURE_OVERRIDES[@]}" + +# Submit the same command. +DRY_RUN=0 SC_EXP_NAME=my-capture-smoke NG_TIC_FP_CANONICAL=1 \ + WALLTIME=1:49:00 bash swe_nano_sc_capture.sh "${CAPTURE_OVERRIDES[@]}" +``` + +`num_prompts_per_step × num_generations_per_prompt` must equal +`train_global_batch_size`; this config has four generations per prompt, hence +`2 × 4 = 8`. The finalizer pool is mandatory whenever capture is enabled, +and `num_finalizer_workers` must be positive. There is no pool enable/disable +or legacy-inline-finalizer flag. The smoke deliberately runs two Gym policy +model workers; their per-rollout ledger files are shared, so it does not rely +on single-worker request affinity. Different rollouts use different locks. + +The 1h49 allocation is suitable when the model and compilation caches are +warm and automatically selects the `short` QOS in `ultra_launch.sh`. For a +cold cache, or when the short-QOS node quota is unavailable, use +`WALLTIME=3:59:00`; `swe_nano.env` leaves `SLURM_QOS` empty for that case. +`WALLTIME` must be a Slurm time string—a value such as `3h` is invalid. + +For a longer run, change `grpo.max_num_steps` and use the longer walltime. Do +not reuse a staging partition concurrently with another run. + +### Interactive iteration + +The interactive launcher accepts the same override array: + +```bash +SC_EXP_NAME=my-capture-smoke-interactive NG_TIC_FP_CANONICAL=1 \ + WALLTIME=3:59:00 bash swe_nano_sc_capture_interactive.sh \ + "${CAPTURE_OVERRIDES[@]}" +``` + +It allocates six nodes, keeps Ray alive, and prints commands equivalent to: + +```bash +bash -attach.sh +source -run-cmd.sh +``` + +Attach to the head node and source the generated run command after Ray is up; +edit and re-source it to iterate in the same allocation. The non-capture +baseline remains `swe_nano_sc.sh` / `swe_nano_sc_interactive.sh`. If batch and +interactive runs overlap, first change the staging-partition value in +`CAPTURE_OVERRIDES` so each live run has its own partition. + +## What the capture launchers add, and why + +Every line of the capture posture in `swe_nano_sc_capture*.sh` exists because +its absence broke a run: + +| Setting | Without it | +|---|---| +| `token_capture.enabled=true` | Capture never engages. The launcher flips the config's default. Enabling it always constructs the fixed CPU finalizer actor pool sized by `token_capture.num_finalizer_workers`; there is no inline fallback. | +| `token_capture.defer_routed_experts_to_policy=true` + `+policy.router_replay.enabled=true` | The validated R3 posture keeps routed-expert tensors out of controller RPCs and canonical rows, then reconstructs them on policy workers from staged-fragment plans. Set both together. | +| `async_rl.sampler.name=windowed` + `max_staleness_versions=1` | The smoke does not exercise the intended bounded-staleness sampling policy. | +| `NG_TIC_FP_CANONICAL=1` | Reasoning models otherwise echo history with `` blocks stripped, so the ledger cannot verify a unique parent and fails the call as `UNRESOLVED`. With canonical fingerprints, `token_in_rate ≈ 0.9999`. | +| `NRL_DRIVER_PYTHONPATH=/opt/nemo-rl/3rdparty/Gym-workspace/Gym` | Driver `ModuleNotFoundError: nemo_gym` — the driver imports the staging record schema, and the baked driver venv has no nemo_gym. | +| `NRL_DRIVER_PIP_INSTALL=orjson` | Driver `ModuleNotFoundError: orjson` — Gym's `token_id_capture/__init__` eagerly imports the store. | +| `NRL_DRIVER_UV_RUN_FLAGS="--locked --no-sync"` | `uv run` otherwise replaces the prefetched driver environment and can give the driver a different Python/Ray version from the already-running Ray cluster. Lock mutation is forbidden; worker-specific environments are still rebuilt. | +| `VllmAsyncGenerationWorker` in `NRL_FORCE_REBUILD_VENVS_LIST` | Worker `ModuleNotFoundError: orjson` — venv caching is spec-unaware and silently reuses a non-capture worker venv built by an earlier job. | +| capture env set *after* sourcing `swe_nano.env` | `swe_nano.env` exports `NRL_FORCE_REBUILD_VENVS_LIST` unconditionally and clobbers an env-prefix value — which is why these are dedicated launchers rather than an env prefix on `swe_nano_sc.sh`. | +| `CALL_TIMING=0` (optional, batch) | Per-call latency JSONL is on by default in the batch launcher (`NRL_CALL_TIMING_DIR`/`NG_CALL_TIMING_DIR`); set 0 to disable. All probes are env-gated and dormant without the dir. | + +## Verifying capture is really engaged + +Config echo is not evidence. Check, in order: + +1. **Ledger-derived admission counters in the finalize metrics.** Each + manifest row records its admission mode, and the finalizer aggregates them + per group into `step_metrics` (W&B prefix `train`): + + ``` + finalize/token_in_calls, finalize/text_root_calls, + finalize/token_in_rate, finalize/capture_poisoned_rollouts + ``` + + `finalize/token_in_rate` should be ≥ 0.99 after the root calls (each chain + opens with exactly one `text` root). A rate near 0 with a large + `text_root_calls` count means canonical fingerprints are off (see above). + Nonzero `capture_poisoned_rollouts` means calls are failing admission or + commit — check the model-server logs for `unresolved_parent` / + `worker_capture_failed` poison reasons. + +2. **Finalizer-pool health.** In `step_metrics`, require + `finalize/invalid_row_rate=0` and, for the R3 command above, + `finalize/routed_experts_row_coverage=1`. The one-step smoke reported + `finalize/queue_depth=0` and `finalize/active_actor_count=1`; queue depth can + be nonzero under heavier load. A `finalizer actor RPC failed after + submission` message is fatal because the publication outcome is unknown + and actors are deliberately not retried. + +3. **TQ staging traffic**: `PUT_DATA` on the staging partition fires per + model call (tens of thousands per run), not just per training batch. + +4. **Training equivalence**: `token_mult_prob_error` should sit near 1.0 + (max ≲ 3), `gen_kl_error` in the same band as a non-capture run (~0.004). + +## Known limits + +- **Grade runs from the SC worker `.out` or W&B, never the driver log** — + Ray's driver-log stdout forwarding dropped entire actors in testing (runs + looked stalled while training normally). +- **Receipt-mode W&B rollout metrics are not yet comparable to legacy**: + `gen_tokens_per_sample` counts the carried prompt tail and + `truncation_rate` is constant on the capture arm. +- **Weight-version mixing** across a spliced chain is tag-checked per call. + The recipe defaults to `mixed_weight_version_policy=allow` and stamps the + row with the group's oldest version for staleness accounting; set it to + `reject` to emit a placeholder for a mixed-version rollout. +- **Router replay (R3)** is enabled in the verified pool-only smoke. Routed + experts are staged beside token deltas and reconstructed on policy workers; + keep `defer_routed_experts_to_policy=true` paired with + `policy.router_replay.enabled=true`. +- **Shutdown noise after a completed smoke** can include forced Ray/Gym actor + teardown because the asynchronous rollout pump may have work beyond the + final requested train step. Grade the run from the completed `train step + 1/1`, metrics, and Slurm exit code rather than teardown warnings alone. + +## Related + +- [Nano SWE with TransferQueue](nano-swe-transferqueue.md) — base recipe, + cluster shape, SingleController constraints +- [Router Replay](router-replay.md) — R3 background and trainer-side replay +- `nemo_rl/data_plane/tq_token_sink.py` — the staging sink/source over TQ +- `nemo_rl/experience/blackbox_finalizer.py` — receipt → training row +- Gym `nemo_gym/token_id_capture/staging/records.py` — the wire schema diff --git a/docs/guides/nano-swe-transferqueue.md b/docs/guides/nano-swe-transferqueue.md new file mode 100644 index 00000000000..7f39bc14bbf --- /dev/null +++ b/docs/guides/nano-swe-transferqueue.md @@ -0,0 +1,200 @@ +# Nano SWE RL with the TransferQueue Data Plane + +A reproducible 6-node smoke recipe that runs agentic SWE RL on +Nemotron-3-Nano-30B-A3B with rollouts flowing through the **TransferQueue (TQ) +data plane** instead of an in-process buffer. + +It exists to answer one question end to end: *which NeMo-RL entrypoint actually +puts SWE rollouts through TransferQueue, and what does a config need for that to +work?* The answer is `examples/run_grpo_single_controller.py`, and this recipe is +the smallest configuration where you can watch it happen. + +## Verified result + +Run on 6 GB200 NVL72 nodes (Slurm job 5648757, 2026-07-28): + +| | | +|---|---| +| Entrypoint | `examples/run_grpo_single_controller.py` | +| Config | `examples/configs/ultra/nano_swe_teacher_sc.yaml` | +| Model | `nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16` | +| Shape | train 4 nodes (TP2·PP2·CP4, EP1) / gen 2 nodes (vLLM TP2 → 4 engines) | +| Overrides | `grpo.num_prompts_per_step=2 policy.train_global_batch_size=8 grpo.max_num_steps=5` | +| Outcome | train steps 1–5 completed, no traceback, no OOM | +| Launched via | `swe_nano_sc_interactive.sh` (attach + run the driver by hand); `swe_nano_sc.sh` submits the same driver command as a batch job | + +The TQ actors were live and serving the rollout→train hop throughout: +`TransferQueueController` plus two `SimpleStorageUnit` actors logging `PUT_DATA`, +`GET_DATA`, `KV_RETRIEVE_META`, `SET_CUSTOM_META` and `CLEAR_DATA`. + +The final step reported `loss=7.7e-05`, `grad_norm=0.013`, +`global_valid_toks=112555`. **Rewards were 0.0 for every rollout** — the base +Nano model solves no SWE-bench instance in a 5-step smoke. This recipe validates +the *loop*, not model quality. + +## Which paths honour the data plane + +This is the part that is easy to get wrong: enabling `data_plane` in a config +does not mean it is used. Verified against the source: + +| Entrypoint / mode | TQ honoured? | Why | +|---|---|---| +| `run_grpo_nemo_gym.py`, `async_grpo.enabled=true` | **No** | `async_grpo_train` builds the in-memory `ReplayBuffer` (`nemo_rl/algorithms/grpo.py:3898`); the TQ-backed `TQReplayBuffer` (`nemo_rl/algorithms/async_utils/replay_buffer.py:638`) is never constructed. The `data_plane` block is silently a no-op. | +| `run_grpo_nemo_gym.py`, `async_grpo.enabled=false` | Yes | `grpo_train_sync` reads/writes through the data plane. | +| `run_grpo_single_controller.py` | Yes, and **required** | The entrypoint raises `ValueError` unless `data_plane.enabled=true`, and `SingleControllerActor` commits each group via `TQReplayBuffer` → `dp_client.put_samples` (`nemo_rl/algorithms/single_controller.py:259`). | + +Since the production SWE paradigm is async (sync leaves the 16 training GPUs +idle for the ~11 minutes a 64-rollout SWE batch takes), **SingleController is the +only path that gives async *and* a real data plane** — hence the recipe below. + +## Quick start + +All launchers share `swe_nano.env` and override the entrypoint and config. Run +them from a networked shell at the repo root (the Slurm controller is +unreachable from sandboxes). + +Unattended reproduction, one command: + +```bash +DRY_RUN=0 bash swe_nano_sc.sh grpo.num_prompts_per_step=2 policy.train_global_batch_size=8 +``` + +Without `DRY_RUN=0` it prints the resolved driver command and exits, which is +worth doing once. Interactive mode — allocate, keep Ray up, run the driver by +hand, edit, re-run — is what the verified run used and is the better choice +whenever you expect to change anything: + +```bash +# async GRPO via SingleController + honoured TransferQueue (the verified recipe) +bash swe_nano_sc_interactive.sh +``` + +The batch and interactive paths build the identical driver command; batch just +lets `ray.sub` run it instead of handing it to you. + +Each interactive launcher allocates 6 nodes, starts Ray, idles, and prints: + +```bash +bash -attach.sh # shell on the head node, Ray already up +source -run-cmd.sh # run the driver; edit + re-source to iterate +``` + +Iterating inside one allocation is the point — a cold start pays for the ~60 GB +checkpoint download plus Megatron conversion and vLLM graph capture before the +first rollout, so you do not want to requeue for every config fix. + +To reach a training step quickly instead of waiting for the default 64-rollout +batch: + +```bash +bash swe_nano_sc_interactive.sh grpo.num_prompts_per_step=2 policy.train_global_batch_size=8 +``` + +Cancel with `scancel `. + +## What to change before you run it + +`swe_nano.env` splits its paths into two blocks: + +**Shared read-only — reuse as is.** The training container, the nemo-skills +sandbox image, the SWE prompt set (`swe.jsonl`, 7816 instances) and the SWE-bench +`.sif` images are all world-readable on Lustre. They total hundreds of GB; do not +copy them. + +**Per-user write — change all of these.** They are owned by `zhiyul` and you +cannot write to them: + +| Variable | What lands there | +|---|---| +| `CODE_DIR` | your checkout of this branch — it is mounted into the container, so it must be the tree you are editing | +| `WORKSPACE_DIR` | results, Ray logs, checkpoints | +| `HF_HOME` | HuggingFace cache (~60 GB for the Nano BF16 checkpoint) | +| `PERSISTENT_CACHE` | vLLM / Triton / Inductor compile caches, reused across jobs | +| `SLURM_ACCOUNT` | an account you can charge | +| `EXP_NAME` | namespaces `results/`, `ray_logs/` and the W&B run | + +Export `HF_TOKEN` too if you can — the verified run downloaded the checkpoint +unauthenticated and was rate limited. + +## Config chain + +``` +nano_swe_teacher_sc.yaml SingleController + TQ (the recipe) +└── nano_swe_teacher_qwen3mesh.yaml TP2·PP2·CP4, EP1, alltoall, vLLM TP2, data_plane block + └── nano_swe_teacher.yaml 49k context, gym venv reuse, sif_dir + ├── swe_teacher.yaml the Ultra SWE stage (rewards, data, optimizer) + └── _nano_smoke_gb200.yaml.inc Nano/GB200 mesh + smoke sizing +``` + +### Why each SingleController-specific setting is there + +Every one of these came from a crash on the way to the working run: + +| Setting | Without it | +|---|---| +| `policy.draft.enabled: false` | `KeyError: 'draft'` — `run_grpo_single_controller.py` reads `config.policy["draft"]["enabled"]` unconditionally, and the SWE/Ultra chain has no draft block. | +| `loss_fn.reference_policy_kl_penalty: 0.01` | `AttributeError: 'MegatronPolicyWorker' object has no attribute 'reference_state_dict'` — the SC refit swaps in reference weights, but the reference model is only initialized when the KL penalty is > 0 (`single_controller_utils/setup.py:208`). The SWE base sets it to 0. | +| `grpo.val_period: 0`, `checkpointing.enabled: false` | SC supports neither validation nor checkpointing yet. | +| `async_rl` block | SC uses its own async config, not `grpo.async_grpo`. `min_groups_for_streaming_train` decides how many groups buffer before the first training step. | +| `moe_token_dispatcher_type: alltoall` (in the mesh config) | `AssertionError: hybrid-ep kernel ... at least 2 ranks, but got 1` — the inherited flex/hybridep dispatcher requires EP≥2, and this recipe runs EP1. | +| `data_plane.global_segment_size` / `local_buffer_size` | `DataPlaneConfig` requires them even though only the `mooncake_cpu` backend reads them; the `simple` backend ignores the values. | + +Two more constraints that are not config fields: + +- **Batch invariant.** `num_prompts_per_step × num_generations_per_prompt` must + equal `train_global_batch_size` on the SC split path (one RL step = one + optimizer step). Overriding one without the other raises `ValueError`. +- **Absolute entrypoint path.** `run_grpo_single_controller.py` must be spawned + from `${CODE_DIR}` by absolute path. The launcher mounts only `nemo_rl/` and + `examples/configs` over the container, so the container's baked `examples/` has + no such file and `uv run ./examples/...` fails with "Failed to spawn". + +## Verifying TQ is really engaged + +Config echo alone is not evidence — the async `run_grpo_nemo_gym.py` path prints +`data_plane={'enabled': True, ...}` and then ignores it. Look for **runtime +actors** in the driver log: + +``` +(TransferQueueController pid=...) Per-operation statistics: NOTIFY_DATA_UPDATE: ... +(SimpleStorageUnit pid=..., ip=...) Per-operation statistics: PUT_DATA: req_count=... GET_DATA: ... +(TransferQueueController pid=...) ... KV_RETRIEVE_META: ... SET_CUSTOM_META: ... +``` + +`PUT_DATA` is a rollout committing to TQ; `KV_RETRIEVE_META` is the trainer +consuming it. Then confirm training progresses: + +``` +🚀 Launching SingleControllerActor +train step 5/5 trainer_v=5 lag=1 +step_metrics={'loss': ..., 'grad_norm': ..., ...} +``` + +## Known limits + +- **Rewards are 0.** The base model solves no SWE instance in a smoke run. Real + signal needs a capable policy and many more steps. +- **EP1 is memory-hungry.** Expert parallelism of 1 replicates every expert on + each MP rank. With the reference model loaded (required by SC, see above) peak + usage was ~170 GB/rank of 189 GB. If you scale up context, batch, or model, + expect the initial weight sync to OOM first; raise EP, enable + `optimizer_cpu_offload`, or shorten the sequence. +- **Walltime and QOS.** `WALLTIME=3:59:00` (the `batch` partition maximum) with + no QOS. Two hours is the practical floor — shorter allocations expired + mid-warmup during development. The recipe deliberately avoids the `short` QOS: + it trades a priority boost for a 2h ceiling and a per-user node cap, and a + 6-node job here was held with `Reason=QOSMaxNodePerUserLimit` while a larger + job of the same user was running. For scale: with `num_prompts_per_step=2` a + 4-rollout batch took 6–8 minutes; the default 64-rollout batch took ~11. +- **Submodule pin.** This branch pins `3rdparty/Gym-workspace/Gym` to `v0.4.0` to + match the container's prebuilt gym venvs (`skip_venv_if_present: true`). + Bumping it without rebuilding the container forces a concurrent nemo-gym + rebuild and a uv cache lock timeout. + +## Related + +- `nemo_rl/data_plane/factory.py` — data-plane client construction and backends +- `nemo_rl/data_plane/docs/data-plane-async-proposal.md` — design notes +- `docs/guides/async-grpo.md` — the async trainer this recipe replaces +- `examples/configs/grpo_math_1B_megatron_single_controller.yaml` — the math + SingleController reference config the SC blocks were adapted from diff --git a/docs/index.md b/docs/index.md index 5d066eb95db..9fe88ddc5f7 100644 --- a/docs/index.md +++ b/docs/index.md @@ -142,6 +142,20 @@ Train Qwen2.5-Omni-7B with GRPO on PhilipC/IntentTrain (audio-visual intent reco Train Qwen3-30B-A3B-Thinking into a SWE agent with a pivot stage plus end-to-end agentic RL on SWE-bench. ::: +:::{grid-item-card} {octicon}`database` Nano SWE with TransferQueue +:link: guides/nano-swe-transferqueue +:link-type: doc + +Six-node SWE RL smoke on Nemotron-3-Nano-30B-A3B with rollouts flowing through the TransferQueue data plane via SingleController. +::: + +:::{grid-item-card} {octicon}`shield-check` Nano SWE with Token Capture +:link: guides/nano-swe-token-capture +:link-type: doc + +Exact-token SWE RL: the worker stages each call's token delta into TransferQueue and the capture ledger serves verified prefixes back — no token echo, no re-tokenization. +::: + :::{grid-item-card} {octicon}`plus-circle` Adding New Models :link: adding-new-models :link-type: doc @@ -297,6 +311,8 @@ guides/lora.md guides/cispo.md guides/prorlv2.md guides/swe-rl-qwen3.md +guides/nano-swe-transferqueue.md +guides/nano-swe-token-capture.md guides/grpo.md guides/ppo.md guides/grpo-deepscaler.md @@ -368,6 +384,8 @@ design-docs/training-backends.md design-docs/sequence-packing-and-dynamic-batching.md design-docs/env-vars.md design-docs/nemo-gym-integration.md +design-docs/token-capture-ledger.md +design-docs/rollout-verification-boundary.md design-docs/modelopt-real-quant-architecture.md design-docs/nccl-reshard-refit.md ``` diff --git a/examples/configs/ultra/_nano_smoke_gb200.yaml.inc b/examples/configs/ultra/_nano_smoke_gb200.yaml.inc new file mode 100644 index 00000000000..25a2fef553d --- /dev/null +++ b/examples/configs/ultra/_nano_smoke_gb200.yaml.inc @@ -0,0 +1,97 @@ +# Shared Nemotron 3 Nano v3 (30B-A3B) overrides for smoke-testing the Ultra +# pipeline on GB200 NVL72 nodes (4 GPUs/node). Include this AFTER the Ultra +# stage config so it overrides the Ultra-scale mesh and sizing, e.g.: +# +# defaults: +# - student_rlvr1.yaml # Ultra stage config (rewards, data, optimizer) +# - _nano_smoke_gb200.yaml.inc # this file (shrink to a Nano smoke shape) +# +# Stage-specific sequence lengths, rewards, data routing, and optimizer settings +# are inherited from the corresponding Ultra config. This file ONLY changes what +# is common to every Nano smoke run: the GPU mesh, the smoke sizing, and the +# Nano-vs-Ultra architecture toggles (alltoall dispatcher, no MTP, no fused +# weighted-squared-ReLU). Per-stage judge shrinking lives in the nano_.yaml +# files, because each stage serves a different set of Gym judge models. + +# ============================================================================= +# Cluster — GB200 NVL72, 4 GPUs/node. num_nodes is overridden by the launcher +# (NUM_ACTOR_NODES). +# +# segment_size MUST be disabled (null). It is an NVL72 topology filter: at Ultra +# scale (segment_size=16) the virtual cluster keeps only nodes that form a +# complete 16-node NVLink segment and DISCARDS the rest — per +# virtual_cluster._filter_bundles_by_segment: +# usable_nodes = (domain_node_count // segment_size) * segment_size +# On a ~5-node smoke allocation that is (5 // 16) * 16 = 0 usable nodes, so every +# placement bundle is discarded and nothing can schedule. null takes the early +# return in build_topology_aware_ordering and skips topology filtering entirely. +# (Distinct from the launcher's SEGMENT_SIZE env var, which is only the sbatch +# divisibility check.) +# ============================================================================= +cluster: + gpus_per_node: 4 + num_nodes: 5 # train(4) + gen(1); overridden by ultra_launch.sh + segment_size: null # disable NVLink-domain segment filtering for small allocs + +# ============================================================================= +# GRPO — shrink the step so a smoke run completes in minutes. +# ============================================================================= +grpo: + num_prompts_per_step: 16 + num_generations_per_prompt: 4 # GBS = 16 * 4 = 64 (see policy below) + num_val_generations_per_prompt: 2 + val_period: 100000 # validation effectively off for the smoke run + val_at_start: false + val_at_end: false + overlong_filtering: true # drop prompts longer than the (small) context + +# ============================================================================= +# Checkpointing — save less often so the 30B save doesn't dominate a short run. +# ============================================================================= +checkpointing: + save_period: 100 + checkpoint_must_save_by: null # let walltime end the job naturally + +# ============================================================================= +# Policy — Nano v3 30B-A3B, small batch + short context. +# model_name / tokenizer are overridden by the launcher (MODEL_PATH). +# ============================================================================= +policy: + train_global_batch_size: 64 # = num_prompts_per_step * num_generations_per_prompt + max_total_sequence_length: 16384 # down from the Ultra stage lengths + + megatron_cfg: + # --- GPU mesh grid (the core smoke-test change) ------------------------- + # world = TP*PP*CP*DP = 2*1*4*2 = 16 (4 GB200 nodes); experts sharded over + # EP=8 (EDP=2). Nano has far fewer experts than Ultra, so EP shrinks too. + tensor_model_parallel_size: 2 # was 8 + expert_tensor_parallel_size: 1 + expert_model_parallel_size: 8 # was 32/64 + context_parallel_size: 4 # was 8/32 + pipeline_model_parallel_size: 1 # unchanged (mirror Ultra) + + # Use the proven Nano dispatcher path instead of Ultra's flex/hybridep, + # which depends on the custom vLLM/kernels build. EP ranks must land in one + # NVLink domain (set SEGMENT_SIZE= in the env so sbatch + # --segment keeps the job on one NVL72 rack), else the EP collective hangs. + # moe_token_dispatcher_type: "alltoall" + # use_fused_weighted_squared_relu: false + + # --- MTP off for the Nano base checkpoint ------------------------------- + # The Nano base checkpoint is not guaranteed to ship MTP heads. + mtp_num_layers: 0 + mtp_loss_scaling_factor: 0.0 + + generation: + max_new_tokens: 4096 # cap generation length for a fast smoke run + vllm_cfg: + tensor_parallel_size: 4 # was 8 -> one GB200 node + expert_parallel_size: 1 # was 8 (EP=TP not needed at this scale) + pipeline_parallel_size: 1 + max_model_len: 16384 + + colocated: + enabled: false + resources: + gpus_per_node: 4 + num_nodes: 1 # overridden by launcher (NUM_GEN_NODES) diff --git a/examples/configs/ultra/nano_swe_teacher.yaml b/examples/configs/ultra/nano_swe_teacher.yaml new file mode 100644 index 00000000000..e88899bc031 --- /dev/null +++ b/examples/configs/ultra/nano_swe_teacher.yaml @@ -0,0 +1,36 @@ +# ============================================================================= +# Nemotron 3 NANO v3 (30B-A3B) — SWE teacher smoke test — GB200 +# ============================================================================= +# Inherits, in order: +# 1. swe_teacher.yaml — the Ultra stage +# 2. _nano_smoke_gb200.yaml.inc — Nano+GB200 mesh & smoke sizing (shared) +# +# SWE has NO GPU-served Gym judges (env.nemo_gym.num_gpu_nodes = 0). Rewards come +# from apptainer sandboxes, so there is no judge parallelism to shrink here. The +# launcher must still set SIF_DIR (sif_dir=${SIF_DIR}) — the SWE agents resolve +# `${sif_dir}//{instance_id}.sif`. +# +# Everything smoke-relevant (mesh, batch, context, MTP off, alltoall) comes from +# _nano_smoke_gb200.yaml.inc, so this file only wires the inheritance. +# ============================================================================= + +defaults: + - swe_teacher.yaml + - _nano_smoke_gb200.yaml.inc + +env: + nemo_gym: + # Reuse the container's baked gym venvs (swe_teacher.yaml sets this false, + # which forces a concurrent nemo-gym rebuild -> uv cache lock timeout). + skip_venv_if_present: true + +policy: + # For smoke, don't keep SWE's full 192k context unless you specifically want + # to test long-context memory pressure. Keep generation max_model_len in sync + # (the .inc pins it to 16384, which would reject longer SWE prompts). + max_total_sequence_length: 49152 + generation: + vllm_cfg: + max_model_len: 49152 + +sif_dir: /lustre/fsw/portfolios/llmservice/users/sdevare/images \ No newline at end of file diff --git a/examples/configs/ultra/nano_swe_teacher_qwen3mesh.yaml b/examples/configs/ultra/nano_swe_teacher_qwen3mesh.yaml new file mode 100644 index 00000000000..29c3593b8f5 --- /dev/null +++ b/examples/configs/ultra/nano_swe_teacher_qwen3mesh.yaml @@ -0,0 +1,71 @@ +# ============================================================================= +# Nemotron 3 NANO v3 (30B-A3B) — SWE Teacher — Qwen3-MoE parallelism (GB200) +# ============================================================================= +# Same Nano SWE smoke as nano_swe_teacher.yaml, but with the TRAIN/GEN mesh +# mirrored from the tuned Qwen3-30B-A3B MoE SWE recipe (both are ~30B MoE, async +# GRPO, non-colocated), adapted to GB200 NVL72 (4 GPUs/node). +# +# Reference: examples/nemo_gym/grpo_qwen3_30ba3b_thinking_swe1.yaml (16n×8g H100) +# Train: TP2 · PP2 · CP4 (no expert-parallel), MoE router frozen +# Gen: vLLM TP2, non-colocated +# +# GB200 adaptation (minimal, DP1): +# Train 4 nodes (16 GPUs) = TP2·PP2·CP4·DP1 +# Gen 2 nodes ( 8 GPUs) = vLLM TP2 -> 4 engines +# Total 6 nodes (set SEGMENT_SIZE=6 so sbatch keeps it on one rack) +# +# NOTE: PP2 and EP1 are inherited from the Qwen3 reference and are UNVERIFIED for +# the specific Nano v3 checkpoint — confirm on a DRY_RUN / first smoke (the model +# must support 2 pipeline stages; EP1 replicates experts per MP rank). +# ============================================================================= + +defaults: + - nano_swe_teacher.yaml + +policy: + megatron_cfg: + tensor_model_parallel_size: 2 # Qwen3: 2 + pipeline_model_parallel_size: 2 # Qwen3: 2 (nano .inc had 1) + context_parallel_size: 4 # Qwen3: 4 + expert_model_parallel_size: 1 # Qwen3: no EP (nano .inc had 8) + # swe_teacher.yaml inherits flex/hybridep, whose deep_ep kernel REQUIRES EP>=2 + # (AssertionError "hybrid-ep kernel ... at least 2 ranks, but got 1"). With EP1 + # use the proven Nano alltoall dispatcher (the _nano_smoke .inc's intent). + moe_token_dispatcher_type: "alltoall" + + # MoE router settings from the Qwen3 reference + freeze_moe_router: true + moe_router_bias_update_rate: 0.001 + moe_aux_loss_coeff: 0.0 + moe_router_enable_expert_bias: true + + generation: + vllm_cfg: + tensor_parallel_size: 2 # Qwen3: 2 (nano .inc had 4) + colocated: + enabled: false + resources: + gpus_per_node: 4 + num_nodes: 2 # overridden by launcher (NUM_GEN_NODES) + +# TransferQueue data plane. Defined here so the children can inherit it, but +# note WHICH entrypoint actually honours it: +# run_grpo_nemo_gym.py + async_grpo.enabled=true -> IGNORES this block. The +# async trainer builds the in-memory ReplayBuffer (grpo.py:3898), not the +# TQ-backed TQReplayBuffer (async_utils/replay_buffer.py:638). +# run_grpo_nemo_gym.py + async_grpo.enabled=false -> honoured by grpo_train_sync +# (sync path; this recipe uses the SingleController below). +# run_grpo_single_controller.py -> honoured, and REQUIRED +# (the entrypoint raises when data_plane.enabled is false). This is the +# async + TQ combination — see nano_swe_teacher_sc.yaml. +# Leaving it enabled on this async config is a harmless no-op; it is kept so the +# two child configs get the same storage settings. +data_plane: + enabled: true + impl: transfer_queue + backend: "simple" + storage_capacity: 1000000 + num_storage_units: 2 + claim_meta_poll_interval_s: 0.5 + global_segment_size: 549755813888 # 512 GiB (only used by mooncake_cpu backend) + local_buffer_size: 68719476736 # 64 GiB (only used by mooncake_cpu backend) diff --git a/examples/configs/ultra/nano_swe_teacher_sc.yaml b/examples/configs/ultra/nano_swe_teacher_sc.yaml new file mode 100644 index 00000000000..706242e5d21 --- /dev/null +++ b/examples/configs/ultra/nano_swe_teacher_sc.yaml @@ -0,0 +1,110 @@ +# ============================================================================= +# Nano v3 30B-A3B SWE — SingleController async GRPO + HONOURED TransferQueue +# ============================================================================= +# THE path that gives all three: SWE (NeMo-Gym) + async + real TQ mediation. +# Runs via examples/run_grpo_single_controller.py (NOT run_grpo_nemo_gym.py) → +# SingleControllerActor, which drives async rollout↔train through TQReplayBuffer +# (dp_client.put_samples → TransferQueue). `run_grpo_nemo_gym.py`'s async path +# (async_grpo_train) uses an in-memory ReplayBuffer and does NOT honour TQ. +# +# uv run ./examples/run_grpo_single_controller.py --config +# +# data_plane.enabled=true is MANDATORY for this launcher (it raises otherwise); +# inherited from nano_swe_teacher_qwen3mesh.yaml (impl: transfer_queue). +# +# VERIFIED on 6 GB200 nodes (train 4 / gen 2), Nemotron-3-Nano-30B-A3B-BF16: +# ran train steps 1-5 with live TransferQueueController + 2 SimpleStorageUnit +# actors serving PUT_DATA / KV_RETRIEVE_META / SET_CUSTOM_META. Launch with +# swe_nano_sc_interactive.sh; see docs/guides/nano-swe-transferqueue.md. +# ============================================================================= + +defaults: + - nano_swe_teacher_qwen3mesh.yaml # SWE env + Nano mesh (TP2·PP2·CP4, alltoall, EP1) + data_plane:transfer_queue + +# SingleController's own async config (SC uses this, not grpo.async_grpo). Mirrors +# examples/configs/grpo_math_1B_megatron_single_controller.yaml. +async_rl: + sampler: + name: in_order + max_lookahead_versions: 1 # how far generation may run ahead of the trainer + recompute_kv_cache_after_weight_updates: false + min_groups_for_streaming_train: ${grpo.num_prompts_per_step} + max_inflight_prompts: ${grpo.num_prompts_per_step} + max_buffered_rollouts: 64 # DataPlane backpressure cap + diagnostics: false + +grpo: + val_period: 0 # SC does not support validation yet + # Main-era SC refuses legacy async config (async_rl above is the SC + # equivalent); the block is inherited from swe_teacher.yaml. + async_grpo: null + +checkpointing: + enabled: false # SC does not support checkpointing yet + +policy: + generation: + # SingleController has no validation loop yet, so keep the required + # validation sampling profile identical to rollout sampling. + val_temperature: ${.temperature} + val_top_p: ${.top_p} + val_top_k: ${.top_k} + vllm_kwargs: + # vLLM 0.25's auto MoE backend picks FLASHINFER_TRTLLM on GB200, which + # re-lays w13/w2 into a 4D block format at init; the RL refit + # (model.load_weights of HF per-expert 2D weights) then dies with + # "shard_dim=0 is not a valid data dimension for a 3D tensor" + # (smoke 6107601, all 8 workers; same class as sglang#27787). Triton + # keeps the canonical [E, I, H] layout that refit can load — matches + # the other MoE recipes (e.g. grpo-nemotron3-super-120BA12B). + moe_backend: triton + # run_grpo_single_controller.py:111 reads policy.draft.enabled (speculative + # draft weights). The Ultra/SWE chain has no draft block; add it, disabled. + draft: + enabled: false + model_name: null + loss_weight: 0.1 + num_layers: null + aux_layer_indices: null + +loss_fn: + # SC initializes the reference model only when kl_penalty>0 (single_controller_utils/ + # setup.py:208: init_reference_model = reference_policy_kl_penalty > 0), and its refit + # unconditionally swaps in reference_state_dict (megatron_policy_worker.py:1611). The SWE + # base sets kl_penalty=0 → no reference model → AttributeError 'reference_state_dict'. + # Enable a small KL (matches grpo_math_1B_megatron_single_controller.yaml). + reference_policy_kl_penalty: 0.01 + +# Ledger-authoritative token capture (token-in/token-out via NeMo-Gym; see +# docs/guides/nano-swe-token-capture.md). Dormant by default: with +# enabled=false every legacy codepath behaves exactly as before — flip to +# true (or override token_capture.enabled=true) for the capture arm. +# Requires this SingleController path with data_plane.enabled=true and the +# async vLLM backend; defaults live on TokenCaptureConfig +# (nemo_rl/algorithms/single_controller_utils/config.py). +token_capture: + enabled: false + # TQ partition holding per-call staged token deltas. Deferred-route mode + # retains these through policy consumption; direct mode clears in finalizer. + staging_partition: "rollout_staging" + # continue: a failed worker-side stage poisons the rollout (placeholder row); + # abort: fails the whole rollout in the ledger. + on_capture_failure: "continue" + # allow: train groups whose calls span a refit (staleness = group's oldest + # call version); reject: placeholder such rollouts. + mixed_weight_version_policy: "allow" + # Drop the whole group when fewer than this fraction of its rollouts + # produced valid rows (null keeps every group). + min_valid_fraction_per_group: null + # Bearer token for Gym's token-capture control routes. null = minted per run. + control_auth_token: null + # Hard deadline per control-plane call (control-plane death must surface as failed + # dispatches + placeholders, not a silent retry stall). + control_timeout_s: 60.0 + # Process-shared per-rollout ledger and capture root. null resolves to + # /gym_token_capture. + capture_dir: null + # Keep route bytes out of canonical rows and assemble them on policy workers. + defer_routed_experts_to_policy: false + # Fixed CPU finalizer pool; actors have no automatic restart/task retry. + num_finalizer_workers: 2 diff --git a/examples/configs/ultra/swe_teacher.yaml b/examples/configs/ultra/swe_teacher.yaml new file mode 100644 index 00000000000..cc129c192ae --- /dev/null +++ b/examples/configs/ultra/swe_teacher.yaml @@ -0,0 +1,478 @@ +# ============================================================================= +# Nemotron 3 Ultra — SWE Teacher (code-execution RLVR, GBS=512, 192k context) +# ============================================================================= +# RLVR teacher for software-engineering benchmarks. Trains the policy against +# multi-turn coding agents (`swe_agents`) that execute candidate fixes inside +# apptainer (.sif) container images for SWE-Bench, SWE-Gym, etc., +# and reward based on test pass/fail — no LLM judges in this stage. +# +# Trained at 192k context with a +# small batch (GBS=512, PPS=32, GPP=16), large CP=32, and EP=32. +# +# Cluster shape (128 nodes × 4 GPUs on GB200 NVL72 = 512 GPUs): +# Training 64 / vLLM 64 / Gym 0 (no LLM judges). Override via +# NUM_TRAIN_NODES, NUM_GEN_NODES, NUM_GYM_NODES. +# +# SIF images: the SWE agent's `container_formatter` paths use the top-level +# `sif_dir` Hydra key — set via `SIF_DIR=...` in the launcher. The released +# blend uses two benchmarks, so the directory needs `swerebench/` and `swegym/` +# subdirectories with per-instance `{instance_id}.sif` files. See the public +# guide for how to build these for ARM64 (GB200). +# ============================================================================= +# +# - gpus_per_node: 4 +# - TP: 8, CP: 8, EP: 32, PP: 1 +# - vLLM TP: 8 +# - Non-colocated async inference with 64 generation nodes +# - Max sequence length: 131072 +# ============================================================================= + +# ============================================================================= +# Cluster — overridden by launch script for parallelism changes +# ============================================================================= +cluster: + gpus_per_node: 4 + num_nodes: 128 + segment_size: 16 + +# ============================================================================= +# SIF image root — required, set by the launcher via SIF_DIR. The SWE agent's +# container_formatter (in env.nemo_gym.swe_agents_train) resolves +# `${sif_dir}//{instance_id}.sif` per SWE benchmark family. +# ============================================================================= +sif_dir: ??? + +# ============================================================================= +# Checkpointing +# ============================================================================= +checkpointing: + enabled: true + checkpoint_dir: "results/grpo_ultra_e2e" + metric_name: "val:total_reward/mean" + higher_is_better: true + keep_top_k: 1000000 + save_period: 5 + ft_keep_latest_k: 1 + ft_save_period: 1 + checkpoint_must_save_by: "00:03:30:00" + model_save_format: "safetensors" + save_consolidated: false + save_optimizer: true + +# ============================================================================= +# GRPO Algorithm +# ============================================================================= +grpo: + num_prompts_per_step: 32 + num_generations_per_prompt: 16 + num_val_generations_per_prompt: 2 + max_rollout_turns: 1 + max_num_epochs: 4 + max_num_steps: 1000000 + normalize_rewards: true + use_leave_one_out_baseline: true + advantage_clip_low: -100 + advantage_clip_high: 100 + val_period: 10000 + val_at_start: false + val_at_end: false + overlong_filtering: false + max_val_samples: null + val_batch_size: 256 + seed: 42 + + use_dynamic_sampling: false + dynamic_sampling_max_gen_batches: 10 + batch_multiplier: 1 + + penalize_invalid_tool_call: true + invalid_tool_call_advantage: -5.0 + penalize_malformed_thinking: true + malformed_thinking_advantage: -5.0 + + reward_shaping: + enabled: false + overlong_buffer_length: 128 + overlong_buffer_penalty: 1 + max_response_length: ${policy.max_total_sequence_length} + stop_properly_penalty_coef: null + reward_scaling: + enabled: false + source_min: 0.0 + source_max: 1.0 + target_min: 0.0 + target_max: 1.0 + + async_grpo: + enabled: true + max_trajectory_age_steps: 1 + in_flight_weight_updates: true + recompute_kv_cache_after_weight_updates: false + + use_best_at_k: false + best_at_k_k: 8 + best_at_k_m: 1000 + + use_combined_training: false + combined_training_weight_mode: "auto" + combined_training_best_at_k_weight: 0.2 + combined_training_pass_at_1_weight: 1.0 + + dynamic_sampling_oversample_ratio: 1.0 + seq_logprob_error_threshold: 2 + +# ============================================================================= +# Loss Function +# ============================================================================= +loss_fn: + reference_policy_kl_penalty: 0.0 + reference_policy_kl_type: "k3" + kl_input_clamp_value: null + kl_output_clamp_value: null + + ratio_clip_min: 0.2 + ratio_clip_max: 0.28 + ratio_clip_c: null + use_on_policy_kl_approximation: true + use_importance_sampling_correction: true + truncated_importance_sampling_ratio: 5 + truncated_importance_sampling_ratio_min: 0.2 + truncated_importance_sampling_type: tis + sequence_level_importance_ratios: false + token_level_loss: true + force_on_policy_ratio: true + use_kl_in_reward: false + +# ============================================================================= +# Policy +# ============================================================================= +policy: + model_name: null + tokenizer: + name: ${policy.model_name} + chat_template_kwargs: null + hf_config_overrides: {} + + train_global_batch_size: 512 + train_micro_batch_size: 1 + generation_batch_size: 64 + logprob_batch_size: 1 + max_total_sequence_length: 196608 + precision: "bfloat16" + logprob_chunk_size: 1024 + offload_optimizer_for_logprob: false + + dtensor_cfg: + _v2: true + enabled: false + cpu_offload: false + sequence_parallel: false + activation_checkpointing: false + tensor_parallel_size: 1 + context_parallel_size: 1 + custom_parallel_plan: null + + megatron_cfg: + enabled: true + empty_unused_memory_level: 2 + activation_checkpointing: true + + # TP=8 spans 2 GB200 nodes (4 GPUs each) via intra-rack NVLink. + tensor_model_parallel_size: 8 + expert_tensor_parallel_size: 1 + expert_model_parallel_size: 32 + pipeline_model_parallel_size: 1 + num_layers_in_first_pipeline_stage: null + num_layers_in_last_pipeline_stage: null + context_parallel_size: 32 + pipeline_dtype: ${policy.precision} + sequence_parallel: true + + # MoE + freeze_moe_router: true + moe_router_dtype: "fp32" + moe_router_load_balancing_type: "none" + moe_router_bias_update_rate: 1.0e-3 + moe_router_enable_expert_bias: true + moe_permute_fusion: true + moe_enable_deepep: false + moe_token_dispatcher_type: "flex" + moe_flex_dispatcher_backend: "hybridep" + moe_hybridep_num_sms: 32 + moe_aux_loss_coeff: 0.0 + moe_shared_expert_overlap: false + use_gloo_process_groups: false + + # Compute + apply_rope_fusion: true + use_fused_weighted_squared_relu: true + bias_activation_fusion: false + gradient_accumulation_fusion: false + defer_fp32_logits: true + + # Logging + track_moe_metrics: true + moe_per_layer_logging: true + do_not_average_loss: true + cp_normalize: true + calculate_per_token_loss: true + scale_loss_by_dp_cp_size: false + + # MTP — disabled + mtp_loss_scaling_factor: 0.0 + mtp_use_repeated_layer: true + mtp_num_layers: 5 + mtp_detach_heads: true + + optimizer: + optimizer: "adam" + lr: 3.0e-6 + min_lr: 3.0e-6 + weight_decay: 0.0 + bf16: true + fp16: false + params_dtype: "float32" + + adam_beta1: 0.9 + adam_beta2: 0.999 + adam_eps: 1e-8 + + sgd_momentum: 0.9 + + use_distributed_optimizer: true + use_precision_aware_optimizer: true + + clip_grad: ${policy.max_grad_norm} + + optimizer_cpu_offload: false + optimizer_offload_fraction: 0.0 + + scheduler: + start_weight_decay: ${policy.megatron_cfg.optimizer.weight_decay} + end_weight_decay: ${policy.megatron_cfg.optimizer.weight_decay} + weight_decay_incr_style: "constant" + lr_decay_style: "constant" + lr_decay_iters: null + lr_warmup_iters: 10 + lr_warmup_init: 3e-7 + override_opt_param_scheduler: true + + distributed_data_parallel_config: + grad_reduce_in_fp32: false + overlap_grad_reduce: false + overlap_param_gather: true + average_in_collective: false + use_custom_fsdp: false + data_parallel_sharding_strategy: "optim_grads_params" + + # FP8 — overridden by precision recipe in launch script + fp8_cfg: + enabled: false + fp8: "e4m3" + fp8_recipe: "mxfp8" + fp8_param: false + + first_last_layers_bf16: true + num_layers_at_start_in_bf16: 1 + num_layers_at_end_in_bf16: 1 + + checkpoint: + async_save: true + ckpt_assume_constant_structure: true + fully_parallel_save_process_group: "ep_dp" + fully_parallel_load_process_group: "ep_dp" + fully_parallel_load_exchange_algo: "broadcast" + + env_vars: null + + # --------------------------------------------------------------------------- + # Sequence Packing + # --------------------------------------------------------------------------- + dynamic_batching: + enabled: false + train_mb_tokens: ${mul:${policy.max_total_sequence_length}, ${policy.train_micro_batch_size}} + logprob_mb_tokens: ${mul:${policy.max_total_sequence_length}, ${policy.logprob_batch_size}} + sequence_length_round: 64 + + sequence_packing: + enabled: true + train_mb_tokens: ${mul:${policy.max_total_sequence_length}, ${policy.train_micro_batch_size}} + logprob_mb_tokens: ${mul:${policy.max_total_sequence_length}, ${policy.logprob_batch_size}} + algorithm: "modified_first_fit_decreasing" + sequence_length_round: 64 + fuse_loss: true + + make_sequence_length_divisible_by: ${mul:${policy.megatron_cfg.context_parallel_size}, ${mul:2, ${policy.megatron_cfg.tensor_model_parallel_size}}} + max_grad_norm: 1.0 + optimizer: null + scheduler: null + + # --------------------------------------------------------------------------- + # Generation (vLLM) — Non-colocated, async + # --------------------------------------------------------------------------- + generation: + port_range_low: 11001 + port_range_high: 15000 + backend: "vllm" + max_new_tokens: ${policy.max_total_sequence_length} + temperature: 1.0 + top_p: 1.0 + top_k: null + stop_token_ids: null + stop_strings: null + vllm_cfg: + async_engine: true + precision: ${policy.precision} + kv_cache_dtype: "auto" + tensor_parallel_size: 8 + pipeline_parallel_size: 1 + expert_parallel_size: 1 + gpu_memory_utilization: 0.8 + max_model_len: ${policy.max_total_sequence_length} + enforce_eager: false + use_deep_gemm: false + num_last_layers_in_bf16: 0 + num_first_layers_in_bf16: 0 + enable_vllm_metrics_logger: true + vllm_metrics_logger_interval: 0.5 + expose_http_server: true + skip_tokenizer_init: false + reasoning_parser_plugin: nemo_rl/models/generation/vllm/reasoning_parsers/nano_v3_reasoning_parser.py + http_server_serving_chat_kwargs: + enable_auto_tools: true + tool_parser: qwen3_coder + reasoning_parser: nano_v3 + + vllm_kwargs: + attention_backend: FLASH_ATTN + mamba_ssm_cache_dtype: "float32" + # Hybrid Mamba: one state slot per running seq; the vLLM default (1024) can + # exceed available slots. 256 matches the Ultra GB200 serving reference + # (Gym nemotron_3_ultra_dev_nemorl_gb200.yaml). + max_num_seqs: 256 + compilation_config: + cudagraph_capture_sizes: [1,2,4,8,16,32,64] + cudagraph_mode: PIECEWISE + pass_config: + fuse_allreduce_rms: false + + colocated: + enabled: false + resources: + gpus_per_node: 4 + num_nodes: 64 + +# ============================================================================= +# Data +# ============================================================================= +data: + max_input_seq_length: null + shuffle: false + num_workers: 1 + use_multiple_dataloader: false + train: + data_path: null # Set by launch script + validation: + data_path: null # Set by launch script + default: + dataset_name: NemoGymDataset + env_name: "nemo_gym" + prompt_file: null + system_prompt_file: null + processor: "nemo_gym_data_processor" + +# ============================================================================= +# Environment — NeMo Gym + SWE Agents +# ============================================================================= +env: + should_use_nemo_gym: true + should_log_nemo_gym_responses: true + nemo_gym: + nemo_gym_log_dir: "logs/nemo_gym" + skip_venv_if_present: false + # NeMo-RL-side effort shaping; stripped before forwarding to Gym. + effort_levels: + low_string: "{reasoning effort: efficient}" + low_weight: 0.1 + low_penalty: 1 + low_ub: 15000 + num_gpu_nodes: 0 + port_range_low: 15001 + port_range_high: 20000 + invalid_tool_call_patterns: + - "" + - "" + - "" + - "" + thinking_tags: + - "" + - "" + config_paths: + - responses_api_models/vllm_model/configs/vllm_model_for_training.yaml + - responses_api_agents/swe_agents/configs/swebench_openhands_training.yaml + # Model-specific (Nemotron chat template): keep reasoning in the multi-turn SWE + # agent history so the trajectory grows monotonically for token-in-token-out RL. + policy_model: + responses_api_models: + vllm_model: + chat_template_kwargs: + enable_thinking: true + truncate_history_thinking: false + sequential_reasoning_allowed: false + swe_agents_train: + responses_api_agents: + swe_agents: + agent_max_turns: 200 + concurrency: ${mul:2, ${mul:${grpo.num_prompts_per_step}, ${grpo.num_generations_per_prompt}}} + swebench_agent_timeout: 3600 + run_with_mixed_prompts: true + dataset_path: ${data.train.data_path} + apptainer_memory_limit_mb: 65536 + container_formatter: + - "${sif_dir}/swerebench/{instance_id}.sif" + - "${sif_dir}/swegym/sweb.eval.arm64.{instance_id}.sif" + swe_agents_val: + responses_api_agents: + swe_agents: + agent_max_turns: 200 + concurrency: ${mul:2, ${mul:${grpo.num_prompts_per_step}, ${grpo.num_generations_per_prompt}}} + swebench_agent_timeout: 3600 + dataset_path: ${data.validation.data_path} + apptainer_memory_limit_mb: 65536 + container_formatter: ${env.nemo_gym.swe_agents_train.responses_api_agents.swe_agents.container_formatter} + use_absolute_ip: true + +# ============================================================================= +# Logger +# ============================================================================= +logger: + log_dir: "logs" + num_val_samples_to_print: 0 + wandb_enabled: false + tensorboard_enabled: false + mlflow_enabled: false + monitor_gpus: true + swanlab_enabled: false + wandb: + project: "ultra-v3-swe-e2e" + name: "grpo-ultra-e2e" + tensorboard: {} + mlflow: + experiment_name: "grpo-ultra-e2e" + run_name: "grpo-ultra-e2e" + gpu_monitoring: + collection_interval: 10 + flush_interval: 10 + +# ============================================================================= +# Reward Penalties (set reward to 0 when triggered) +# ============================================================================= +reward_penalties: + penalize_duplicated_reasoning: true # reasoning content == final answer + penalize_empty_final_answer: true # last message output has empty content + penalize_unwanted_tokens: true # unwanted token appears in generation + penalize_malformed_think_tag: true # /<\/think> count != 1 per turn + token_ids: + unwanted: [2] # + think_open: 12 # + think_close: 13 # diff --git a/examples/nemo_gym/nemotron-3-ultra/ultra_launch.sh b/examples/nemo_gym/nemotron-3-ultra/ultra_launch.sh index ed7e64268fd..5a7f1ef0009 100755 --- a/examples/nemo_gym/nemotron-3-ultra/ultra_launch.sh +++ b/examples/nemo_gym/nemotron-3-ultra/ultra_launch.sh @@ -753,14 +753,33 @@ if [[ -d "${OVERLAY_SOURCE}/nemo_rl" ]]; then _append_mount "${OVERLAY_SOURCE}/nemo_rl:/opt/nemo-rl/nemo_rl" echo " Mount: nemo_rl → /opt/nemo-rl/nemo_rl" fi +# The worktree's project files must ride along with the overlaid sources: +# node-side venv rebuilds run `uv sync --locked` against /opt/nemo-rl, and a +# mounted Gym/pyproject paired with the container's baked uv.lock fails the +# lock check (job 6368547). +for _project_file in pyproject.toml uv.lock .python-version; do + if [[ -f "${OVERLAY_SOURCE}/${_project_file}" ]]; then + _append_mount "${OVERLAY_SOURCE}/${_project_file}:/opt/nemo-rl/${_project_file}" + echo " Mount: ${_project_file} → /opt/nemo-rl/${_project_file}" + fi +done if [[ -d "${OVERLAY_SOURCE}/examples/configs" ]]; then _append_mount "${OVERLAY_SOURCE}/examples/configs:/opt/nemo-rl/examples/configs" echo " Mount: configs → /opt/nemo-rl/examples/configs" fi -if [[ -d "${OVERLAY_SOURCE}/3rdparty/Gym-workspace/Gym" ]]; then - _append_mount "${OVERLAY_SOURCE}/3rdparty/Gym-workspace/Gym:/opt/nemo-rl/3rdparty/Gym-workspace/Gym" - echo " Mount: Gym → /opt/nemo-rl/3rdparty/Gym-workspace/Gym" -fi +_local_project_paths=( + "3rdparty/TensorRT-LLM-workspace" + "3rdparty/Automodel-workspace/Automodel" + "3rdparty/Megatron-Bridge-workspace/Megatron-Bridge" + "3rdparty/Gym-workspace/Gym" + "research/template_project" +) +for _local_project_path in "${_local_project_paths[@]}"; do + if [[ -d "${OVERLAY_SOURCE}/${_local_project_path}" ]]; then + _append_mount "${OVERLAY_SOURCE}/${_local_project_path}:/opt/nemo-rl/${_local_project_path}" + echo " Mount: ${_local_project_path} → /opt/nemo-rl/${_local_project_path}" + fi +done if [[ "${USE_SNAPSHOT}" == "1" ]]; then _append_mount "${SNAPSHOT_DIR}:${SNAPSHOT_DIR}" @@ -855,7 +874,16 @@ export SETUP_COMMAND # learning rate, etc.) live in CONFIG_PATH. The launcher only passes the # per-run overrides: cluster shape, paths, judge endpoints, logging. # ============================================================================= +# A shared UV cache (e.g. prewarmed on Lustre) takes precedence; the /tmp +# fallback expands ${SLURM_JOB_ID} at runtime, hence the single quotes. +if [ -n "${UV_CACHE_DIR:-}" ]; then + _TRAIN_UV_CACHE_DIR="${UV_CACHE_DIR}" +else + _TRAIN_UV_CACHE_DIR='/tmp/nemo-gym-uv-cache-${SLURM_JOB_ID:-default}' +fi TRAIN_CMD="cd ${CODE_ROOT} && date ; \ +${NRL_DRIVER_PIP_INSTALL:+uv pip install --python /opt/nemo_rl_venv/bin/python ${NRL_DRIVER_PIP_INSTALL} ; }\ +${NRL_DRIVER_PYTHONPATH:+PYTHONPATH=${NRL_DRIVER_PYTHONPATH} }\ OMP_NUM_THREADS=16 \ RAY_DEDUP_LOGS=1 \ WANDB_INIT_TIMEOUT=300 \ @@ -864,7 +892,7 @@ NRL_VLLM_CACHE_SEED_DIR=${NRL_VLLM_CACHE_SEED_DIR} \ DG_JIT_CACHE_DIR=${NRL_VLLM_LOCAL_CACHE_DIR}/deep_gemm \ TORCHINDUCTOR_CACHE_DIR=${INDUCTOR_CACHE_DIR} \ TRITON_CACHE_DIR=${TRITON_CACHE_DIR} \ -UV_CACHE_DIR=/tmp/nemo-gym-uv-cache-\${SLURM_JOB_ID:-default} \ +UV_CACHE_DIR=${_TRAIN_UV_CACHE_DIR} \ UV_LOCK_TIMEOUT=1800 \ RAY_ENABLE_UV_RUN_RUNTIME_ENV=0 \ UV_HTTP_TIMEOUT=10 \ @@ -875,7 +903,7 @@ NRL_WG_USE_RAY_REF=1 \ HF_HOME=${HF_HOME:-} \ HF_TOKEN=${HF_TOKEN:-} \ NRL_USE_FASTOKENS=${NRL_USE_FASTOKENS:-1} \ -uv run ./examples/nemo_gym/run_grpo_nemo_gym.py \ +uv run ${NRL_DRIVER_UV_RUN_FLAGS:-} ${NRL_ENTRYPOINT:-./examples/nemo_gym/run_grpo_nemo_gym.py} \ --config ${CONFIG_PATH} \ policy.model_name=${MODEL_PATH} \ cluster.num_nodes=${NUM_ACTOR_NODES} \ diff --git a/examples/nemo_gym/nemotron-3.5-nano/nano35_dolphin_launch.sh b/examples/nemo_gym/nemotron-3.5-nano/nano35_dolphin_launch.sh new file mode 100755 index 00000000000..107efa32697 --- /dev/null +++ b/examples/nemo_gym/nemotron-3.5-nano/nano35_dolphin_launch.sh @@ -0,0 +1,417 @@ +#!/bin/bash +# Copyright (c) 2026, NVIDIA CORPORATION. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +set -euo pipefail + +# ============================================================================= +# nano35_dolphin_launch.sh +# +# Nemotron 3.5 Nano — RLVR, legacy async-1, honest-dolphin warm start. +# Reproduces the internal reference run +# geshen-ultra-rl-nano-honest-dolphin-v10-iter6000-mopd-rlvr +# (launch_nano_honest_dolphin.sh + grpo_ultra_512n4g_bf16.yaml on +# nemo-rl-internal @ 97c55ee2) on public NeMo-RL main. +# +# This is a thin wrapper over examples/nemo_gym/nemotron-3-ultra/ultra_launch.sh. +# That launcher is fully parameterised and already handles code snapshotting, +# persistent-cache seeding, container mounts, Ray/Gym orchestration, and the +# OccupiedIdleGPUsJobReaper --comment exemption — so we set environment and +# delegate rather than forking 800+ lines. +# +# Usage. Judges run one of two ways; see the GenRM section below for the +# trade-off between them. +# +# Against a warm out-of-band GenRM pool (default; must already be serving): +# GENRM_BASE_URL=http://:9213/v1 \ +# bash examples/nemo_gym/nemotron-3.5-nano/nano35_dolphin_launch.sh +# +# Hosting GenRM and the NL2Bash judge inside the job, as the 6K recipe does: +# EXTERNAL_JUDGES=1 \ +# bash examples/nemo_gym/nemotron-3.5-nano/nano35_dolphin_launch.sh +# +# DRY_RUN=1 GENRM_BASE_URL=... bash .../nano35_dolphin_launch.sh # inspect only +# +# Extra Hydra overrides are forwarded verbatim: +# GENRM_BASE_URL=... bash .../nano35_dolphin_launch.sh grpo.max_num_steps=2 +# ============================================================================= + +REPO_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/../../.." && pwd)" +cd "${REPO_ROOT}" # ultra_launch.sh derives PROJECT_ROOT from $PWD + +# ----------------------------------------------------------------------------- +# GenRM hosting. Two shapes, and this recipe supports both. +# +# EXTERNAL_JUDGES=1 is the 6K shape: GenRM and the NL2Bash judge are +# co-scheduled in a second Slurm hetgroup inside this job. The allocation +# wrapper brings up every replica plus one load balancer per pool, waits for +# health, then substitutes the resolved URLs into the driver command. The job +# owns its judges, so the reward model lands in provenance and no run can +# outlive, mismatch, or be starved by a pool it does not control. +# +# EXTERNAL_JUDGES=0 (default) keeps GenRM on a separately managed warm pool +# reached through GENRM_BASE_URL, and leaves the NL2Bash judge in the Gym pool. +# Partition `batch` caps at 4 h, so a warm pool amortizes the 470 GB bf16 +# Qwen3-235B load across a whole chain of jobs. The in-job path instead pays +# that load once per job — about ten minutes before training starts, measured +# on the 6K run — in exchange for being self-contained. +# +# To stand up the out-of-band pool (copy the dir first — it holds .lb_pid_*, +# logs/ and a flock'd registry, so running someone else's in place collides): +# +# cp -r /lustre/fs1/portfolios/llmservice/projects/llmservice_modelalignment_ppo/\ +# users/geshen/mopd_nano_fast/genrm_serving /genrm_serving +# cd /genrm_serving +# MODEL=/lustre/fsw/portfolios/llmservice/users/ansubramania/models/qwen235b_principle_comparison_genrm_step1230 \ +# ACCOUNT=nemotron_sw_post PARTITION=batch_long TIME=1-12:00:00 \ +# LB_PORT=9213 GENRM_GROUP_ID=nano35_dolphin \ +# ./genrm_server_manager.sh launch N +# ./genrm_server_manager.sh url +# +# NOTE: that script's default MODEL is the *ultra* GenRM (step_720) — override it. +# Each worker is 2 nodes x 4 GPUs at TP=8, separate from this job's 64 nodes. +# +# The `model` field must equal the pool's --served-model-name ("model" in +# genrm_worker.sh). ultra_launch.sh sets base_url XOR model, never both, so the +# name is pinned in rlvr_dolphin.yaml instead of passed here. +# ----------------------------------------------------------------------------- +export EXTERNAL_JUDGES="${EXTERNAL_JUDGES:-0}" +_DEFAULT_GYM_NODES=16 +if [[ "${EXTERNAL_JUDGES}" == "1" ]]; then + # Both pools reproduce the control run's serving shape, not just its GPU count, + # so only the judges' location changes. GenRM is 2 replicas at TP=8 spanning + # two nodes each, exactly as genrm_worker.sh ran them in the warm pool, filling + # the same four nodes. NL2Bash is eight TP=4 replicas, equalling the TP4 x DP8 + # deployment it replaces in Gym. + # + # GenRM's TP is the one number worth not "simplifying" to a single node. The + # checkpoint is 438 GB, so it is resident once per replica: 4 x TP4 would hold + # four copies (1752 GB) on the same 16 GPUs where 2 x TP8 holds two (876 GB), + # and the difference comes straight out of KV cache -- roughly 1.1 TB against + # 2.0 TB pool-wide. Four replicas would buy more schedulers and more concurrent + # slots, but GenRM's long prompts are KV-bound, so the control's shape wins. + export GENRM_MODEL="${GENRM_MODEL:-/lustre/fsw/portfolios/llmservice/users/ansubramania/models/qwen235b_principle_comparison_genrm_step1230}" + export GENRM_REPLICAS="${GENRM_REPLICAS:-2}" + export GENRM_TENSOR_PARALLEL_SIZE="${GENRM_TENSOR_PARALLEL_SIZE:-8}" + # The warm pool serves this checkpoint through the ultra_v3 parser plugin, so + # carry both the plugin and its name over rather than vLLM's built-ins. + export GENRM_REASONING_PARSER="${GENRM_REASONING_PARSER:-/lustre/fsw/portfolios/llmservice/users/lvega/evals/ultra_v3_reasoning_parser.py}" + export GENRM_REASONING_PARSER_NAME="${GENRM_REASONING_PARSER_NAME:-ultra_v3}" + # The control's genrm_worker.sh passes --enable-expert-parallel; the 6K + # deployment of this checkpoint does not. Follow the control. + export GENRM_ENABLE_EXPERT_PARALLEL="${GENRM_ENABLE_EXPERT_PARALLEL:-1}" + export NL2BASH_REPLICAS="${NL2BASH_REPLICAS:-8}" + export NL2BASH_TENSOR_PARALLEL_SIZE="${NL2BASH_TENSOR_PARALLEL_SIZE:-4}" + export EXTERNAL_VLLM_SEGMENT_SIZE="${EXTERNAL_VLLM_SEGMENT_SIZE:-2}" + # Gym drops to 8 nodes because the NL2Bash judge vacates exactly the 8 it was + # filling: TP=4 puts one replica on each four-GPU node, and DP=8 means eight + # of them. Every Gym node that was not serving NL2Bash stays, so the safety + # judge and the CPU-side env servers are untouched. + # + # That keeps the GPU allocation comparable to the control run. Akash's + # baseline was a 64-node job (5931924: 8 train + 40 gen + 16 gym) alongside a + # 4-node GenRM pool of its own (5931683 and 5931688, 2 nodes each, spanning + # the full training window) -- 68 nodes, 272 GB200 GPUs. This path is + # 8 + 40 + 8 in hetgroup 0 plus 12 service nodes, which is the same 68. + _DEFAULT_GYM_NODES=8 + # + # CHECK THIS FIRST IF A POOL FAILS TO START. serve_vllm_on_ray.py imports + # nemo_rl before vLLM's serve CLI, so pools must run the RL venv; they cannot + # use the Gym venv that rlvr_dolphin.yaml deliberately gives the in-Gym + # NL2Bash judge. On a vLLM 0.25 container that RL venv pairs vLLM with the + # openai release uv.lock pins, and `vllm serve` dies importing NamespaceTool + # from openai.types.responses (job 5943331). Judges share nothing with the + # trainer, so the fix is to serve them from an image whose RL venv is + # self-consistent rather than to match CONTAINER. Both pools default to + # CONTAINER in ultra_launch.sh and are overridable independently: + # GENRM_CONTAINER= NL2BASH_CONTAINER= +else + : "${GENRM_BASE_URL:?GENRM_BASE_URL must point to the external GenRM /v1 endpoint (or set EXTERNAL_JUDGES=1 to host GenRM in-job)}" + export GENRM_BASE_URL + unset GENRM_MODEL +fi + +# ----------------------------------------------------------------------------- +# Experiment identity +# EXP_NAME drives the W&B run name, the singleton job name, and the checkpoint +# and log dirs — so changing it starts a *new* run rather than resuming. +# ----------------------------------------------------------------------------- +export EXP_NAME="${EXP_NAME:-akamehra-nano35-honest-dolphin-v10-iter6000-rlvr-async1-tp4_cp4_ep8_pp1_gpp16_pps128_gbs2048}" +export CONFIG_PATH="${CONFIG_PATH:-examples/nemo_gym/nemotron-3.5-nano/rlvr_dolphin.yaml}" + +# ----------------------------------------------------------------------------- +# Model and data +# ----------------------------------------------------------------------------- +# honest-dolphin SFT v10 (closethink unmask, from midtrain 100B LC), iter_0006000. +export MODEL_PATH="${MODEL_PATH:-/lustre/fsw/portfolios/llmservice/users/venkats/training_actual_0603/nano_n3_post/checkpoints/nano-3.5-sft-v10-closethink-unmask-orig6k-from-midtrain-100B-lc-lr2e-5/eval/iter_0006000/hf}" + +# trusty_viper: 199,680 prompts / 24 agent families, carrying agent_ref per row. +# VAL_PATH intentionally equals TRAIN_PATH, as in the reference — validation is +# effectively disabled (grpo.val_period is very large) because the genrm cohort +# envs are train-only and would hang under eval. +_BLEND="${_BLEND:-/lustre/fs1/portfolios/llmservice/projects/llmservice_modelalignment_ppo/users/geshen/rl-data-tools/blends/curriculum_honest_dolphin_v41_trusty_viper.train.jsonl}" +export TRAIN_PATH="${TRAIN_PATH:-${_BLEND}}" +export VAL_PATH="${VAL_PATH:-${_BLEND}}" + +# ----------------------------------------------------------------------------- +# Judge checkpoints. Where they run depends on EXTERNAL_JUDGES above: the safety +# judge is always served from the Gym pool, and the NL2Bash judge joins it there +# unless the in-job hetgroup is enabled, in which case Gym skips its local +# launch and proxies to the pool's load balancer instead. +# ----------------------------------------------------------------------------- +export NL2BASH_JUDGE_MODEL="${NL2BASH_JUDGE_MODEL:-/lustre/fsw/portfolios/llmservice/users/ansubramania/models/Qwen3-235B-A22B-Instruct-2507-FP8}" +export SAFETY_JUDGE_MODEL="${SAFETY_JUDGE_MODEL:-/lustre/fsw/portfolios/llmservice/users/ansubramania/super_v3/model_checkpoints/Nemotron-Content-Safety-Reasoning-4B}" + +# ----------------------------------------------------------------------------- +# Containers +# Built 2026-08-14 with prefetched venvs, on vllm 0.25.1. +# +# The image supplies Megatron, and that is why this pin has to track the repo. +# ultra_launch.sh overlays only nemo_rl/, examples/configs and Gym from the +# worktree, so megatron.core always comes from the container. `46ab18ce1 ci: bump +# Megatron-Bridge to 0c565c9a0 (#3568)` landed on main 2026-08-11 and made +# megatron_policy_worker.py import FullyShardedDataParallelV1, which no image +# built before that date has -- including the 2026-07-30 one that used to be the +# default here and the 2026-08-07 main build. Both fail at +# `from megatron.core.distributed.fsdp.mcore_fsdp_adapter import ...` while the +# Megatron workers come up, ~20 min in (job 6231494). To check a candidate before +# spending an allocation on it: +# +# unsquashfs -l | grep mcore_fsdp_adapter +# unsquashfs -d /tmp/probe -f \ +# opt/nemo-rl/3rdparty/Megatron-Bridge-workspace/Megatron-Bridge/3rdparty/\ +# Megatron-LM/megatron/core/distributed/fsdp/mcore_fsdp_adapter.py +# +# Keep vllm >= 0.25.1 when moving this pin: vllm_worker_async.py imports +# ServingTokenization from vllm.entrypoints.serve.tokenize.serving, added in +# 0.25.x, and a 0.20.0 image kills every generation worker at engine init ~25 min +# in, after the judges and the Megatron workers have already loaded (jobs 5942981 +# and 5981739). +# +# The image's vllm and its nvidia-cutlass-dsl also have to agree, which is why +# this is the 08-14 build and not the 08-15 one that first got us past the +# Megatron import. The 08-15 image carries vllm 0.25.1+precompiled, whose +# vllm/vllm_flash_attn/cute/utils.py adds +# `from cutlass._mlir_helpers.arith import recast_type`; the cutlass-dsl 4.5.2 +# it ships exposes that module only as cutlass.base_dsl._mlir_helpers, so the +# import raises ModuleNotFoundError. utils.py is imported lazily, from the FA4 +# branch of flash_attn_varlen_func, so nothing surfaces until the first real +# attention forward: engine init and the weight sync both succeed, then every +# TP worker dies on the first rollout wave, EngineCore goes down, every +# generation 500s and the infra budget aborts the run ~3 min into serving (jobs +# 6233682 and 6234556, which is also where the scheduler's req_id_to_index +# KeyError came from -- collateral from the workers dying mid-batch, not a +# cause). FA4 is the default on these GB200s, so there is no getting to it. +# Add to the candidate probe above: +# +# unsquashfs -d /tmp/probe -f \ +# opt/ray_venvs/nemo_rl.models.generation.vllm.vllm_worker_async.\ +# VllmAsyncGenerationWorker/lib/python3.13/site-packages/vllm/vllm_flash_attn/\ +# cute/utils.py +# # follow the symlink into /root/.cache/uv, then: +# grep -n 'from cutlass' # must not name _mlir_helpers +# +# The 07-30 image passes that check but fails the Megatron one; 08-14 is the +# only build in yifuw's directory that passes both. +# ----------------------------------------------------------------------------- +if [[ -z "${CONTAINER:-}" ]]; then + : "${LUSTRE:?Set LUSTRE or provide CONTAINER explicitly}" + export CONTAINER="${LUSTRE}/enroot-images/nvcr.io/nvidian/nemo-rl:nightly-gym.squashfs" +else + export CONTAINER +fi + +# ----------------------------------------------------------------------------- +# Sandbox process DISABLED — this is what killed jobs 5726250 and 5732681. +# +# ray.sub launches the nemo-skills sandbox on every node with +# `--kill-on-bad-exit=1` (ray.sub:967), unlike the Ray worker step which uses +# `--kill-on-bad-exit=0` (ray.sub:1016). So a single node failing during sandbox +# startup makes srun tear down all 64 sandbox tasks; ray.sub then sees its +# background sandbox srun die and exits. At 64 nodes that happened on 3 of 3 +# attempts, on different nodes each time (nvl72133-T01, nvl72141-T07, +# nvl72126-T05/T17) — node exclusion cannot fix it. +# +# Only three Gym servers use the sandbox: competitive_coding_challenges (not in +# our blend), math_formal_lean and ns_tools. Both of the latter are in +# config_paths and are LEFT THERE deliberately: they construct their sandbox +# client lazily (math_formal_lean/app.py:387 builds it, :444 uses it inside a +# request handler; ns_tools only holds sandbox_host/port as config), so they +# boot fine without a sandbox and only contact it if a request routes to them. +# The dolphin blend routes to neither, so the sandbox is never needed. +# +# ray.sub gates everything sandbox-related on SANDBOX_CONTAINER && SANDBOX_COMMAND +# both being non-empty (ray.sub:559) — the ports dir, the 64-instance ready wait, +# and the srun itself. The unmodified ultra launcher replaces an empty +# SANDBOX_COMMAND with its default, so keep SANDBOX_CONTAINER empty instead. +# +# Side benefit: no 16 GB sandbox image extracted on 64 nodes, so faster startup. +# To re-enable (e.g. if a future blend uses Lean4), explicitly set +# SANDBOX_CONTAINER to the sandbox image path. +# ----------------------------------------------------------------------------- +if [[ -z "${SANDBOX_CONTAINER:-}" ]]; then + : "${LUSTRE:?Set LUSTRE or provide SANDBOX_CONTAINER explicitly}" + export SANDBOX_CONTAINER="${LUSTRE}/enroot-images/nvcr.io/nvidian/nemo-rl:skills-sandbox-latest.squashfs" +else + export SANDBOX_CONTAINER +fi + +# ----------------------------------------------------------------------------- +# Caches +# PERSISTENT_CACHE must be set explicitly: ultra_launch.sh requires it, and the +# internal reference's derivation (/lustre/fsw/portfolios/${ACCOUNT%%_*}/users/$USER) +# would resolve to /lustre/fsw/portfolios/nemotron/... which is read-only for us. +# HF_HOME is a *sibling* of the cache, not inside it, because the launcher purges +# vllm_compile_cache* under PERSISTENT_CACHE on every submission. +# HF_HOME also decides where the HF->Megatron conversion of the 62 GB checkpoint +# lands (get_megatron_checkpoint_dir falls back to $HF_HOME/nemo_rl), so keeping +# it on Lustre means the conversion is done once, not once per job. +# ----------------------------------------------------------------------------- +export PERSISTENT_CACHE="${PERSISTENT_CACHE:-/lustre/fs1/portfolios/coreai/projects/coreai_dlalgo_llm/users/akamehra/.cache/nano35-dolphin}" +export HF_HOME="${HF_HOME:-/lustre/fs1/portfolios/coreai/projects/coreai_dlalgo_llm/users/akamehra/hf_home}" + +# ----------------------------------------------------------------------------- +# Container mounts — REQUIRED. +# ultra_launch.sh starts MOUNTS empty and only appends three source overlays +# (nemo_rl, examples/configs, Gym). It never mounts /lustre, so without this the +# container cannot see the checkpoint, the blend jsonl, the judge models, +# HF_HOME or PERSISTENT_CACHE. The internal reference hardcoded this mount. +# ----------------------------------------------------------------------------- +export MOUNTS="${MOUNTS:-/lustre:/lustre}" + +# ----------------------------------------------------------------------------- +# Do not write bytecode into the Lustre-mounted source. +# +# nemo_rl is bind-mounted from Lustre into every container. Without this, all 64 +# nodes write .pyc back into that shared tree (194 files appeared during earlier +# runs, including generation/__pycache__/interfaces.cpython-313.pyc). That is +# metadata churn on the exact directories every node is importing from. +# +# Job 5742619 died when Ray unpickled VllmAsyncGenerationWorker on one node: +# __init__.py executed from the mount, then its sibling interfaces.py was not +# found — a per-node directory-view inconsistency, not a missing file. Reads +# alone are far safer than reads plus concurrent writes. +# ----------------------------------------------------------------------------- +export PYTHONDONTWRITEBYTECODE="${PYTHONDONTWRITEBYTECODE:-1}" + +# ----------------------------------------------------------------------------- +# examples/nemo_gym mount — REQUIRED, and the reason job 5730369 died with +# FileNotFoundError: /opt/nemo-rl/examples/nemo_gym/nemotron-3.5-nano/rlvr_dolphin.yaml +# +# ultra_launch.sh overlays only nemo_rl, examples/configs and Gym. Everything +# else under /opt/nemo-rl — including examples/nemo_gym, where this recipe and +# the nemotron-3-ultra base config live — comes from the container image. +# Two things are therefore invisible without this mount: +# 1. rlvr_dolphin.yaml itself (written here, never in any image), and +# 2. nemotron-3-ultra/student_rlvr1.yaml, which it inherits — that landed on +# main in 64cb9f985 (Jul 29-30) but this image was built Jul 26, so the +# image genuinely does not contain it (verified by inspecting the image). +# Mounting the directory fixes both, and also removes a version skew: nemo_rl +# would otherwise come from this checkout while examples/nemo_gym came from a +# Jul-26 image. +# +# Scoped to this one directory rather than mounting the repo root over +# /opt/nemo-rl, to avoid shadowing anything the image builds in place. +# ----------------------------------------------------------------------------- +_NEMO_GYM_MOUNT="${REPO_ROOT}/examples/nemo_gym:/opt/nemo-rl/examples/nemo_gym" +if [[ -n "${EXTRA_MOUNTS:-}" ]]; then + export EXTRA_MOUNTS="${EXTRA_MOUNTS},${_NEMO_GYM_MOUNT}" +else + export EXTRA_MOUNTS="${_NEMO_GYM_MOUNT}" +fi + +# ----------------------------------------------------------------------------- +# Snapshotting is OFF because tools/code_snapshot.sh copies only *git-tracked* +# files, and this recipe is untracked (`?? examples/nemo_gym/nemotron-3.5-nano/`). +# With snapshotting on, the mount above would point into a snapshot that does +# not contain the config. To restore frozen provenance, `git add` the recipe and +# set USE_SNAPSHOT=1. +# ----------------------------------------------------------------------------- +export USE_SNAPSHOT="${USE_SNAPSHOT:-0}" + +# ----------------------------------------------------------------------------- +# Results root — MUST be absolute. +# ultra_launch.sh defaults RESULTS_DIR to the relative "results/${EXP_NAME}". +# The host mkdir would land it in the repo, but TRAIN_CMD does `cd /opt/nemo-rl` +# inside the container, so the same relative string resolves to +# /opt/nemo-rl/results/... — the container's ephemeral overlay. Checkpoints +# would vanish at job end and the singleton auto-resume would never find them, +# so on a 4 h wall the run would restart from the SFT checkpoint forever. +# An absolute Lustre path fixes checkpoints, logs, ray_logs and slurm output. +# ----------------------------------------------------------------------------- +export RESULTS_DIR="${RESULTS_DIR:-/lustre/fs1/portfolios/coreai/projects/coreai_dlalgo_llm/users/akamehra/runs/${EXP_NAME}}" + +# ----------------------------------------------------------------------------- +# SLURM +# Job shape: 8 train + 40 gen + 16 gym = 64 GB200 nodes (4 GPUs each), a 5:1 +# generation-to-training split. GenRM runs in its own allocation (2 replicas x +# 2 nodes), so the campaign footprint is 68 nodes. +# SEGMENT_SIZE=2 is the nano value; ultra defaults to 16. +# Partition `batch` caps at 4 h (batch_long is 7 d), so WALLTIME is 4 h and +# CHECKPOINTING_SAVE_BY keeps the reference's 25-minute teardown margin. +# ----------------------------------------------------------------------------- +export SLURM_ACCOUNT="${SLURM_ACCOUNT:-nemotron_sw_post}" +export SLURM_PARTITION="${SLURM_PARTITION:-batch}" +export WALLTIME="${WALLTIME:-4:00:00}" +# Async rollout collection can leave the training GPUs idle while judges work, +# and startup is idle end to end: job 5965796 needed 56 minutes to reach its +# first optimizer step and the reaper took it at 60. This is the name +# ultra_launch.sh actually reads -- JOB_REAPER_EXEMPT_IDLE_MINS was never +# consumed by anything, so the banner reported a number with no effect. +export JOB_REAPER_EXEMPT_MINS="${JOB_REAPER_EXEMPT_MINS:-120}" +export CHECKPOINTING_SAVE_BY="${CHECKPOINTING_SAVE_BY:-00:03:35:00}" +export NUM_TRAIN_NODES="${NUM_TRAIN_NODES:-8}" +export NUM_GEN_NODES="${NUM_GEN_NODES:-40}" +export NUM_GYM_NODES="${NUM_GYM_NODES:-${_DEFAULT_GYM_NODES}}" +export SEGMENT_SIZE="${SEGMENT_SIZE:-2}" + +# ----------------------------------------------------------------------------- +# W&B. WANDB_API_KEY must already be in the environment — ultra_launch.sh needs +# it. If it is exported only from ~/.zshrc, submit from zsh; a bash context +# will not see it. +# ----------------------------------------------------------------------------- +export WANDB_PROJ="${WANDB_PROJ:-ultra-streaming}" +export WANDB_ENTITY="${WANDB_ENTITY:-joc}" + +# MTP: head *training* is on via the config (5 repeated layers, loss 0.3, +# detached heads), matching the reference. MTP *speculative decoding* for vLLM +# is a separate, independent switch and is off, also matching the reference. +export ENABLE_MTP_INFERENCE="${ENABLE_MTP_INFERENCE:-0}" + +echo "================================================================" +echo " Nemotron 3.5 Nano — RLVR async-1 (honest-dolphin)" +echo "================================================================" +echo " Experiment : ${EXP_NAME}" +echo " Config : ${CONFIG_PATH}" +echo " Model : ${MODEL_PATH}" +echo " Blend : ${TRAIN_PATH}" +echo " Container : ${CONTAINER}" +echo " Cache : ${PERSISTENT_CACHE}" +echo " HF_HOME : ${HF_HOME}" +if [[ "${EXTERNAL_JUDGES}" == "1" ]]; then +echo " GenRM : in-job hetgroup — ${GENRM_REPLICAS} x TP=${GENRM_TENSOR_PARALLEL_SIZE}" +echo " ${GENRM_MODEL}" +echo " NL2Bash : in-job hetgroup — ${NL2BASH_REPLICAS} x TP=${NL2BASH_TENSOR_PARALLEL_SIZE}" +else +echo " GenRM : ${GENRM_BASE_URL} (external pool; served model name: model)" +echo " NL2Bash : served in the Gym pool" +fi +echo " SLURM : ${SLURM_ACCOUNT} / ${SLURM_PARTITION} / ${WALLTIME}" +echo " Reaper : ${JOB_REAPER_EXEMPT_MINS} min idle exemption" +echo " Nodes : ${NUM_TRAIN_NODES} train + ${NUM_GEN_NODES} gen + ${NUM_GYM_NODES} gym" +echo " W&B : ${WANDB_ENTITY}/${WANDB_PROJ}" +echo "================================================================" +echo "" + +exec bash examples/nemo_gym/nemotron-3-ultra/ultra_launch.sh "$@" diff --git a/examples/nemo_gym/nemotron-3.5-nano/nano35_dolphin_launch_sc.sh b/examples/nemo_gym/nemotron-3.5-nano/nano35_dolphin_launch_sc.sh new file mode 100755 index 00000000000..214f8309318 --- /dev/null +++ b/examples/nemo_gym/nemotron-3.5-nano/nano35_dolphin_launch_sc.sh @@ -0,0 +1,199 @@ +#!/bin/bash +# Copyright (c) 2026, NVIDIA CORPORATION. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +set -euo pipefail + +# ============================================================================= +# nano35_dolphin_launch_sc.sh +# +# SingleController variant of nano35_dolphin_launch.sh: the same 64-node +# Nemotron 3.5 Nano RLVR pipeclean, driven by run_grpo_single_controller.py +# with streaming forward/backward, the TransferQueue data plane and +# shard-to-shard NCCL-reshard weight refit. +# +# Shape (GB200, 4 GPUs/node -> 256 GPUs), unchanged from the baseline so the +# two runs are comparable: +# 8 train + 40 generation + 16 gym = 64 nodes (5:1 generation-to-training) +# GenRM adds 4 nodes from its own allocation, so the campaign footprint is 68. +# +# This is a thin wrapper over nano35_dolphin_launch.sh, which already carries +# every site default (model, blend, judges, container, mounts, caches, Slurm). +# We only swap the config and the driver, so the two runs differ solely in the +# SC wiring. +# +# This branch supports durable trainer checkpoints plus periodic rollout/TQ +# snapshots. The defaults below exercise that production recovery path; set +# ROLLOUT_CHECKPOINT_INTERVAL_S=null to keep trainer checkpointing but disable +# periodic rollout snapshots for an ablation. +# +# Usage: +# GENRM_BASE_URL=http://:9213/v1 \ +# bash examples/nemo_gym/nemotron-3.5-nano/nano35_dolphin_launch_sc.sh +# +# DRY_RUN=1 GENRM_BASE_URL=... bash .../nano35_dolphin_launch_sc.sh +# +# GenRM must already be serving — see the GenRM section of the baseline script +# for how to stand up the external pool, or pass EXTERNAL_JUDGES=1 to host +# GenRM and the NL2Bash judge in a hetgroup inside this job instead, which is +# the shape the 6K recipe uses. +# +# Optional: +# NRL_MAX_STEPS=10 # short pipeclean +# STREAM_MIN_GROUPS=32 # async_rl.min_groups_for_streaming_train +# SAMPLER=in_order # in_order | weight_fifo | windowed +# MAX_LOOKAHEAD_VERSIONS=4 # the sampler's slack, whatever it spells it +# # 1 restores parity with the async-1 baseline +# BUFFER_RETENTION_MULTIPLIER=2 # max_buffered_rollouts only; gated samplers +# NUM_STORAGE_UNITS=16 # data_plane.num_storage_units +# REFIT_TRANSPORT=null # fall back to the full-tensor NCCL broadcast +# ROLLOUT_CHECKPOINT_INTERVAL_S=120 +# ROLLOUT_TELEMETRY_INTERVAL_S=30 +# +# Extra positional args are forwarded as Hydra overrides, after ours, so they win. +# ============================================================================= + +SCRIPT_DIR=$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" &>/dev/null && pwd) + +export CONFIG_PATH="${CONFIG_PATH:-examples/nemo_gym/nemotron-3.5-nano/rlvr_dolphin_sc.yaml}" + +# The SC driver. This branch's Ultra launcher reads NRL_ENTRYPOINT. +# data_plane.enabled=true (set in the config) is mandatory for it. +export NRL_ENTRYPOINT="${NRL_ENTRYPOINT:-./examples/run_grpo_single_controller.py}" + +# Distinct from the baseline's EXP_NAME so this starts a new W&B run, run dir +# and singleton job name rather than colliding with the async-1 baseline. +export EXP_NAME="${EXP_NAME:-akamehra-nano35-honest-dolphin-v10-iter6000-rlvr-sc-tp4_cp4_ep8_pp1_gpp16_pps128_gbs2048}" + +# The SC knobs worth sweeping without editing the config. +# +# STREAM_MIN_GROUPS starts the optimizer step earlier on partial cohorts. +# NUM_STORAGE_UNITS is the untuned one at this data volume. It is passed as a +# Hydra override unconditionally below, so it beats the recipe and the value in +# rlvr_dolphin_sc.yaml is never what runs — keep the two in step. It shards a +# global pool rather than reserving per unit, so over-provisioning costs only a +# CPU actor each; the windowed sweep's 14336 peak rows are 1.4% of capacity. +# MAX_LOOKAHEAD_VERSIONS is the sampler's slack. in_order spells it +# max_lookahead_versions; weight_fifo/windowed spell it max_staleness_versions. +# The gated samplers also require max_buffered_rollouts >= prompts * (slack + 1). +STREAM_MIN_GROUPS="${STREAM_MIN_GROUPS:-32}" +NUM_STORAGE_UNITS="${NUM_STORAGE_UNITS:-16}" +MAX_LOOKAHEAD_VERSIONS="${MAX_LOOKAHEAD_VERSIONS:-4}" +ROLLOUT_CHECKPOINT_INTERVAL_S="${ROLLOUT_CHECKPOINT_INTERVAL_S:-300}" +ROLLOUT_TELEMETRY_INTERVAL_S="${ROLLOUT_TELEMETRY_INTERVAL_S:-60}" + +# Emit only fields belonging to the selected discriminated sampler config. +# Hydra's `+` form is required when switching away from the in_order block +# declared in rlvr_dolphin_sc.yaml. +SAMPLER="${SAMPLER:-in_order}" +case "${SAMPLER}" in + in_order) + _SAMPLER_OVERRIDES=( + "async_rl.sampler.name=in_order" + "async_rl.sampler.max_lookahead_versions=${MAX_LOOKAHEAD_VERSIONS}" + ) + ;; + weight_fifo) + _SAMPLER_OVERRIDES=( + "async_rl.sampler.name=weight_fifo" + "+async_rl.sampler.max_staleness_versions=${MAX_LOOKAHEAD_VERSIONS}" + ) + ;; + windowed) + _SAMPLER_OVERRIDES=( + "async_rl.sampler.name=windowed" + "+async_rl.sampler.max_staleness_versions=${MAX_LOOKAHEAD_VERSIONS}" + ) + ;; + *) + echo "SAMPLER must be in_order, weight_fifo or windowed, got '${SAMPLER}'" >&2 + exit 1 + ;; +esac + +# Shard-to-shard weight refit, on by default in this variant. It is still +# experimental, so keep the escape hatch one env var away: REFIT_TRANSPORT=null +# restores the full-tensor broadcast that rlvr_dolphin.yaml uses. +REFIT_TRANSPORT="${REFIT_TRANSPORT:-nccl_reshard}" + +_NUM_PROMPTS_PER_STEP="${_NUM_PROMPTS_PER_STEP:-128}" + +# Generation quota: the current cohort plus every lookahead cohort in flight at +# once. This is the number that must not move, because it is what the arms are +# compared on. +_MAX_INFLIGHT_PROMPTS=$(( _NUM_PROMPTS_PER_STEP * (MAX_LOOKAHEAD_VERSIONS + 1) )) + +# Retention headroom, as a multiple of that quota. +# +# _buffer_capacity is a per-group semaphore taken at dispatch and released on +# select, evict, or failure. Zero eviction deletes one of those three release +# paths, so groups that have finished generating but have not yet been trained +# on stay resident holding permits. At a multiplier of 1 they are holding +# permits out of the same pool that admission draws from, so the finished work +# crowds out new generation -- the fix for eviction creates a throughput +# problem one layer down. +# +# A multiplier above 1 gives retention its own headroom, which is what v1 does: +# late_arrival_slack=2 sizes its retention at P*lag*2 against a generation quota +# of P*lag. Retention strictly exceeding what admission can produce is the +# property that stops completed work from starving dispatch. +# +# This is NOT job 6014206 (768 buffered against 384 in flight, 1.8x slower). +# That arm ran WindowedSampler, which derives from BaseSampler and whose admit +# returns None immediately -- "dispatch is bounded by buffer capacity, not by +# version" -- so there the buffer was the only thing limiting dispatch and +# raising it raised dispatch. in_order and weight_fifo are gated samplers, so +# their dispatch windows remain bounded independently of buffer headroom. +BUFFER_RETENTION_MULTIPLIER="${BUFFER_RETENTION_MULTIPLIER:-1}" +_MAX_BUFFERED_ROLLOUTS=$(( _MAX_INFLIGHT_PROMPTS * BUFFER_RETENTION_MULTIPLIER )) + +if (( BUFFER_RETENTION_MULTIPLIER < 1 )); then + echo "BUFFER_RETENTION_MULTIPLIER must be >= 1, got ${BUFFER_RETENTION_MULTIPLIER}." >&2 + echo "Below 1 the buffer sits under the sampler's required floor and the train" >&2 + echo "pump waits for a batch the buffer is too small to ever hold." >&2 + exit 1 +fi + +if (( BUFFER_RETENTION_MULTIPLIER > 1 )) && [[ "${SAMPLER}" == "windowed" ]]; then + echo "BUFFER_RETENTION_MULTIPLIER=${BUFFER_RETENTION_MULTIPLIER} with SAMPLER=windowed is the 6014206 trap." >&2 + echo "WindowedSampler.admit returns None, so the buffer is its only dispatch" >&2 + echo "limit and raising it raises dispatch: that arm ran 1.8x slower. Only the" >&2 + echo "gated samplers (in_order and weight_fifo) can take a multiplier." >&2 + exit 1 +fi + +echo "================================================================" +echo " Nemotron 3.5 Nano — RLVR SingleController (honest-dolphin)" +echo "================================================================" +echo " Entrypoint : ${NRL_ENTRYPOINT}" +echo " Config : ${CONFIG_PATH}" +echo " Refit : ${REFIT_TRANSPORT}" +echo " Streaming : min ${STREAM_MIN_GROUPS} of ${_NUM_PROMPTS_PER_STEP} groups per dispatch" +echo " Sampler : ${SAMPLER} (slack ${MAX_LOOKAHEAD_VERSIONS})" +echo " Capacity : buffer ${_MAX_BUFFERED_ROLLOUTS} groups (x${BUFFER_RETENTION_MULTIPLIER}), ${_MAX_INFLIGHT_PROMPTS} in flight" +echo " TQ units : ${NUM_STORAGE_UNITS}" +echo " Rollout ckpt: interval=${ROLLOUT_CHECKPOINT_INTERVAL_S}s, telemetry=${ROLLOUT_TELEMETRY_INTERVAL_S}s" +echo "================================================================" +echo "" + +exec bash "${SCRIPT_DIR}/nano35_dolphin_launch.sh" \ + "async_rl.min_groups_for_streaming_train=${STREAM_MIN_GROUPS}" \ + "${_SAMPLER_OVERRIDES[@]}" \ + "async_rl.max_inflight_prompts=${_MAX_INFLIGHT_PROMPTS}" \ + "async_rl.max_buffered_rollouts=${_MAX_BUFFERED_ROLLOUTS}" \ + "data_plane.num_storage_units=${NUM_STORAGE_UNITS}" \ + "policy.generation.refit_transport=${REFIT_TRANSPORT}" \ + "rollout_checkpointing.interval_s=${ROLLOUT_CHECKPOINT_INTERVAL_S}" \ + "rollout_checkpointing.telemetry_interval_s=${ROLLOUT_TELEMETRY_INTERVAL_S}" \ + "$@" diff --git a/examples/nemo_gym/nemotron-3.5-nano/nano35_dolphin_launch_sc_smoke.sh b/examples/nemo_gym/nemotron-3.5-nano/nano35_dolphin_launch_sc_smoke.sh new file mode 100755 index 00000000000..8980df8760b --- /dev/null +++ b/examples/nemo_gym/nemotron-3.5-nano/nano35_dolphin_launch_sc_smoke.sh @@ -0,0 +1,118 @@ +#!/bin/bash +# Copyright (c) 2026, NVIDIA CORPORATION. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +# ============================================================================= +# Nemotron 3.5 Nano — 8-node SingleController pipeclean +# +# Scaled-down nano35_dolphin_launch_sc.sh for validating the stack end to end +# before spending a 68-node allocation: SingleController, the TransferQueue +# data plane, nccl_reshard refit with MTP-head gating, and the in-job judge +# hetgroup this recipe gained from the 6K side. +# +# Usage: +# bash examples/nemo_gym/nemotron-3.5-nano/nano35_dolphin_launch_sc_smoke.sh +# DRY_RUN=1 bash .../nano35_dolphin_launch_sc_smoke.sh # inspect only +# +# Training parallelism is deliberately NOT scaled. Four nodes is the floor that +# preserves it exactly: TP=4 x CP=4 x PP=1 fills 16 GPUs, and EP=8 still divides +# 16 at ETP=1, so every rank owns the same 16 of the 128 routed experts as the +# full-scale run. The only difference is DP, which drops from 2 to 1. +# Going to 2 training nodes would force CP below 4, and CP=4 is what makes the +# 73728-token context tractable, so the comparison would stop being meaningful. +# +# Sequence length is left at the production 73728 on purpose. Truncating it +# would risk failures in the blend rather than in the code under test, and +# generation length is bounded by EOS in practice, not by the cap. +# +# What gets scaled instead is the cohort, from 128 prompts to 8, and the step +# count. NRL_MAX_STEPS must stay >= 2: the first weight sync only happens +# between steps, and refit is the main thing this run exists to exercise. +# +# The default shape is 8 nodes total. Raise NUM_GEN_NODES / NUM_GYM_NODES to 2 +# each (10 nodes) if rollouts are throughput-bound or Gym's env servers are +# cramped on a single node. +# ============================================================================= + +set -euo pipefail + +SCRIPT_DIR=$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" &>/dev/null && pwd) + +# 4 + 1 + 1 in hetgroup 0, plus 2 service nodes below = 8. Both counts have to +# stay divisible by their segment size, which is 2 on nano. +export NUM_TRAIN_NODES="${NUM_TRAIN_NODES:-4}" +export NUM_GEN_NODES="${NUM_GEN_NODES:-1}" +export NUM_GYM_NODES="${NUM_GYM_NODES:-1}" +export SEGMENT_SIZE="${SEGMENT_SIZE:-2}" + +# Judges in-job, which is the path that has never been run and the main reason +# this profile exists. One replica each at the production TP, so the servers +# are sharded exactly as they would be at full scale -- there are just fewer. +# This also frees Gym down to a single node, since only the safety judge +# (TP=1, DP=1) still runs there. +export EXTERNAL_JUDGES="${EXTERNAL_JUDGES:-1}" +export GENRM_REPLICAS="${GENRM_REPLICAS:-1}" +export GENRM_TENSOR_PARALLEL_SIZE="${GENRM_TENSOR_PARALLEL_SIZE:-4}" +export NL2BASH_REPLICAS="${NL2BASH_REPLICAS:-1}" +export NL2BASH_TENSOR_PARALLEL_SIZE="${NL2BASH_TENSOR_PARALLEL_SIZE:-4}" +export EXTERNAL_VLLM_SEGMENT_SIZE="${EXTERNAL_VLLM_SEGMENT_SIZE:-2}" + +# 8 prompts x 16 generations. Over DP=1 that is 128 sequences per rank against +# 1024 at full scale, so this is strictly lighter per GPU, not heavier. +NUM_PROMPTS_PER_STEP="${NUM_PROMPTS_PER_STEP:-8}" +TRAIN_GLOBAL_BATCH_SIZE="${TRAIN_GLOBAL_BATCH_SIZE:-$((NUM_PROMPTS_PER_STEP * 16))}" + +# Production lookahead, so the sampler and its capacity checks behave as they +# would at scale; the buffer floor scales with the cohort. +MAX_LOOKAHEAD_VERSIONS="${MAX_LOOKAHEAD_VERSIONS:-4}" +export MAX_LOOKAHEAD_VERSIONS +export _NUM_PROMPTS_PER_STEP="${NUM_PROMPTS_PER_STEP}" + +# Start the optimizer on a quarter cohort so streaming is actually exercised +# rather than trivially satisfied by the full cohort arriving at once. +export STREAM_MIN_GROUPS="${STREAM_MIN_GROUPS:-$((NUM_PROMPTS_PER_STEP / 4))}" +export NUM_STORAGE_UNITS="${NUM_STORAGE_UNITS:-2}" + +# The full-scale recipe inherits akamehra's results and cache directories from +# the reference run, and they are not group-writable. Point both at the +# submitter's own scratch so this profile runs as-is for whoever launches it. +_USER_SCRATCH="/lustre/fsw/portfolios/coreai/users/${USER}" +export RESULTS_DIR="${RESULTS_DIR:-${_USER_SCRATCH}/runs/nano35-sc-pipeclean-n8}" +export PERSISTENT_CACHE="${PERSISTENT_CACHE:-${_USER_SCRATCH}/.cache/nano35-dolphin}" + +export EXP_NAME="${EXP_NAME:-${USER}-nano35-sc-pipeclean-n8}" +export NRL_MAX_STEPS="${NRL_MAX_STEPS:-3}" +export WALLTIME="${WALLTIME:-2:00:00}" +export SLURM_QOS="${SLURM_QOS:-short}" +# Run the live worktree rather than a submission-time copy, so a fix can be +# retried without re-snapshotting between attempts. +export USE_SNAPSHOT="${USE_SNAPSHOT:-0}" + +if (( NRL_MAX_STEPS < 2 )); then + echo "ERROR: NRL_MAX_STEPS=${NRL_MAX_STEPS} never reaches a weight sync, which" >&2 + echo " leaves nccl_reshard refit -- the point of this run -- untested." >&2 + exit 1 +fi + +echo "Nemotron 3.5 Nano — SingleController pipeclean profile" +echo " hetgroup 0: train=${NUM_TRAIN_NODES}, generation=${NUM_GEN_NODES}, Gym=${NUM_GYM_NODES}" +echo " hetgroup 1: GenRM=${GENRM_REPLICAS}xTP${GENRM_TENSOR_PARALLEL_SIZE}, NL2Bash=${NL2BASH_REPLICAS}xTP${NL2BASH_TENSOR_PARALLEL_SIZE}" +echo " parallelism: TP4 x CP4 x EP8 x PP1 (ETP1) — unchanged, DP 2 -> 1" +echo " batch: prompts=${NUM_PROMPTS_PER_STEP}, generations=16, global=${TRAIN_GLOBAL_BATCH_SIZE}" +echo " steps: ${NRL_MAX_STEPS}" + +exec bash "${SCRIPT_DIR}/nano35_dolphin_launch_sc.sh" \ + "grpo.num_prompts_per_step=${NUM_PROMPTS_PER_STEP}" \ + "policy.train_global_batch_size=${TRAIN_GLOBAL_BATCH_SIZE}" \ + "$@" diff --git a/examples/nemo_gym/nemotron-3.5-nano/rlvr_dolphin.yaml b/examples/nemo_gym/nemotron-3.5-nano/rlvr_dolphin.yaml new file mode 100644 index 00000000000..b51704eb103 --- /dev/null +++ b/examples/nemo_gym/nemotron-3.5-nano/rlvr_dolphin.yaml @@ -0,0 +1,336 @@ +# ============================================================================= +# Nemotron 3.5 Nano — RLVR, legacy async-1, honest-dolphin warm start +# ============================================================================= +# Reproduces the internal reference run +# geshen-ultra-rl-nano-honest-dolphin-v10-iter6000-mopd-rlvr +# (launch_nano_honest_dolphin.sh + examples/configs/grpo_ultra_512n4g_bf16.yaml +# on nemo-rl-internal @ 97c55ee2, branch geshen/nano_training) +# on public NeMo-RL main. +# +# Inherits examples/nemo_gym/nemotron-3-ultra/student_rlvr1.yaml. "ultra" there +# names the RLVR *recipe lineage* — the 34-env Gym blend, GenRM comparison +# judging, judge topology, reward penalties, async-1 — not the model. The +# reference does the same thing: it runs the nano checkpoint against the ultra +# config and overrides parallelism. Everything ultra-shaped is overridden below. +# +# Model: NemotronH hybrid-Mamba MoE, 52 layers, hidden 2688, 128 routed experts +# top-6 + 1 shared, 32 heads / 2 KV, mamba n_groups=8, MTP 1 layer, vocab 131072. +# +# Node budget (GB200, 4 GPU/node), sbatch --nodes=64: +# 8 train + 40 generation -> cluster.num_nodes = 48 +# 16 gym -> env.nemo_gym.num_gpu_nodes = 16 +# +# A 5:1 generation-to-training split. GenRM is served from a SEPARATE +# allocation (2 replicas x 2 nodes = 4 nodes), so the campaign footprint is 68 +# nodes even though this job asks for 64. +# ============================================================================= +defaults: ../nemotron-3-ultra/student_rlvr1.yaml + +# ----------------------------------------------------------------------------- +# Cluster — ultra was 256 nodes / segment 16; nano uses 48 actor nodes / segment 2 +# ----------------------------------------------------------------------------- +cluster: + gpus_per_node: 4 + num_nodes: 48 + segment_size: 2 + +checkpointing: + # 4 h wall on partition `batch` (batch_long's 7 d is not what we're using), + # minus the reference's 25 min teardown margin. + checkpoint_must_save_by: "00:03:35:00" + # Reference used 10 against a 24 h wall. At 4 h a slow step can mean a job + # restarts having saved nothing and re-does the same work forever. Raise back + # toward 10 once step time is measured. + save_period: 5 + save_optimizer: true + +grpo: + # Stability setting: cut the prompt groups while retaining 16 generations per + # prompt. This yields 2048 samples/step and an async replay-buffer capacity + # of 256 groups. + num_prompts_per_step: 128 + # Validation is effectively off in the reference; genrm cohort envs are + # train-only and would hang under eval. + val_period: 10000000 + # REQUIRED by GRPOConfig, and absent from the inherited student_rlvr1.yaml — + # upstream inconsistency: 83753ed56 (Jul 29-30) added val_start_at as a required + # field while 64cb9f985 added the ultra recipes without it, so student_rlvr1.yaml + # on main cannot be validated by MasterConfig. Killed job 5733424 with + # "ValidationError: grpo.val_start_at Field required". + # -1 disables the delay, matching grpo_nanov3.yaml and grpo_math_1B.yaml. + val_start_at: -1 + +# ----------------------------------------------------------------------------- +# Policy — nano parallelism (reference: TP=4 CP=4 EP=16 PP=1 ETP=1) +# ----------------------------------------------------------------------------- +policy: + model_name: /lustre/fsw/portfolios/llmservice/users/venkats/training_actual_0603/nano_n3_post/checkpoints/nano-3.5-sft-v10-closethink-unmask-orig6k-from-midtrain-100B-lc-lr2e-5/eval/iter_0006000/hf + + # Must equal num_prompts_per_step * num_generations_per_prompt: 128 * 16. + # Over DP=2 (32 training GPUs / TP4 x CP4 x PP1) this is 1024 sequences per + # data-parallel rank — the same per-rank load as the 16-node / GBS 4096 shape + # this replaces, so per-GPU memory pressure is unchanged. + train_global_batch_size: 2048 + + # Ultra base is 49152. Reference nano runs 73728. + max_total_sequence_length: 73728 + + megatron_cfg: + tensor_model_parallel_size: 4 + expert_tensor_parallel_size: 1 + # 128 routed experts / EP8 = 16 experts per rank. Matches the control run + # (job 5931924) so the judge relocation is the only variable between them. + expert_model_parallel_size: 8 + pipeline_model_parallel_size: 1 + context_parallel_size: 4 + moe_token_dispatcher_type: "alltoall" + moe_flex_dispatcher_backend: "alltoall" + # MTP head training is inherited unchanged from the base and matches the + # reference exactly: mtp_num_layers 5, loss_scaling 0.3, use_repeated_layer + # true, detach_heads true. detach_heads keeps the MTP loss off the backbone, + # so it cannot perturb GRPO. use_repeated_layer reconciles 5 with the + # checkpoint's num_nextn_predict_layers=1 (one layer's weights reused 5x). + + generation: + # Base hardcodes 49152; re-tie to the policy value. + max_new_tokens: ${policy.max_total_sequence_length} + + vllm_cfg: + tensor_parallel_size: 4 + # Reference runs EP=1 for nano. vLLM's EP = DP * TP, and EP > TP is + # blocked upstream (NVIDIA-NeMo/RL#1101). + expert_parallel_size: 1 + max_model_len: ${policy.max_total_sequence_length} + # 0.85 -> 0.80. Job 5744748 died in weight refit: + # update_weights_from_collective: CUDA out of memory. + # Tried to allocate 3.69 GiB. GPU has 184.31 GiB total, 1.61 GiB free; + # this process holds 163.22 GiB. + # The OOM killed that vLLM rank, which was a participant in the NCCL + # collective, so MegatronPolicyWorker rank=0/22 aborted with it. + # 0.80 frees ~9 GiB per GPU against a 3.69 GiB staging buffer. + # The 40-node smoke run refit fine at 0.85 because it had 32 generation + # ranks; the buffer grows with that count. This shape has 160, more than + # the 128 that OOM'd, so 0.80 is a floor to watch rather than a settled + # value. Costs some KV cache, so slightly lower generation throughput. + gpu_memory_utilization: 0.80 + # nano_v3 reasoning parser + qwen3_coder tool parser are inherited and are + # correct for this checkpoint: its chat template emits and the + # XML form. + + vllm_kwargs: + # Hybrid Mamba allocates one state slot per running sequence. Base uses + # 256 (sized for ultra); the reference nano run uses 64 at 73728 tokens. + max_num_seqs: 64 + # Required on vLLM 0.25 and NOT accepted by 0.20 -- this branch only runs + # against a 0.25 container. Left at the default "auto", 0.25 picks the + # FlashInfer TRT-LLM BF16 MoE, whose expert weights are 3D grouped + # tensors under the new routed_experts submodule. Refit then hands them to + # a loader that rejects the expert dimension: job 5944595 died in + # update_weights_from_collective with "shard_dim=0 is not a valid data + # dimension for a 3D tensor (expected 1 or 2)". flashinfer_cutlass keeps + # the layout refit can address and is the backend the 64-node Ultra smoke + # run (job 5939591) refit through step 4 with on this same container. + moe_backend: flashinfer_cutlass + + colocated: + enabled: false + resources: + gpus_per_node: 4 + # Overridden by the launch script via NUM_GEN_NODES. + num_nodes: 40 + +# ----------------------------------------------------------------------------- +# Data — the launcher overrides these; defaults point at the reference blend. +# trusty_viper carries agent_ref per row, so NeMo-Gym routes each row to its +# family directly (no BlendDispatchAgent — that is a nemogym2mrl construct). +# ----------------------------------------------------------------------------- +data: + train: + data_path: /lustre/fs1/portfolios/llmservice/projects/llmservice_modelalignment_ppo/users/geshen/rl-data-tools/blends/curriculum_honest_dolphin_v41_trusty_viper.train.jsonl + validation: + data_path: /lustre/fs1/portfolios/llmservice/projects/llmservice_modelalignment_ppo/users/geshen/rl-data-tools/blends/curriculum_honest_dolphin_v41_trusty_viper.train.jsonl + +env: + nemo_gym: + num_gpu_nodes: 16 + + # ------------------------------------------------------------------------- + # Env blend. Identical to the base except rdkit_chemistry, which the base + # lists but the pinned Gym submodule (473f446) does not ship — it would fail + # at Gym startup. Unused by this blend, so dropped. + # ------------------------------------------------------------------------- + config_paths: + - responses_api_models/vllm_model/configs/vllm_model_for_training.yaml + - resources_servers/math_with_judge/configs/math_with_judge.yaml + - resources_servers/code_gen/configs/code_gen.yaml + - resources_servers/workplace_assistant/configs/workplace_assistant.yaml + - resources_servers/mcqa/configs/mcqa.yaml + - resources_servers/instruction_following/configs/instruction_following.yaml + - resources_servers/equivalence_llm_judge/configs/lc_judge.yaml + - resources_servers/calendar/configs/calendar.yaml + - resources_servers/genrm_compare/configs/genrm_compare.yaml + - resources_servers/equivalence_llm_judge/configs/nl2bash-equivalency.yaml + - resources_servers/equivalence_llm_judge/configs/equivalence_llm_judge.yaml + - resources_servers/single_step_tool_use_with_argument_comparison/configs/single_step_tool_use_with_argument_comparison.yaml + - resources_servers/reasoning_gym/configs/reasoning_gym.yaml + - resources_servers/terminus_judge/configs/terminus_judge_string_only.yaml + - resources_servers/ns_tools/configs/ns_tools.yaml + - resources_servers/math_formal_lean/configs/math_formal_lean_multi_turn.yaml + - resources_servers/multichallenge/configs/multichallenge.yaml + - resources_servers/inverse_if/configs/inverse_if.yaml + - resources_servers/single_step_tool_use_with_argument_comparison/configs/search_pivot_single_step_tool_use_with_argument_comparison.yaml + - resources_servers/single_step_tool_use_with_argument_comparison/configs/toolcall_schema_single_step_tool_use_with_argument_comparison.yaml + - resources_servers/single_step_tool_use_with_argument_comparison/configs/swe_pivot_single_step_tool_use_with_argument_comparison.yaml + - resources_servers/abstention/configs/abstention.yaml + - resources_servers/nvarc/configs/inductive.yaml + - resources_servers/nvarc/configs/transductive.yaml + - resources_servers/single_step_tool_use_with_argument_comparison/configs/droid_pivot_single_step_tool_use_with_argument_comparison.yaml + - resources_servers/equivalence_rule/configs/lc.yaml + - resources_servers/ether0/configs/ether0.yaml + - resources_servers/structured_outputs/configs/structured_outputs_json_yaml_xml_v1.yaml + - resources_servers/structured_outputs/configs/structured_outputs_v3.yaml + - resources_servers/format_verification/configs/freeform_formatting.yaml + - resources_servers/format_verification/configs/citation_format.yaml + - resources_servers/jailbreak_detection/configs/jailbreak_detection_nemotron_combined_reward_tp8.yaml + - resources_servers/indirect_prompt_injection/configs/indirect_prompt_injection.yaml + + # ------------------------------------------------------------------------- + # GenRM — EXTERNAL. The launcher sets base_url to the compute-side load + # balancer. `model` must match the external vLLM --served-model-name. + # Local vLLM settings below are inert whenever base_url is set. + # ------------------------------------------------------------------------- + genrm_model: + _override_: true + responses_api_models: + genrm_model: + entrypoint: app.py + api_key: dummy_key + base_url: null # set by the launcher via GENRM_BASE_URL + model: "model" + uses_reasoning_parser: true + return_token_id_information: false + debug: true + ray_worker_py_executable: /opt/ray_venvs/nemo_rl.models.generation.vllm.vllm_worker_async.VllmAsyncGenerationWorker/bin/python + # Aligned to nl2bash_judge_model where their roles overlap, except for + # GenRM-specific data parallelism, parsing, batching, and load format. + # Job 5746440's external GenRM predecessor died with a Triton illegal + # memory access; that incident is separate from job 5755734's internal + # nl2bash collective stall. + vllm_serve_env_vars: + VLLM_RAY_DP_PACK_STRATEGY: strict + NCCL_MNNVL_ENABLE: "1" + # Disable the blocked-GEMM MoE kernels. Job 5743685 died here: + # AssertionError: K must be divisible by blockK + # Error executing method 'load_model' + # genrm_model was the only vLLM service with none of these set, and + # it inherits VLLM_USE_FLASHINFER_MOE_FP8=1 from the driver's + # TRAIN_CMD. nl2bash is the control: same Qwen3-235B-A22B family, + # same TP4, loads fine with all four disabled. safety disables + # DeepGEMM too. This makes GenRM match them. + VLLM_USE_DEEP_GEMM: "0" + VLLM_MOE_USE_DEEP_GEMM: "0" + VLLM_USE_FLASHINFER_MOE_FP8: "0" + VLLM_USE_FLASHINFER_MOE_FP16: "0" + vllm_serve_kwargs: + attention_backend: FLASH_ATTN # was TRITON_ATTN; matches nl2bash + tensor_parallel_size: 4 + data_parallel_size: 4 + data_parallel_size_local: 1 + pipeline_parallel_size: 1 + enable_expert_parallel: true # matches nl2bash on the same MoE family + reasoning_parser: deepseek_r1 # GenRM-specific, nl2bash has none + gpu_memory_utilization: 0.85 + max_model_len: 131072 + max_num_seqs: 256 + max_num_batched_tokens: 8192 + enable_prefix_caching: true + enable_chunked_prefill: true + # Back to 112 (upstream default). The os error 108 in job 5744482 + # was traced to a single bad node, nvl72138-T01 (10.109.24.155), + # which also caused the ModuleNotFoundError in 5742619 and 5745204 — + # not loader-thread pressure. That node is now excluded at submit. + model_loader_extra_config: + enable_multithread_load: true + num_threads: 112 + load_format: auto + compilation_config: # matches nl2bash + cudagraph_capture_sizes: [1, 2, 4, 8, 16, 32] + + genrm_compare_resources_server: + resources_servers: + genrm_compare: + # Base ships 0.1 / 0.25 (retuned for ultra). Reference dolphin used + # 0.12 / 0.12 — restored, since we are reproducing that run. + group_reasoning_length_penalty_coeff: 0.12 + group_answer_length_penalty_coeff: 0.12 + + # Qwen3-235B-A22B-Instruct-2507-FP8. TP4 x DP8 = 32 GPUs = 8 of the 16 gym + # nodes. Base ships DP4 (sized for 256n ultra); reference uses DP8. + # Backs equivalence_llm_judge, lc_judge, math_with_judge, multichallenge, + # inverse_if and abstention — so this sizing gates six families. + nl2bash_judge_model: + responses_api_models: + local_vllm_model: + model: /lustre/fsw/portfolios/llmservice/users/ansubramania/models/Qwen3-235B-A22B-Instruct-2507-FP8 + # Launch this judge from the Gym venv, not the RL one. On a 0.25 + # container the RL venv pairs vLLM 0.25 with the openai version + # uv.lock pins, and `vllm serve` then fails to import NamespaceTool + # from openai.types.responses (job 5943331). The Gym venv is + # self-consistent. Judges need no weight sync, so they gain nothing + # from sharing the RL venv. + ray_worker_py_executable: /opt/gym_venvs/responses_api_models/local_vllm_model/.venv/bin/python + vllm_serve_env_vars: + # Matches the known-working internal dolphin/bear judge recipe. + NCCL_MNNVL_ENABLE: "0" + # Keep Ray's compiled-DAG wait aligned with vLLM's distributed wait. + RAY_CGRAPH_get_timeout: "2400" + # Off for every judge that runs from the Gym venv; see the note on + # safety_judge_model below. + VLLM_USE_FASTOKENS: "0" + vllm_serve_kwargs: + tensor_parallel_size: 4 + data_parallel_size: 8 + distributed_timeout_seconds: 2400 + # Back to 112 (upstream default); see genrm note above. + model_loader_extra_config: + enable_multithread_load: true + num_threads: 112 + + # Nemotron-Content-Safety-Reasoning-4B — 1 GPU, per the reference. + safety_judge_model: + responses_api_models: + local_vllm_model: + model: /lustre/fsw/portfolios/llmservice/users/ansubramania/super_v3/model_checkpoints/Nemotron-Content-Safety-Reasoning-4B + vllm_serve_env_vars: + # Some images bake VLLM_USE_FASTOKENS=1 into /etc/environment, so + # every process inherits it, but only the RL venv carries the wheel + # (fastokens 0.3.1). This judge has no ray_worker_py_executable, so + # it runs from the Gym venv, whose vLLM ships the integration module + # (vllm/tokenizers/fastokens.py) without the package behind it, and + # exits at import with "The 'fastokens' package (>= 0.2.0) is + # required when VLLM_USE_FASTOKENS=1" -- taking Gym spinup down with + # it (job 6232802, on the 08-15 build). The Jul-30 image's Gym vLLM + # had no fastokens support at all and ignored the flag, which is why + # this appeared only once the pin moved forward. Pinned off rather + # than left to the image so a future pin cannot reintroduce it. + # Merged onto the base's env vars rather than replacing them. + VLLM_USE_FASTOKENS: "0" + vllm_serve_kwargs: + tensor_parallel_size: 1 + data_parallel_size: 1 + +# ----------------------------------------------------------------------------- +# Logger — log_dir / wandb.name are set by the launcher. +# ----------------------------------------------------------------------------- +logger: + wandb_enabled: true + tensorboard_enabled: true + monitor_gpus: true + wandb: + project: "ultra-streaming" + +# reward_penalties inherited unchanged and already match the reference: +# penalize_duplicated_reasoning / empty_final_answer / unwanted_tokens / +# malformed_think_tag all true, token_ids {unwanted: [2], think_open: 12, +# think_close: 13}. This is the upstreamed form of the internal config's +# top-level penalize_* + token_ids block. diff --git a/examples/nemo_gym/nemotron-3.5-nano/rlvr_dolphin_sc.yaml b/examples/nemo_gym/nemotron-3.5-nano/rlvr_dolphin_sc.yaml new file mode 100644 index 00000000000..b3d203dd62e --- /dev/null +++ b/examples/nemo_gym/nemotron-3.5-nano/rlvr_dolphin_sc.yaml @@ -0,0 +1,341 @@ +# ============================================================================= +# Nemotron 3.5 Nano — RLVR on the SingleController path +# ============================================================================= +# Overlay on rlvr_dolphin.yaml that swaps the legacy async-GRPO driver for the +# SingleController (SC) architecture: streaming forward/backward, the +# TransferQueue data plane, gated in-order rollout sampling, and shard-to-shard +# NCCL-reshard weight refit. +# +# Everything about the model, parallelism, batch shape, sequence length, node +# split and the Gym blend is inherited unchanged, so a run of this config is +# directly comparable against a rlvr_dolphin.yaml run on the same 64 nodes +# (8 train + 40 generation + 16 gym, plus GenRM's own 4). +# +# Entrypoint: examples/run_grpo_single_controller.py (NOT run_grpo_nemo_gym.py). +# nano35_dolphin_launch_sc.sh sets that via NRL_ENTRYPOINT. +# +# Group vs sample units: async_rl counts PROMPT GROUPS, not generations. One +# optimizer cohort here is 2048 / 16 = 128 groups. +# ============================================================================= +defaults: rlvr_dolphin.yaml + +# ============================================================================= +# Checkpointing — on, because a 100-step run outlives the 4 h wall +# ============================================================================= +# The SC path saves and resumes now: _save_checkpoint writes policy weights, the +# optimizer, the tokenizer, the dataloader state, the replay buffer and the +# step/epoch/consumed-sample counters, and setup_single_controller resumes from +# get_latest_checkpoint_path. Restoring the buffer matters more here than the +# weights do -- a resumed run comes back with its lag-4 staleness distribution +# instead of ramping from zero, so the convergence curve has no seam at the job +# boundary. The buffer restore is skipped if the sampler name changed, so every +# job in a chain must keep the same async_rl.sampler.name. +# +# grpo.max_num_steps is compared against the *restored* step count, so it is a +# whole-campaign target: NRL_MAX_STEPS=100 once, and each requeued job continues +# toward it rather than running 100 more steps. +# +# Inherited checkpoint settings are overridden for SC recovery: +# +# metric_name student_rlvr1.yaml sets "val:total_reward/mean", which +# validate_single_controller_config rejects outright -- SC has +# no validation loop, so a "val:" metric is never collected and +# top-k retention would silently degrade to a no-op. null here +# makes retention purely recency-based, which is what we want. +# save periods Full trainer state is saved every step. Periodic rollout +# snapshots after bootstrap require a durable checkpoint at +# the matching trainer step, so a wider period would leave +# most of a short pipeclean without an eligible anchor. +# keep_top_k Inherited as 1000000, i.e. keep everything. With no metric, +# "best" is recency, so this keeps the newest 3 periodic +# checkpoints and lets the rest go. +# +# checkpoint_must_save_by comes from the launcher (CHECKPOINTING_SAVE_BY, +# 00:03:35:00) and remains a wall-clock backstop. +# ============================================================================= +checkpointing: + enabled: true + metric_name: null + save_period: 1 + ft_save_period: 1 + ft_keep_latest_k: 1 + keep_top_k: 3 + save_optimizer: true + +grpo: + # setup_single_controller rejects val_period > 0 outright, so the baseline's + # "effectively disabled" 10000000 is not enough — validation has to be off. + # val_start_at: -1, val_at_start: false and val_at_end: false are inherited + # and already satisfy the rest of the guard. + val_period: 0 + + # Required because loss_fn.reference_policy_kl_penalty is 0.0 (inherited from + # student_rlvr1.yaml). No reference model is built, but the SC train pump + # still computes reference logprobs unless this is set, and then fails with + # AttributeError: 'MegatronPolicyWorker' has no 'reference_state_dict'. + skip_reference_policy_logprobs_calculation: true + + # run_grpo_single_controller.py raises unless this is null; the SC knobs live + # under async_rl. student_rlvr1.yaml interpolates + # ${add:${grpo.async_grpo.max_trajectory_age_steps}, 1} for num_groups_nemo_rl + # in two places, so both are pinned to the value that interpolation resolved + # to (max_trajectory_age_steps: 1) under env.nemo_gym below. Nothing may + # reference this block once it is null or config load fails. + async_grpo: null + +policy: + # --------------------------------------------------------------------------- + # Speculative-decoding draft head. run_grpo_single_controller.py reads + # config.policy["draft"]["enabled"] unconditionally, so omitting this block is + # a KeyError before Ray starts. The nemo_gym runner guards it with an `in` + # check, which is why neither rlvr_dolphin.yaml nor student_rlvr1.yaml needs it. + # + # Unrelated to the MTP head this recipe trains: MTP head training is on via + # megatron_cfg (5 repeated layers, loss 0.3, detached), and MTP speculative + # decoding in vLLM is off (ENABLE_MTP_INFERENCE=0), both inherited. + # --------------------------------------------------------------------------- + draft: + enabled: false + model_name: null + loss_weight: 0.1 + num_layers: null + aux_layer_indices: null + + generation: + # ------------------------------------------------------------------------- + # Pinned, not interpolated. rlvr_dolphin.yaml ties max_new_tokens to + # ${policy.max_total_sequence_length}, which also feeds vllm_cfg.max_model_len + # — so the generation budget and the context window are the same number and + # a prompt at exactly that length can never emit a token. It is rejected + # deterministically instead, and because Gym prepends system prompts and tool + # schemas at rollout time and rollouts are multi-turn, no first-turn filter + # bounds it: that pairing rejected 2108 prompts in one 8-node ultra probe arm + # and 1344 in another at a 16384 window. + # + # 73728 is the value the interpolation already resolves to here, kept + # identical on purpose so these arms stay comparable with the v1 and windowed + # arms they are measured against. The pin buys decoupling, not headroom: what + # actually keeps rejections at zero on this recipe is that the window is 4.5x + # the ultra one and no row of the dolphin blend approaches it (job 5995586 + # logged no context rejections at this window). Anyone lowering + # max_total_sequence_length here must now choose a smaller max_new_tokens + # deliberately, which is the step the ultra fan-in config skipped. + # ------------------------------------------------------------------------- + max_new_tokens: 73728 + + # ------------------------------------------------------------------------- + # Weight refit transport. The inherited default (null) broadcasts every full + # parameter tensor from the 32 training ranks to all 160 generation ranks. + # nccl_reshard instead moves each parameter shard-to-shard between the two + # parallelism layouts over NCCL's M2N reshard, so no rank ever materializes + # a full tensor. The bulk path covers the MoE FFN projections, which are + # 97-98% of the weights at EP=16. + # + # Declared here rather than passed with `+` because refit_transport is + # NotRequired in VllmConfig; with the key present, REFIT_TRANSPORT=null in + # nano35_dolphin_launch_sc.sh falls back to the broadcast path. + # + # This config satisfies every config-checkable constraint in + # check_nccl_reshard_refit_support: non-colocated, Megatron train, vLLM gen, + # ETP=1, PP=1 with no custom layout, BF16 end to end, no EPLB, and vLLM + # expert_parallel_size=1 (allowed alongside any TP; the other legal value is + # EP == TP). + # + # STILL WORTH CHECKING ON THE FIRST RUN: if the container lacks the nccl4py + # M2N integration, xferdtensor.py logs "nccl.m2n.reshard not found" and + # silently falls back to a Python implementation. Grep the logs for it. + # + # The MTP-head risk flagged here before is now handled: this checkpoint's HF + # export does emit MTP expert projections (mtp.layers.N.mixer.experts.*), and + # they did take the bulk path and fail (job 5943756, "layer_prefix mismatch: + # mtp != backbone"). is_nccl_reshard_param now routes anything under mtp. to + # the misc path, matching what the broadcast baseline does with them. + # REFIT_TRANSPORT=null remains the escape hatch. + # ------------------------------------------------------------------------- + refit_transport: nccl_reshard + + # vllm_cfg.gpu_memory_utilization stays at the inherited 0.80. That was + # lowered from 0.85 for the broadcast path's staging buffer (job 5744748 + # OOM'd in update_weights_from_collective at 128 generation ranks, and this + # shape has 160) — the exact pressure nccl_reshard removes. 0.85 may be + # recoverable here once a reshard run is confirmed healthy; left alone so + # the two recipes differ only in the transport. + + vllm_cfg: + # ----------------------------------------------------------------------- + # Samples vllm:num_requests_running / num_requests_waiting / + # kv_cache_usage_perc off each engine so a step's exposed_generation can + # be attributed. Every run so far shows generation as the binding + # constraint (0.18 groups/s produced against a 0.26 groups/s trainer + # appetite) without showing why: a compute-bound fleet and an + # under-subscribed one are indistinguishable from the trainer's side, and + # only the second gets faster when max_inflight_prompts rises. A waiting + # queue that is busy most of the step means the engines are the ceiling + # and deeper lag cannot help. + # + # Matches the nemotron-3-ultra recipes, which all enable this. Costs one + # Prometheus snapshot per engine every 0.5 s on a daemon thread. + # ----------------------------------------------------------------------- + enable_vllm_metrics_logger: true + vllm_metrics_logger_interval: 0.5 + +# ============================================================================= +# Loss — off-policy correction, required by every arm past lag 1 +# ============================================================================= +# student_rlvr1.yaml pins the importance ratio at 1, which makes +# use_importance_sampling_correction inert: the sampled tokens come from an +# older policy but the loss is computed as though they came from the current +# one. A lag sweep run that way measures the uncorrected loss, not the +# staleness, so the pin is lifted here for every sampler. This costs the +# prev_logprobs forward pass that the inherited pin was skipping. +# ============================================================================= +loss_fn: + force_on_policy_ratio: false + +# ============================================================================= +# SingleController async RL — required by MasterConfig +# ============================================================================= +async_rl: + # in_order is checkpoint-replay capable and is the gated sampler used by the + # partial-rollout recovery implementation on this branch. The launcher can + # switch to weight_fifo or windowed while emitting that sampler's correctly + # named slack field. + sampler: + name: in_order + max_lookahead_versions: 4 + + recompute_kv_cache_after_weight_updates: false + + # Begin forward/backward once this many groups are ready rather than waiting + # for all 128; the remainder streams in behind it and accumulates into the + # single GBS=2048 optimizer step. 32 is 25% of the cohort, the ratio the + # validated smaller-scale streaming runs used to get 2-3 chunks per step. + min_groups_for_streaming_train: 32 + + # Let the current cohort and all four lookahead cohorts generate + # concurrently: num_prompts_per_step * (max_staleness_versions + 1) = 128 * 5. + max_inflight_prompts: 640 + + # Hard floor, not a preference: validate_sampler_buffer_capacity raises if + # this is below num_prompts_per_step * (max_staleness_versions + 1) = 640, + # because the rollout pump would deadlock waiting for buffer slots. It scales + # with the slack, so raising one raises both. + # + # Keep it EQUAL to max_inflight_prompts, never above. The buffer permit is + # acquired before dispatch, so it is a dispatch semaphore rather than passive + # capacity: job 6014206 set 768 against 384 in flight and ran 1.8x slower. + max_buffered_rollouts: 640 + + # Rollout fault tolerance. Gym retries dropped connections itself, but hands + # us env-server 500s and truncated response bodies, and one of those used to + # abort the whole run: jobs 5966951 and 5967263 died that way after one and + # six steps while the legacy stack absorbed 400 of the same faults and kept + # training. + # + # Retrying matters more here than it does in v1. A dropped prompt leaves its + # batch a group short, and the in_order sampler only ever offers the trainer + # groups stamped for the current step, so that step can never close: job + # 5982927 wedged exactly this way after step 7 when two prompts hit 500s from + # a single sick env server, then sat idle for 104 minutes. Keeping the batch + # whole is worth waiting for; the trainer falls back to a shortened batch only + # once every attempt is gone. + # + # Keys follow the post-#3589 taxonomy: infra failures (dead shard, timeout, + # transport) re-dispatch onto a different shard, data failures do not. The two + # budgets are independent counters, not a total and a sub-total. + rollout_failure: + # 5 infra attempts with a 1s base gives 1s/2s/4s/8s, bounded at 30s. v1's + # AsyncTrajectoryCollector retries a comparable number of times, so neither + # stack absorbs more than the other. + max_infra_attempts_per_prompt: 5 + max_data_attempts_per_prompt: 2 + backoff_base_s: 1.0 + max_backoff_s: 30.0 + + # This branch has no replacement-reserve policy. Fail after exhausting the + # per-prompt budgets instead of silently accepting an in_order cohort hole. + max_skipped_prompts: 0 + + # Deadlines. This is the NeMo-Gym rollout path, so only the nemo_gym block + # applies; validate_single_controller_config rejects a populated native + # block here rather than letting it look effective. Nothing below SC bounds + # a rollout -- Gym retries its server-to-server hops in an uncapped loop + # against an aiohttp session with no ClientTimeout -- so without this a + # wedged env server holds its in_order slot until the wall clock. + nemo_gym: + rollout_timeout_s: 1800.0 + + # Must outlast every deadline above, which _check_watchdog_outlasts_rollouts + # enforces: below it, a rollout that is merely slow reads as a stall. + watchdog: + interval_s: 30.0 + stall_timeout_s: 2400.0 + stall_action: warn + gym_subprocess_check: true + +# ============================================================================= +# NeMo-Gym rollout actor +# ============================================================================= +env: + nemo_gym: + # Pinned literals for what student_rlvr1.yaml interpolated off + # grpo.async_grpo.max_trajectory_age_steps, which is null on this path. + policy_model: + responses_api_models: + vllm_model: + num_groups_nemo_rl: 2 + policy_model_reasoning_off: + responses_api_models: + vllm_model: + num_groups_nemo_rl: 2 + +# ============================================================================= +# TransferQueue data plane — mandatory for the SC path +# ============================================================================= +# run_grpo_single_controller.py refuses to start unless enabled is true, and +# DataPlaneConfig requires every key below with no Python-side defaults. +# global_segment_size / local_buffer_size are read only when backend is +# mooncake_cpu, but must still be present. +# ============================================================================= +data_plane: + enabled: true + impl: transfer_queue + backend: "simple" + storage_capacity: 1000000 + # Sharding only — storage_capacity above is global and TQ splits it, as + # ceil(total_storage_size / num_data_storage_units), so this trades unit count + # against per-unit size and reserves nothing extra. Raising it costs one CPU + # actor each and buys read/write fan-out. + # + # 16 is headroom, not a fix for anything measured. The windowed sweep sizes + # the buffer as num_prompts_per_step * (staleness + 1), so the arms hold 256 / + # 384 / 640 / 896 groups resident — up to 14336 rows at staleness 6, which is + # 40% past the 10240 the previous value of 8 was reasoned against. Peak + # resident is still only ~1.4% of the 1000000-row global capacity, so the + # concern is the actors' aggregate throughput rather than their space. Held + # identical across arms deliberately: varying it would confound the staleness + # comparison the sweep exists to make. + num_storage_units: 16 + claim_meta_poll_interval_s: 0.5 + # Mooncake-only: _init_tq reads these solely on the mooncake_cpu branch, and + # this recipe is backend "simple". Inert here, kept for backend switches. + global_segment_size: 549755813888 # 512 GiB + local_buffer_size: 68719476736 # 64 GiB + # Required for restoring replay rows and token-capture receipts alongside + # trainer state and for periodic rollout snapshots. + checkpointing_enabled: true + +# Gate-authoritative token capture gives every logical sibling a durable TQ +# receipt that can be reused after restart. +token_capture: + enabled: true + on_capture_failure: continue + mixed_weight_version_policy: allow + +# Periodic data-plane/ledger snapshots. The launcher exposes both intervals as +# environment variables and forwards them as Hydra overrides. +rollout_checkpointing: + interval_s: 120 + telemetry_interval_s: 30 + keep_latest_k: 2 + restore_mode: latest diff --git a/nano35_ledger_smoke.sh b/nano35_ledger_smoke.sh new file mode 100644 index 00000000000..62f59cf2966 --- /dev/null +++ b/nano35_ledger_smoke.sh @@ -0,0 +1,93 @@ +#!/bin/bash +# ============================================================================= +# nano35_ledger_smoke.sh — BATCH rerun of the nano-35-rlvr-sc-tc-smoke shape +# (failed job 6300221) on the lineage-ledger token-capture code. +# +# Purpose: regression validation of the multi-worker capture path. Job 6300221 +# died at the first rollout batch because the old per-process gate registry +# (register-before-dispatch) split across policy_model num_workers=16 / +# policy_model_reasoning_off num_workers=4 — 409 Conflict on unregistered +# workers, then UnknownRolloutError from ingest_coords mid-flight. The ledger +# replaced that with a process-shared FileLineageStore, so the SAME shape with +# num_workers deliberately left at 16/4 must now pass. +# +# Deviations from the original run, both deliberate: +# - container: zhiyul nightly-gym.2026-08-10 (the bake this branch's launch +# fixes were validated against), not amahishi's nightly-gym squashfs. +# - paths/W&B/secrets: pthombre-owned. +# Everything else (node shape, judges, batch geometry, walltime, QOS, model, +# blend, num_workers defaults) mirrors job 6300221. +# +# Run from a NETWORKED shell at the repo root (fsw twin path): +# DRY_RUN=1 bash nano35_ledger_smoke.sh # inspect only +# DRY_RUN=0 bash nano35_ledger_smoke.sh # submit +# ============================================================================= +set -euo pipefail +HERE="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" + +_DRY_RUN_IN="${DRY_RUN:-}" + +set -a +# Secrets (WANDB_API_KEY, HF_TOKEN): untracked mode-600 file, never in git. +# shellcheck disable=SC1091 +source "${HERE}/swe_nano.secrets.env" + +# ---- container / shared read-only assets ------------------------------------ +CONTAINER=/lustre/fsw/portfolios/llmservice/users/zhiyul/enroot-images/nvcr.io+nvidian+nemo-rl+nightly-gym.2026-08-10.squashfs +SANDBOX_CONTAINER=/lustre/fsw/portfolios/coreai/users/cye/enroot/nemo-rl:skills-sandbox-latest.squashfs +# Model/blend/judge paths are baked into the dolphin launcher + recipe +# defaults (all verified readable); HF_HUB_CACHE keeps reading zhiyul's +# already-downloaded hub shards. +HF_HUB_CACHE=/lustre/fsw/portfolios/llmservice/users/zhiyul/hf_cache/hub + +# ---- per-user write paths ---------------------------------------------------- +WORKSPACE_DIR=/lustre/fsw/portfolios/llmservice/users/pthombre/sweRun/RL-pr3456-delta-smoke-workspace +HF_HOME=/lustre/fsw/portfolios/llmservice/users/pthombre/hf_cache +PERSISTENT_CACHE=/lustre/fsw/portfolios/llmservice/users/pthombre/persistent_cache +NRL_MEGATRON_CHECKPOINT_DIR=${PERSISTENT_CACHE}/megatron_ckpt_cache + +EXP_NAME="${SC_EXP_NAME:-nano35-rlvr-sc-tc-smoke-ledger}" +RESULTS_DIR=${WORKSPACE_DIR}/results/${EXP_NAME} +BASE_LOG_DIR=${WORKSPACE_DIR}/ray_logs/${EXP_NAME} +WANDB_PROJ=nano-35-rlvr + +# ---- Slurm shape: exactly job 6300221 ---------------------------------------- +SLURM_PARTITION=batch +SLURM_ACCOUNT=nemotron_sw_post +SLURM_QOS=short +GPUS_PER_NODE=4 # GB200 NVL72 +WALLTIME=2:00:00 +# Smoke wrapper defaults reproduce the rest: 4 train + 1 gen + 1 gym nodes, +# EXTERNAL_JUDGES=1 (GenRM 1xTP4 + NL2Bash 1xTP4 hetgroup), 8 prompts x 16 +# generations, NRL_MAX_STEPS=3, STREAM_MIN_GROUPS=2, NUM_STORAGE_UNITS=2. +# num_workers stays at the recipe's 16/4 — the regression trigger under test. + +USE_SNAPSHOT=0 # live worktree, same as the original run + +# ---- launch plumbing proven by the 0820 ledger A/B smokes -------------------- +# (swe_nano.env / swe_nano_sc_capture.sh; see session/20260820_004547) +NRL_FORCE_REBUILD_VENVS=false +NRL_FORCE_REBUILD_VENVS_LIST="nemo_rl.environments.nemo_gym.NemoGym,nemo_rl.models.generation.vllm.vllm_worker_async.VllmAsyncGenerationWorker" +NRL_DRIVER_PYTHONPATH="/opt/nemo-rl/3rdparty/Gym-workspace/Gym" +NRL_DRIVER_PIP_INSTALL="orjson" +NRL_DRIVER_UV_RUN_FLAGS="--locked --no-sync" +NRL_TQ_SKIP_RUNTIME_ENV_PIN=1 +NRL_VENV_SYNC_FROZEN=1 +NRL_WG_USE_RAY_REF=1 +NRL_REFIT_ERRORS_FATAL=1 +# Shared prewarmed uv cache — NEVER /tmp; and never UV_CACHE_DIR_OVERRIDE with +# prefetch-venvs containers (it severs the baked venvs' hardlinks). +UV_CACHE_DIR=/lustre/fsw/portfolios/llmservice/users/pthombre/uv + +DRY_RUN="${_DRY_RUN_IN:-1}" +set +a + +# Same trailing override as the original invocation. rollout_checkpointing is +# an extra-allowed (ignored) block in this tree; kept for command fidelity. +# checkpointing.enabled=false: this tree's SingleController raises +# NotImplementedError on trainer checkpointing (the recovery-refresh branch +# supports it); irrelevant to the multi-worker capture path under test. +exec bash "${HERE}/examples/nemo_gym/nemotron-3.5-nano/nano35_dolphin_launch_sc_smoke.sh" \ + rollout_checkpointing.restore_mode=none \ + checkpointing.enabled=false \ + "$@" diff --git a/nemo_rl/algorithms/advantage_estimator.py b/nemo_rl/algorithms/advantage_estimator.py index 491de0d9414..3167603967d 100644 --- a/nemo_rl/algorithms/advantage_estimator.py +++ b/nemo_rl/algorithms/advantage_estimator.py @@ -66,7 +66,7 @@ def __init__( self.use_leave_one_out_baseline = estimator_config.use_leave_one_out_baseline self.normalize_rewards = estimator_config.normalize_rewards - def compute_advantage(self, prompt_ids, rewards, mask, **kwargs): + def compute_advantage(self, prompt_ids, rewards, mask, valid_mask=None, **kwargs): """Compute GRPO advantages. Args: @@ -74,6 +74,11 @@ def compute_advantage(self, prompt_ids, rewards, mask, **kwargs): rewards: Tensor of shape [batch_size] containing reward for each sample. mask: Response token mask of shape [batch_size, seq_len], 1 for valid response tokens, 0 for padding. Used only for expanding advantages to token-level shape. + valid_mask: Optional tensor of shape [batch_size], 1.0 for samples whose + reward should participate in the per-prompt baseline/std. Token-capture + placeholder rows carry 0.0 (their sample_mask already excludes them + from the loss; excluding them here keeps siblings' baselines unbiased). + None keeps the legacy all-valid behavior. **kwargs: Additional arguments (unused). Returns: @@ -82,7 +87,7 @@ def compute_advantage(self, prompt_ids, rewards, mask, **kwargs): baseline, std = calculate_baseline_and_std_per_prompt( prompt_ids, rewards, - torch.ones_like(rewards), + torch.ones_like(rewards) if valid_mask is None else valid_mask.float(), leave_one_out_baseline=self.use_leave_one_out_baseline, ) advantages = (rewards - baseline).unsqueeze(-1) diff --git a/nemo_rl/algorithms/async_utils/replay_buffer.py b/nemo_rl/algorithms/async_utils/replay_buffer.py index 7d408d7b4fe..a37cf5a61c6 100644 --- a/nemo_rl/algorithms/async_utils/replay_buffer.py +++ b/nemo_rl/algorithms/async_utils/replay_buffer.py @@ -13,7 +13,6 @@ # limitations under the License. import asyncio -import gc import statistics import threading as _threading import uuid @@ -22,17 +21,13 @@ from typing import Any, Iterable, Optional import ray -import torch from nemo_rl.algorithms.async_utils.interfaces import ReplayBufferProtocol from nemo_rl.data_plane import KVBatchMeta -from nemo_rl.data_plane.schema import ROUTED_EXPERTS_FIELD -from nemo_rl.experience.interfaces import ( - NEMO_GYM_TASK_INDEX_KEY, - NEXT_NEMO_GYM_TASK_INDEX_KEY, - PromptGroupRecord, -) +from nemo_rl.data_plane.schema import ROUTE_PLAN_TAG, ROUTED_EXPERTS_FIELD +from nemo_rl.experience.interfaces import PromptGroupRecord from nemo_rl.experience.payload import pack_payload, record_to_train_batch +from nemo_rl.experience.row_dump import maybe_dump_train_rows from nemo_rl.utils.r3_trace import trace_rollout_payload @@ -346,44 +341,6 @@ def state_dict(self) -> dict[str, Any]: "max_size": self.max_size, } - def save_to_path(self, path: str) -> int: - """Serialize inside the actor without materializing the buffer on the driver.""" - state = self.state_dict() - torch.save(state, path) - num_trajectories = len(state["trajectories"]) - del state - gc.collect() - return num_trajectories - - def load_from_path( - self, - path: str, - num_prompts_per_step: int | None = None, - current_training_step: int | None = None, - max_age_steps: int | None = None, - ) -> dict[str, int]: - """Restore inside the actor and return only compact coordination metadata.""" - state = torch.load(path, weights_only=False) - saved_task_indices = [ - int(trajectory[NEMO_GYM_TASK_INDEX_KEY]) - for trajectory in state.get("trajectories", []) - if trajectory.get(NEMO_GYM_TASK_INDEX_KEY) is not None - ] - next_task_index = max(saved_task_indices, default=-1) + 1 - num_trajectories = len(state["trajectories"]) - self.load_state_dict( - state, - num_prompts_per_step=num_prompts_per_step, - current_training_step=current_training_step, - max_age_steps=max_age_steps, - ) - del state - gc.collect() - return { - "num_trajectories": num_trajectories, - NEXT_NEMO_GYM_TASK_INDEX_KEY: next_task_index, - } - def load_state_dict( self, state: dict[str, Any], @@ -694,11 +651,16 @@ def __init__( partition_id: str, *, pad_value_dict: Mapping[str, int], + staging_partition_id: Optional[str] = None, require_routed_experts: bool = False, ): self._dp_client = dp_client self._partition_id = partition_id self._pad_value_dict = dict(pad_value_dict) + # Token-capture mode only (docs/design-docs/token-capture-ledger.md): + # the staging partition whose per-call delta rows `remove` must clear + # alongside the canonical rows. None on the legacy path. + self._staging_partition_id = staging_partition_id self._require_routed_experts = require_routed_experts self.meta_list: list[Optional[KVBatchMeta]] = [] self.start_weight_list: list[int] = [] @@ -707,6 +669,9 @@ def __init__( self.target_step_list: list[Optional[int]] = [] self.ready_list: list[bool] = [] self._group_ids: list[str] = [] + # Parallel to the lists above; populated only in token-capture mode. + self._rollout_ids_list: list[Optional[list[str]]] = [] + self._staging_keys_list: list[Optional[list[str]]] = [] def reserve( self, @@ -714,6 +679,7 @@ def reserve( weight_version: int, target_step: Optional[int] = None, group_id: Optional[str] = None, + rollout_ids: Optional[list[str]] = None, ) -> str: """Append an unready slot tagged with weight_version. @@ -721,6 +687,9 @@ def reserve( weight_version: Weight version stamped on the slot. target_step: Training step this slot targets; only consulted by StalenessSampler.force_in_order. group_id: Per-group sample_id prefix; defaults to a fresh uuid4. + rollout_ids: Token-capture mode: the ledger-registered rollout ids + this slot dispatched, recorded so cleanup can name what it + owns even before a receipt exists. Returns: group_id used by the matching commit. @@ -733,6 +702,10 @@ def reserve( self.target_step_list.append(target_step) self.ready_list.append(False) self._group_ids.append(group_id) + self._rollout_ids_list.append( + list(rollout_ids) if rollout_ids is not None else None + ) + self._staging_keys_list.append(None) return group_id async def commit( @@ -758,12 +731,12 @@ async def commit( ValueError: group_id has no live slot (removed or never reserved). RuntimeError: router replay is enabled but the payload has no routes. """ - # Precondition: reserve() must have registered this group_id. Raise - # before any side effects so a stray commit doesn't leak orphan DP rows. + # Check the slot is still live BEFORE writing: a slot evicted while + # its rollout was in flight must not orphan rows into the partition. if group_id not in self._group_ids: raise ValueError( - f"commit called with unknown group_id={group_id!r}; " - f"reserve() must precede commit() (or the slot was already removed)" + f"TQReplayBuffer.commit: group {group_id} has no live slot " + "(evicted or never reserved); nothing written" ) train_batch = record_to_train_batch(record, pad_value_dict=self._pad_value_dict) sample_ids, fields, tags = pack_payload( @@ -776,6 +749,13 @@ async def commit( "not produce that field. Check vLLM routed-expert capture and " "the async message-log flattening path." ) + maybe_dump_train_rows( + source="legacy_commit", + group_id=group_id, + sample_ids=list(sample_ids), + train_batch=train_batch, + weight_version=start_weight_version, + ) trace_rollout_payload(keys=sample_ids, data=train_batch) try: await self._call_dp( @@ -785,7 +765,6 @@ async def commit( fields=fields, tags=tags, ) - # mirrors kv_first_write lengths = train_batch["input_lengths"] meta = KVBatchMeta( @@ -797,7 +776,15 @@ async def commit( tags=[dict(t) for t in tags], ) - idx = self._group_ids.index(group_id) + try: + idx = self._group_ids.index(group_id) + except ValueError: + # Evicted during the awaited write: un-write the rows so the + # partition holds nothing the buffer no longer tracks. + raise ValueError( + f"TQReplayBuffer.commit: group {group_id} was evicted " + "during the write; rows cleared" + ) from None self.meta_list[idx] = meta self.end_weight_list[idx] = end_weight_version self.ready_list[idx] = True @@ -839,9 +826,113 @@ async def remove_group(self, group_id: str, *, remove_in_dp: bool = False) -> in raise ValueError(f"unknown group_id={group_id!r}") from error return await self.remove([idx], remove_in_dp=remove_in_dp) + async def commit_finalized( + self, + group_id: str, + meta: KVBatchMeta, + group_min_wv: int, + group_max_wv: int, + *, + staging_keys: Optional[list[str]] = None, + ) -> KVBatchMeta: + """Mark a slot ready from finalizer output (token-capture mode). + + Unlike :meth:`commit`, the canonical rows are already in TQ — the + finalizer tensorized and put them — so this only fills the slot. + The slot's effective version is the group's OLDEST call version + (``group_min_wv``): staleness accounting stays conservative when a + rollout straddles a refit. + + Args: + group_id: group_id returned by the matching reserve call. + meta: KVBatchMeta the finalizer built over its published rows. + group_min_wv: Oldest weight version any call in the group used. + group_max_wv: Newest weight version any call in the group used. + staging_keys: The group's staged delta keys, recorded so + :meth:`remove` can clear the staging partition too. + + Raises: + ValueError: group_id has no live slot (removed or never reserved). + """ + try: + idx = self._group_ids.index(group_id) + except ValueError: + raise ValueError( + f"TQReplayBuffer.commit_finalized: group {group_id} has no " + "live slot (evicted or never reserved)" + ) from None + tagged_plans = [ + tag[ROUTE_PLAN_TAG] for tag in (meta.tags or []) if ROUTE_PLAN_TAG in tag + ] + if tagged_plans: + if len(tagged_plans) != len(meta.sample_ids): + raise ValueError( + "commit_finalized received mixed deferred/direct route plans" + ) + from nemo_rl.experience.route_plan import decode_route_plan + + plan_cleanup_keys = { + key + for encoded in tagged_plans + for key in decode_route_plan(encoded).cleanup_staging_keys + } + provided_staging_keys = list(staging_keys or []) + if len(provided_staging_keys) != len(set(provided_staging_keys)): + raise ValueError("commit_finalized staging_keys contains duplicates") + if set(provided_staging_keys) != plan_cleanup_keys: + raise ValueError( + "commit_finalized staging ownership does not match route plans: " + f"provided={sorted(provided_staging_keys)!r}, " + f"planned={sorted(plan_cleanup_keys)!r}" + ) + self.meta_list[idx] = meta + self.start_weight_list[idx] = group_min_wv + self.end_weight_list[idx] = group_max_wv + self.ready_list[idx] = True + self._staging_keys_list[idx] = ( + list(staging_keys) if staging_keys is not None else None + ) + return meta + + def abort(self, group_id: str) -> bool: + """Drop an unready slot whose dispatch failed or was cancelled. + + Token-capture mode; called from the failed dispatch path. + No DataPlane rows are cleared here. Callers with sealed receipts must + clear their deterministic canonical IDs and full staging manifests + before dropping this ownership record. Before receipt sealing, orphan + cleanup remains an explicit controlled-validation limitation. + + Returns: + True when a slot was dropped; False when the group_id has no + live slot (already committed+consumed or never reserved). + """ + try: + idx = self._group_ids.index(group_id) + except ValueError: + return False + if self.ready_list[idx]: + return False + self._delete_slot(idx) + return True + + def _delete_slot(self, idx: int) -> None: + del self.meta_list[idx] + del self.start_weight_list[idx] + del self.end_weight_list[idx] + del self.target_step_list[idx] + del self.ready_list[idx] + del self._group_ids[idx] + del self._rollout_ids_list[idx] + del self._staging_keys_list[idx] + async def remove(self, idxs: list[int], remove_in_dp: bool) -> int: """Drop entries at the given indices and optionally clear them from DataPlane. + In token-capture mode (``staging_partition_id`` set), clearing a + group also clears its recorded staged delta rows, so eviction leaves + neither canonical nor staging bytes behind. + Args: idxs: Entry indices to drop. Must be within [0, size). remove_in_dp: If True, also clear the dropped rows from DataPlane. @@ -860,23 +951,46 @@ async def remove(self, idxs: list[int], remove_in_dp: bool) -> int: ) dropped_sample_ids: list[str] = [] + dropped_staging_keys: list[str] = [] for i in drop_idxs: meta = self.meta_list[i] if meta is not None: dropped_sample_ids.extend(meta.sample_ids) - del self.meta_list[i] - del self.start_weight_list[i] - del self.end_weight_list[i] - del self.target_step_list[i] - del self.ready_list[i] - del self._group_ids[i] + staging_keys = self._staging_keys_list[i] + if staging_keys: + dropped_staging_keys.extend(staging_keys) if remove_in_dp: - await self._call_dp( - "clear_samples", - sample_ids=dropped_sample_ids, - partition_id=self._partition_id, - ) + if dropped_sample_ids: + try: + await self._call_dp( + "clear_samples", + sample_ids=dropped_sample_ids, + partition_id=self._partition_id, + ) + except Exception as error: + raise RuntimeError( + "canonical cleanup failed; retained replay-buffer ownership " + f"partition={self._partition_id!r}, " + f"sample_ids={dropped_sample_ids!r}" + ) from error + if dropped_staging_keys and self._staging_partition_id is not None: + try: + await self._call_dp( + "clear_samples", + sample_ids=dropped_staging_keys, + partition_id=self._staging_partition_id, + ) + except Exception as error: + raise RuntimeError( + "staging cleanup failed; retained replay-buffer ownership " + f"partition={self._staging_partition_id!r}, " + f"staging_keys={dropped_staging_keys!r}; canonical rows " + "may already be cleared" + ) from error + + for i in drop_idxs: + self._delete_slot(i) return len(drop_idxs) diff --git a/nemo_rl/algorithms/single_controller.py b/nemo_rl/algorithms/single_controller.py index 3a6a60e8f08..e604d8c5168 100644 --- a/nemo_rl/algorithms/single_controller.py +++ b/nemo_rl/algorithms/single_controller.py @@ -35,9 +35,10 @@ from __future__ import annotations import asyncio +import statistics import time from functools import partial -from typing import Any, Optional, Union, cast +from typing import TYPE_CHECKING, Any, Optional, Union, cast import ray import torch @@ -61,13 +62,18 @@ from nemo_rl.data.interfaces import DatumSpec from nemo_rl.data_plane import KVBatchMeta from nemo_rl.data_plane.schema import DP_CALIB_INPUT_FIELDS +from nemo_rl.data_plane.schema import ROUTE_PLAN_TAG from nemo_rl.distributed.batched_data_dict import BatchedDataDict +from nemo_rl.experience.route_plan import decode_route_plan from nemo_rl.models.generation.sglang.sglang_generation import SGLangGeneration from nemo_rl.models.generation.vllm import VllmGeneration from nemo_rl.models.policy.tq_policy import TQPolicy from nemo_rl.utils.logger import Logger from nemo_rl.utils.timer import Timer +if TYPE_CHECKING: + from nemo_rl.experience.finalizer_actor import FinalizationRequest + Generation = Union[VllmGeneration, SGLangGeneration] @@ -123,6 +129,14 @@ def __init__( # Rebind so writer and sampler share one buffer instance even # when Ray deserializes rollout_manager and tq_buffer separately. self._rollout_manager._tq_buffer = self._buffer + self._finalizer_actors = list(actor_args.finalizer_actors) + self._available_finalizers: asyncio.Queue[Any] = asyncio.Queue() + for actor in self._finalizer_actors: + self._available_finalizers.put_nowait(actor) + self._active_finalizers = 0 + self._finalizer_waiters = 0 + self._finalizer_unknown_outcomes = 0 + self._finalizer_metrics_by_group: dict[str, dict[str, float]] = {} # Built here, not on the driver: Logger backends (wandb/tb/...) hold # _thread.lock that Ray can't cloudpickle into the actor. @@ -215,6 +229,11 @@ async def run(self) -> dict[str, Any]: rollout_task.cancel() train_task.cancel() await asyncio.gather(rollout_task, train_task, return_exceptions=True) + for actor in self._finalizer_actors: + try: + ray.kill(actor, no_restart=True) + except Exception as error: + print(f"finalizer actor termination failed: {error}", flush=True) self._logger.finish() return { @@ -231,6 +250,10 @@ async def ping(self) -> dict[str, Any]: "inflight_rollouts": self._inflight_rollouts, "rollout_permitted": self._rollout_permitted.is_set(), "epoch": self._current_epoch, + "active_finalizers": self._active_finalizers, + "finalizer_waiters": self._finalizer_waiters, + "finalizer_queue_depth": self._available_finalizers.qsize(), + "finalizer_unknown_outcomes": self._finalizer_unknown_outcomes, } # ── internal helpers ─────────────────────────────────────────────────── @@ -250,6 +273,203 @@ async def _call_dp(self, method_name: str, **kwargs) -> Any: return await result return result + @staticmethod + def _request_staging_keys(request: "FinalizationRequest") -> list[str]: + """Return the full receipt-manifest staging ownership for a request.""" + keys: list[str] = [] + for receipt in request.receipts: + if receipt is None: + continue + manifest = receipt.get("manifest") + if not isinstance(manifest, list): + continue + for record in manifest: + if isinstance(record, dict) and isinstance( + record.get("staging_key"), str + ): + keys.append(record["staging_key"]) + return list(dict.fromkeys(keys)) + + async def _cleanup_known_finalization_request( + self, request: "FinalizationRequest" + ) -> None: + """Clear known request ownership after a pre-publication/known outcome.""" + errors: list[BaseException] = [] + try: + await self._call_dp( + "clear_samples", + sample_ids=list(request.rollout_ids), + partition_id=self._partition_id, + ) + except Exception as error: + errors.append( + RuntimeError( + "pre-publication canonical cleanup failed for " + f"group={request.group_id!r}, ids={request.rollout_ids!r}" + ) + ) + errors[-1].__cause__ = error + staging_keys = self._request_staging_keys(request) + if staging_keys: + try: + await self._call_dp( + "clear_samples", + sample_ids=staging_keys, + partition_id=self._master_config.token_capture.staging_partition, + ) + except Exception as error: + errors.append( + RuntimeError( + "pre-publication staging cleanup failed for " + f"group={request.group_id!r}, keys={staging_keys!r}" + ) + ) + errors[-1].__cause__ = error + if errors: + raise BaseExceptionGroup( + f"known-outcome cleanup failed for group {request.group_id}", + errors, + ) + self._buffer.abort(request.group_id) + + async def _finalize_with_actor(self, request: "FinalizationRequest") -> None: + """Submit one metadata request to the bounded fixed actor pool.""" + self._finalizer_waiters += 1 + queue_depth = max( + 0, + self._finalizer_waiters - self._available_finalizers.qsize(), + ) + queue_start = time.perf_counter() + try: + actor = await self._available_finalizers.get() + except asyncio.CancelledError: + await self._cleanup_known_finalization_request(request) + raise + finally: + self._finalizer_waiters -= 1 + queue_wait_ms = (time.perf_counter() - queue_start) * 1000.0 + self._active_finalizers += 1 + active_actor_count = self._active_finalizers + finalize_start = time.perf_counter() + try: + finalized = await actor.finalize.remote(request) + except BaseException: + self._finalizer_unknown_outcomes += 1 + print( + "FATAL: finalizer actor RPC failed after submission; canonical " + f"publication outcome is unknown for group {request.group_id}. " + "Stopping validation without actor replacement or retry.", + flush=True, + ) + raise + else: + self._available_finalizers.put_nowait(actor) + finally: + self._active_finalizers -= 1 + finalize_total_ms = (time.perf_counter() - finalize_start) * 1000.0 + + if finalized.dropped: + try: + await self._cleanup_known_finalization_request(request) + except BaseException as cleanup_error: + raise RuntimeError( + "finalizer dropped the group and known-key cleanup failed " + f"for group {request.group_id}" + ) from cleanup_error + raise RuntimeError( + f"token capture: group {request.group_id} dropped " + "(min_valid_fraction_per_group)" + ) + if finalized.meta is None: + try: + await self._cleanup_known_finalization_request(request) + except BaseException as cleanup_error: + raise RuntimeError( + "finalizer returned no metadata and known-key cleanup failed " + f"for group {request.group_id}" + ) from cleanup_error + raise RuntimeError( + f"finalizer returned no metadata for non-dropped group {request.group_id}" + ) + try: + await self._buffer.commit_finalized( + request.group_id, + finalized.meta, + finalized.group_min_wv, + finalized.group_max_wv, + staging_keys=finalized.staging_keys, + ) + except BaseException as commit_error: + try: + await self._cleanup_known_finalization_request(request) + except BaseException as cleanup_error: + raise BaseExceptionGroup( + f"finalizer commit and known-key cleanup failed for " + f"group {request.group_id}", + [commit_error, cleanup_error], + ) + raise + finalized.metrics.update( + { + "finalize/queue_wait_ms": queue_wait_ms, + "finalize/total_ms": finalize_total_ms, + "finalize/queue_depth": float(queue_depth), + "finalize/active_actor_count": float(active_actor_count), + } + ) + self._finalizer_metrics_by_group[request.group_id] = dict(finalized.metrics) + + async def _cleanup_consumed_metas(self, metas: list[KVBatchMeta]) -> None: + """Clear canonical rows and full-manifest staging keys after train success.""" + canonical_by_partition: dict[str, list[str]] = {} + staging_by_partition: dict[str, list[str]] = {} + for meta in metas: + canonical_by_partition.setdefault(meta.partition_id, []).extend( + meta.sample_ids + ) + for tag in meta.tags or []: + encoded_plan = tag.get(ROUTE_PLAN_TAG) + if encoded_plan is None: + continue + plan = decode_route_plan(encoded_plan) + staging_by_partition.setdefault(plan.staging_partition, []).extend( + plan.cleanup_staging_keys + ) + + errors: list[BaseException] = [] + for partition_id, sample_ids in canonical_by_partition.items(): + unique_ids = list(dict.fromkeys(sample_ids)) + try: + await self._call_dp( + "clear_samples", + sample_ids=unique_ids, + partition_id=partition_id, + ) + except Exception as error: + cleanup_error = RuntimeError( + "post-train canonical cleanup failed: " + f"partition={partition_id!r}, sample_ids={unique_ids!r}" + ) + cleanup_error.__cause__ = error + errors.append(cleanup_error) + for partition_id, staging_keys in staging_by_partition.items(): + unique_keys = list(dict.fromkeys(staging_keys)) + try: + await self._call_dp( + "clear_samples", + sample_ids=unique_keys, + partition_id=partition_id, + ) + except Exception as error: + cleanup_error = RuntimeError( + "post-train staging cleanup failed: " + f"partition={partition_id!r}, staging_keys={unique_keys!r}" + ) + cleanup_error.__cause__ = error + errors.append(cleanup_error) + if errors: + raise BaseExceptionGroup("post-train DataPlane cleanup failed", errors) + # ── the three pumps + the inline advantage stage ─────────────────────── async def _rollout_pump(self) -> None: @@ -263,9 +483,8 @@ async def _rollout_pump(self) -> None: 1. Acquire _buffer_capacity slot (backpressure) 2. Acquire sem (cap concurrent in-flight rollouts) 3. Wait for _rollout_permitted (paused during weight sync) - 4. Call rollout_manager.generate_and_push(prompt) — local async - RolloutManager reserves a slot, runs the rollout, then commits the - group via TQReplayBuffer (→ dp_client.put_samples + mark ready) + 4. Run the rollout, then either commit it directly or submit its + metadata-only request to the finalizer actor pool. 5. Decrement _inflight_rollouts """ sem = asyncio.Semaphore(self._async_cfg.max_inflight_prompts) @@ -279,20 +498,39 @@ async def _dispatch_one_prompt( ) -> None: task_started_event.set() self._inflight_rollouts += 1 + generation_permit_released = False + inflight_count_released = False + ownership_transferred = False try: - await self._rollout_manager.generate_and_push( - prompt, - target_step=target_step, - inflight_registry=self._inflight_by_group_id, - ) + if self._finalizer_actors: + request = await self._rollout_manager.generate_for_finalization( + prompt, + target_step=target_step, + inflight_registry=self._inflight_by_group_id, + ) + self._inflight_rollouts -= 1 + inflight_count_released = True + sem.release() + generation_permit_released = True + await self._finalize_with_actor(request) + else: + await self._rollout_manager.generate_and_push( + prompt, + target_step=target_step, + inflight_registry=self._inflight_by_group_id, + ) + ownership_transferred = True except BaseException: # On success ownership transfers to the train pump, which # releases this permit after consuming the committed group. - self._buffer_capacity.release() + if not ownership_transferred: + self._buffer_capacity.release() raise finally: - self._inflight_rollouts -= 1 - sem.release() + if not inflight_count_released: + self._inflight_rollouts -= 1 + if not generation_permit_released: + sem.release() if self._async_cfg.diagnostics: content = "" @@ -379,6 +617,9 @@ async def _train_pump(self) -> None: min_sample_version = None step_open = False calibration_batches: list[BatchedDataDict[Any]] = [] + consumed_metas: list[KVBatchMeta] = [] + consumed_group_count = 0 + step_finalizer_metrics: dict[str, list[float]] = {} with self._timer.time("total_step_time"): while groups_dispatched < grpo_cfg.num_prompts_per_step: @@ -435,9 +676,19 @@ async def _train_pump(self) -> None: await asyncio.sleep(0.005) continue - # Release buffer capacity - for _ in range(num_groups): - self._buffer_capacity.release() + consumed_metas.append(train_meta) + consumed_group_count += num_groups + selected_group_ids = { + sample_id.rsplit("_g", 1)[0] + for sample_id in train_meta.sample_ids + } + for group_id in selected_group_ids: + for name, value in self._finalizer_metrics_by_group.pop( + group_id, {} + ).items(): + step_finalizer_metrics.setdefault(name, []).append( + float(value) + ) # Compute prev_logprobs / ref_logprobs if ( @@ -509,19 +760,23 @@ async def _train_pump(self) -> None: else: min_sample_version = curr_min_sample_version - # Remove consumed sample_ids from the buffer - await self._call_dp( - "clear_samples", - sample_ids=list(train_meta.sample_ids), - partition_id=self._partition_id, - ) - groups_dispatched += num_groups with self._timer.time("policy_training"): result = await asyncio.to_thread(self._trainer.finish_train_step) + await self._cleanup_consumed_metas(consumed_metas) + for _ in range(consumed_group_count): + self._buffer_capacity.release() + step_metrics = aggregate_step_metrics(result) + step_metrics.update( + { + name: statistics.fmean(values) + for name, values in step_finalizer_metrics.items() + if values + } + ) step_metrics.update( reduce_advantage_pump_metrics(**self._step_log_dict) ) @@ -694,6 +949,12 @@ async def _sync_weights( print(f" _sync_weights: sync done in {elapsed:.3f}s", flush=True) self._rollout_manager.set_weight_version(self._trainer_version) + if self._master_config.token_capture.enabled: + # Rotate the version vLLM workers stamp on captured model calls + # (per-call tagging; group staleness = min over the group's calls). + await asyncio.to_thread( + self._gen.set_rollout_weight_version, self._trainer_version + ) self._rollout_permitted.set() return aborted_stale_inflight_groups @@ -752,6 +1013,9 @@ async def _advantage_stage(self, meta: KVBatchMeta) -> KVBatchMeta: rewards=rewards, mask=mask, repeated_batch=repeated_batch, + # Real validity (token-capture placeholders carry sample_mask 0) + # instead of the hardwired all-ones — § 9.1, advantage_estimator. + valid_mask=sample_mask, **kwargs, ) response_advantages = torch.masked_select(advantages, mask.bool()) diff --git a/nemo_rl/algorithms/single_controller_utils/config.py b/nemo_rl/algorithms/single_controller_utils/config.py index 031b478f0ee..ca9a6973bde 100644 --- a/nemo_rl/algorithms/single_controller_utils/config.py +++ b/nemo_rl/algorithms/single_controller_utils/config.py @@ -14,10 +14,11 @@ from __future__ import annotations +import warnings from dataclasses import dataclass, field -from typing import Any, Optional +from typing import Any, Literal, Optional -from pydantic import BaseModel, Field +from pydantic import BaseModel, Field, PositiveInt from nemo_rl.algorithms.async_utils.staleness_sampler import ( InOrderSamplerConfig, @@ -52,6 +53,48 @@ class AsyncRLConfig(BaseModel, extra="allow"): diagnostics: bool = False +class TokenCaptureConfig(BaseModel, extra="allow"): + """Ledger-authoritative token capture (token-in/token-out via NeMo-Gym). + + Dormant by default: with ``enabled=False`` every legacy codepath behaves + exactly as before — no staging partition is registered, no ledger is + installed, and rollouts ride the token-echo path. See + docs/design-docs/token-capture-ledger.md. + """ + + enabled: bool = False + # TQ partition holding per-call staged token deltas (cleared by the + # finalizer; distinct from the canonical rollout partition). + staging_partition: str = "rollout_staging" + # A failed worker-side stage poisons the rollout; "continue" serves the + # completion and lets the finalizer emit a placeholder row, "abort" fails + # the whole rollout in the ledger. + on_capture_failure: Literal["continue", "abort"] = "continue" + # "allow" trains groups whose calls span a refit (staleness accounted via + # group_min_wv); "reject" placeholders them. Strict modes beyond the MVP + # matrix raise NotImplementedError at setup. + mixed_weight_version_policy: Literal["allow", "reject"] = "allow" + # Drop the whole group when fewer than this fraction of its rollouts + # produced valid rows (None keeps every group). + min_valid_fraction_per_group: Optional[float] = None + # Bearer token for Gym's token-capture control routes. None = + # minted per run at setup; set explicitly only for multi-controller + # setups that must share one ledger. + control_auth_token: Optional[str] = None + # Hard deadline per control-plane call (S5 finding: control-plane death must + # surface as a failed dispatch, not a silent retry stall). + control_timeout_s: float = 60.0 + # Root for Gym's per-rollout capture ledgers and base capture layer. None = + # derived at setup + # under the run's log dir. + capture_dir: Optional[str] = None + # Keep routed_experts out of canonical rows and assemble them on policy + # workers from strict staged-fragment plans. + defer_routed_experts_to_policy: bool = False + # Fixed CPU finalizer pool size; actors are never automatically replaced. + num_finalizer_workers: PositiveInt = 2 + + class MasterConfig(BaseModel, extra="allow"): policy: PolicyConfig loss_fn: ClippedPGLossConfig @@ -63,6 +106,7 @@ class MasterConfig(BaseModel, extra="allow"): checkpointing: CheckpointingConfig data_plane: DataPlaneConfig async_rl: AsyncRLConfig + token_capture: TokenCaptureConfig = Field(default_factory=TokenCaptureConfig) def validate_sampler_buffer_capacity( @@ -118,6 +162,25 @@ def validate_single_controller_config(master_config: MasterConfig) -> None: sampler_name=async_config.sampler.name, ) + token_capture_config = master_config.token_capture + if token_capture_config.defer_routed_experts_to_policy and not ( + token_capture_config.enabled + ): + raise ValueError( + "token_capture.defer_routed_experts_to_policy requires " + "token_capture.enabled=true" + ) + if ( + token_capture_config.enabled + and token_capture_config.num_finalizer_workers + > async_config.max_buffered_rollouts + ): + warnings.warn( + "token_capture.num_finalizer_workers exceeds " + "async_rl.max_buffered_rollouts; excess finalizer actors cannot be busy", + stacklevel=2, + ) + # A non-zero reference-policy KL penalty makes the loss read # ``reference_policy_logprobs``, but the SC train pump only computes them # when ``skip_reference_policy_logprobs_calculation`` is false (see diff --git a/nemo_rl/algorithms/single_controller_utils/setup.py b/nemo_rl/algorithms/single_controller_utils/setup.py index 96eff61231d..27096f1e550 100644 --- a/nemo_rl/algorithms/single_controller_utils/setup.py +++ b/nemo_rl/algorithms/single_controller_utils/setup.py @@ -21,6 +21,7 @@ from __future__ import annotations +import os import time from concurrent.futures import ThreadPoolExecutor from dataclasses import dataclass @@ -92,6 +93,7 @@ class SingleControllerActorArgs: rollout_manager: RolloutManager tq_buffer: TQReplayBuffer partition_id: str + finalizer_actors: list[Any] def _build_clusters( @@ -300,6 +302,12 @@ def _spinup_gym(master_config: MasterConfig, base_urls: list[str]) -> tuple[Any, enable_router_replay=enable_router_replay, routed_experts_dtype=routed_experts_dtype, use_fastokens=bool(policy_config["tokenizer"].get("use_fastokens")), + # Ledger config rides into Gym's policy model server (§ 9.1). + token_capture=( + master_config.token_capture.model_dump() + if master_config.token_capture.enabled + else None + ), ) return actor, time.perf_counter() - t0 @@ -400,6 +408,53 @@ def setup_single_controller( "data.use_multiple_dataloader=True yet." ) + # Token capture: validate the MVP matrix loudly at setup (§ 6, § 10) and + # give capture-enabled vLLM workers a venv that carries nemo_gym (the + # worker hosts Gym's capture core + adapter in-process). + token_capture_cfg = master_config.token_capture + if token_capture_cfg.enabled: + if not _should_use_nemo_gym(master_config): + raise ValueError( + "token_capture.enabled requires the NeMo-Gym rollout path " + "(env.should_use_nemo_gym=true) — the ledger lives in Gym's " + "policy model server" + ) + if generation_config["backend"] != "vllm": + raise NotImplementedError( + "token_capture.enabled supports the vllm backend only; got " + f"{generation_config['backend']!r}" + ) + if not generation_config["vllm_cfg"]["async_engine"]: + raise ValueError( + "token_capture.enabled requires " + "policy.generation.vllm_cfg.async_engine=true (the capture " + "host is the worker's in-process HTTP server)" + ) + from nemo_rl.distributed.ray_actor_environment_registry import ( + ACTOR_ENVIRONMENT_REGISTRY, + ) + from nemo_rl.distributed.virtual_cluster import PY_EXECUTABLES + + ACTOR_ENVIRONMENT_REGISTRY[ + "nemo_rl.models.generation.vllm.vllm_worker_async.VllmAsyncGenerationWorker" + ] = PY_EXECUTABLES.VLLM_GYM + + # Fill the derived ledger-hosting fields (see TokenCaptureConfig): a + # per-run control-plane bearer token and the process-shared capture + # directory used by every Gym worker. + if token_capture_cfg.control_auth_token is None: + # Deferred import: only needed on the capture path. + import secrets + + token_capture_cfg.control_auth_token = secrets.token_hex(32) + if token_capture_cfg.capture_dir is None: + token_capture_cfg.capture_dir = os.path.abspath( + os.path.join( + master_config.logger.get("log_dir") or "logs", + "gym_token_capture", + ) + ) + set_seed(grpo_config.seed) # ========================== @@ -560,6 +615,71 @@ def _build_generation_then_trainer( # Connect-only DP client; TQPolicy already bootstrapped the controller. dp_client = build_data_plane_client(dp_config, bootstrap=False) + # Token-capture mode: pre-register both rollout partitions from this + # single driver thread before any producer is live. TQ's controller + # registers unseen field names lazily inside update_production_status + # without a lock, so the first concurrent puts into an unregistered + # partition can race kv_retrieve_meta and kill the controller thread + # (see TQDataPlaneClient.register_partition). + token_capture_cfg = master_config.token_capture + if token_capture_cfg.enabled: + from nemo_rl.data_plane.schema import ( + DP_TRAIN_FIELDS, + fields_with_optional_routed_experts, + ) + from nemo_rl.data_plane.schema import ( + ROUTED_EXPERTS_FIELD as STAGING_ROUTED_EXPERTS_FIELD, + ) + from nemo_rl.data_plane.tq_token_sink import STAGING_FIELDS + + r3_enabled = router_replay_enabled(master_config.policy) + if token_capture_cfg.defer_routed_experts_to_policy and not r3_enabled: + raise ValueError( + "token_capture.defer_routed_experts_to_policy requires " + "policy.router_replay.enabled=true" + ) + group_size = grpo_config.num_generations_per_prompt + num_rollout_samples = master_config.async_rl.max_buffered_rollouts * group_size + dp_client.register_partition( + partition_id=partition_id, + fields=fields_with_optional_routed_experts( + DP_TRAIN_FIELDS, + enabled=r3_enabled + and not token_capture_cfg.defer_routed_experts_to_policy, + ), + num_samples=num_rollout_samples, + consumer_tasks=["prev_lp", "ref_lp", "train"], + grpo_group_size=group_size, + ) + dp_client.register_partition( + partition_id=token_capture_cfg.staging_partition, + fields=list(STAGING_FIELDS) + + ([STAGING_ROUTED_EXPERTS_FIELD] if r3_enabled else []), + num_samples=num_rollout_samples, + consumer_tasks=["finalize", "prev_lp", "train"], + ) + # Host Gym's capture core in every vLLM DP leader (in-worker DP + # client + TQTokenSink + the single install_capture call), and give + # workers the initial weight version to stamp on captured calls. + try: + generation.setup_token_capture( + dp_config, token_capture_cfg.staging_partition + ) + except Exception as error: + if "No module named 'nemo_gym'" in str(error): + # Worker venvs are cached by actor class name + # (nemo_rl/utils/venvs.py), so a venv prebuilt before token + # capture predates the nemo_gym extra and is reused as-is. + raise RuntimeError( + "token_capture.enabled requires nemo_gym inside the vLLM " + "worker venv, but the cached worker venv predates it. " + "Rebuild worker venvs (NRL_FORCE_REBUILD_VENVS=true) or " + "delete $NEMO_RL_VENV_DIR/nemo_rl.models.generation.vllm." + "vllm_worker_async.VllmAsyncGenerationWorker and rerun." + ) from error + raise + generation.set_rollout_weight_version(0) + t0 = time.perf_counter() weight_synchronizer = create_weight_synchronizer( policy=trainer, @@ -587,7 +707,30 @@ def _build_generation_then_trainer( partition_id=partition_id, pad_value_dict={"token_ids": pad_id, "input_ids": pad_id}, require_routed_experts=router_replay_enabled(policy_config), + staging_partition_id=( + token_capture_cfg.staging_partition if token_capture_cfg.enabled else None + ), ) + finalizer_actors: list[Any] = [] + if token_capture_cfg.enabled: + from nemo_rl.experience.finalizer_actor import ( + FinalizerActorConfig, + create_finalizer_actors, + ) + + finalizer_actors = create_finalizer_actors( + dp_config, + FinalizerActorConfig( + partition_id=partition_id, + staging_partition=token_capture_cfg.staging_partition, + pad_token_id=pad_id, + mixed_weight_version_policy=token_capture_cfg.mixed_weight_version_policy, + min_valid_fraction_per_group=token_capture_cfg.min_valid_fraction_per_group, + router_replay_enabled=router_replay_enabled(policy_config), + defer_routed_experts_to_policy=token_capture_cfg.defer_routed_experts_to_policy, + ), + num_workers=token_capture_cfg.num_finalizer_workers, + ) rollout_manager = RolloutManager( tokenizer=tokenizer, task_to_env=env_handles, @@ -622,5 +765,6 @@ def _build_generation_then_trainer( rollout_manager=rollout_manager, tq_buffer=tq_buffer, partition_id=partition_id, + finalizer_actors=finalizer_actors, ) return actor_args, setup_timing_metrics diff --git a/nemo_rl/data_plane/adapters/transfer_queue.py b/nemo_rl/data_plane/adapters/transfer_queue.py index 995cfa24c37..a125f2418e3 100644 --- a/nemo_rl/data_plane/adapters/transfer_queue.py +++ b/nemo_rl/data_plane/adapters/transfer_queue.py @@ -23,6 +23,7 @@ from __future__ import annotations import ipaddress +import logging import os import socket import subprocess @@ -159,6 +160,18 @@ class layout — if TQ restructures, this becomes a no-op with a if _TQ_RUNTIME_ENV_PATCHED: return + # Skip the pin when the deployment declares the base env already ships + # TQ on every node (container-baked, or single-node test runs whose venv + # is shared): the per-actor pip env Ray would build does not inherit the + # parent site-packages and breaks on nodes without network egress. + if os.environ.get("NRL_TQ_SKIP_RUNTIME_ENV_PIN", "") == "1": + logging.getLogger(__name__).info( + "NRL_TQ_SKIP_RUNTIME_ENV_PIN=1: not injecting the TransferQueue " + "pip pin into TQ actor runtime_envs (base-env TQ assumed)." + ) + _TQ_RUNTIME_ENV_PATCHED = True + return + runtime_env = {"pip": [_resolve_tq_pin()]} def _install(cls) -> bool: diff --git a/nemo_rl/data_plane/preshard.py b/nemo_rl/data_plane/preshard.py index f9ce2fdc6c7..9790cdd60cb 100644 --- a/nemo_rl/data_plane/preshard.py +++ b/nemo_rl/data_plane/preshard.py @@ -142,6 +142,7 @@ def shard_meta_for_dp( flat_idx.extend(idx_list) rank_sample_ids = [meta.sample_ids[i] for i in idx_list] rank_seqlens = [seq_lens[i] for i in idx_list] + rank_tags = [meta.tags[i] for i in idx_list] if meta.tags is not None else None rank_extra = dict(base_extra) # Per-shard packing metadata — set by ``shard_by_batch_size`` when # sequence_packing or dynamic_batching is enabled. Workers' @@ -165,6 +166,7 @@ def shard_meta_for_dp( sample_ids=rank_sample_ids, fields=meta.fields, sequence_lengths=rank_seqlens, + tags=rank_tags, extra_info=rank_extra, ) ) diff --git a/nemo_rl/data_plane/schema.py b/nemo_rl/data_plane/schema.py index 49cf79422e7..68aed992bc5 100644 --- a/nemo_rl/data_plane/schema.py +++ b/nemo_rl/data_plane/schema.py @@ -62,6 +62,15 @@ DP_CALIB_INPUT_FIELDS = (INPUT_IDS, INPUT_LENGTHS, "multi_modal_inputs") ROUTED_EXPERTS_FIELD = "routed_experts" +ROUTED_LEN_FIELD = "routed_len" +ROUTED_EXPERTS_ENCODING_FIELD = "routed_experts_encoding" +ROUTED_EXTRAS_METADATA_FIELD = "extras_metadata_json" + +# Deferred route storage. Canonical rows carry one strict encoded route plan +# per tag; policy workers omit the absent canonical route column and assemble +# it from staging immediately before previous-policy logprob or training. +ROUTE_PLAN_TAG = "route_assembly_plan" +ROUTE_PASSTHROUGH_FLAG = "route_passthrough" # Per-sample 1D scalar fields. The TQ adapter promotes these to ``(N, 1)`` # on write to work around TQ v0.1.9's KVStorageManager schema/data mismatch on @@ -79,6 +88,19 @@ INPUT_LENGTHS, "total_reward", SAMPLE_MASK, + ROUTED_LEN_FIELD, + ROUTED_EXPERTS_ENCODING_FIELD, + "schema_version", + "digest_version", + "extras_digest_version", + "parent_call_id_present", + "capture_mode", + "prev_len", + "delta_len", + "cum_len", + "weight_version", + "chain_hash_present", + "cumulative_hash_present", } ) diff --git a/nemo_rl/data_plane/tq_token_sink.py b/nemo_rl/data_plane/tq_token_sink.py new file mode 100644 index 00000000000..b38bdf7a34d --- /dev/null +++ b/nemo_rl/data_plane/tq_token_sink.py @@ -0,0 +1,560 @@ +# Copyright (c) 2026, NVIDIA CORPORATION. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +"""TransferQueue implementations of NeMo-Gym's token staging protocols. + +``TQTokenSink``/``TQTokenSource`` are NeMo-RL's providers for the +ledger-authoritative capture design (docs/design-docs/token-capture-ledger.md): +the sink is the worker-side write of one model call's token delta to the +``rollout_staging`` partition — the design's only heavy token hop — and the +source is the finalizer's read-back of those rows by staging key. This module +is the only hot-path file that knows tokens live in TQ; Gym sees opaque +staging keys. + +Each staged row carries three jagged columns (``token_ids_delta``, +``token_mask_delta``, ``generation_logprobs_delta``), the complete receipt +identity/lineage metadata, and all digest inputs so it round-trips to a +complete ``StagedCallSnapshot``. Masks/logprobs are float32 on the wire, +matching ``compute_staging_digest``'s float32-bit-pattern scheme, so the +finalizer's digest recomputation over fetched values is byte-exact. +""" + +from __future__ import annotations + +import json +import logging +from dataclasses import dataclass +from typing import Any + +import ray +import torch +from tensordict import TensorDict + +from nemo_gym.token_id_capture.staging.records import ( + StagedCallRecord, + StagedCallSnapshot, + StageResult, +) + +from nemo_rl.data_plane.schema import ( + ROUTED_EXPERTS_ENCODING_FIELD, + ROUTED_EXPERTS_FIELD, + ROUTED_EXTRAS_METADATA_FIELD, + ROUTED_LEN_FIELD, +) + +STAGING_FIELDS = [ + "token_ids_delta", + "token_mask_delta", + "generation_logprobs_delta", + "schema_version", + "digest_version", + "extras_digest_version", + "rollout_id_utf8", + "model_call_id_utf8", + "parent_call_id_utf8", + "parent_call_id_present", + "capture_mode", + "prev_len", + "delta_len", + "cum_len", + "weight_version", + "digest_bytes", + "extras_digest_bytes", + "chain_hash_bytes", + "chain_hash_present", + "cumulative_hash_bytes", + "cumulative_hash_present", + ROUTED_EXTRAS_METADATA_FIELD, + ROUTED_EXPERTS_ENCODING_FIELD, + ROUTED_LEN_FIELD, +] + +_ROUTE_ENCODING_NONE = 0 +_ROUTE_ENCODING_ENVELOPE = 1 +_ROUTE_ENCODING_LIST = 2 +_MODE_TO_CODE = {"text": 0, "token_in": 1} +_CODE_TO_MODE = {code: mode for mode, code in _MODE_TO_CODE.items()} + + +def _bytes_tensor(value: bytes) -> torch.Tensor: + """Encode non-empty bytes as one jagged TQ row.""" + if not value: + raise ValueError("staging byte fields must be non-empty") + return torch.tensor([list(value)], dtype=torch.uint8) + + +def _optional_digest_fields(value: str | None) -> tuple[torch.Tensor, torch.Tensor]: + return ( + _bytes_tensor(bytes.fromhex(value) if value is not None else bytes(32)), + torch.tensor([value is not None], dtype=torch.bool), + ) + + +@dataclass(frozen=True) +class FetchedStagedCall: + """One explicitly identified small-column finalization fetch result.""" + + staging_key: str + snapshot: StagedCallSnapshot + routed_len: int + + +def _call_dp(dp_client: Any, method_name: str, **kwargs: Any) -> Any: + """Call a DataPlaneClient method on a local client or a Ray actor handle.""" + method = getattr(dp_client, method_name) + remote = getattr(method, "remote", None) + if remote is not None: + return ray.get(remote(**kwargs)) + return method(**kwargs) + + +class TQTokenSink: + """Gym ``StagingSink`` over ``DataPlaneClient.put_samples``. + + ``stage`` is synchronous and returns only after TQ acknowledged the + write, so the capture layer's fail-closed ordering (bytes durable before + the model call is acked) holds by construction. Failures are reported in + the ``StageResult`` — the caller decides whether the rollout poisons or + aborts (``token_capture.on_capture_failure``). + """ + + def __init__(self, dp_client: Any, *, staging_partition: str) -> None: + self._dp_client = dp_client + self._staging_partition = staging_partition + + def stage(self, record: StagedCallRecord) -> StageResult: + key = record.staging_key + try: + field_dict = { + "token_ids_delta": torch.tensor( + [record.token_ids_delta], dtype=torch.int64 + ), + "token_mask_delta": torch.tensor( + [record.token_mask_delta], dtype=torch.float32 + ), + "generation_logprobs_delta": torch.tensor( + [record.generation_log_probs_delta], dtype=torch.float32 + ), + "schema_version": torch.tensor( + [record.schema_version], dtype=torch.int64 + ), + "digest_version": torch.tensor( + [record.digest_version], dtype=torch.int64 + ), + "extras_digest_version": torch.tensor( + [record.extras_digest_version], dtype=torch.int64 + ), + "rollout_id_utf8": _bytes_tensor(record.rollout_id.encode("utf-8")), + "model_call_id_utf8": _bytes_tensor( + record.model_call_id.encode("utf-8") + ), + "parent_call_id_utf8": _bytes_tensor( + (record.parent_call_id or "\0").encode("utf-8") + ), + "parent_call_id_present": torch.tensor( + [record.parent_call_id is not None], dtype=torch.bool + ), + "capture_mode": torch.tensor( + [_MODE_TO_CODE[record.mode]], dtype=torch.int64 + ), + "prev_len": torch.tensor([record.prev_len], dtype=torch.int64), + "delta_len": torch.tensor([record.delta_len], dtype=torch.int64), + "cum_len": torch.tensor([record.cum_len], dtype=torch.int64), + "weight_version": torch.tensor( + [record.weight_version], dtype=torch.int64 + ), + "digest_bytes": _bytes_tensor(bytes.fromhex(record.digest)), + "extras_digest_bytes": _bytes_tensor( + bytes.fromhex(record.extras_digest) + ), + } + chain_hash, chain_hash_present = _optional_digest_fields(record.chain_hash) + cumulative_hash, cumulative_hash_present = _optional_digest_fields( + record.cumulative_hash + ) + field_dict.update( + { + "chain_hash_bytes": chain_hash, + "chain_hash_present": chain_hash_present, + "cumulative_hash_bytes": cumulative_hash, + "cumulative_hash_present": cumulative_hash_present, + } + ) + extras_metadata = dict(record.extras) if record.extras is not None else None + routed = ( + extras_metadata.pop("routed_experts", None) + if extras_metadata is not None + else None + ) + field_dict[ROUTED_EXTRAS_METADATA_FIELD] = _bytes_tensor( + json.dumps( + extras_metadata, + sort_keys=True, + separators=(",", ":"), + ensure_ascii=False, + ).encode("utf-8") + ) + routed_len = 0 + routed_encoding = _ROUTE_ENCODING_NONE + if routed is not None: + delta_len = len(record.token_ids_delta) + if isinstance(routed, str): + from nemo_rl.utils.routed_experts_codec import ( + decode_routed_experts, + ) + + dtype_name = routed.split(":", 3)[1] + dtype = { + "int8": torch.int8, + "int16": torch.int16, + "int32": torch.int32, + }.get(dtype_name) + if dtype is None: + raise ValueError( + f"unsupported routed_experts dtype {dtype_name!r}" + ) + experts = decode_routed_experts(routed, dtype) + routed_encoding = _ROUTE_ENCODING_ENVELOPE + else: + experts = torch.tensor(routed, dtype=torch.int16) + routed_encoding = _ROUTE_ENCODING_LIST + if experts.dim() != 3 or experts.shape[0] != delta_len: + raise ValueError( + "routed_experts must already be delta-aligned: " + f"got shape {tuple(experts.shape)} for delta_len={delta_len}" + ) + field_dict[ROUTED_EXPERTS_FIELD] = experts.unsqueeze(0) + routed_len = int(experts.shape[0]) + field_dict[ROUTED_EXPERTS_ENCODING_FIELD] = torch.tensor( + [routed_encoding], dtype=torch.int64 + ) + field_dict[ROUTED_LEN_FIELD] = torch.tensor([routed_len], dtype=torch.int64) + fields = TensorDict(field_dict, batch_size=[1]) + tags = [ + { + "rollout_id": record.rollout_id, + "model_call_id": record.model_call_id, + "parent_call_id": record.parent_call_id, + "prev_len": record.prev_len, + "delta_len": record.delta_len, + "cum_len": record.cum_len, + "weight_version": record.weight_version, + "digest": record.digest, + "schema_version": record.schema_version, + } + ] + _call_dp( + self._dp_client, + "put_samples", + sample_ids=[key], + partition_id=self._staging_partition, + fields=fields, + tags=tags, + ) + except Exception as error: # noqa: BLE001 — any failure must poison, not crash serving + # The reason string is dropped downstream (_failed_coords carries + # only the disposition) — this log line is the only place the + # actual stage failure is visible. + logging.getLogger(__name__).warning( + "TQTokenSink.stage failed for %s: %s: %s", + key, + type(error).__name__, + error, + ) + return StageResult( + ok=False, staging_key=key, error=f"{type(error).__name__}: {error}" + ) + return StageResult(ok=True, staging_key=key) + + def clear(self, staging_keys: list[str]) -> None: + """Drop staged rows (finalizer / eviction cleanup).""" + if not staging_keys: + return + _call_dp( + self._dp_client, + "clear_samples", + sample_ids=list(staging_keys), + partition_id=self._staging_partition, + ) + + +class TQTokenSource: + """Gym ``StagingSource`` over ``DataPlaneClient.get_samples``. + + All requested rows are fetched in a single batched ``get_samples`` call + (TQ returns jagged delta columns as nested tensors; ``_from_wire`` + preserves the raggedness), in the order requested. A missing or + unreadable row raises ``KeyError`` per the protocol — the finalizer maps + that to a placeholder, never a silent skip. TQ's field-readiness check + is all-or-nothing across a batch, so the extras fallback is batch-level: + extras-free runs land in the base schema exactly like the old per-key + probe, but a batch with *mixed* extras presence degrades every row to + the base schema (worker feature-gating makes presence uniform per run). + """ + + def __init__(self, dp_client: Any, *, staging_partition: str) -> None: + self._dp_client = dp_client + self._staging_partition = staging_partition + + def fetch(self, staging_keys: list[str]) -> list[StagedCallSnapshot]: + """Fetch Gym snapshots, retaining legacy optional route materialization.""" + if not staging_keys: + return [] + try: + # Extras are optional per row (feature-gated at the worker); + # try the extended selection first, fall back to the base + # schema so extras-free rows keep fetching. + try: + rows = _call_dp( + self._dp_client, + "get_samples", + sample_ids=list(staging_keys), + partition_id=self._staging_partition, + select_fields=STAGING_FIELDS + [ROUTED_EXPERTS_FIELD], + ) + except Exception: # noqa: BLE001 — field-not-present probe + rows = _call_dp( + self._dp_client, + "get_samples", + sample_ids=list(staging_keys), + partition_id=self._staging_partition, + select_fields=STAGING_FIELDS, + ) + except Exception as error: # noqa: BLE001 — protocol maps any miss to KeyError + raise KeyError( + f"staged rows for {len(staging_keys)} keys could not be " + f"fetched from {self._staging_partition!r}: {error}" + ) from error + # TQ's kv path only errors when *zero* keys resolve; a partial miss + # returns fewer rows with no error. Guard explicitly so a lost row + # rejects the rollout as missing_staging_row instead of surfacing + # later as a misleading digest mismatch from misaligned zipping. + n_rows = int(rows.batch_size[0]) if len(rows.batch_size) else 0 + if n_rows != len(staging_keys): + raise KeyError( + f"staged rows missing: requested {len(staging_keys)} keys " + f"from {self._staging_partition!r}, got {n_rows} rows" + ) + # Row order mirrors the requested key order; the finalizer's digest + # recomputation is the byte-exact backstop if that ever breaks. + snapshots: list[StagedCallSnapshot] = [] + for index, key in enumerate(staging_keys): + snapshot = _row_to_snapshot(_select_row(rows, index), include_routes=True) + if snapshot.staging_key != key: + raise KeyError( + f"staged row identity mismatch: requested {key!r}, got {snapshot.staging_key!r}" + ) + snapshots.append(snapshot) + return snapshots + + def fetch_prefix_token_ids(self, staging_keys: list[str]) -> list[int]: + """Bulk-fetch ordered delta chain and concatenate token_ids_delta into a prefix.""" + if not staging_keys: + return [] + if len(set(staging_keys)) != len(staging_keys): + raise KeyError("prefix fetch: staging_keys contains duplicates") + rows = _call_dp( + self._dp_client, + "get_samples", + sample_ids=list(staging_keys), + partition_id=self._staging_partition, + select_fields=["token_ids_delta"], + ) + n_rows = int(rows.batch_size[0]) if rows.batch_size else 0 + if n_rows != len(staging_keys): + raise KeyError( + f"prefix fetch incomplete: requested {len(staging_keys)} keys, got {n_rows}" + ) + result: list[int] = [] + for index in range(n_rows): + row = _select_row(rows, index) + delta = row["token_ids_delta"].squeeze(0).tolist() + result.extend(int(t) for t in delta) + return result + + def fetch_for_finalization( + self, staging_keys: list[str] + ) -> list[FetchedStagedCall]: + """Fetch only digest-covered columns plus required route length metadata.""" + if not staging_keys: + return [] + if len(set(staging_keys)) != len(staging_keys): + raise KeyError("finalization staging request contains duplicate keys") + try: + rows = _call_dp( + self._dp_client, + "get_samples", + sample_ids=list(staging_keys), + partition_id=self._staging_partition, + select_fields=STAGING_FIELDS, + ) + except Exception as error: # noqa: BLE001 — protocol maps misses to KeyError + raise KeyError( + f"staged rows for {len(staging_keys)} keys could not be " + f"fetched from {self._staging_partition!r}: {error}" + ) from error + n_rows = int(rows.batch_size[0]) if len(rows.batch_size) else 0 + if n_rows != len(staging_keys): + raise KeyError( + f"staged rows missing: requested {len(staging_keys)} keys " + f"from {self._staging_partition!r}, got {n_rows} rows" + ) + fetched: list[FetchedStagedCall] = [] + for index, key in enumerate(staging_keys): + row = _select_row(rows, index) + snapshot = _row_to_snapshot(row, include_routes=False) + if snapshot.staging_key != key: + raise KeyError( + f"staged row identity mismatch: requested {key!r}, got {snapshot.staging_key!r}" + ) + fetched.append( + FetchedStagedCall( + staging_key=key, + snapshot=snapshot, + routed_len=_row_scalar_int(row, ROUTED_LEN_FIELD), + ) + ) + return fetched + + +def _select_row(rows: TensorDict, index: int) -> dict[str, torch.Tensor]: + """Slice one row out of a batched fetch, restoring single-row shapes. + + ``_row_to_snapshot`` predates batching and expects each field with a + leading batch dim of 1 (the shape a single-key ``get_samples`` returns), + so re-add it after indexing. Indexing a nested tensor yields that row's + dense component, which is exactly the jagged-row payload. + """ + row: dict[str, torch.Tensor] = {} + for field in rows.keys(): + row[str(field)] = rows.get(field)[index].unsqueeze(0) + return row + + +def _row_to_snapshot(row: Any, *, include_routes: bool) -> StagedCallSnapshot: + def _leaf(name: str) -> torch.Tensor: + value = row[name] + tensor = value[0] if value.dim() > 1 or value.numel() > 1 else value + return tensor.reshape(-1) + + def _text(name: str) -> str: + return bytes(int(value) for value in _leaf(name).tolist()).decode("utf-8") + + def _digest(name: str) -> str: + value = bytes(int(item) for item in _leaf(name).tolist()) + if len(value) != 32: + raise ValueError(f"{name} must contain exactly 32 bytes") + return value.hex() + + def _optional_digest(name: str, present_name: str) -> str | None: + return _digest(name) if bool(_leaf(present_name)[0].item()) else None + + parent_call_id = ( + _text("parent_call_id_utf8") + if bool(_leaf("parent_call_id_present")[0].item()) + else None + ) + mode_code = int(_leaf("capture_mode")[0].item()) + try: + mode = _CODE_TO_MODE[mode_code] + except KeyError as error: + raise ValueError(f"unknown capture_mode code {mode_code}") from error + extras = json.loads(_text(ROUTED_EXTRAS_METADATA_FIELD)) + routed_encoding = int(_leaf(ROUTED_EXPERTS_ENCODING_FIELD)[0].item()) + if routed_encoding != _ROUTE_ENCODING_NONE and include_routes: + try: + routed = row[ROUTED_EXPERTS_FIELD] + except KeyError as error: + raise KeyError( + "staged row metadata names routed_experts but its field is absent" + ) from error + experts = routed[0] if routed.dim() > 3 or routed.shape[0] == 1 else routed + if extras is None: + extras = {} + if not isinstance(extras, dict): + raise TypeError("staged extras metadata must decode to an object or null") + if routed_encoding == _ROUTE_ENCODING_ENVELOPE: + from nemo_rl.utils.routed_experts_codec import encode_routed_experts + + extras[ROUTED_EXPERTS_FIELD] = encode_routed_experts(experts) + elif routed_encoding == _ROUTE_ENCODING_LIST: + extras[ROUTED_EXPERTS_FIELD] = experts.tolist() + else: + raise ValueError(f"unknown routed_experts_encoding {routed_encoding}") + elif routed_encoding not in ( + _ROUTE_ENCODING_NONE, + _ROUTE_ENCODING_ENVELOPE, + _ROUTE_ENCODING_LIST, + ): + raise ValueError(f"unknown routed_experts_encoding {routed_encoding}") + + values = dict( + schema_version=int(_leaf("schema_version")[0].item()), + digest_version=int(_leaf("digest_version")[0].item()), + extras_digest_version=int(_leaf("extras_digest_version")[0].item()), + rollout_id=_text("rollout_id_utf8"), + model_call_id=_text("model_call_id_utf8"), + parent_call_id=parent_call_id, + mode=mode, + prev_len=int(_leaf("prev_len")[0].item()), + delta_len=int(_leaf("delta_len")[0].item()), + cum_len=int(_leaf("cum_len")[0].item()), + weight_version=int(_leaf("weight_version")[0].item()), + digest=_digest("digest_bytes"), + token_ids_delta=[int(t) for t in _leaf("token_ids_delta").tolist()], + token_mask_delta=[float(m) for m in _leaf("token_mask_delta").tolist()], + generation_log_probs_delta=[ + float(p) for p in _leaf("generation_logprobs_delta").tolist() + ], + extras=extras, + extras_digest=_digest("extras_digest_bytes"), + chain_hash=_optional_digest("chain_hash_bytes", "chain_hash_present"), + cumulative_hash=_optional_digest( + "cumulative_hash_bytes", "cumulative_hash_present" + ), + ) + if include_routes or routed_encoding == _ROUTE_ENCODING_NONE: + return StagedCallSnapshot(**values) + # Metadata-only deferred finalization deliberately leaves the heavy route + # fragment in TQ. The finalizer verifies its digest binding before + # publishing a route assembly plan. + return StagedCallSnapshot.model_construct(**values) + + +def _row_scalar_int(row: Any, field_name: str) -> int: + """Read one required scalar from a single-row TQ result.""" + value = row[field_name] + if not isinstance(value, torch.Tensor): + raise TypeError( + f"staging field {field_name!r} must be a tensor, got {type(value).__name__}" + ) + integer_dtypes = { + torch.uint8, + torch.int8, + torch.int16, + torch.int32, + torch.int64, + } + if value.dtype not in integer_dtypes: + raise TypeError( + f"staging field {field_name!r} must use an integer dtype, got {value.dtype}" + ) + tensor = value[0] if value.dim() > 1 or value.numel() > 1 else value + flattened = tensor.reshape(-1) + if flattened.numel() != 1: + raise ValueError( + f"staging field {field_name!r} must contain one scalar, got " + f"shape {tuple(value.shape)}" + ) + return int(flattened[0].item()) diff --git a/nemo_rl/data_plane/worker_mixin.py b/nemo_rl/data_plane/worker_mixin.py index 1125245c98a..d3c81fc8b99 100644 --- a/nemo_rl/data_plane/worker_mixin.py +++ b/nemo_rl/data_plane/worker_mixin.py @@ -27,6 +27,11 @@ from __future__ import annotations +import json +import logging +import time +from collections import Counter +from dataclasses import dataclass from typing import TYPE_CHECKING, Any, Literal, Optional import torch @@ -39,6 +44,11 @@ GLOBAL_FORWARD_PAD_SEQLEN, MICRO_BATCH_INDICES, MICRO_BATCH_LENGTHS, + ROUTE_PASSTHROUGH_FLAG, + ROUTE_PLAN_TAG, + ROUTED_EXPERTS_ENCODING_FIELD, + ROUTED_EXPERTS_FIELD, + ROUTED_EXTRAS_METADATA_FIELD, Layout, ) from nemo_rl.distributed.batched_data_dict import BatchedDataDict, SequencePackingArgs @@ -50,6 +60,13 @@ from nemo_rl.data_plane.interfaces import DataPlaneClient +@dataclass(frozen=True) +class _DeferredRouteFragment: + routes: torch.Tensor + encoding: int + extras_metadata_json: torch.Tensor + + def _broadcast_batched_data_dict( data: Optional[BatchedDataDict[Any]], *, @@ -105,7 +122,15 @@ def _broadcast_batched_data_dict( dtype = getattr(torch, dtype_str.split(".")[-1]) tensor = torch.empty(shape, dtype=dtype, device=bcast_device) out[key] = tensor - torch.distributed.broadcast(tensor, src=src, group=group) + # NCCL has no int16 ("Short") type; ship as int32 and narrow back + # (routed_experts rides TQ as int16). + if tensor.dtype == torch.int16: + wire = tensor.to(torch.int32) + torch.distributed.broadcast(wire, src=src, group=group) + tensor = wire.to(torch.int16) + out[key] = tensor + else: + torch.distributed.broadcast(tensor, src=src, group=group) # Restore non-leader tensors to the leader's source device # so downstream code sees the same layout pre-broadcast. if ( @@ -128,6 +153,7 @@ class TQWorkerMixin: """ _dp_client: Optional[DataPlaneClient] = None + _route_fallback_counts: Counter[str] = Counter() def setup_data_plane(self, cfg: DataPlaneConfig) -> None: """Connect this worker process's client to the existing TQ controller. @@ -142,6 +168,7 @@ def setup_data_plane(self, cfg: DataPlaneConfig) -> None: ) if self._dp_client is not None: return + self._route_fallback_counts = Counter() from nemo_rl.data_plane import build_data_plane_client # bootstrap=False — the driver already created the named @@ -168,6 +195,12 @@ def _get_replica_group(self) -> Optional[Any]: """ return None + def _routed_experts_dimensions(self) -> tuple[int, int]: + """Return model-owned ``(num_moe_layers, top_k)`` route dimensions.""" + raise NotImplementedError( + "the router-replay policy worker must provide route dimensions" + ) + def _pad_value_dict(self) -> dict[str, Any]: """Per-field pad value used by :func:`materialize` to detile the jagged wire format. @@ -247,6 +280,7 @@ def _fetch( pad_value_dict=pad_value_dict, pad_to_seqlen=pad_to_seqlen, ) + data = self._maybe_assemble_routed_experts(meta, data) else: data = None data = _broadcast_batched_data_dict( @@ -278,6 +312,7 @@ def _fetch( pad_value_dict=pad_value_dict, pad_to_seqlen=pad_to_seqlen, ) + data = self._maybe_assemble_routed_experts(meta, data) attach_message_log_view(data) trace_tq_fetch_payload( stage=meta.task_name or "unknown", @@ -288,6 +323,246 @@ def _fetch( data = preprocess(self, data) return data + def _fetch_route_fragments( + self, + *, + keys: list[str], + partition_id: str, + ) -> dict[str, _DeferredRouteFragment]: + """Fetch a unique key set in one request and preserve request identity.""" + if not keys: + return {} + rows = self._require_dp_client().get_samples( + sample_ids=keys, + partition_id=partition_id, + select_fields=[ + ROUTED_EXPERTS_FIELD, + ROUTED_EXPERTS_ENCODING_FIELD, + ROUTED_EXTRAS_METADATA_FIELD, + ], + ) + n_rows = int(rows.batch_size[0]) if len(rows.batch_size) else 0 + if n_rows != len(keys): + raise KeyError(f"requested {len(keys)} route rows, got {n_rows}") + route_column = rows.get(ROUTED_EXPERTS_FIELD) + encoding_column = rows.get(ROUTED_EXPERTS_ENCODING_FIELD) + metadata_column = rows.get(ROUTED_EXTRAS_METADATA_FIELD) + if route_column is None or encoding_column is None or metadata_column is None: + raise KeyError("deferred route row is missing integrity metadata") + return { + key: _DeferredRouteFragment( + routes=route_column[index], + encoding=int(encoding_column[index].reshape(-1)[0].item()), + extras_metadata_json=metadata_column[index].reshape(-1), + ) + for index, key in enumerate(keys) + } + + @staticmethod + def _verify_route_fragment_integrity( + fragment: _DeferredRouteFragment, + *, + extras_digest_version: int, + expected_extras_digest: str, + ) -> bool: + """Rebuild deferred extras and verify the receipt-bound Gym digest.""" + from nemo_gym.token_id_capture.staging.digest import ( + EXTRAS_DIGEST_VERSION, + compute_extras_digest, + ) + from nemo_rl.utils.routed_experts_codec import encode_routed_experts + + if extras_digest_version != EXTRAS_DIGEST_VERSION: + return False + try: + raw_metadata = bytes( + int(value) for value in fragment.extras_metadata_json.tolist() + ) + decoded = json.loads(raw_metadata.decode("utf-8")) + if decoded is None: + extras: dict[str, Any] = {} + elif isinstance(decoded, dict): + extras = decoded + else: + return False + if fragment.encoding == 1: + extras[ROUTED_EXPERTS_FIELD] = encode_routed_experts(fragment.routes) + elif fragment.encoding == 2: + extras[ROUTED_EXPERTS_FIELD] = fragment.routes.tolist() + else: + return False + return compute_extras_digest(extras) == expected_extras_digest + except (TypeError, ValueError): + return False + + def _route_fragments_by_row( + self, + plans: list[Any], + ) -> tuple[list[dict[str, _DeferredRouteFragment]], int, float]: + """Use one normal-path batch read; isolate error retries per rollout.""" + from nemo_rl.experience.route_plan import decode_route_plan + + decoded = [decode_route_plan(plan) for plan in plans] + partitions = {plan.staging_partition for plan in decoded} + if len(partitions) != 1: + raise RuntimeError( + f"deferred route plans use mixed staging partitions: {partitions}" + ) + partition_id = next(iter(partitions)) + keys = list( + dict.fromkeys( + span.staging_key + for plan in decoded + for span in plan.spans + if span.staged_route_len > 0 + ) + ) + fetch_start = time.perf_counter() + try: + fragments = self._fetch_route_fragments( + keys=keys, + partition_id=partition_id, + ) + except Exception as batch_error: # noqa: BLE001 - isolate fallback by rollout + logging.getLogger(__name__).warning( + "deferred route batch fetch failed; isolating by rollout: %s", + batch_error, + ) + per_row: list[dict[str, _DeferredRouteFragment]] = [] + for plan in decoded: + row_keys = list( + dict.fromkeys( + span.staging_key + for span in plan.spans + if span.staged_route_len > 0 + ) + ) + try: + per_row.append( + self._fetch_route_fragments( + keys=row_keys, + partition_id=partition_id, + ) + ) + except Exception: # noqa: BLE001 - this rollout becomes sentinel + per_row.append({}) + return ( + per_row, + len(keys), + (time.perf_counter() - fetch_start) * 1000.0, + ) + return ( + [fragments for _ in decoded], + len(keys), + (time.perf_counter() - fetch_start) * 1000.0, + ) + + def _maybe_assemble_routed_experts( + self, + meta: "KVBatchMeta", + data: BatchedDataDict[Any], + ) -> BatchedDataDict[Any]: + """Materialize deferred routes at the policy worker consumption boundary.""" + if not (meta.extra_info or {}).get(ROUTE_PASSTHROUGH_FLAG): + return data + + from nemo_rl.experience.route_plan import ( + classify_route_span, + decode_route_plan, + ) + from nemo_rl.models.generation.interfaces import ( + ROUTED_EXPERTS_MISSING_ROUTE_SENTINEL, + ) + + tags = meta.tags or [] + if len(tags) != len(meta.sample_ids): + raise RuntimeError( + "deferred route tags must align with sample_ids: " + f"{len(tags)} tags for {len(meta.sample_ids)} rows" + ) + encoded_plans = [] + for index, tag in enumerate(tags): + if ROUTE_PLAN_TAG not in tag: + raise RuntimeError( + f"deferred route plan missing for row {meta.sample_ids[index]!r}" + ) + encoded_plans.append(tag[ROUTE_PLAN_TAG]) + plans = [decode_route_plan(plan) for plan in encoded_plans] + fragments_by_row, _, _ = self._route_fragments_by_row(encoded_plans) + + num_moe_layers, top_k = self._routed_experts_dimensions() + input_ids = data["input_ids"] + input_lengths = data["input_lengths"].reshape(-1) + routed = torch.full( + ( + len(meta.sample_ids), + int(input_ids.shape[1]), + num_moe_layers, + top_k, + ), + ROUTED_EXPERTS_MISSING_ROUTE_SENTINEL, + dtype=torch.int16, + ) + request_fallbacks: Counter[str] = Counter() + for row_index, (plan, fragments) in enumerate(zip(plans, fragments_by_row)): + reason: Optional[str] = None + canonical_len = int(input_lengths[row_index].item()) + if canonical_len != plan.expected_token_length: + reason = "canonical_length_mismatch" + position = 0 + if reason is None: + for span in plan.spans: + contribution = span.carry_len + span.generation_len + mode = classify_route_span(span) + if mode != "sentinel": + fragment = fragments.get(span.staging_key) + if fragment is None: + reason = "missing_fragment" + break + if not self._verify_route_fragment_integrity( + fragment, + extras_digest_version=span.extras_digest_version, + expected_extras_digest=span.extras_digest, + ): + reason = "fragment_integrity" + break + routes = fragment.routes + if routes.dim() != 3: + reason = "fragment_rank" + break + if int(routes.shape[0]) != span.staged_route_len: + reason = "fragment_length" + break + if tuple(routes.shape[1:]) != (num_moe_layers, top_k): + reason = "fragment_model_shape" + break + if mode == "full": + routed[row_index, position : position + contribution] = ( + routes.to(torch.int16) + ) + else: + tail_start = position + span.carry_len + routed[row_index, tail_start : position + contribution] = ( + routes[-span.generation_len :].to(torch.int16) + ) + position += contribution + if reason is None and plan.spans and position != canonical_len: + reason = "assembled_length_mismatch" + if reason is not None: + routed[row_index].fill_(ROUTED_EXPERTS_MISSING_ROUTE_SENTINEL) + request_fallbacks[reason] += 1 + + self._route_fallback_counts.update(request_fallbacks) + if request_fallbacks: + logging.getLogger(__name__).warning( + "deferred route fallback for %d/%d rollouts: %s", + sum(request_fallbacks.values()), + len(plans), + dict(request_fallbacks), + ) + data[ROUTED_EXPERTS_FIELD] = routed + return data + def _apply_packing_prep(self, data: BatchedDataDict[Any]) -> BatchedDataDict[Any]: """Re-derive ``micro_batch_indices`` / ``micro_batch_lengths`` on the local slice. diff --git a/nemo_rl/distributed/virtual_cluster.py b/nemo_rl/distributed/virtual_cluster.py index 96250b71802..15b32733a31 100644 --- a/nemo_rl/distributed/virtual_cluster.py +++ b/nemo_rl/distributed/virtual_cluster.py @@ -73,6 +73,10 @@ class PY_EXECUTABLES: # Use NeMo-Gym dependencies NEMO_GYM = f"uv run --locked --extra nemo_gym --directory {git_root}" + # vLLM worker hosting Gym's token capture (token_capture.enabled): the + # worker imports nemo_gym's dependency-free capture core + vLLM adapter. + VLLM_GYM = f"uv run --locked --extra vllm --extra nemo_gym --directory {git_root}" + # Use NeMo-RL direct dependencies and SGLang. SGLANG = f"uv run --locked --extra sglang --directory {git_root}" diff --git a/nemo_rl/environments/nemo_gym.py b/nemo_rl/environments/nemo_gym.py index 3eb9adbf700..306036a1787 100644 --- a/nemo_rl/environments/nemo_gym.py +++ b/nemo_rl/environments/nemo_gym.py @@ -11,6 +11,7 @@ # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License. +import asyncio import math import os import subprocess @@ -21,6 +22,7 @@ from pathlib import Path from typing import Any, Dict, List, NotRequired, Optional, TypedDict +import aiohttp import ray import torch from PIL import Image @@ -126,6 +128,25 @@ class NemoGymConfig(TypedDict): tokenizer_config: NotRequired[ Optional[TokenizerConfig] ] # For processor reconstruction inside the actor + # Ledger-authoritative token capture (token_capture.enabled): the dumped + # TokenCaptureConfig. Turns on external staging in Gym's policy model + # server, switches run_rollouts to receipt mode, and assembles receipts + # from the manifest control route. None/absent = legacy token-echo path. + token_capture: NotRequired[Dict[str, Any] | None] + + +# Gym control-plane server name (the model server hosting the ledger) and the +# opaque run-body key rollout ids ride on (Gym's ROLLOUT_ID_KEY_NAME): the +# agent derives the id from the run body and stamps /ng-rollout/ on every +# model call, so the TQ sample id IS the capture key end to end. +_POLICY_SERVER_NAME = "policy_model" +_NG_ROLLOUT_ID_BODY_KEY = "_ng_rollout_id" +_TOKEN_CAPTURE_CONTROL_PREFIX = "/training-token-capture/control" +_TOKEN_CAPTURE_CONTROL_ENV = "NEMO_GYM_TOKEN_CAPTURE_CONTROL_TOKEN" +# Mirrors nemo_gym.token_id_capture.sink.UNCOMMITTED_CALL_REASON: the poison +# reason for an admitted call that finished without worker coordinates (no +# completion was ever served for it). +_UNCOMMITTED_CALL_REASON = "request_finished_without_staged_coordinates" def _detect_invalid_tool_call_and_malformed_thinking( @@ -149,11 +170,10 @@ def _detect_invalid_tool_call_and_malformed_thinking( ) thinking_tags = thinking_tags or DEFAULT_THINKING_TAGS - is_output_message = ( - "content" in output_item_dict - and len(output_item_dict["content"]) > 0 - and "text" in output_item_dict["content"][0] - ) + # A tool-call-only assistant item carries content: None — not a final + # content message. + item_content = output_item_dict.get("content") or [] + is_output_message = len(item_content) > 0 and "text" in item_content[0] # NeMo-Gym only attaches generation_token_ids to the last output item of a # model call (see vllm_model/app.py postprocess_chat_response). So this item # is guaranteed to be the final thing the model produced for this turn. @@ -452,6 +472,69 @@ def _spinup(self) -> None: "port": self.head_server_port, } + self.rollout_max_attempts_to_avoid_lp_nan = initial_global_config_dict.pop( + "rollout_max_attempts_to_avoid_lp_nan", 1 + ) + + assert self.rollout_max_attempts_to_avoid_lp_nan >= 1, ( + "`rollout_max_attempts_to_avoid_lp_nan` must be at least 1" + ) + + # Ledger-authoritative token capture: enable external staging in the + # policy model server (via the policy_model global-config override + # block the env yamls already use) and disable the legacy token echo. + # Receipt mode is incompatible with re-dispatching a batch under the + # same rollout ids — the retry's calls would resolve against the + # first attempt's ledger rows — so the NaN retry must be exactly 1. + token_capture = self.cfg.get("token_capture") or None + self._token_capture_enabled = bool( + token_capture and token_capture.get("enabled") + ) + self._server_client = None + self._control_headers: Dict[str, str] = {} + self._control_timeout_s = 60.0 + if self._token_capture_enabled: + if self.rollout_max_attempts_to_avoid_lp_nan != 1: + raise ValueError( + "token_capture.enabled requires " + "rollout_max_attempts_to_avoid_lp_nan == 1: a NaN retry " + "would resolve against the first attempt's ledger rows" + ) + policy_overrides = ( + initial_global_config_dict.setdefault("policy_model", {}) + .setdefault("responses_api_models", {}) + .setdefault("vllm_model", {}) + ) + policy_overrides["return_token_id_information"] = False + capture_dir = os.path.abspath(token_capture["capture_dir"]) + initial_global_config_dict["token_id_capture"] = { + "enabled": True, + "all_agents": True, + "rebuild_response": False, + "dir": capture_dir, + # The lineage store is process-shared and doubles as the + # per-rollout capture ledger. Every uvicorn worker builds its + # own handle over the same root, so token-in ancestry remains + # valid when consecutive calls land on different workers. + "lineage_store": ("nemo_gym.token_id_capture.lineage:FileLineageStore"), + "lineage_store_kwargs": {"root": os.path.join(capture_dir, "lineage")}, + "external_staging": True, + "control_auth_token_env": _TOKEN_CAPTURE_CONTROL_ENV, + } + # Gym resolves the credential inside each serving process. Keep + # only the variable name in serialized config and inherit the + # secret through the server process environment. + os.environ[_TOKEN_CAPTURE_CONTROL_ENV] = token_capture["control_auth_token"] + self._control_headers = { + "Authorization": f"Bearer {token_capture['control_auth_token']}" + } + # S5 chaos finding: Gym's shared request() retries connection + # errors indefinitely; a dead control plane must surface as a + # failed manifest fetch (placeholder row), not a silent stall. + self._control_timeout_s = float( + token_capture.get("control_timeout_s") or 60.0 + ) + self.rh = RunHelper() self.rh.start( global_config_dict_parser_config=GlobalConfigDictParserConfig( @@ -469,6 +552,43 @@ def _spinup(self) -> None: ) self.rch = RolloutCollectionHelper() + # ── ledger control plane (token-capture mode) ─────────────────────────── + + def _control_client(self): + """Gym ServerClient resolving servers by name from the head server.""" + if self._server_client is None: + from nemo_gym.server_utils import ServerClient + + self._server_client = ServerClient.load_from_global_config( + self.head_server_config + ) + return self._server_client + + async def _control(self, method: str, path: str, **kwargs: Any) -> dict: + headers = {**kwargs.pop("headers", {}), **self._control_headers} + try: + response = await asyncio.wait_for( + self._control_client().request( + server_name=_POLICY_SERVER_NAME, + url_path=path, + method=method, + headers=headers, + **kwargs, + ), + timeout=self._control_timeout_s, + ) + except asyncio.TimeoutError: + raise RuntimeError( + f"ledger control call {method} {path} exceeded " + f"{self._control_timeout_s}s (control plane unreachable or stalled)" + ) from None + if response.status != 200: + raise RuntimeError( + f"ledger control call {method} {path} failed: " + f"HTTP {response.status} {await response.text()}" + ) + return await response.json() + async def run_rollouts( self, nemo_gym_examples: list[dict], @@ -502,6 +622,15 @@ async def run_rollouts( with timer.time(label=f"{timer_prefix}/await_results"): try: nemo_gym_row, nemo_gym_result = await task + except aiohttp.ClientResponseError as e: + # aiohttp exceptions carry CIMultiDictProxy headers that + # Ray cannot pickle across the actor boundary, masking the + # real error with a TypeError; re-raise as a plain, + # picklable RuntimeError. + raise RuntimeError( + f"NemoGym rollout HTTP error: {e.status} " + f"{e.message} url={e.request_info.real_url}" + ) from None except Exception as error: if hasattr(error, "response_content"): print( @@ -512,15 +641,23 @@ async def run_rollouts( raise with timer.time(label=f"{timer_prefix}/postprocess_results"): - nemo_rl_result = self._postprocess_nemo_gym_to_nemo_rl_result( - nemo_gym_row, - nemo_gym_result, - tokenizer, - include_initial_multimodal_data=not deduplicate_multimodal_data, - ) - if _has_nan_generation_logprobs(nemo_rl_result): - raise RuntimeError("Generation logprobs contain NaN") - + if self._token_capture_enabled: + # Receipt mode: fetch the ledger manifest and assemble the + # receipt locally; token-free result. The canonical row is + # rebuilt by the finalizer, so no message_log walk (and no + # NaN check) applies here. + nemo_rl_result = await self._postprocess_receipt_mode( + nemo_gym_row, nemo_gym_result + ) + else: + nemo_rl_result = self._postprocess_nemo_gym_to_nemo_rl_result( + nemo_gym_row, + nemo_gym_result, + tokenizer, + include_initial_multimodal_data=not deduplicate_multimodal_data, + ) + if _has_nan_generation_logprobs(nemo_rl_result): + raise RuntimeError("Generation logprobs contain NaN") num_results += 1 timing_metrics = None if num_results == len(nemo_gym_examples): @@ -551,6 +688,170 @@ async def run_rollouts( yield nemo_gym_row["_rowidx"], nemo_rl_result, timing_metrics + async def _postprocess_receipt_mode( + self, nemo_gym_row: dict, nemo_gym_result: dict + ) -> dict: + """Fetch the ledger manifest and assemble the receipt locally. + + The legacy token walk (and its contiguity assert) does not run: the + capture ledger owns lineage, output items carry no token arrays, and + the canonical row is rebuilt by the finalizer from staged deltas. The + Ray return carries only the receipt (~100 B/call) beside the + agent-level result. + """ + assert isinstance(nemo_gym_result, dict), ( + f"Hit a non-successful response when querying NeMo Gym for rollouts: {nemo_gym_result}" + ) + rollout_id = nemo_gym_row[_NG_ROLLOUT_ID_BODY_KEY] + terminal_logical_request_id = nemo_gym_result.get("terminal_logical_request_id") + if not ( + isinstance(terminal_logical_request_id, str) and terminal_logical_request_id + ): + # A harness that reports no terminal id still gets its manifest + # fetched; receipt assembly attributes the terminal from the + # scored response, falling back to heuristic selection. + terminal_logical_request_id = None + scored_response = nemo_gym_result.get("response") + if not isinstance(scored_response, dict): + scored_response = None + receipt = None + try: + manifest = await self._control( + "GET", + f"{_TOKEN_CAPTURE_CONTROL_PREFIX}" + f"/rollouts/{rollout_id}/manifest", + ) + receipt = self._assemble_receipt( + rollout_id, + manifest, + terminal_logical_request_id=terminal_logical_request_id, + scored_response=scored_response, + reward=float(nemo_gym_result.get("reward") or 0.0), + ) + except (RuntimeError, OSError) as error: + # An unfetchable manifest finalizes as a placeholder row. + print(f"manifest({rollout_id}) fetch failed: {error}", flush=True) + return { + "message_log": [], + "input_message_log": [], + "full_result": nemo_gym_result, + "rollout_id": rollout_id, + "receipt": receipt, + } + + @staticmethod + def _assemble_receipt( + rollout_id: str, + manifest: dict, + *, + terminal_logical_request_id: Optional[str], + scored_response: Optional[dict] = None, + reward: float, + ) -> dict: + """Build the token-free RolloutReceipt payload from a ledger manifest. + + Terminal selection is staged, fail-closed at every stage: + + 1. Witness attribution (``resolve_terminal``): the declared response + id (``terminal_logical_request_id``), the scored response's + envelope id, and its content fingerprints each independently name + a manifest row through ``CallRecord.response_id`` and the recorded + fingerprints. Agreeing witnesses attribute; a declared id that + matches no row masks and never falls back; disagreeing witnesses + attribute nothing. + 2. Heuristic fallback (``select_terminal_call``): with no witness, + the manifest's explicit parent links infer the terminal + (fail-closed: ambiguity masks). + + The receipt records the resolving stage in ``terminal_selection`` + (``declared``/``response_id``/``content``/``heuristic`` — failed + selections stamp the last stage attempted) and the witness trail in + ``terminal_attribution_reason``. Retry duplicates are dead-branch + rows: they stay in the manifest (their staged rows are fetched, + verified, and cleaned) but never join the terminal chain — + ``verify_and_linearize`` tolerates rows unreferenced by the terminal + chain. + + Poisoning is fail-closed with one carve-out. A failure row whose + reason is ``request_finished_without_staged_coordinates`` is a call + that never returned a completion (the ledger commit precedes the + response leaving the server) and can never be a lineage parent (an + uncommitted call has no row to resolve against) — e.g. the doomed + final call of a rollout that exhausted the model's context window. + Such rows are structurally off-chain and do not poison; if the + *terminal* request itself died this way, the missing-terminal-row + check below still masks the rollout. Every other failure reason + (``worker_capture_failed``, ``invalid_worker_commit_coordinates``) + marks a call whose completion WAS served — a hole in the chain — + and poisons. + """ + # Deferred: nemo_gym is an optional extra absent in non-gym runs. + from nemo_gym.token_id_capture.staging.attribution import resolve_terminal + from nemo_gym.token_id_capture.staging.records import CallRecord + from nemo_gym.token_id_capture.staging.terminal import select_terminal_call + + records = [dict(record) for record in manifest.get("records") or []] + failures = list(manifest.get("failures") or []) + deduped: dict[str, dict] = {} + for record in records: + deduped.setdefault(str(record.get("model_call_id")), record) + terminal_record = None + selection_reason = None + attribution_reason = None + terminal_selection = "heuristic" + parsed_records = None + try: + parsed_records = [ + CallRecord.model_validate(record) for record in deduped.values() + ] + except ValueError: + selection_reason = "invalid_manifest_row" + if parsed_records is not None: + attribution = resolve_terminal( + parsed_records, + scored_response, + declared_response_id=terminal_logical_request_id, + ) + attribution_reason = attribution.reason or None + if attribution.attributed: + terminal_selection = attribution.method + terminal_record = deduped[attribution.model_call_id] + elif terminal_logical_request_id is not None: + # A declaration is authoritative: a declared id the ledger + # cannot confirm masks and never falls back to the heuristic. + terminal_selection = "declared" + selection_reason = None + else: + selection = select_terminal_call(parsed_records) + if selection.terminal_model_call_id is not None: + terminal_record = deduped[selection.terminal_model_call_id] + else: + selection_reason = selection.reason + poisoning_failures = [ + failure + for failure in failures + if str(failure.get("reason") or "") != _UNCOMMITTED_CALL_REASON + ] + failure_reason = None + if poisoning_failures: + failure_reason = str(poisoning_failures[0].get("reason") or "capture_failed") + elif terminal_record is None: + failure_reason = selection_reason or "missing_terminal_row" + return { + "rollout_id": rollout_id, + "reward": reward, + "terminal_model_call_id": ( + terminal_record.get("model_call_id") + if terminal_record is not None + else None + ), + "manifest": list(deduped.values()), + "capture_poisoned": failure_reason is not None, + "failure_reason": failure_reason, + "terminal_selection": terminal_selection, + "terminal_attribution_reason": attribution_reason, + } + def _postprocess_nemo_gym_to_nemo_rl_result( self, nemo_gym_row: dict, @@ -680,12 +981,22 @@ def _postprocess_nemo_gym_to_nemo_rl_result( f"{expected_tokens}." ) elif self.cfg.get("require_routed_experts", False): - raise ValueError( - "policy.router_replay.enabled=true requires NeMo Gym output " - "items to include routed_experts, but the field was missing. " - "Make sure the Gym repo includes routed_experts propagation " - "and the NeMo-RL vLLM OpenAI-compatible server is configured " - "with enable_return_routed_experts." + # Routes can be legitimately unrecoverable on the echo path + # (e.g. a context-overflow rollout whose only persisted + # completion record is the gate's synthetic empty response). + # Leave the message routeless: backfill_missing_routed_experts + # sentinel-fills it at flatten and Megatron self-routes those + # tokens; total absence across a batch still fails loudly at + # the rollout actor's ROUTED_EXPERTS_FIELD guard. + print( + "router_replay: trainable Gym output item without " + "routed_experts — falling back to the missing-route " + "sentinel for this message " + f"[item_idx={len(nemo_rl_message_log) // 2}, " + f"item_type={output_item_dict.get('type')!r}, " + f"n_prompt={len(prompt_token_ids)}, " + f"n_gen={len(generation_token_ids)}]", + flush=True, ) # The next prompt prefill supplies the real route for the previous @@ -928,6 +1239,7 @@ def spinup_nemo_gym_actor( enable_router_replay: bool, routed_experts_dtype: str, use_fastokens: bool, + token_capture: Optional[dict[str, Any]] = None, ) -> Any: """Spin up the NeMo-Gym actor against the given generation server URLs. @@ -977,6 +1289,7 @@ def spinup_nemo_gym_actor( routed_experts_dtype=routed_experts_dtype, use_fastokens=use_fastokens, initial_global_config_dict=nemo_gym_dict, + token_capture=token_capture, ) nemo_gym_py_exec = get_actor_python_env("nemo_rl.environments.nemo_gym.NemoGym") diff --git a/nemo_rl/experience/blackbox_finalizer.py b/nemo_rl/experience/blackbox_finalizer.py new file mode 100644 index 00000000000..3c74530c23d --- /dev/null +++ b/nemo_rl/experience/blackbox_finalizer.py @@ -0,0 +1,849 @@ +# Copyright (c) 2026, NVIDIA CORPORATION. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +"""Blackbox finalization: token-free receipts + staged deltas -> canonical rows. + +Orchestration only (docs/design-docs/token-capture-ledger.md): +per rollout, fetch the staged rows the receipt manifest names through the +``TokenSource``, re-verify them (digest recomputation over fetched values, +shape/mask/finite-logprob checks, length chaining, weight-version tag +equality), then delegate semantics to Gym's pure ``linearize`` +(``main_chain_only`` + ``terminal_hint``). Any rejection becomes a masked +placeholder row — the group always publishes exactly N rows so GRPO group +shape survives; validity folds into ``sample_mask`` (no new train field) and +placeholders copy ``prompt_ids_for_adv`` from a valid sibling so per-prompt +baselines stay well-formed. + +In deferred-route mode the finalizer reads only small columns, publishes a +strict route plan beside each canonical row, and leaves staged route fragments +live until policy consumption completes. The direct-route rollback path keeps +the prior finalizer-side materialization behavior. +""" + +from __future__ import annotations + +import math +import time +from dataclasses import dataclass, field +from typing import Any, Optional + +import torch + +from nemo_rl.data_plane import KVBatchMeta +from nemo_rl.data_plane.schema import ROUTE_PLAN_TAG +from nemo_rl.data_plane.tq_token_sink import TQTokenSink, TQTokenSource +from nemo_rl.experience.route_plan import ( + ROUTE_PLAN_SCHEMA_VERSION, + RouteAssemblyPlan, + RouteSpan, + classify_route_span, + encode_route_plan, + encoded_route_plan_size_bytes, +) +from nemo_rl.experience.payload import pack_payload +from nemo_rl.experience.row_dump import maybe_dump_train_rows + +# Keep the finalizer importable in its CPU-only actor without importing the +# generation package (which eagerly loads backend dependencies). This value is +# the shared router-replay missing-route wire sentinel. +_ROUTED_EXPERTS_SENTINEL = -1 + + +@dataclass(frozen=True) +class FinalizedRollout: + """One rollout's canonical row, or its rejection.""" + + rollout_id: str + valid: bool + rejection_reason: Optional[str] + token_ids: list[int] + token_mask: list[float] + logprobs: list[float] + prompt_len: int + reward: float + staging_keys: list[str] + min_wv: Optional[int] = None + max_wv: Optional[int] = None + # Router replay (R3): [len(token_ids)][num_moe_layers][topk] from the + # rebuilt chain; None when the rollout staged no extras. + routed_experts: Optional[list] = None + route_plan: Optional[RouteAssemblyPlan] = None + + +@dataclass +class FinalizedGroup: + """What ``finalize_group`` hands back for ``commit_finalized``.""" + + meta: Optional[KVBatchMeta] + group_min_wv: int + group_max_wv: int + staging_keys: list[str] + metrics: dict[str, float] = field(default_factory=dict) + # True when min_valid_fraction_per_group rejected the whole group; the + # caller aborts the slot instead of committing it. + dropped: bool = False + + +def _linearize_metadata_only(receipt: Any, snapshots: list[Any]) -> Any: + """Linearize a verified chain while leaving digest-bound route bytes in TQ.""" + from nemo_gym.token_id_capture.staging.rebuild import ( + LinearizedRow, + RebuildError, + WeightVersionSpan, + ) + + records = {record.model_call_id: record for record in receipt.manifest} + staged = {snapshot.model_call_id: snapshot for snapshot in snapshots} + for model_call_id, record in records.items(): + if record.parent_call_id is not None: + parent = records.get(record.parent_call_id) + if parent is None: + raise RebuildError( + "missing_parent", + f"call {model_call_id} names absent parent {record.parent_call_id}", + ) + if parent.cum_len != record.prev_len: + raise RebuildError( + "parent_length_mismatch", + f"call {model_call_id} starts at {record.prev_len}, parent ends at {parent.cum_len}", + ) + visited: set[str] = set() + cursor = record + while cursor is not None: + if cursor.model_call_id in visited: + raise RebuildError( + "lineage_cycle", f"cycle reaches call {cursor.model_call_id}" + ) + visited.add(cursor.model_call_id) + cursor = ( + records.get(cursor.parent_call_id) + if cursor.parent_call_id is not None + else None + ) + + terminal_id = receipt.terminal_model_call_id + if terminal_id is None or terminal_id not in records: + raise RebuildError( + "missing_terminal", "successful receipt has no terminal call" + ) + chain = [] + cursor = records[terminal_id] + while cursor is not None: + chain.append(cursor) + cursor = ( + records.get(cursor.parent_call_id) + if cursor.parent_call_id is not None + else None + ) + chain.reverse() + + token_ids: list[int] = [] + token_mask: list[float] = [] + logprobs: list[float] = [] + model_call_ids: list[str] = [] + weight_versions: list[int] = [] + weight_version_spans = [] + link_spans: list[tuple[str, int, int]] = [] + prompt_len = 0 + for index, record in enumerate(chain): + snapshot = staged[record.model_call_id] + boundary = 0 + for mask in snapshot.token_mask_delta: + if mask != 0.0: + break + boundary += 1 + if boundary == len(snapshot.token_mask_delta) or any( + mask != 1.0 for mask in snapshot.token_mask_delta[boundary:] + ): + raise RebuildError( + "invalid_mask_order", + f"call {record.model_call_id} mask is not carry-then-generation", + ) + start = len(token_ids) + token_ids.extend(snapshot.token_ids_delta) + token_mask.extend(snapshot.token_mask_delta) + logprobs.extend(snapshot.generation_log_probs_delta) + end = len(token_ids) + if index == 0: + prompt_len = boundary + model_call_ids.append(record.model_call_id) + weight_versions.append(record.weight_version) + weight_version_spans.append( + WeightVersionSpan( + model_call_id=record.model_call_id, + start=start, + end=end, + weight_version=record.weight_version, + ) + ) + link_spans.append((record.model_call_id, boundary, record.delta_len - boundary)) + return LinearizedRow( + rollout_id=receipt.rollout_id, + token_ids=token_ids, + token_mask=token_mask, + logprobs=logprobs, + model_call_ids=model_call_ids, + prompt_len=prompt_len, + weight_versions=weight_versions, + weight_version_spans=weight_version_spans, + link_spans=link_spans, + ) + + +class BlackboxFinalizer: + """Receipts -> verified rows -> N-row publish, off the generation hot path.""" + + def __init__( + self, + dp_client: Any, + *, + partition_id: str, + staging_partition: str, + pad_token_id: int, + mixed_weight_version_policy: str, + min_valid_fraction_per_group: Optional[float], + router_replay_enabled: bool = False, + defer_routed_experts_to_policy: bool = False, + ) -> None: + self._dp_client = dp_client + self._partition_id = partition_id + self._pad_token_id = int(pad_token_id) + self._mixed_weight_version_policy = mixed_weight_version_policy + self._min_valid_fraction = min_valid_fraction_per_group + self._router_replay_enabled = router_replay_enabled + self._defer_routed_experts_to_policy = defer_routed_experts_to_policy + if self._defer_routed_experts_to_policy and not self._router_replay_enabled: + raise ValueError( + "defer_routed_experts_to_policy requires router replay to be enabled" + ) + self._staging_partition = staging_partition + # (num_moe_layers, topk), learned from the first rebuilt row that + # carries routes; placeholder-only groups need it to shape their + # sentinel tensors consistently with the model. + self._routed_dims: Optional[tuple[int, int]] = None + self._source = TQTokenSource(dp_client, staging_partition=staging_partition) + # The sink's clear() is the staging-partition delete; no staging + # writes happen here. + self._staging = TQTokenSink(dp_client, staging_partition=staging_partition) + + # ── per rollout ───────────────────────────────────────────────────────── + + def finalize_rollout( + self, rollout_id: str, receipt: Optional[dict[str, Any]], *, reward: float + ) -> FinalizedRollout: + """Verify one receipt against its staged rows and linearize the main chain. + + Never raises for rollout-level problems: every rejection returns an + invalid row whose reason feeds the metrics; the group publisher + substitutes a placeholder. + """ + # Deferred: nemo_gym is an optional extra absent in non-gym runs. + from nemo_gym.token_id_capture.staging.digest import compute_staging_digest + from nemo_gym.token_id_capture.staging.rebuild import ( + ReceiptVerificationError, + RebuildError, + verify_and_linearize, + ) + from nemo_gym.token_id_capture.staging.records import RolloutReceipt + + def rejected(reason: str, staging_keys: list[str]) -> FinalizedRollout: + return FinalizedRollout( + rollout_id=rollout_id, + valid=False, + rejection_reason=reason, + token_ids=[], + token_mask=[], + logprobs=[], + prompt_len=0, + reward=reward, + staging_keys=staging_keys, + ) + + if receipt is None: + return rejected("missing_receipt", []) + try: + parsed = RolloutReceipt.model_validate(receipt) + except ValueError as error: + return rejected(f"invalid_receipt:{error}", []) + staging_keys = [record.staging_key for record in parsed.manifest] + if parsed.rollout_id != rollout_id: + return rejected(f"identity_mismatch:{parsed.rollout_id}", staging_keys) + if parsed.failure_reason is not None: + return rejected(f"rollout_failed:{parsed.failure_reason}", staging_keys) + if parsed.capture_poisoned: + return rejected("capture_poisoned", staging_keys) + if not parsed.manifest: + return rejected("empty_manifest", staging_keys) + if len(set(staging_keys)) != len(staging_keys): + return rejected( + "duplicate_staging_key", + list(dict.fromkeys(staging_keys)), + ) + records_by_call = {record.model_call_id: record for record in parsed.manifest} + if len(records_by_call) != len(parsed.manifest): + return rejected("duplicate_manifest_call_id", staging_keys) + + try: + fetched = ( + self._source.fetch_for_finalization(staging_keys) + if self._defer_routed_experts_to_policy + else None + ) + snapshots = ( + [item.snapshot for item in fetched] + if fetched is not None + else self._source.fetch(staging_keys) + ) + except KeyError as error: + return rejected(f"missing_staging_row:{error}", staging_keys) + except (TypeError, ValueError) as error: + return rejected(f"invalid_staging_row:{error}", staging_keys) + fetched_by_call = {} + if fetched is not None: + for record, item in zip(parsed.manifest, fetched): + if item.staging_key != record.staging_key: + return rejected( + f"staging_key_mismatch:{record.model_call_id}", staging_keys + ) + if item.snapshot.model_call_id != record.model_call_id: + return rejected( + f"call_id_mismatch:{record.model_call_id}", staging_keys + ) + fetched_by_call[record.model_call_id] = item + if len(fetched_by_call) != len(fetched): + return rejected("duplicate_fetched_call_id", staging_keys) + + for record, snapshot in zip(parsed.manifest, snapshots): + if not ( + len(snapshot.token_ids_delta) + == len(snapshot.token_mask_delta) + == len(snapshot.generation_log_probs_delta) + ): + return rejected( + f"misaligned_delta:{record.model_call_id}", staging_keys + ) + if any(m not in (0.0, 1.0) for m in snapshot.token_mask_delta): + return rejected( + f"invalid_token_mask:{record.model_call_id}", staging_keys + ) + if any(not math.isfinite(p) for p in snapshot.generation_log_probs_delta): + return rejected( + f"non_finite_logprob:{record.model_call_id}", staging_keys + ) + if record.delta_len != len(snapshot.token_ids_delta) or ( + snapshot.prev_len + record.delta_len != record.cum_len + ): + return rejected(f"length_mismatch:{record.model_call_id}", staging_keys) + compared_fields = ( + "schema_version", + "digest_version", + "extras_digest_version", + "rollout_id", + "model_call_id", + "parent_call_id", + "mode", + "prev_len", + "delta_len", + "cum_len", + "weight_version", + "digest", + "extras_digest", + "chain_hash", + "cumulative_hash", + ) + mismatch = next( + ( + field_name + for field_name in compared_fields + if getattr(snapshot, field_name) + != ( + rollout_id + if field_name == "rollout_id" + else getattr(record, field_name) + ) + ), + None, + ) + if mismatch is not None: + return rejected( + f"{mismatch}_mismatch:{record.model_call_id}", staging_keys + ) + try: + digest = compute_staging_digest( + schema_version=snapshot.schema_version, + digest_version=snapshot.digest_version, + extras_digest_version=snapshot.extras_digest_version, + rollout_id=rollout_id, + model_call_id=record.model_call_id, + parent_call_id=record.parent_call_id, + mode=record.mode, + prev_len=snapshot.prev_len, + delta_len=snapshot.delta_len, + cum_len=snapshot.cum_len, + weight_version=snapshot.weight_version, + token_ids_delta=snapshot.token_ids_delta, + token_mask_delta=snapshot.token_mask_delta, + generation_log_probs_delta=(snapshot.generation_log_probs_delta), + extras_digest=snapshot.extras_digest, + chain_hash=snapshot.chain_hash, + cumulative_hash=snapshot.cumulative_hash, + ) + except (TypeError, ValueError, OverflowError) as error: + return rejected( + f"invalid_digest_input:{record.model_call_id}:{error}", + staging_keys, + ) + if digest != record.digest: + return rejected(f"digest_mismatch:{record.model_call_id}", staging_keys) + + try: + row = ( + _linearize_metadata_only(parsed, snapshots) + if self._defer_routed_experts_to_policy + else verify_and_linearize(parsed, snapshots) + ) + except ( + KeyError, + ValueError, + ReceiptVerificationError, + RebuildError, + NotImplementedError, + ) as error: + return rejected(f"rebuild_failed:{error}", staging_keys) + weight_versions = [record.weight_version for record in parsed.manifest] + min_wv, max_wv = min(weight_versions), max(weight_versions) + if self._mixed_weight_version_policy == "reject" and min_wv != max_wv: + return rejected(f"mixed_weight_versions:{min_wv}..{max_wv}", staging_keys) + + route_plan = None + if self._router_replay_enabled and self._defer_routed_experts_to_policy: + link_spans = row.link_spans + if link_spans is None: + return rejected("missing_link_spans", staging_keys) + route_spans: list[RouteSpan] = [] + seen_span_call_ids: set[str] = set() + for call_id, carry_len, generation_len in link_spans: + if call_id in seen_span_call_ids: + return rejected(f"duplicate_route_span:{call_id}", staging_keys) + seen_span_call_ids.add(call_id) + record = records_by_call.get(call_id) + item = fetched_by_call.get(call_id) + if record is None or item is None: + return rejected(f"route_span_identity:{call_id}", staging_keys) + if item.routed_len not in (0, record.delta_len): + return rejected(f"routed_len_mismatch:{call_id}", staging_keys) + if generation_len < 0 or generation_len > record.delta_len: + return rejected( + f"route_generation_span_mismatch:{call_id}", staging_keys + ) + if carry_len < 0: + return rejected( + f"route_carry_span_mismatch:{call_id}", staging_keys + ) + span = RouteSpan( + staging_key=record.staging_key, + carry_len=int(carry_len), + generation_len=int(generation_len), + staged_route_len=item.routed_len, + extras_digest_version=record.extras_digest_version, + extras_digest=record.extras_digest, + ) + classify_route_span(span) + route_spans.append(span) + if sum(span.carry_len + span.generation_len for span in route_spans) != len( + row.token_ids + ): + return rejected("route_span_length_mismatch", staging_keys) + route_plan = RouteAssemblyPlan( + schema_version=ROUTE_PLAN_SCHEMA_VERSION, + staging_partition=self._staging_partition, + spans=tuple(route_spans), + cleanup_staging_keys=tuple(staging_keys), + expected_token_length=len(row.token_ids), + ) + encode_route_plan(route_plan) + + return FinalizedRollout( + rollout_id=rollout_id, + valid=True, + rejection_reason=None, + token_ids=row.token_ids, + token_mask=row.token_mask, + logprobs=row.logprobs, + prompt_len=row.prompt_len, + reward=reward, + staging_keys=staging_keys, + min_wv=min_wv, + max_wv=max_wv, + # getattr: pre-R3 Gym pins' LinearizedRow has no routed_experts; + # capture-R3 then degrades to the sentinel/group-drop path. + routed_experts=( + None + if self._defer_routed_experts_to_policy + else getattr(row, "routed_experts", None) + ), + route_plan=route_plan, + ) + + # ── per group ─────────────────────────────────────────────────────────── + + def finalize_group( + self, + group_id: str, + rollout_ids: list[str], + receipts: list[Optional[dict[str, Any]]], + rewards: list[float], + *, + fallback_weight_version: int, + ) -> FinalizedGroup: + """Publish exactly N canonical rows for one prompt group. + + Blocking (TQ round trips); run via ``asyncio.to_thread`` from the + dispatch task. ``fallback_weight_version`` stamps a group none of + whose rollouts produced a valid row (placeholder-only groups still + need a staleness tag). + """ + assert len(rollout_ids) == len(receipts) == len(rewards), ( + "rollout_ids, receipts, and rewards must be parallel" + ) + _group_t0 = time.perf_counter() + rows = [ + self.finalize_rollout(rollout_id, receipt, reward=reward) + for rollout_id, receipt, reward in zip(rollout_ids, receipts, rewards) + ] + _rollouts_ms = (time.perf_counter() - _group_t0) * 1000.0 + valid_rows = [row for row in rows if row.valid] + staging_keys = [key for row in rows for key in row.staging_keys] + metrics = { + "finalize/invalid_row_rate": 1.0 - len(valid_rows) / len(rows), + "finalize/calls_per_rollout": ( + sum(len(row.staging_keys) for row in rows) / len(rows) + ), + } + # Ledger-derived admission counters (per group): each manifest row + # carries its admission mode. token_in_rate near 1.0 is the capture + # health signal (a text root only opens each chain); this replaces the + # deleted gate metrics route. + manifest_rows = [ + record + for receipt in receipts + if isinstance(receipt, dict) + for record in (receipt.get("manifest") or []) + if isinstance(record, dict) + ] + if manifest_rows: + token_in_calls = sum( + 1 for record in manifest_rows if record.get("mode") == "token_in" + ) + metrics["finalize/token_in_calls"] = float(token_in_calls) + metrics["finalize/text_root_calls"] = float( + len(manifest_rows) - token_in_calls + ) + metrics["finalize/token_in_rate"] = token_in_calls / len(manifest_rows) + metrics["finalize/capture_poisoned_rollouts"] = float( + sum( + 1 + for receipt in receipts + if isinstance(receipt, dict) and receipt.get("capture_poisoned") + ) + ) + # Per-method terminal-selection breakdown. Witness methods + # (declared/response_id/content) resolve from evidence; heuristic is + # the no-witness parent-link fallback — a nonzero heuristic fraction + # on a declaring harness is a regression signal. Failed selections + # stamp the last stage attempted, so masked rollouts stay visible in + # their method's bucket (cross-reference finalize/invalid_row_rate). + for method in ("declared", "response_id", "content", "heuristic"): + method_receipts = sum( + 1 + for receipt in receipts + if isinstance(receipt, dict) + and receipt.get("terminal_selection") == method + ) + metrics[f"finalize/terminal_selection_{method}_count"] = float( + method_receipts + ) + metrics[f"finalize/terminal_selection_{method}_fraction"] = ( + method_receipts / len(receipts) + ) + witness_disagreements = sum( + 1 + for receipt in receipts + if isinstance(receipt, dict) + and "witness_disagreement" in str(receipt.get("terminal_attribution_reason") or "") + ) + metrics["finalize/terminal_witness_disagreement_count"] = float( + witness_disagreements + ) + for row in rows: + if not row.valid: + print( + f" finalize: rollout {row.rollout_id} rejected " + f"({row.rejection_reason}) — placeholder", + flush=True, + ) + + group_min_wv = min( + (r.min_wv for r in valid_rows), default=fallback_weight_version + ) + group_max_wv = max( + (r.max_wv for r in valid_rows), default=fallback_weight_version + ) + + valid_fraction = len(valid_rows) / len(rows) + if ( + self._min_valid_fraction is not None + and valid_fraction < self._min_valid_fraction + ): + self._clear_staging(staging_keys) + metrics["finalize/group_dropped"] = 1.0 + return FinalizedGroup( + meta=None, + group_min_wv=group_min_wv, + group_max_wv=group_max_wv, + staging_keys=[], + metrics=metrics, + dropped=True, + ) + + _tensorize_t0 = time.perf_counter() + # Placeholders borrow a valid sibling's prompt ids so per-prompt + # baselines group correctly; an all-placeholder group uses a single + # pad token (its rows all carry sample_mask 0 and never train). + sibling_prompt = ( + valid_rows[0].token_ids[: valid_rows[0].prompt_len] if valid_rows else [] + ) or [self._pad_token_id] + + n = len(rows) + seq_lens = [max(1, len(row.token_ids)) for row in rows] + max_len = max(seq_lens) + input_ids = torch.full((n, max_len), self._pad_token_id, dtype=torch.int64) + token_mask = torch.zeros((n, max_len), dtype=torch.float32) + logprobs = torch.zeros((n, max_len), dtype=torch.float32) + prompt_ids_for_adv = torch.tensor([sibling_prompt] * n, dtype=torch.int64) + sample_mask = torch.zeros(n, dtype=torch.float32) + lengths = torch.tensor(seq_lens, dtype=torch.long) + rewards_t = torch.tensor([row.reward for row in rows], dtype=torch.float32) + for i, row in enumerate(rows): + if not row.valid: + continue + length = len(row.token_ids) + input_ids[i, :length] = torch.tensor(row.token_ids, dtype=torch.int64) + token_mask[i, :length] = torch.tensor(row.token_mask, dtype=torch.float32) + logprobs[i, :length] = torch.tensor(row.logprobs, dtype=torch.float32) + sample_mask[i] = 1.0 + + train_batch = { + "input_ids": input_ids, + "input_lengths": lengths, + "generation_logprobs": logprobs, + "token_mask": token_mask, + "sample_mask": sample_mask, + "prompt_ids_for_adv": prompt_ids_for_adv, + "total_reward": rewards_t, + } + if self._router_replay_enabled and not self._defer_routed_experts_to_policy: + has_routed_row = any(r.valid and r.routed_experts for r in rows) + if not has_routed_row and self._routed_dims is None and not valid_rows: + # Nothing to learn (L, K) from yet — e.g. an all-poisoned + # group before the first healthy rollout. Dropping loses no + # training signal (no valid rows or routes) and keeps the + # partition schema consistent for groups that do publish. + print( + f" finalize: group {group_id} dropped — router replay on " + "but no rollout carried routed_experts and (L, K) is " + "unknown yet", + flush=True, + ) + self._clear_staging(staging_keys) + metrics["finalize/group_dropped"] = 1.0 + return FinalizedGroup( + meta=None, + group_min_wv=group_min_wv, + group_max_wv=group_max_wv, + staging_keys=[], + metrics=metrics, + dropped=True, + ) + train_batch["routed_experts"] = self._build_routed_experts_tensor( + rows, max_len=max_len, metrics=metrics + ) + sample_ids, fields, tags = pack_payload( + train_batch, weight_version=group_min_wv, group_id=group_id + ) + if self._defer_routed_experts_to_policy: + encoded_sizes = 0 + span_count = 0 + for tag, row, expected_length in zip(tags, rows, seq_lens): + plan = row.route_plan + if plan is None: + plan = RouteAssemblyPlan( + schema_version=ROUTE_PLAN_SCHEMA_VERSION, + staging_partition=self._staging_partition, + spans=(), + cleanup_staging_keys=tuple(row.staging_keys), + expected_token_length=expected_length, + ) + encoded = encode_route_plan(plan) + tag[ROUTE_PLAN_TAG] = encoded + encoded_sizes += encoded_route_plan_size_bytes(plan) + span_count += len(plan.spans) + metrics["finalize/route_plan_span_count"] = float(span_count) + metrics["finalize/route_plan_encoded_bytes"] = float(encoded_sizes) + valid_route_rows = sum( + 1 for row in valid_rows if row.route_plan and row.route_plan.spans + ) + if valid_rows: + metrics["finalize/routed_experts_row_coverage"] = ( + valid_route_rows / len(valid_rows) + ) + maybe_dump_train_rows( + source="finalizer", + group_id=group_id, + sample_ids=list(sample_ids), + train_batch=train_batch, + weight_version=group_min_wv, + ) + assert sample_ids == rollout_ids, ( + "canonical sample ids must equal the ledger-registered rollout ids: " + f"{sample_ids} != {rollout_ids}" + ) + _tensorize_ms = (time.perf_counter() - _tensorize_t0) * 1000.0 + _put_t0 = time.perf_counter() + self._call_dp( + "put_samples", + sample_ids=sample_ids, + partition_id=self._partition_id, + fields=fields, + tags=tags, + ) + _put_ms = (time.perf_counter() - _put_t0) * 1000.0 + _clear_ms = 0.0 + if not self._defer_routed_experts_to_policy: + _clear_t0 = time.perf_counter() + self._clear_staging(staging_keys) + _clear_ms = (time.perf_counter() - _clear_t0) * 1000.0 + # Per-step W&B breakdown of training-row assembly (capture arm) rides + # FinalizedGroup.metrics into the controller's rollout metrics. + metrics["row_assembly/rollouts_ms"] = _rollouts_ms + metrics["row_assembly/tensorize_ms"] = _tensorize_ms + metrics["row_assembly/tq_put_ms"] = _put_ms + if not self._defer_routed_experts_to_policy: + metrics["row_assembly/clear_staging_ms"] = _clear_ms + meta = KVBatchMeta( + partition_id=self._partition_id, + task_name="train", + sample_ids=list(sample_ids), + fields=list(fields.keys()), + sequence_lengths=[int(s) for s in lengths.tolist()], + tags=[dict(t) for t in tags], + ) + return FinalizedGroup( + meta=meta, + group_min_wv=group_min_wv, + group_max_wv=group_max_wv, + staging_keys=(staging_keys if self._defer_routed_experts_to_policy else []), + metrics=metrics, + ) + + # ── internals ─────────────────────────────────────────────────────────── + + def _build_routed_experts_tensor( + self, + rows: list[FinalizedRollout], + *, + max_len: int, + metrics: dict[str, float], + ) -> torch.Tensor: + """[n, max_len, L, K] int16 routes for the group; sentinel elsewhere. + + Padding, placeholder rows, and valid rows whose rebuild carried no + routes are all-sentinel: Megatron's replay falls back to its own + router for exactly those positions. (L, K) is learned from the first + rebuilt row that carries routes and cached for placeholder-only + groups; a group arriving before any routed row has been seen cannot + be shaped and fails loudly (unreachable once the first real rollout + of the run finalizes). + """ + for row in rows: + if row.valid and row.routed_experts: + first = row.routed_experts[0] + self._routed_dims = (len(first), len(first[0])) + break + if self._routed_dims is None: + raise RuntimeError( + "policy.router_replay.enabled=true (token-capture mode) but no " + "finalized rollout has carried routed_experts yet, so the " + "placeholder group tensor cannot be shaped. Check vLLM " + "enable_return_routed_experts and the staging-extras path." + ) + num_moe_layers, topk = self._routed_dims + routed = torch.full( + (len(rows), max_len, num_moe_layers, topk), + _ROUTED_EXPERTS_SENTINEL, + dtype=torch.int16, + ) + rows_with_routes = 0 + valid_rows = 0 + sentinel_tokens = 0 + covered_tokens = 0 + for i, row in enumerate(rows): + if not row.valid: + continue + valid_rows += 1 + covered_tokens += len(row.token_ids) + if not row.routed_experts: + sentinel_tokens += len(row.token_ids) + continue + rows_with_routes += 1 + row_routes = torch.tensor(row.routed_experts, dtype=torch.int16) + if row_routes.shape != (len(row.token_ids), num_moe_layers, topk): + raise RuntimeError( + "rebuilt routed_experts shape " + f"{tuple(row_routes.shape)} does not match " + f"({len(row.token_ids)}, {num_moe_layers}, {topk}) for " + f"rollout {row.rollout_id}" + ) + routed[i, : row_routes.shape[0]] = row_routes + sentinel_tokens += int( + row_routes.eq(_ROUTED_EXPERTS_SENTINEL).all(-1).all(-1).sum().item() + ) + if valid_rows: + metrics["finalize/routed_experts_row_coverage"] = ( + rows_with_routes / valid_rows + ) + if covered_tokens: + metrics["finalize/routed_experts_sentinel_token_fraction"] = ( + sentinel_tokens / covered_tokens + ) + return routed + + def _clear_staging(self, staging_keys: list[str]) -> None: + if not staging_keys: + return + try: + self._staging.clear(staging_keys) + except Exception as error: + raise RuntimeError( + "finalizer staging cleanup failed for known keys " + f"partition={self._staging_partition!r}, keys={staging_keys!r}" + ) from error + + def _call_dp(self, method_name: str, **kwargs: Any) -> Any: + import ray + + method = getattr(self._dp_client, method_name) + remote = getattr(method, "remote", None) + if remote is not None: + return ray.get(remote(**kwargs)) + return method(**kwargs) diff --git a/nemo_rl/experience/finalizer_actor.py b/nemo_rl/experience/finalizer_actor.py new file mode 100644 index 00000000000..3502eee0807 --- /dev/null +++ b/nemo_rl/experience/finalizer_actor.py @@ -0,0 +1,151 @@ +# Copyright (c) 2026, NVIDIA CORPORATION. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +"""CPU Ray actors for metadata-only token-capture finalization.""" + +from __future__ import annotations + +from dataclasses import dataclass, fields, is_dataclass +from typing import Any, Optional + +import ray +import torch + +from nemo_rl.data_plane import DataPlaneConfig, build_data_plane_client +from nemo_rl.experience.blackbox_finalizer import BlackboxFinalizer, FinalizedGroup + +_FORBIDDEN_RPC_KEYS = frozenset( + { + "input_ids", + "token_ids", + "token_ids_delta", + "token_mask", + "token_mask_delta", + "generation_logprobs", + "generation_logprobs_delta", + "generation_log_probs_delta", + "logprobs_delta", + "routed_experts", + } +) + + +@dataclass(frozen=True) +class FinalizationRequest: + """Metadata-only input for one prompt group's finalization.""" + + group_id: str + rollout_ids: tuple[str, ...] + receipts: tuple[Optional[dict[str, Any]], ...] + rewards: tuple[float, ...] + fallback_weight_version: int + + +@dataclass(frozen=True) +class FinalizerActorConfig: + """Internal constructor values shared by every finalizer actor.""" + + partition_id: str + staging_partition: str + pad_token_id: int + mixed_weight_version_policy: str + min_valid_fraction_per_group: Optional[float] + router_replay_enabled: bool + defer_routed_experts_to_policy: bool + + +def assert_metadata_only(value: Any, *, path: str = "rpc") -> None: + """Reject tensors and known heavy row fields reachable from an RPC graph.""" + if isinstance(value, torch.Tensor): + raise TypeError( + f"{path} contains a torch.Tensor with shape {tuple(value.shape)}" + ) + if value is None or isinstance(value, (str, int, float, bool)): + return + if is_dataclass(value) and not isinstance(value, type): + for field_info in fields(value): + assert_metadata_only( + getattr(value, field_info.name), + path=f"{path}.{field_info.name}", + ) + return + if isinstance(value, dict): + for key, item in value.items(): + if key in _FORBIDDEN_RPC_KEYS: + raise TypeError(f"{path} contains forbidden heavy field {key!r}") + assert_metadata_only(key, path=f"{path}.key") + assert_metadata_only(item, path=f"{path}[{key!r}]") + return + if isinstance(value, (list, tuple)): + for index, item in enumerate(value): + assert_metadata_only(item, path=f"{path}[{index}]") + return + raise TypeError(f"{path} contains unsupported RPC type {type(value).__name__}") + + +@ray.remote( + num_cpus=1, + num_gpus=0, + max_restarts=0, + max_task_retries=0, +) +class FinalizerActor: # pragma: no cover + """Own a connect-only TQ client and lightweight finalizer in one process.""" + + def __init__( + self, + dp_config: DataPlaneConfig, + config: FinalizerActorConfig, + ) -> None: + dp_client = build_data_plane_client(dp_config, bootstrap=False) + self._finalizer = BlackboxFinalizer( + dp_client, + partition_id=config.partition_id, + staging_partition=config.staging_partition, + pad_token_id=config.pad_token_id, + mixed_weight_version_policy=config.mixed_weight_version_policy, + min_valid_fraction_per_group=config.min_valid_fraction_per_group, + router_replay_enabled=config.router_replay_enabled, + defer_routed_experts_to_policy=config.defer_routed_experts_to_policy, + ) + + def finalize(self, request: FinalizationRequest) -> FinalizedGroup: + """Finalize one request without allowing tensor payloads across Ray RPC.""" + assert_metadata_only(request) + if not ( + len(request.rollout_ids) == len(request.receipts) == len(request.rewards) + ): + raise ValueError( + "finalizer request rollout_ids, receipts, and rewards must be parallel" + ) + result = self._finalizer.finalize_group( + request.group_id, + list(request.rollout_ids), + list(request.receipts), + list(request.rewards), + fallback_weight_version=request.fallback_weight_version, + ) + assert_metadata_only(result) + return result + + +def create_finalizer_actors( + dp_config: DataPlaneConfig, + config: FinalizerActorConfig, + *, + num_workers: int, +) -> list[Any]: + """Construct the fixed validation pool after TQ partitions are registered.""" + if num_workers <= 0: + raise ValueError(f"num_finalizer_workers must be positive, got {num_workers}") + return [FinalizerActor.remote(dp_config, config) for _ in range(num_workers)] diff --git a/nemo_rl/experience/rollout_manager.py b/nemo_rl/experience/rollout_manager.py index 87c27954901..f8f3bbaa0c4 100644 --- a/nemo_rl/experience/rollout_manager.py +++ b/nemo_rl/experience/rollout_manager.py @@ -15,7 +15,8 @@ import asyncio import copy import json -from typing import Any, Optional +import uuid +from typing import TYPE_CHECKING, Any, Optional import torch from transformers import PreTrainedTokenizerBase @@ -43,6 +44,9 @@ TokenizerType = PreTrainedTokenizerBase +if TYPE_CHECKING: + from nemo_rl.experience.finalizer_actor import FinalizationRequest + class AsyncRolloutImpl: """Manages per-prompt multi-turn rollouts, producing a PromptGroupRecord per call. @@ -68,15 +72,21 @@ def __init__( self._max_rollout_turns = max_rollout_turns self._policy_generation = policy_generation - async def run_rollout(self, input_sample: DatumSpec) -> PromptGroupRecord: + async def run_rollout( + self, input_sample: DatumSpec, *, rollout_ids: Optional[list[str]] = None + ) -> PromptGroupRecord: """Run num_generations_per_prompt rollouts for one prompt. Args: input_sample: A single prompt (one DatumSpec entry). + rollout_ids: Unsupported here — token capture is NeMo-Gym only. Returns: PromptGroupRecord with num_generations_per_prompt completions. """ + assert rollout_ids is None, ( + "token capture (rollout_ids) is only supported on the NeMo-Gym path" + ) timer = Timer() timer_prefix = "timing/rollout" timer.start(f"{timer_prefix}/total") @@ -439,11 +449,17 @@ def __init__( self._validate_init_params() - async def run_rollout(self, input_sample: DatumSpec) -> PromptGroupRecord: + async def run_rollout( + self, input_sample: DatumSpec, *, rollout_ids: Optional[list[str]] = None + ) -> PromptGroupRecord: """Run num_generations_per_prompt rollouts for one prompt. Args: input_sample: A single prompt (one DatumSpec entry). + rollout_ids: Token-capture mode: gate-registered rollout ids, one + per generation, riding each row's run body as the opaque + ``_ng_rollout_id`` key (agents stamp /ng-rollout/ from it; + zero agent changes). Returns: PromptGroupRecord with num_generations_per_prompt completions. @@ -452,7 +468,7 @@ async def run_rollout(self, input_sample: DatumSpec) -> PromptGroupRecord: timer_prefix = "timing/rollout" timer.start(f"{timer_prefix}/total") - rollout_inputs = self._build_inputs(input_sample) + rollout_inputs = self._build_inputs(input_sample, rollout_ids=rollout_ids) completions, prompt_message_log, rollout_metrics = await self._run_rollouts( rollout_inputs, timer, timer_prefix ) @@ -483,7 +499,9 @@ def _validate_init_params(self) -> None: "Please set `max_rollout_turns` to 1." ) - def _build_inputs(self, input_sample: DatumSpec) -> list[dict]: + def _build_inputs( + self, input_sample: DatumSpec, *, rollout_ids: Optional[list[str]] = None + ) -> list[dict]: """Build N row dicts from input_sample, applying generation config params.""" # Build a template row from the input_sample's extra_env_info, applying generation params. template_row: dict = copy.deepcopy(input_sample["extra_env_info"]) # type: ignore @@ -504,10 +522,19 @@ def _build_inputs(self, input_sample: DatumSpec) -> list[dict]: ) # Build N rows with distinct rowidxs so run_rollouts can sort them correctly. + if rollout_ids is not None: + assert len(rollout_ids) == self._num_generations_per_prompt, ( + "token-capture rollout ids must be one per generation" + ) rows = [] for i in range(self._num_generations_per_prompt): row = copy.deepcopy(template_row) row["_rowidx"] = i + if rollout_ids is not None: + # Opaque run-body carrier (Gym's _ng_rollout_id key): the agent + # derives the id from the run body and stamps /ng-rollout/ + # on every model call, so the TQ sample id IS the capture key. + row["_ng_rollout_id"] = rollout_ids[i] rows.append(row) return rows @@ -564,6 +591,21 @@ async def _run_rollouts( def _result_to_completion(self, result: dict) -> Completion: """Convert one run_rollouts result dict into a Completion.""" + if "receipt" in result: + # Receipt mode (token capture): the result is token-free — the + # message_log is empty and the canonical row is rebuilt by the + # finalizer from staged deltas. The receipt and rollout id ride + # env_extras for the finalize step. + env_extras = dict(result["full_result"]) + env_extras["ng_receipt"] = result["receipt"] + env_extras["ng_rollout_id"] = result["rollout_id"] + return Completion( + message_log=result["message_log"], + env_extras=env_extras, + truncated=False, + reward=float(result["full_result"]["reward"]), + ) + # Tensorize token fields. _tensorize_by_key(result["message_log"], "token_ids") _tensorize_by_key( @@ -598,29 +640,54 @@ def _compute_rollout_metrics( """Aggregate per-sample and per-agent metrics.""" # Prepare lists of values for each metric. total_reward = [c.reward for c in completions] - turn_count = [ - sum(1 for m in c.message_log if m["role"] == "user") for c in completions - ] - # token metrics - total_tokens = [ - sum(len(m["token_ids"]) for m in c.message_log) for c in completions - ] - assistant_tokens = [ - sum(len(m["token_ids"]) for m in c.message_log if m["role"] == "assistant") - for c in completions - ] - # max_gen_tokens_per_turn: Diagnostic for long single generations - max_gen_tokens_per_turn = [ - max( - ( + receipt_mode = bool(completions) and "ng_receipt" in completions[0].env_extras + if receipt_mode: + # Token-free receipts: token accounting comes from the manifest + # (cum_len of the deepest chain; delta sums as the generation + # proxy) instead of a message_log walk. + manifests = [ + ((c.env_extras.get("ng_receipt") or {}).get("manifest") or []) + for c in completions + ] + turn_count = [len(m) for m in manifests] + total_tokens = [ + max((entry["cum_len"] for entry in m), default=0) for m in manifests + ] + assistant_tokens = [ + sum(entry["delta_len"] for entry in m) for m in manifests + ] + max_gen_tokens_per_turn = [ + max((entry["delta_len"] for entry in m), default=0) for m in manifests + ] + else: + turn_count = [ + sum(1 for m in c.message_log if m["role"] == "user") + for c in completions + ] + # token metrics + total_tokens = [ + sum(len(m["token_ids"]) for m in c.message_log) for c in completions + ] + assistant_tokens = [ + sum( len(m["token_ids"]) for m in c.message_log if m["role"] == "assistant" - ), - default=0, - ) - for c in completions - ] + ) + for c in completions + ] + # max_gen_tokens_per_turn: Diagnostic for long single generations + max_gen_tokens_per_turn = [ + max( + ( + len(m["token_ids"]) + for m in c.message_log + if m["role"] == "assistant" + ), + default=0, + ) + for c in completions + ] # truncated metrics truncated = [c.truncated for c in completions] @@ -644,8 +711,12 @@ def _compute_rollout_metrics( "truncation_rate": sum(truncated) / n, } - # Agent-level metrics. - agent_extras = [c.env_extras for c in completions] + # Agent-level metrics. Receipts are lineage records, not agent + # results — keep them (and their manifests) out of the logged table. + agent_extras = [ + {k: v for k, v in c.env_extras.items() if k not in ("ng_receipt",)} + for c in completions + ] for key in agent_extras[0].keys(): values = [ float(r[key]) # type: ignore @@ -687,7 +758,6 @@ def __init__( assert num_generations_per_prompt >= 1, ( "num_generations_per_prompt must be >= 1" ) - if not use_nemo_gym: rollout_cls = AsyncRolloutImpl assert policy_generation is not None, ( @@ -713,6 +783,7 @@ def __init__( self._tokenizer = tokenizer self._num_generations_per_prompt = num_generations_per_prompt self._tq_buffer = tq_buffer + self._env_handles = task_to_env self._weight_version: int = 0 def set_weight_version(self, version: int) -> None: @@ -723,8 +794,13 @@ def set_weight_version(self, version: int) -> None: """ self._weight_version = int(version) - async def run_rollout(self, input_sample: DatumSpec) -> PromptGroupRecord: - return await self._impl.run_rollout(input_sample) + async def run_rollout( + self, input_sample: DatumSpec, *, rollout_ids: Optional[list[str]] = None + ) -> PromptGroupRecord: + if rollout_ids is None: + # Legacy path: keep the impl call signature byte-identical. + return await self._impl.run_rollout(input_sample) + return await self._impl.run_rollout(input_sample, rollout_ids=rollout_ids) async def generate_and_push( self, @@ -777,3 +853,67 @@ async def generate_and_push( flush=True, ) raise + + async def generate_for_finalization( + self, + input_sample: DatumSpec, + *, + target_step: Optional[int] = None, + inflight_registry: Optional[dict[str, tuple[asyncio.Task[None], int]]] = None, + ) -> "FinalizationRequest": + """Run capture generation and return a metadata-only actor request. + + The replay-buffer slot remains reserved and unready. The caller owns + finalizer submission and must either commit the returned group or stop + the validation run on an unknown publication outcome. + """ + from nemo_rl.experience.finalizer_actor import FinalizationRequest + + assert self._tq_buffer is not None, ( + "generate_for_finalization requires tq_buffer to be set at __init__" + ) + start_version = self._weight_version + group_id = str(uuid.uuid4()) + rollout_ids = tuple( + f"{group_id}_g{i}" for i in range(self._num_generations_per_prompt) + ) + self._tq_buffer.reserve( + weight_version=start_version, + target_step=target_step, + group_id=group_id, + rollout_ids=list(rollout_ids), + ) + try: + if inflight_registry is not None: + current_task = asyncio.current_task() + assert current_task is not None + inflight_registry[group_id] = (current_task, start_version) + try: + record = await self.run_rollout( + input_sample, + rollout_ids=list(rollout_ids), + ) + finally: + if inflight_registry is not None: + inflight_registry.pop(group_id, None) + receipts = tuple(c.env_extras.get("ng_receipt") for c in record.completions) + rewards = tuple(float(c.reward) for c in record.completions) + request = FinalizationRequest( + group_id=group_id, + rollout_ids=rollout_ids, + receipts=receipts, + rewards=rewards, + fallback_weight_version=start_version, + ) + from nemo_rl.experience.finalizer_actor import assert_metadata_only + + assert_metadata_only(request) + return request + except BaseException: + # Abandoned dispatch: no receipt will name these rollouts' staged + # rows, so they leak until the staging partition is torn down at + # run end (there is no prefix-clear primitive in the data plane + # yet). Their ledger files are inert — failure rows or missing + # terminal rows keep any later read fail-closed. + self._tq_buffer.abort(group_id) + raise diff --git a/nemo_rl/experience/route_plan.py b/nemo_rl/experience/route_plan.py new file mode 100644 index 00000000000..ff31c39fa91 --- /dev/null +++ b/nemo_rl/experience/route_plan.py @@ -0,0 +1,270 @@ +# Copyright (c) 2026, NVIDIA CORPORATION. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +"""Strict metadata contract for deferred routed-expert assembly.""" + +from __future__ import annotations + +import json +from dataclasses import dataclass +from typing import Any, Literal + +ROUTE_PLAN_SCHEMA_VERSION = 2 +_EXTRAS_DIGEST_VERSION = 1 +_SHA256_HEX_LENGTH = 64 + + +@dataclass(frozen=True) +class RouteSpan: + """One root-first route contribution to a canonical rollout row.""" + + staging_key: str + carry_len: int + generation_len: int + staged_route_len: int + extras_digest_version: int + extras_digest: str + + +@dataclass(frozen=True) +class RouteAssemblyPlan: + """How a policy worker reconstructs routes from staged fragments.""" + + schema_version: int + staging_partition: str + spans: tuple[RouteSpan, ...] + cleanup_staging_keys: tuple[str, ...] + expected_token_length: int + + +RouteSpanMode = Literal["full", "tail", "sentinel"] + + +def classify_route_span(span: RouteSpan) -> RouteSpanMode: + """Apply Gym's route-linearization decision table using metadata only.""" + expected = span.carry_len + span.generation_len + if span.staged_route_len > 0 and span.staged_route_len == expected: + return "full" + if span.staged_route_len > 0 and 0 < span.generation_len <= span.staged_route_len: + return "tail" + return "sentinel" + + +def _require_exact_keys( + value: dict[str, Any], expected: set[str], *, where: str +) -> None: + actual = set(value) + if actual != expected: + raise ValueError( + f"{where} fields must be exactly {sorted(expected)}, got {sorted(actual)}" + ) + + +def _require_int(value: Any, *, where: str) -> int: + if type(value) is not int: + raise TypeError(f"{where} must be int, got {type(value).__name__}") + return value + + +def _require_nonnegative_int(value: Any, *, where: str) -> int: + parsed = _require_int(value, where=where) + if parsed < 0: + raise ValueError(f"{where} must be non-negative, got {parsed}") + return parsed + + +def _require_string(value: Any, *, where: str) -> str: + if not isinstance(value, str) or not value: + raise TypeError(f"{where} must be a non-empty string") + return value + + +def _validate_plan(plan: RouteAssemblyPlan) -> None: + if plan.schema_version != ROUTE_PLAN_SCHEMA_VERSION: + raise ValueError( + "unsupported route plan schema version " + f"{plan.schema_version}; expected {ROUTE_PLAN_SCHEMA_VERSION}" + ) + _require_string(plan.staging_partition, where="route_plan.staging_partition") + _require_nonnegative_int( + plan.expected_token_length, + where="route_plan.expected_token_length", + ) + cleanup_keys = set(plan.cleanup_staging_keys) + if len(cleanup_keys) != len(plan.cleanup_staging_keys): + raise ValueError("route_plan.cleanup_staging_keys contains duplicates") + for index, key in enumerate(plan.cleanup_staging_keys): + _require_string(key, where=f"route_plan.cleanup_staging_keys[{index}]") + for index, span in enumerate(plan.spans): + _require_string( + span.staging_key, where=f"route_plan.spans[{index}].staging_key" + ) + _require_nonnegative_int( + span.carry_len, where=f"route_plan.spans[{index}].carry_len" + ) + _require_nonnegative_int( + span.generation_len, + where=f"route_plan.spans[{index}].generation_len", + ) + _require_nonnegative_int( + span.staged_route_len, + where=f"route_plan.spans[{index}].staged_route_len", + ) + if ( + type(span.extras_digest_version) is not int + or span.extras_digest_version != _EXTRAS_DIGEST_VERSION + ): + raise ValueError( + f"route_plan.spans[{index}].extras_digest_version must be " + f"{_EXTRAS_DIGEST_VERSION}" + ) + if ( + not isinstance(span.extras_digest, str) + or len(span.extras_digest) != _SHA256_HEX_LENGTH + or any( + character not in "0123456789abcdef" for character in span.extras_digest + ) + ): + raise ValueError( + f"route_plan.spans[{index}].extras_digest must be a lowercase " + "SHA-256 hex digest" + ) + if span.staging_key not in cleanup_keys: + raise ValueError( + f"route_plan.spans[{index}] key {span.staging_key!r} is outside " + "cleanup_staging_keys" + ) + classify_route_span(span) + if plan.spans: + contribution = sum(span.carry_len + span.generation_len for span in plan.spans) + if contribution != plan.expected_token_length: + raise ValueError( + f"route plan spans contribute {contribution} tokens, expected " + f"{plan.expected_token_length}" + ) + + +def encode_route_plan(plan: RouteAssemblyPlan) -> dict[str, Any]: + """Encode a validated plan into primitive ``KVBatchMeta.tags`` data.""" + _validate_plan(plan) + return { + "schema_version": plan.schema_version, + "staging_partition": plan.staging_partition, + "spans": [ + { + "staging_key": span.staging_key, + "carry_len": span.carry_len, + "generation_len": span.generation_len, + "staged_route_len": span.staged_route_len, + "extras_digest_version": span.extras_digest_version, + "extras_digest": span.extras_digest, + } + for span in plan.spans + ], + "cleanup_staging_keys": list(plan.cleanup_staging_keys), + "expected_token_length": plan.expected_token_length, + } + + +def decode_route_plan(value: Any) -> RouteAssemblyPlan: + """Strictly decode a plan without defaults or compatibility guesses.""" + if not isinstance(value, dict): + raise TypeError(f"route plan must be a dict, got {type(value).__name__}") + _require_exact_keys( + value, + { + "schema_version", + "staging_partition", + "spans", + "cleanup_staging_keys", + "expected_token_length", + }, + where="route_plan", + ) + spans_value = value["spans"] + if not isinstance(spans_value, list): + raise TypeError("route_plan.spans must be a list") + spans: list[RouteSpan] = [] + for index, span_value in enumerate(spans_value): + if not isinstance(span_value, dict): + raise TypeError(f"route_plan.spans[{index}] must be a dict") + _require_exact_keys( + span_value, + { + "staging_key", + "carry_len", + "generation_len", + "staged_route_len", + "extras_digest_version", + "extras_digest", + }, + where=f"route_plan.spans[{index}]", + ) + spans.append( + RouteSpan( + staging_key=_require_string( + span_value["staging_key"], + where=f"route_plan.spans[{index}].staging_key", + ), + carry_len=_require_nonnegative_int( + span_value["carry_len"], + where=f"route_plan.spans[{index}].carry_len", + ), + generation_len=_require_nonnegative_int( + span_value["generation_len"], + where=f"route_plan.spans[{index}].generation_len", + ), + staged_route_len=_require_nonnegative_int( + span_value["staged_route_len"], + where=f"route_plan.spans[{index}].staged_route_len", + ), + extras_digest_version=_require_int( + span_value["extras_digest_version"], + where=f"route_plan.spans[{index}].extras_digest_version", + ), + extras_digest=_require_string( + span_value["extras_digest"], + where=f"route_plan.spans[{index}].extras_digest", + ), + ) + ) + cleanup_value = value["cleanup_staging_keys"] + if not isinstance(cleanup_value, list): + raise TypeError("route_plan.cleanup_staging_keys must be a list") + cleanup_keys = tuple( + _require_string(key, where=f"route_plan.cleanup_staging_keys[{index}]") + for index, key in enumerate(cleanup_value) + ) + plan = RouteAssemblyPlan( + schema_version=_require_int( + value["schema_version"], where="route_plan.schema_version" + ), + staging_partition=_require_string( + value["staging_partition"], where="route_plan.staging_partition" + ), + spans=tuple(spans), + cleanup_staging_keys=cleanup_keys, + expected_token_length=_require_nonnegative_int( + value["expected_token_length"], + where="route_plan.expected_token_length", + ), + ) + _validate_plan(plan) + return plan + + +def encoded_route_plan_size_bytes(plan: RouteAssemblyPlan) -> int: + """Return the compact UTF-8 encoded size used for observability.""" + return len( + json.dumps(encode_route_plan(plan), separators=(",", ":")).encode("utf-8") + ) diff --git a/nemo_rl/experience/row_dump.py b/nemo_rl/experience/row_dump.py new file mode 100644 index 00000000000..12b831a6e11 --- /dev/null +++ b/nemo_rl/experience/row_dump.py @@ -0,0 +1,77 @@ +# Copyright (c) 2026, NVIDIA CORPORATION. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +"""Env-gated dump of canonical training rows at their TQ publish sites. + +Set ``NRL_SC_DUMP_TRAIN_ROWS=`` to append one JSON line per training row +whenever a group is published to the ``rollout_data`` partition — from the +legacy ``TQReplayBuffer.commit`` path and from the token-capture +``BlackboxFinalizer`` publish. Off (no I/O, no imports of the payload) unless +the env var is set. Used by the S5 legacy-vs-capture offline row diff. +""" + +import json +import os +import threading +from collections.abc import Mapping +from typing import Any, Optional + +import torch + +_DUMP_ENV_VAR = "NRL_SC_DUMP_TRAIN_ROWS" +# The finalizer publishes from a worker thread (asyncio.to_thread) while the +# legacy path publishes from the SC event loop; serialize appends. +_G_WRITE_LOCK = threading.Lock() + + +def _row_value(tensor: torch.Tensor, row: int) -> Any: + value = tensor[row] + if value.dim() == 0: + return value.item() + return value.tolist() + + +def maybe_dump_train_rows( + *, + source: str, + group_id: str, + sample_ids: list[str], + train_batch: Mapping[str, torch.Tensor], + weight_version: Optional[int], +) -> None: + """Append each row of a published group to the dump file, if enabled. + + Args: + source: Publish site tag (``"legacy_commit"`` or ``"finalizer"``). + group_id: Prompt-group id the rows belong to. + sample_ids: Canonical per-row sample ids (``{group_id}_g{i}``). + train_batch: Column tensors as passed to ``pack_payload``. + weight_version: Weight version stamped on the rows' tags. + """ + dump_dir = os.environ.get(_DUMP_ENV_VAR) + if not dump_dir: + return + os.makedirs(dump_dir, exist_ok=True) + path = os.path.join(dump_dir, f"train_rows_{source}.jsonl") + lines = [] + for i, sample_id in enumerate(sample_ids): + record = { + "source": source, + "group_id": group_id, + "sample_id": sample_id, + "weight_version": weight_version, + **{name: _row_value(tensor, i) for name, tensor in train_batch.items()}, + } + lines.append(json.dumps(record)) + with _G_WRITE_LOCK, open(path, "a") as f: + f.write("\n".join(lines) + "\n") diff --git a/nemo_rl/models/generation/vllm/vllm_generation.py b/nemo_rl/models/generation/vllm/vllm_generation.py index 888e2539ce4..f71b329475e 100644 --- a/nemo_rl/models/generation/vllm/vllm_generation.py +++ b/nemo_rl/models/generation/vllm/vllm_generation.py @@ -533,6 +533,37 @@ def _post_init(self): results = ray.get(futures) return results + def setup_token_capture( + self, dp_cfg: dict[str, Any], staging_partition: str + ) -> None: + """Install ledger-authoritative token capture in every DP-leader worker. + + Called once at setup when ``token_capture.enabled``; each async worker + builds its in-worker data-plane client + TQTokenSink and makes the + single Gym ``install_capture`` call (see + docs/design-docs/token-capture-ledger.md). + """ + assert self.cfg["vllm_cfg"]["async_engine"], ( + "token capture requires the async vLLM engine (the capture host " + "is the worker's in-process HTTP server)" + ) + futures = self.worker_group.run_all_workers_single_data( + "setup_token_capture", + dp_cfg=dp_cfg, + staging_partition=staging_partition, + run_rank_0_only_axes=["tensor_parallel", "pipeline_parallel"], + ) + ray.get(futures) + + def set_rollout_weight_version(self, version: int) -> None: + """Rotate the weight version workers stamp on captured model calls.""" + futures = self.worker_group.run_all_workers_single_data( + "set_rollout_weight_version", + version=version, + run_rank_0_only_axes=["tensor_parallel", "pipeline_parallel"], + ) + ray.get(futures) + def _get_raw_spec_counters(self) -> dict[str | tuple[str, int], float]: """Collect raw spec decode counters from workers.""" futures = self.worker_group.run_all_workers_single_data( diff --git a/nemo_rl/models/generation/vllm/vllm_worker_async.py b/nemo_rl/models/generation/vllm/vllm_worker_async.py index eec501eb609..8991f3d59ac 100644 --- a/nemo_rl/models/generation/vllm/vllm_worker_async.py +++ b/nemo_rl/models/generation/vllm/vllm_worker_async.py @@ -16,6 +16,7 @@ import copy import gc import logging +import os import threading import time import uuid @@ -100,6 +101,19 @@ def __init__( self.base_url = None self.http_server = None + # Ledger-authoritative token capture (dormant until the + # setup_token_capture fan-out runs; see + # docs/design-docs/token-capture-ledger.md). The weight + # version is stamped per model call at begin_call time and rotated by + # the set_rollout_weight_version fan-out from the SC's _sync_weights. + self.token_capture = None + self._rollout_weight_version = 0 + # In-flight captured calls keyed by id(request): (ActiveCall, the + # exact engine prompt ids recorded at preprocess time). + self._capture_calls: dict[int, tuple[Any, list[int]]] = {} + self._staging_source: Any | None = None + self._prefix_cache: dict[str, list[int]] = {} + super().__init__( config, bundle_indices, @@ -345,8 +359,201 @@ async def get_reserved_url(self) -> Optional[str]: async def report_dp_openai_server_base_url(self) -> Optional[str]: return self.base_url + def install_token_capture(self, capture: Any) -> None: + """Gym's ``install_capture`` seam (the ``CaptureHost`` contract).""" + self.token_capture = capture + + async def setup_token_capture( + self, dp_cfg: dict[str, Any], staging_partition: str + ) -> bool: + """Host ledger-authoritative token capture in this worker. + + Fan-out target (token_capture.enabled only): builds the in-worker + data-plane client and TQTokenSink, then makes the single + ``install_capture`` call wiring Gym's engine-blind capture core + + vLLM adapter into this worker. Returns whether capture was installed + (False on non-model-owner ranks, which serve no HTTP). + """ + if not self.is_model_owner: + return False + # Deferred: nemo_gym is an optional extra absent in non-gym runs. + from nemo_gym.token_id_capture.adapters.vllm import VLLMCaptureAdapter + from nemo_gym.token_id_capture.staging import install_capture + + from nemo_rl.data_plane import build_data_plane_client + from nemo_rl.data_plane.tq_token_sink import TQTokenSink, TQTokenSource + + dp_client = build_data_plane_client(dp_cfg, bootstrap=False) + sink = TQTokenSink(dp_client, staging_partition=staging_partition) + self._staging_source = TQTokenSource( + dp_client, staging_partition=staging_partition + ) + self._prefix_cache.clear() + install_capture( + self, + sink=sink, + weight_version_fn=lambda: self._rollout_weight_version, + adapter=VLLMCaptureAdapter(), + ) + return True + + async def set_rollout_weight_version(self, version: int) -> None: + """Rotate the weight version stamped on subsequent captured calls.""" + self._rollout_weight_version = int(version) + + def _begin_request_capture(self, request: Any, prompt_token_ids: list[int]) -> None: + """Admit one ledger-forwarded call into the capture layer. + + Called from preprocess_chat once the exact engine prompt is known + (post-splice in token-in mode, full render in text mode). No-op + unless capture is installed and the request carries the ledger's + ``ng_capture`` context. + """ + capture = self.token_capture + context = getattr(request, "ng_capture", None) + if capture is None or not context: + return + from nemo_gym.token_id_capture.staging.records import CaptureAdmission + + call = capture.begin_call( + CaptureAdmission.model_validate(context), + stream=bool(getattr(request, "stream", False)), + ) + self._capture_calls[id(request)] = (call, list(prompt_token_ids)) + + def _fetch_chain_prefix(self, staging_chain: list[str]) -> list[int]: + """Assemble prefix token ids from staging_chain, with a worker-local LRU cache.""" + cache = self._prefix_cache + cached_ids: list[int] = [] + miss_start = 0 + for i, key in enumerate(staging_chain): + if key in cache: + cached_ids = cache[key] + miss_start = i + 1 + miss_keys = staging_chain[miss_start:] + if not miss_keys: + return list(cached_ids) + if self._staging_source is None: + raise RuntimeError( + "_staging_source not initialized; call setup_token_capture() first" + ) + fetched = self._staging_source.fetch_prefix_token_ids(miss_keys) + result = cached_ids + fetched + last_key = staging_chain[-1] + cache[last_key] = result + if len(cache) > 256: + del cache[next(iter(cache))] + return result + + def _patch_chain_prefix(self, ng_capture: dict[str, Any]) -> list[int] | None: + """Fetch a staged prefix and patch a raw capture admission in place.""" + staging_chain = ng_capture.get("staging_chain") or [] + if not staging_chain: + return None + prefix_token_ids = self._fetch_chain_prefix(staging_chain) + prev_len = ng_capture.get("prev_len") + if type(prev_len) is not int or prev_len <= 0: + raise ValueError("staging_chain admission requires prev_len > 0") + if len(prefix_token_ids) != prev_len: + raise ValueError( + "staging_chain prefix length mismatch: " + f"expected {prev_len}, fetched {len(prefix_token_ids)}" + ) + ng_capture["required_prefix_token_ids"] = prefix_token_ids + return prefix_token_ids + + @staticmethod + def _delta_align_routed_experts( + payload: dict[str, Any], *, prev_len: int, prompt_len: int, generated_len: int + ) -> None: + """Normalize optional vLLM routes to the exact staged token delta.""" + choices = payload.get("choices") or [] + if len(choices) != 1 or not isinstance(choices[0], dict): + return + choice = dict(choices[0]) + message = dict(choice.get("message") or {}) + routed = message.get("routed_experts") + if routed is None: + return + try: + from nemo_rl.utils.routed_experts_codec import ( + decode_routed_experts, + encode_routed_experts, + ) + + if isinstance(routed, str): + dtype_name = routed.split(":", 3)[1] + dtype = { + "int8": torch.int8, + "int16": torch.int16, + "int32": torch.int32, + }.get(dtype_name) + if dtype is None: + raise ValueError(f"unsupported routed_experts dtype {dtype_name!r}") + else: + dtype = torch.int16 + experts = decode_routed_experts(routed, dtype) + expected_full_len = prompt_len + generated_len + if experts.dim() != 3 or experts.shape[0] != expected_full_len: + raise ValueError( + f"route length {experts.shape[0]} does not match engine sequence " + f"length {expected_full_len}" + ) + message["routed_experts"] = encode_routed_experts(experts[prev_len:]) + except (IndexError, TypeError, ValueError) as error: + LOGGER.warning( + "dropping invalid routed_experts from staged capture: %s", error + ) + message.pop("routed_experts", None) + choice["message"] = message + payload["choices"] = [choice] + + def _finish_request_capture(self, request: Any, content: dict) -> dict: + """Stage the finished call and ride its coords on the response. + + Fail-closed (§ 3.5): the sink write happens inside complete_call — + the coords exist only after the bytes are durable, and any capture + failure degrades to capture_failed coords without breaking the + completion. Token ids and logprobs are stripped: the staged delta is + the only token store on this path, so the worker->gate hop carries + text + delta ids + coords only (§ 3.2). + """ + state = self._capture_calls.pop(id(request), None) + if state is None: + return content + call, prompt_token_ids = state + payload = dict(content) + # vLLM's OpenAI response carries no prompt ids; the adapter reads the + # preprocess-time engine prompt off the payload (see + # nemo_gym.token_id_capture.adapters.vllm.extract_prompt_ids). + payload["prompt_token_ids"] = prompt_token_ids + adapter = self.token_capture.adapter + if adapter is not None: + try: + generated_token_ids, _ = adapter.extract_generation(payload) + except Exception: # capture core will report the authoritative failure + generated_token_ids = [] + self._delta_align_routed_experts( + payload, + prev_len=call.admission.prev_len, + prompt_len=len(prompt_token_ids), + generated_len=len(generated_token_ids), + ) + coords = self.token_capture.complete_call_from_response(call, payload) + for choice in content.get("choices") or []: + choice.pop("logprobs", None) + content["ng_commit_coords"] = coords.model_dump() + return content + + def _abort_request_capture(self, request: Any, *, reason: str) -> None: + """Drop the in-flight capture state for a request that errored.""" + state = self._capture_calls.pop(id(request), None) + if state is not None and self.token_capture is not None: + self.token_capture.fail_call(state[0], reason=reason) + # ruff: noqa def _setup_vllm_openai_api_server(self, app: FastAPI) -> FastAPI: + worker_self = self from copy import deepcopy from logging import Filter as LoggingFilter from logging import LogRecord @@ -503,7 +710,16 @@ async def preprocess_chat( ) raise - if ( + # Check staging_chain in ng_capture before required_prefix_token_ids. + ng_capture_dict = getattr(request, "ng_capture", None) or {} + chain_prefix_token_ids = worker_self._patch_chain_prefix( + ng_capture_dict + ) + + if chain_prefix_token_ids is not None: + # External staging, token-in mode: fetch prefix from TQ. + model_prefix_token_ids = chain_prefix_token_ids + elif ( not hasattr(request, "required_prefix_token_ids") or request.required_prefix_token_ids is None ): @@ -514,8 +730,16 @@ async def preprocess_chat( actual_request_max_tokens, res[1][0]["prompt_token_ids"], ) + # Token capture, text mode: the full render is the exact + # engine prompt. + worker_self._begin_request_capture( + request, res[1][0]["prompt_token_ids"] + ) return res + else: + model_prefix_token_ids = list(request.required_prefix_token_ids) + # Token-in splice path — shared by staging_chain and direct prefix. last_assistant_message_idx = None for i in reversed(range(len(messages_for_replace_prefix_tokens))): if messages_for_replace_prefix_tokens[i]["role"] == "assistant": @@ -555,7 +779,7 @@ async def preprocess_chat( final_prompt_token_ids = replace_prefix_tokens( tokenizer=self.renderer.tokenizer, - model_prefix_token_ids=request.required_prefix_token_ids, + model_prefix_token_ids=model_prefix_token_ids, template_prefix_token_ids=actual_corresponding_token_ids, template_token_ids=engine_prompt["prompt_token_ids"], ) @@ -570,6 +794,10 @@ async def preprocess_chat( final_prompt_token_ids, ) + # Token capture, token-in mode: the spliced prompt is the + # exact engine prompt. + worker_self._begin_request_capture(request, final_prompt_token_ids) + return res ######################################## @@ -581,6 +809,9 @@ class NeMoRLChatCompletionRequest( NeMoRLOpenAIChatRequestMixin, ChatCompletionRequest ): required_prefix_token_ids: Optional[List[int]] = None + # Ledger-authoritative token capture: the call identity the ledger + # attaches (rollout_id, call_id, parent_call_id, prev_len, mode). + ng_capture: Optional[dict[str, Any]] = None # vLLM 0.25 routes both /v1/chat/completions and /tokenize through # OnlineRenderer.preprocess_chat, so the prefix-token override @@ -745,6 +976,7 @@ async def create_chat_completion( # max_model_len during tokenization, instead of returning an # ErrorResponse. Convert to HTTP 400 so the Gym proxy can # detect context-length overflow and handle it gracefully. + worker_self._abort_request_capture(request, reason="context_length") return JSONResponse( content={ "error": { @@ -755,19 +987,26 @@ async def create_chat_completion( }, status_code=400, ) + except BaseException: + worker_self._abort_request_capture(request, reason="engine_error") + raise if isinstance(generator, ErrorResponse): + worker_self._abort_request_capture(request, reason="error_response") return JSONResponse( content=generator.model_dump(), status_code=generator.error.code ) elif isinstance(generator, ChatCompletionResponse): - return JSONResponse( - content=model_dump_chat_response_with_dynamic_message_fields( - generator - ) + content = model_dump_chat_response_with_dynamic_message_fields( + generator ) + # Token capture: stage the delta and ride the coords on the + # response; strips logprobs/ids (no-op when capture is off). + content = worker_self._finish_request_capture(request, content) + return JSONResponse(content=content) + worker_self._abort_request_capture(request, reason="streaming_response") return StreamingResponse(content=generator, media_type="text/event-stream") ######################################## @@ -899,6 +1138,13 @@ def _setup_vllm_server(self) -> "tuple[threading.Thread, str, uvicorn.Server]": base_url = f"http://{node_ip}:{free_port}/v1" print(f"Starting server on {base_url}") + byte_dir = os.environ.get("NRL_HTTP_BYTES_DIR") + if byte_dir: + # Perf-measurement tooling only (see nemo_rl/utils/http_byte_counter.py). + from nemo_rl.utils.http_byte_counter import HttpByteCounterMiddleware + + app = HttpByteCounterMiddleware(app, "vllm_worker", byte_dir) # type: ignore[assignment] + config = uvicorn.Config( app, host="0.0.0.0", diff --git a/nemo_rl/models/megatron/router_replay.py b/nemo_rl/models/megatron/router_replay.py index 0dc3dfb8c23..6ffe0b6e4cd 100644 --- a/nemo_rl/models/megatron/router_replay.py +++ b/nemo_rl/models/megatron/router_replay.py @@ -122,6 +122,18 @@ def _global_moe_layer_numbers(model_config: Any) -> list[int]: return [layer_idx + 1 for layer_idx, is_moe in enumerate(pattern) if is_moe] +def router_replay_dimensions(model_config: Any) -> tuple[int, int]: + """Return model-owned ``(num_moe_layers, top_k)`` route dimensions.""" + num_moe_layers = len(_global_moe_layer_numbers(model_config)) + top_k = int(getattr(model_config, "moe_router_topk")) + if num_moe_layers <= 0 or top_k <= 0: + raise ValueError( + "router replay requires positive route dimensions, got " + f"num_moe_layers={num_moe_layers}, top_k={top_k}" + ) + return num_moe_layers, top_k + + def _router_replay_instances_for_model(model: Any) -> list[tuple[Any, int]]: instances: list[tuple[Any, int]] = [] seen: set[int] = set() diff --git a/nemo_rl/models/policy/tq_policy.py b/nemo_rl/models/policy/tq_policy.py index 13f04d7d77c..6dc3458a18d 100644 --- a/nemo_rl/models/policy/tq_policy.py +++ b/nemo_rl/models/policy/tq_policy.py @@ -45,6 +45,8 @@ DP_TRAIN_FIELDS, GLOBAL_FORWARD_PAD_SEQLEN, LP_SEED_FIELDS, + ROUTE_PASSTHROUGH_FLAG, + ROUTE_PLAN_TAG, fields_with_optional_routed_experts, ) from nemo_rl.distributed.batched_data_dict import BatchedDataDict @@ -267,6 +269,38 @@ def _packing_args( return args, None return None, None + def _with_route_fields( + self, + meta: KVBatchMeta, + base_fields: tuple[str, ...], + *, + task_name: str, + want_routes: bool, + ) -> KVBatchMeta: + """Resolve direct versus deferred route storage for one worker request.""" + want = self._router_replay_enabled and want_routes + plan_presence = [ROUTE_PLAN_TAG in tag for tag in (meta.tags or [])] + if want and any(plan_presence) and not all(plan_presence): + raise RuntimeError( + "router replay does not support mixed direct/deferred route " + "storage in one worker fetch" + ) + passthrough = bool(want and plan_presence and all(plan_presence)) + extra_info = dict(meta.extra_info or {}) + if passthrough: + extra_info[ROUTE_PASSTHROUGH_FLAG] = True + else: + extra_info.pop(ROUTE_PASSTHROUGH_FLAG, None) + return replace( + meta, + fields=fields_with_optional_routed_experts( + base_fields, + enabled=want and not passthrough, + ), + task_name=task_name, + extra_info=extra_info, + ) + def _logprob_dispatch( self, meta: KVBatchMeta, @@ -289,13 +323,11 @@ def _logprob_dispatch( """ self._stamp_pad_seqlen(meta) spa, dba = self._packing_args("logprob_mb_tokens") - lp_meta = replace( + lp_meta = self._with_route_fields( meta, - fields=fields_with_optional_routed_experts( - LP_SEED_FIELDS, - enabled=self._router_replay_enabled and include_router_replay, - ), + LP_SEED_FIELDS, task_name=task_name, + want_routes=include_router_replay, ) with timer.time(f"{timer_prefix}/shard_meta") if timer else nullcontext(): metas, _ = shard_meta_for_dp( @@ -397,12 +429,11 @@ def train_from_meta( # default ``DP_TRAIN_FIELDS``) must be in TQ before this call — written # by workers + driver delta-writes. Caller may narrow to drop columns # skipped this step (e.g. ``prev_logprobs`` under force_on_policy_ratio). - train_meta = replace( + train_meta = self._with_route_fields( meta, - fields=fields_with_optional_routed_experts( - train_fields, enabled=self._router_replay_enabled - ), + train_fields, task_name="train", + want_routes=True, ) with timer.time("policy_training/shard_meta") if timer else nullcontext(): dp_metas, _ = shard_meta_for_dp( @@ -521,12 +552,11 @@ def train_microbatches_from_meta( """ self._stamp_pad_seqlen(meta) spa, dba = self._packing_args("train_mb_tokens") - train_meta = replace( + train_meta = self._with_route_fields( meta, - fields=fields_with_optional_routed_experts( - DP_TRAIN_FIELDS, enabled=self._router_replay_enabled - ), + DP_TRAIN_FIELDS, task_name="train", + want_routes=True, ) with timer.time("policy_training/shard_meta") if timer else nullcontext(): dp_metas, _ = shard_meta_for_dp( diff --git a/nemo_rl/models/policy/workers/megatron_policy_worker.py b/nemo_rl/models/policy/workers/megatron_policy_worker.py index 3a9ff57dbfe..2c1e6882442 100644 --- a/nemo_rl/models/policy/workers/megatron_policy_worker.py +++ b/nemo_rl/models/policy/workers/megatron_policy_worker.py @@ -69,7 +69,10 @@ broadcast_obj_from_pp_rank, broadcast_tensors_from_last_stage, ) -from nemo_rl.models.megatron.router_replay import router_replay_enabled +from nemo_rl.models.megatron.router_replay import ( + router_replay_dimensions, + router_replay_enabled, +) from nemo_rl.models.megatron.setup import ( finalize_megatron_setup, handle_model_import, @@ -278,6 +281,10 @@ def _local_coords(self) -> dict[str, int]: "pipeline_parallel": parallel_state.get_pipeline_model_parallel_rank(), } + def _routed_experts_dimensions(self) -> tuple[int, int]: + """Return route dimensions from the initialized Megatron model config.""" + return router_replay_dimensions(self._get_model_config()) + def _get_replica_group(self) -> Optional[Any]: """Replica group = TP × CP × PP siblings within this DP rank. diff --git a/nemo_rl/utils/http_byte_counter.py b/nemo_rl/utils/http_byte_counter.py new file mode 100644 index 00000000000..7038d35b053 --- /dev/null +++ b/nemo_rl/utils/http_byte_counter.py @@ -0,0 +1,82 @@ +# Copyright (c) 2026, NVIDIA CORPORATION. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +"""Env-gated per-route HTTP byte counter (``NRL_HTTP_BYTES_DIR``). + +Measurement tooling for the token-capture perf comparison: sums request-body +and response-body bytes per path on the vLLM worker's in-process HTTP server +and periodically flushes an aggregate JSON. Mirrors Gym's +``HttpByteCounterMiddleware`` (separate copy: worker venvs cannot assume +``nemo_gym`` is installed on the legacy path). Never installed unless the env +var is set. +""" + +import json +import os +from typing import Any, Awaitable, Callable + +Scope = dict[str, Any] +Message = dict[str, Any] +Receive = Callable[[], Awaitable[Message]] +Send = Callable[[Message], Awaitable[None]] + + +class HttpByteCounterMiddleware: + """Pure ASGI wrapper counting per-path request/response body bytes.""" + + FLUSH_EVERY = 25 + + def __init__(self, app: Any, server_name: str, out_dir: str) -> None: + self.app = app + self.out_path = os.path.join(out_dir, f"{server_name}_{os.getpid()}.json") + os.makedirs(out_dir, exist_ok=True) + self.counts: dict[str, list[int]] = {} + self._events = 0 + + def _flush(self) -> None: + with open(self.out_path, "w") as f: + json.dump( + { + path: { + "requests": c[0], + "req_bytes": c[1], + "resp_bytes": c[2], + } + for path, c in self.counts.items() + }, + f, + ) + + async def __call__(self, scope: Scope, receive: Receive, send: Send) -> None: + if scope["type"] != "http": + return await self.app(scope, receive, send) + entry = self.counts.setdefault(scope["path"], [0, 0, 0]) + entry[0] += 1 + + async def counting_receive() -> Message: + message = await receive() + if message["type"] == "http.request": + entry[1] += len(message.get("body", b"")) + return message + + async def counting_send(message: Message) -> None: + if message["type"] == "http.response.body": + entry[2] += len(message.get("body", b"")) + await send(message) + + try: + await self.app(scope, counting_receive, counting_send) + finally: + self._events += 1 + if self._events % self.FLUSH_EVERY == 0: + self._flush() diff --git a/nemo_rl/utils/venvs.py b/nemo_rl/utils/venvs.py index fa5b8acb738..a5a3eca9692 100644 --- a/nemo_rl/utils/venvs.py +++ b/nemo_rl/utils/venvs.py @@ -94,7 +94,9 @@ def create_local_venv( exec_cmd.extend(["echo", f"Finished creating venv {venv_path}"]) # Always run uv sync first to ensure the build requirements are set (for --no-build-isolation packages) - subprocess.run(["uv", "sync", "--directory", git_root], env=env, check=True) + subprocess.run( + ["uv", "sync", "--locked", "--directory", git_root], env=env, check=True + ) subprocess.run(exec_cmd, env=env, check=True) # Return the path to the python executable in the virtual environment @@ -172,6 +174,15 @@ def create_local_venv_on_each_node(py_executable: str, venv_name: str): ray.get(pg.ready()) force_rebuild = os.environ.get("NRL_FORCE_REBUILD_VENVS", "false").lower() == "true" + # NRL_FORCE_REBUILD_VENVS_LIST: comma-separated venv names to rebuild even + # when the global flag is off — for containers whose baked venvs are only + # partially compatible with the checked-out branch. + rebuild_list = { + name + for name in os.environ.get("NRL_FORCE_REBUILD_VENVS_LIST", "").split(",") + if name + } + force_rebuild = force_rebuild or venv_name in rebuild_list # Launch one actor per node actors = [ _env_builder.options(placement_group=pg).remote( diff --git a/nemo_rl/weight_sync/nccl_reshard_utils.py b/nemo_rl/weight_sync/nccl_reshard_utils.py index 8dd978e6772..f4914ac3136 100644 --- a/nemo_rl/weight_sync/nccl_reshard_utils.py +++ b/nemo_rl/weight_sync/nccl_reshard_utils.py @@ -191,9 +191,13 @@ def is_nccl_reshard_param(param_name: str) -> bool: ``load_weights`` path. Shared-expert FFN weights (``*.shared_expert.*``) are routed to misc path. + Bare ``mtp.``-prefixed HF names are routed to misc too, because vLLM keeps + the MTP drafter separate and updates it through ``load_weights``. """ if "shared_expert" in param_name: return False + if param_name.startswith("mtp."): + return False return param_name.endswith(FFN_PROJ_WEIGHT_SUFFIXES) or param_name.endswith( FFN_GROUPED_EXPERT_SUFFIXES ) diff --git a/pyproject.toml b/pyproject.toml index f290d42d8e5..8a08cb91190 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -265,7 +265,11 @@ test = [ tensorrt-llm = { path = "3rdparty/TensorRT-LLM-workspace" } nemo-automodel = { path = "3rdparty/Automodel-workspace/Automodel", editable = true } megatron-bridge = { path = "3rdparty/Megatron-Bridge-workspace/Megatron-Bridge", editable = true } -nemo_gym = { workspace = true } +# Keep Gym as an editable path dependency rather than a workspace member. Both +# projects define a ``vllm`` extra with intentionally different server stacks; +# uv applies a requested extra name to every selected workspace member, which +# makes the RL vLLM + base Gym worker environment impossible to select. +nemo_gym = { path = "3rdparty/Gym-workspace/Gym", editable = true } nemo_run = { git = "https://github.com/NVIDIA-NeMo/Run", rev = "414f0077c648fde2c71bb1186e97ccbf96d6844c" } # torch/torchvision/triton all come from the torch index in order to pick up aarch64 wheels torch = [ @@ -289,7 +293,6 @@ flashinfer-jit-cache = { index = "flashinfer-cu130" } [tool.uv.workspace] members = [ - "3rdparty/Gym-workspace/Gym", # Research projects are also added here in order for them to share the global root level uv.lock. # If we don't do this, the research projects do not see the global uv.lock, and may mistakenly # install numpy>=2.0 because nemo-rl's core [dependencies] do not pin numpy, but when you inspect @@ -442,26 +445,6 @@ constraint-dependencies = [ exclude-dependencies = ["nvidia-cutlass-dsl-libs-base"] conflicts = [ - [ - { package = "nemo-rl", extra = "automodel" }, - { package = "nemo-gym", extra = "vllm" }, - ], - [ - { package = "nemo-rl", extra = "vllm" }, - { package = "nemo-gym", extra = "vllm" }, - ], - [ - { package = "nemo-rl", extra = "sglang" }, - { package = "nemo-gym", extra = "vllm" }, - ], - [ - { package = "nemo-rl", extra = "mcore" }, - { package = "nemo-gym", extra = "vllm" }, - ], - [ - { package = "nemo-rl", extra = "trtllm" }, - { package = "nemo-gym", extra = "vllm" }, - ], [ { extra = "fsdp" }, { extra = "sglang" }, diff --git a/pyrefly.toml b/pyrefly.toml index e0c35ebf542..71eddf3b3d1 100644 --- a/pyrefly.toml +++ b/pyrefly.toml @@ -9,6 +9,7 @@ replace-imports-with-any = [ "tensorrt_llm.*", "vllm.*", "math_verify.*", + "nemo_gym.*", "sympy.*", "torchdata.*", "torchaudio.*", @@ -123,6 +124,7 @@ project-includes = [ "nemo_rl/data_plane/observability.py", "nemo_rl/data_plane/preshard.py", "nemo_rl/data_plane/schema.py", + "nemo_rl/data_plane/tq_token_sink.py", "nemo_rl/data_plane/worker_mixin.py", "nemo_rl/distributed/__init__.py", "nemo_rl/distributed/collectives.py", @@ -143,11 +145,15 @@ project-includes = [ "nemo_rl/evals/__init__.py", "nemo_rl/evals/answer_parsing.py", "nemo_rl/experience/__init__.py", + "nemo_rl/experience/blackbox_finalizer.py", + "nemo_rl/experience/finalizer_actor.py", + "nemo_rl/experience/row_dump.py", "nemo_rl/experience/interfaces.py", "nemo_rl/experience/metric_utils.py", "nemo_rl/experience/payload.py", "nemo_rl/experience/rollout_manager.py", "nemo_rl/experience/rollouts.py", + "nemo_rl/experience/route_plan.py", "nemo_rl/modelopt/__init__.py", "nemo_rl/modelopt/models/__init__.py", "nemo_rl/modelopt/models/generation/__init__.py", @@ -214,6 +220,7 @@ project-includes = [ "nemo_rl/utils/config.py", "nemo_rl/utils/fastokens.py", "nemo_rl/utils/grad_norm.py", + "nemo_rl/utils/http_byte_counter.py", "nemo_rl/utils/multimodal_payload_metrics.py", "nemo_rl/utils/native_checkpoint.py", "nemo_rl/utils/nsys.py", diff --git a/reports/auto_research/lineage-ledger-0820/experiments.tsv b/reports/auto_research/lineage-ledger-0820/experiments.tsv new file mode 100644 index 00000000000..41a7158da39 --- /dev/null +++ b/reports/auto_research/lineage-ledger-0820/experiments.tsv @@ -0,0 +1,16 @@ +index branch parent_commit commit recipe metric_name metric_value elapsed_min launcher job_id command log_path status description +1 autoresearch/2026-08-20-lineage-ledger/impl c9602d447 1f9af7987 nano_swe_teacher_sc smoke_5step_dynamics PENDING swe batch 6358267 DRY_RUN=0 SC_EXP_NAME=ledger-capture-s43-0820 NG_TIC_FP_CANONICAL=1 WALLTIME=3:59:00 WANDB_PROJ=PR3456-Ledger-AB-0820 bash swe_nano_sc_capture.sh grpo.max_num_steps=5 grpo.seed=43 grpo.num_prompts_per_step=2 policy.train_global_batch_size=8 token_capture.num_finalizer_workers=2 token_capture.defer_routed_experts_to_policy=true token_capture.staging_partition=rollout_staging_ledger_smoke_0820 +env.nemo_gym.policy_model.responses_api_models.vllm_model.num_workers=2 +policy.router_replay.enabled=true async_rl.sampler.name=windowed +async_rl.sampler.max_staleness_versions=1 +env.nemo_gym.model_endpoint_readiness_timeout_seconds=1800 policy.generation.vllm_cfg.reasoning_parser_plugin=/opt/nemo-rl/nemo_rl/models/generation/vllm/reasoning_parsers/nano_v3_reasoning_parser.py /lustre/fsw/portfolios/llmservice/users/pthombre/sweRun/RL-pr3456-delta-smoke-workspace/results/ledger-capture-s43-0820 running Capture arm: ledger-authoritative token capture + R3, num_workers=2, seed 43, 5 steps +2 autoresearch/2026-08-20-lineage-ledger/impl c9602d447 890a485c7 nano_swe_teacher_sc smoke_5step_dynamics PENDING swe batch 6358268 DRY_RUN=0 SC_EXP_NAME=ledger-legacy-s43-0820 WALLTIME=3:59:00 WANDB_PROJ=PR3456-Ledger-AB-0820 bash swe_nano_sc.sh grpo.max_num_steps=5 grpo.seed=43 grpo.num_prompts_per_step=2 policy.train_global_batch_size=8 +env.nemo_gym.policy_model.responses_api_models.vllm_model.num_workers=2 +policy.router_replay.enabled=true async_rl.sampler.name=windowed +async_rl.sampler.max_staleness_versions=1 +env.nemo_gym.model_endpoint_readiness_timeout_seconds=1800 policy.generation.vllm_cfg.reasoning_parser_plugin=/opt/nemo-rl/nemo_rl/models/generation/vllm/reasoning_parsers/nano_v3_reasoning_parser.py /lustre/fsw/portfolios/llmservice/users/pthombre/sweRun/RL-pr3456-delta-smoke-workspace/results/ledger-legacy-s43-0820 running Legacy arm: token-echo path + R3, num_workers=2, seed 43, 5 steps +3 autoresearch/2026-08-20-lineage-ledger/impl 890a485c7 (venv+legacy fix) nano_swe_teacher_sc smoke_5step_dynamics FAILED swe batch 6358267 (same as row 1) .../ledger-capture-s43-0820 crash setup_token_capture ModuleNotFoundError orjson: NRL_FORCE_REBUILD_VENVS_LIST unconsumed in this tree (stale node venv). Fixed in venvs.py. +4 autoresearch/2026-08-20-lineage-ledger/impl 890a485c7 (venv+legacy fix) nano_swe_teacher_sc smoke_5step_dynamics FAILED swe batch 6358268 (same as row 2) .../ledger-legacy-s43-0820 crash Pre-existing legacy bug: tool-call-only output item content=None -> TypeError in invalid-tool-call detector (also killed 0819 legacy runs). Fixed. +5 autoresearch/2026-08-20-lineage-ledger/impl 890a485c7 84feac48b nano_swe_teacher_sc smoke_5step_dynamics PASS 5/5 tmpe 1.0137-1.0147 gen_kl ~0.001 token_in_rate 0.95-0.99 59 swe batch 6359951 SC_EXP_NAME=ledger-capture-s43-0820-r2 ... (row 1 overrides, staging partition r3) .../ledger-capture-s43-0820-r2 keep Capture arm PASS: dynamics tighter than legacy; 16/40 rollouts fail-closed placeholders (per-call aborts; retry-idempotency follow-up); see ledger-ab-report.md +6 autoresearch/2026-08-20-lineage-ledger/impl 890a485c7 84feac48b nano_swe_teacher_sc smoke_5step_dynamics PASS 5/5 tmpe 1.020-1.066 gen_kl ~0.004 all-8-valid 67 swe batch 6359952 SC_EXP_NAME=ledger-legacy-s43-0820-r2 ... (row 2 overrides) .../ledger-legacy-s43-0820-r2 keep Legacy arm PASS: looser dynamics (echo re-tokenization); no rollout loss; see ledger-ab-report.md +7 autoresearch/2026-08-20-lineage-ledger/impl 5bc2b50d1 b39dbd8cd nano_swe_teacher_sc smoke_5step_dynamics PASS 5/5 valid 32/40 (was 24/40) tmpe 1.0137-1.0171 72 swe batch short-QOS 6365594 SC_EXP_NAME=ledger-capture-s43-0820-r3 WALLTIME=1:59:00 ... (row 5 overrides, staging partition r4) .../ledger-capture-s43-0820-r3 keep Carve-out validated: +8 valid rows; residue 9x missing_terminal_row (2 first-call deaths + ~7 harness-reported doomed-attempt terminal ids); dynamics unchanged +8 autoresearch/2026-08-20-lineage-ledger/mw-smoke-revalidation b39dbd8cd 02d700023 rlvr_dolphin_sc mw_smoke_revalidation CRASH-setup 14 dolphin batch short-QOS 6368085 DRY_RUN=0 bash nano35_ledger_smoke.sh .../ray_logs/nano35-rlvr-sc-tc-smoke-ledger/6368085-logs crash SC checkpointing NotImplementedError in this tree; recipe had checkpointing.enabled=true. Fixed: checkpointing.enabled=false override. 409/UnknownRollout greps trivially 0 (died pre-gym). +9 autoresearch/2026-08-20-lineage-ledger/mw-smoke-revalidation b39dbd8cd rlvr_dolphin_sc mw_smoke_revalidation CRASH-setup 18 dolphin batch short-QOS 6368547 DRY_RUN=0 bash nano35_ledger_smoke.sh .../6368547-logs crash uv sync --locked failed in _env_builder: examples ultra_launch overlaid rebased Gym pyproject without mounting worktree uv.lock. Ported root launcher mount block. +10 autoresearch/2026-08-20-lineage-ledger/mw-smoke-revalidation b39dbd8cd rlvr_dolphin_sc mw_smoke_revalidation CRASH-setup 22 dolphin batch short-QOS 6370717 DRY_RUN=0 bash nano35_ledger_smoke.sh .../6370717-logs crash nccl_reshard refit-info assert: mtp.-prefixed FFN weights matched bulk-reshard whitelist (layer_prefix mismatch mtp != backbone). Ported upstream mtp guard to is_nccl_reshard_param. +11 autoresearch/2026-08-20-lineage-ledger/mw-smoke-revalidation b39dbd8cd rlvr_dolphin_sc mw_smoke_revalidation PARTIAL-PASS mw-regression 0x409 0xUnknownRollout 47x100%-collections 4 syncs 3/3 steps 52 dolphin batch short-QOS 6371991 DRY_RUN=0 bash nano35_ledger_smoke.sh .../6371991-logs keep Multi-worker gate regression GONE at num_workers=16/4. BUT finalize/invalid_row_rate=1.0: every rollout rejected missing_receipt (terminal_logical_request_id->manifest handoff empty), loss=0 on placeholders. User has a known fix; standing by. +12 autoresearch/2026-08-20-lineage-ledger/mw-smoke-revalidation 02d700023 c8ca62e7f rlvr_dolphin_sc mw_smoke_revalidation PASS 5/5 valid 638/640 (3x unresolved_parent only) tmpe 1.028-1.038 loss nonzero 0x409 0xUnknownRollout 58 dolphin batch short-QOS 6381019 DRY_RUN=0 NRL_MAX_STEPS=5 SC_EXP_NAME=nano35-rlvr-sc-tc-smoke-ledger-heur bash nano35_ledger_smoke.sh /lustre/fsw/portfolios/llmservice/users/pthombre/sweRun/RL-pr3456-delta-smoke-workspace/ray_logs/nano35-rlvr-sc-tc-smoke-ledger-heur/6381019-logs keep Heuristic terminal selection validated e2e: zero missing_receipt (row-12 was 128/128 per step), num_valid_samples 126,128,128,128,128; rewards 0.41-1.37; multi-worker regression still absent. Gym 3117ca30+0edca059, RL c8ca62e7f. +13 autoresearch/2026-08-23-token-chain/baseline 91fdfbbf4 e8494630c nano_swe_teacher_sc smoke_5step_dynamics PASS 5/5 valid 40/40 tmpe 1.021-1.032 gen_kl 0.00220-0.00352 57 swe batch 6465030 DRY_RUN=0 USE_SNAPSHOT=0 SC_EXP_NAME=token-chain-baseline-s43-0823 WANDB_PROJ=PR3456-Token-Chain-AB-0823 bash swe_nano_sc.sh grpo.max_num_steps=5 grpo.seed=43 grpo.num_prompts_per_step=2 policy.train_global_batch_size=8 +env.nemo_gym.policy_model.responses_api_models.vllm_model.num_workers=2 +policy.router_replay.enabled=true async_rl.sampler.name=windowed +async_rl.sampler.max_staleness_versions=1 +env.nemo_gym.model_endpoint_readiness_timeout_seconds=1800 policy.generation.vllm_cfg.reasoning_parser_plugin=/opt/nemo-rl/nemo_rl/models/generation/vllm/reasoning_parsers/nano_v3_reasoning_parser.py /lustre/fsw/portfolios/llmservice/users/pthombre/sweRun/RL-pr3456-delta-smoke-workspace/ray_logs/token-chain-baseline-s43-0823/6465030-logs keep Matched no-token-capture baseline PASS: Slurm 0:0 in 56:30, W&B dyb1em3g, finite loss and 8 valid rows every step, 0 stale/aborted groups, 0 fatal/integrity errors. +14 autoresearch/2026-08-23-token-chain/capture c8ca62e7f 91fdfbbf4 nano_swe_teacher_sc smoke_5step_dynamics PASS 5/5 valid 31/40 tmpe 1.011-1.016 gen_kl 0.00074-0.00109 token_in 0.981-0.989 53 swe batch 6465058 DRY_RUN=0 USE_SNAPSHOT=0 SC_EXP_NAME=token-chain-capture-s43-0823 NG_TIC_FP_CANONICAL=1 WANDB_PROJ=PR3456-Token-Chain-AB-0823 bash swe_nano_sc_capture.sh grpo.max_num_steps=5 grpo.seed=43 grpo.num_prompts_per_step=2 policy.train_global_batch_size=8 token_capture.num_finalizer_workers=2 token_capture.defer_routed_experts_to_policy=true token_capture.staging_partition=rollout_staging_token_chain_0823_capture +env.nemo_gym.policy_model.responses_api_models.vllm_model.num_workers=2 +policy.router_replay.enabled=true async_rl.sampler.name=windowed +async_rl.sampler.max_staleness_versions=1 +env.nemo_gym.model_endpoint_readiness_timeout_seconds=1800 policy.generation.vllm_cfg.reasoning_parser_plugin=/opt/nemo-rl/nemo_rl/models/generation/vllm/reasoning_parsers/nano_v3_reasoning_parser.py /lustre/fsw/portfolios/llmservice/users/pthombre/sweRun/RL-pr3456-delta-smoke-workspace/ray_logs/token-chain-capture-s43-0823/6465058-logs keep Chain-in-request PASS: Slurm 0:0 in 53:09, W&B wiyqqjau; 3440 committed nodes + 13 explicit max-context failure records across 44 ledgers, chain max 200, zero chain/fetch/staging/coordinate errors. Nine trained missing-terminal placeholders were fail-closed; all 5 steps retained 5-8 valid rows. +15 autoresearch/2026-08-23-token-chain/capture 91fdfbbf4 6d12d9663+6061dadb nano_swe_teacher_sc smoke_5step_dynamics PASS 5/5 valid 5-8 token_in 0.968-0.986 0 integrity errors custody=3081 chain_hash=3081 cum_tok=0 51 6490451 DRY_RUN=0 SC_EXP_NAME=swe-token-free-ledger-s43-0824 NG_TIC_FP_CANONICAL=1 WANDB_PROJ=nemo-rl-swe-research bash swe_nano_sc_capture.sh grpo.max_num_steps=5 grpo.seed=43 grpo.num_prompts_per_step=2 policy.train_global_batch_size=8 token_capture.num_finalizer_workers=2 token_capture.defer_routed_experts_to_policy=true token_capture.staging_partition=rollout_staging_token_free_s43_0824 +env.nemo_gym.policy_model.responses_api_models.vllm_model.num_workers=2 +policy.router_replay.enabled=true async_rl.sampler.name=windowed +async_rl.sampler.max_staleness_versions=1 /lustre/fsw/portfolios/llmservice/users/pthombre/sweRun/RL-pr3456-delta-smoke-workspace/ray_logs/swe-token-free-ledger-s43-0824/6490451-logs running Token-free ledger smoke: custody rows must have no cumulative_token_ids, non-null chain_hash/cumulative_hash, token_in≈1, zero chain/staging/coordinate errors. Gym 6061dadb + NeMo-RL 6d12d9663. diff --git a/reports/auto_research/lineage-ledger-0820/ledger-ab-report.md b/reports/auto_research/lineage-ledger-0820/ledger-ab-report.md new file mode 100644 index 00000000000..11ba6ff9ace --- /dev/null +++ b/reports/auto_research/lineage-ledger-0820/ledger-ab-report.md @@ -0,0 +1,129 @@ +# Lineage-Ledger Token Capture — 5-Step Smoke A/B (2026-08-20) + +**Goal:** validate the gate→ledger replacement (`token-capture-lineage-ledger-approach.md`) +end to end: external token capture ON (ledger) vs OFF (legacy token echo), both with +router replay (R3) enabled and the Gym policy model server at `num_workers=2`. + +**Setup** (identical across arms, per `docs/guides/nano-swe-token-capture.md`): +Nemotron-3-Nano-30B-A3B on 6 GB200 nodes (train 4 / gen 2), `grpo.seed=43`, +`num_prompts_per_step=2`, GBS 8, 5 steps, `+policy.router_replay.enabled=true`, +windowed sampler `max_staleness_versions=1`. Capture arm additionally: +`token_capture.enabled=true`, `defer_routed_experts_to_policy=true`, +`num_finalizer_workers=2`, `NG_TIC_FP_CANONICAL=1`. +Code: RL `autoresearch/2026-08-20-lineage-ledger/impl`, Gym `token-capture-lineage-ledger`. +W&B: `PR3456-Ledger-AB-0820`. + +| Arm | Job | Result | +|---|---|---| +| capture (ledger) | 6359951 | COMPLETED, 5/5 steps, 58:44 | +| legacy (token echo) | 6359952 | COMPLETED, 5/5 steps, 1:07:03 | + +## Training dynamics (per step, 1→5) + +| Metric | Capture (ledger) | Legacy (echo) | +|---|---|---| +| `token_mult_prob_error` | 1.0147, 1.0145, 1.0137, 1.0144, 1.0147 | 1.0199, 1.0319, 1.0657, 1.0347, 1.0283 | +| `gen_kl_error` | 0.00090, 0.00092, 0.00100, 0.00117, 0.00106 | 0.00216, 0.00403, 0.00371, 0.00377, 0.00384 | +| `reward` (mean) | 0, 0, 0.25, 0, 0 | 0, 0, 0.375, 0, 0 | +| `probs_ratio` | 1.0 all steps | 1.0 all steps | + +**Verdict: training dynamics are similar and the capture arm is strictly tighter** — +the same exact-token + exact-route property the gate-based runs showed +(prior campaign bands: capture 1.012–1.017 / ~0.001 vs legacy 1.023–1.027 / ~0.003). +Both arms are far below the `seq_logprob_error_threshold=2.0` health gate, rewards +spike on the same step (3), and the ratio stays pinned at 1.0. + +## Ledger health (capture arm) + +- **2,821 ledger rows across 44 rollouts; zero `unresolved_parent`, zero + `worker_capture_failed`, zero `invalid_worker_commit_coordinates`.** Lineage + resolution and tri-state admission worked flawlessly across 2 uvicorn workers + (no request affinity): `finalize/token_in_rate` 0.954–0.987 per group + (last step: 306 token-in calls vs 4 text roots), `routed_experts_row_coverage=1.0`. +- **16 poison rows, all `request_finished_without_staged_coordinates`** → + 16/40 trained rollouts masked as placeholders (`invalid_row_rate` 0–0.625 per + group; `num_valid_samples` 3–8 of 8). Root cause: ~0.57% of model calls + (16/2,821) finish without a worker ack — agent-side aborts/timeouts plus two + engine `ClientOSError` retry bursts — and one such call fail-closes its whole + rollout. With ~64 calls/rollout this amplifies to ~36% rollout loss. This is + the *designed* fail-closed behavior (identical semantics to the gate's + `fail_call`), and the documented retry-idempotency follow-up (harness-minted + logical ids + deterministic `model_call_id`) is the recovery path. The legacy + arm keeps such rollouts because re-tokenized echo text doesn't need per-call + acks — at the cost of the looser dynamics above. + +## Perf + +| Metric (step 5 / summary) | Capture | Legacy | +|---|---|---| +| `timing/train/total_step_time` (s) | 392.4 | 573.3 | +| `timing/train/exposed_generation` (s) | 368.6 | 542.3 | +| `timing/train/policy_training` (s) | 9.5 | 14.4 | +| `timing/train/valid_tokens_per_sec_per_gpu` | 6.84 | 8.08 | +| Row assembly (finalizer, per group, ms) | fetch+verify+linearize 250–650; tensorize 11–21; tq_put 41–44; total 313–725 | n/a (inline echo path) | +| `finalize/queue_wait_ms` | ≤0.013 | n/a | +| Wall clock, 5 steps e2e | 58:44 | 1:07:03 | + +Reading: the capture arm's off-hot-path finalizer costs are sub-second per group +and its queue never backs up. Its lower `valid_tokens_per_sec_per_gpu` is an +artifact of masked placeholder rows (fewer valid tokens per identical step), not +slower machinery — total step time is *shorter* than legacy, partly because +poisoned rollouts contribute less generation. At smoke scale (2 prompts/step) +these arms are not a rigorous throughput comparison; the prior campaign's +15-step pairs remain the perf reference. + +## Fixes landed to get here (all committed on the impl branch) + +1. `venvs.py`: `NRL_FORCE_REBUILD_VENVS_LIST` was never consumed in this tree — + every 08-19/08-20 capture smoke (8 jobs) died on stale worker venvs + (`orjson` missing in `setup_token_capture`). +2. `nemo_gym.py`: tool-call-only assistant items (`content: None`) crashed the + legacy postprocess (`TypeError`) — killed every legacy run on this branch; + regression tests added. +3. `swe_nano.env`: `UV_CACHE_DIR_OVERRIDE` must not mount over the prefetch-venvs + container's `/root/.cache/uv` (severs baked-venv hardlinks). Driver uv cache + pinned to `/lustre/fsw/portfolios/llmservice/users/pthombre/uv` (never /tmp). +4. `transfer_queue.py`: ported `NRL_TQ_SKIP_RUNTIME_ENV_PIN`. +5. `swe_nano_sc.sh`: honour `SC_EXP_NAME`. + +## Follow-ups + +- Retry idempotency (plan's explicit deferral): recover the ~36% rollout + poisoning from per-call aborts. +- Investigate the vLLM engine `ClientOSError` bursts (engine 13015) — connection + drops under concurrent capture traffic. +- CI (`/ok to test`) to run the Ray-heavy RL suites (blackbox_finalizer, + tq_token_sink, vllm hosting) that cannot run outside the container. + +## Addendum: r3 rerun with the off-chain carve-out (job 6365594) + +`_assemble_receipt` now ignores `request_finished_without_staged_coordinates` +failure rows off the terminal chain (commit b39dbd8cd). Capture arm rerun, +same posture, short QOS, COMPLETED 5/5 in 1:11:49. + +| Metric | r2 (blanket poison) | r3 (carve-out) | +|---|---|---| +| valid rows / 40 | 24 (60%) | **32 (80%)** | +| rejection reason | 16× failure-row poison | 9× missing_terminal_row only | +| `token_mult_prob_error` | 1.0137–1.0147 | 1.0137–1.0171 | +| `gen_kl_error` | ~0.001 | ~0.001 | +| `finalize/token_in_rate` | 0.95–0.99 | 0.98–0.99 | +| ledger census | 2,821 rows / 16 uncommitted failures | 3,525 rows / 19 uncommitted failures | + +The carve-out worked exactly as designed: zero rollouts were rejected for +failure rows. The remaining 9 `missing_terminal_row` rejections split into: + +- 2 rollouts whose *first* call died (empty ledger — nothing to train; + correctly masked). +- ~7 overflow rollouts where the terminal id OpenHands reported does not + match any committed row: the harness derives it from the last llm_completion + file (`swe_agents/app.py:3101`), which for these episodes corresponds to the + doomed final attempt rather than the last successful completion. (In the + other ~9 overflow rollouts this run, the last file was the last successful + call and they trained — the carve-out's win.) + +Options for the residue (not taken here): (a) harness-side — report the last +*successful* response id when the final call errors; (b) receipt-side — fall +back to the deepest committed row when the terminal is missing and all +failures are uncommitted. (b) weakens the agent-kept-response attestation, so +(a) is the better follow-up alongside retry idempotency. diff --git a/session/20260820_004547/handoff.md b/session/20260820_004547/handoff.md new file mode 100644 index 00000000000..6d06541a1f3 --- /dev/null +++ b/session/20260820_004547/handoff.md @@ -0,0 +1,18 @@ +# Handoff + +## Resume From Here +CAMPAIGN COMPLETE. The lineage-ledger plan is implemented, unit-tested, and validated by a passing 5-step smoke A/B. + +- Gym branch `token-capture-lineage-ledger` (head: lint fix on 1e1a16cb): gate deleted, ledger implemented, 436 tests green. +- RL branch `autoresearch/2026-08-20-lineage-ledger/impl` (head 5bc2b50d1): receipt assembly from manifest, gate plumbing removed, ledger metrics in finalizer, launch-infra fixes (venv rebuild-list, UV cache, content-None legacy fix, TQ pin skip, SC_EXP_NAME), docs + design doc, A/B report. +- Smoke A/B (seed 43, R3 on, num_workers=2, 5 steps): capture 6359951 PASS (58:44), legacy 6359952 PASS (1:07:03). Capture dynamics strictly tighter (tmpe 1.0137-1.0147 vs 1.020-1.066; gen_kl ~0.001 vs ~0.004). Ledger: 2821 rows, 0 unresolved/worker failures, token_in_rate 0.95-0.99. +- Full report: reports/auto_research/lineage-ledger-0820/ledger-ab-report.md; ledger TSV in the same dir. + +## Next Actions +- Push both branches and update Gym PR #2278 / RL PR #3456; run CI (/ok to test) — CI covers the Ray-heavy RL suites that cannot run outside the container. +- Follow-up items (report §Follow-ups): retry idempotency (recovers ~36% fail-closed rollout loss from per-call aborts), vLLM engine ClientOSError bursts, ledger-file drop on publish. + +## Watch Outs +- swe_nano.env now targets pthombre paths + nightly-gym 08-10; never set UV_CACHE_DIR_OVERRIDE with prefetch-venvs containers; uv cache = /lustre/fsw/portfolios/llmservice/users/pthombre/uv (never /tmp). +- swe_nano.secrets.env is untracked (git-excluded via .git/info/exclude), mode 600. +- Lustre git operations intermittently time out (~2 min) — retry with timeout. diff --git a/session/20260820_004547/session_state.md b/session/20260820_004547/session_state.md new file mode 100644 index 00000000000..5db2c73442b --- /dev/null +++ b/session/20260820_004547/session_state.md @@ -0,0 +1,37 @@ +# Session State + +- Session: 20260820_004547 +- Repo: /lustre/fs1/portfolios/llmservice/projects/llmservice_fm_text/users/pthombre/sweRun/RL-pr3456-delta-staging +- Branch: autoresearch/2026-08-18-nano35-dolphin-delta-r3/stage1-treatment-gate (start) +- Started: 2026-08-20 00:45 +- Updated: 2026-08-20 00:45 + +## Goal +Implement `token-capture-lineage-ledger-approach.md` (replace Gym's RolloutCaptureGate/GateStateStore with a LineageStore-based capture ledger; full gate deletion; NeMo RL companion changes). Validate via unit tests, then run a 5-step smoke A/B (external token capture on vs off, router replay enabled, Gym model server num_workers>1) and compare training dynamics + perf metrics per docs/guides/nano-swe-token-capture.md. uv cache: /lustre/fsw/portfolios/llmservice/users/pthombre/uv (never /tmp). + +## Current Subtask +Reading target code paths in Gym + RL (task #1). + +## Loaded Skills +- `nemo-rl-auto-research` — campaign workflow, branching, TSV ledger, stop rules. +- `nemo-rl-session-memory` — this record. + +## Current Status +- RL worktree has pre-existing UNCOMMITTED gate→ledger doc/comment renames across 12 files (verified: no functional changes) + untracked docs/design-docs/rollout-verification-boundary.md and two plan MDs. These are prep for this plan; carry them onto the impl branch. +- Gym checkout (3rdparty/Gym-workspace/Gym) clean at 10b34908 on token-capture-worker-custody-rebased; gate.py/gate_store.py still present. +- Prior campaign reference: /lustre/.../sweRun/RL/reports/auto_research/swe-r3-capture (CAMPAIGN.md read — smoke pattern: nano_swe_teacher_sc.yaml, pps=2/GBS=8/3-5 steps, Slurm, shared uv cache via NRL_UV_CACHE_DIR, NG_TIC_FP_CANONICAL=1, metrics token_mult_prob_error/gen_kl). + +## Plan +- [ ] #1 Read code paths, verify plan line refs +- [ ] #2 Gym ledger implementation + gate deletion + tests +- [ ] #3 NeMo RL companion changes +- [ ] #4 Unit tests green +- [ ] #5 5-step smoke A/B (capture on/off, R3 on, num_workers>1) +- [ ] #6 Compare dynamics + perf + +## Assumptions +- Smoke runs go to Slurm via the pattern in nano-swe-token-capture.md / prior campaign (slurm-broker MCP available). +- Pre-existing dirty rename files are intentional prep and should be committed with the work. + +## Blockers +- None known. diff --git a/session/20260820_004547/timeline.md b/session/20260820_004547/timeline.md new file mode 100644 index 00000000000..17939ac0db3 --- /dev/null +++ b/session/20260820_004547/timeline.md @@ -0,0 +1,36 @@ +# Timeline + +## 2026-08-20 00:45 +- User asked: implement token-capture-lineage-ledger-approach.md, unit-test, then 5-step smoke A/B (capture on/off, R3 on, num_workers>1), compare dynamics+perf. +- Context gathered: plan doc; CAMPAIGN.md of prior swe-r3-capture campaign; Gym gate/lineage/sink/control_routes/records/rebuild/config/base_responses_api_model/vllm app; RL nemo_gym.py/blackbox_finalizer/rollout_manager/single_controller. +- Key findings: + - Pre-existing uncommitted RL changes = pure gate→ledger doc renames (safe to carry). + - `_validate_manifest_graph` (staging/rebuild.py:159) tolerates dead-branch rows → plan's step-8 open check resolved: no pruning needed in receipt assembly. + - RL vLLM worker imports only `token_id_capture.staging.*` → worker path untouched by gate deletion. + - Lineage JSONL files were never cleaned in gate mode either → skipping ledger-file drop is no regression (documented follow-up). + - Gate metrics route dies with the gate → replace with finalizer-derived mode counters (manifest rows carry mode) logged as finalize/token_in_rate. +- Design decisions: + - New Gym config flag `token_id_capture.external_staging: bool` replaces `gate:` block (extra="forbid" on TokenIdCaptureSettings makes leftover gate keys fail loudly). Requires enabled + rebuild_response=false; keeps control_auth_token_env for manifest-route bearer auth. + - Ledger row: existing 6 lineage keys + parent_call_id/staging_key/weight_version/prev_len/delta_len/cum_len/extras_digest/mode/logical_request_id + staging_digest (coords.digest; distinct from lineage digest). Failure rows: model_call_id+failure_reason, no fingerprint (resolve uses .get so they can't match). + - CallRecord gains optional logical_request_id; new RolloutManifest/ManifestFailure wire models in staging/records.py; manifest route GET /training-token-capture/rollouts/{id}/manifest. + - CaptureContext: staging_gate/data_capability deleted; adds external_staging bool + request_items (stashed in resolve_parent for commit-time record()). + - Data-capability plumbing fully removed (rollout_correlation, server_utils, base agents, swe sandbox mount); swe sandbox token-capture URL prefix re-keyed to agent-level token_id_capture enabled. + - Abandoned-rollout staging rows: accept leak until partition teardown (plan option b), fail_rollouts deleted. +- Result: starting Gym implementation. +- 2026-08-20 01:39:46 Gym side committed (1e1a16cb): ledger implemented, gate deleted, 436 Gym tests green (ledger 23, capture 112, swe 158, vllm 143). Starting RL companion. + +## 2026-08-20 06:30 +- RL companion committed (1f9af7987 + launcher fixes 890a485c7, uv-cache fix, Gym pin bumps; Gym lint fix 74d4ef85). +- Smoke A/B submitted: capture=6358267, legacy=6358268 (first pair 6357295/6357320 failed/cancelled — UV_CACHE_DIR_OVERRIDE mount severed the container venv's hardlinked packages; removed from swe_nano.env). +- RL unit tests: receipt-assembly 6/6 green (--noconftest). Ray-fixture suites hang on the login node (conftest autouse init_ray + TQ actors); moved to compute-node job 6359381. + +## 2026-08-20 07:40 +- Unit-test scope closed: Gym capture suites all green (436); RL receipt-assembly (6) + rollout-manager (16) green. Ray-heavy RL suites (blackbox_finalizer/tq_token_sink/vllm hosting) cannot run on login node (conftest autouse Ray wedge) nor bare/pyxis compute shells (uv-run worker env drift; /opt/nemo_rl_venv is ray.sub-materialized). They are exercised end-to-end by the smoke and will run in CI. +- Ported NRL_TQ_SKIP_RUNTIME_ENV_PIN into transfer_queue.py (committed) while diagnosing. +- Smoke r2: capture 6359951 TRAINING (step 2/5 at ~37 min — venv rebuild-list fix confirmed). Legacy 6359952 still queued. + +## 2026-08-20 09:10 +- BOTH SMOKE ARMS PASSED 5/5 steps. Capture: tmpe 1.0137-1.0147, gen_kl ~0.001, token_in_rate 0.95-0.99, 2821 ledger rows / 0 unresolved / 0 worker failures; 16 rollouts fail-closed on aborted calls (request_finished_without_staged_coordinates — retry-idempotency follow-up). Legacy: tmpe 1.020-1.066, gen_kl ~0.004, all 8 samples valid every step. Capture strictly tighter — matches gate-era campaign bands. +- Review of legacy fix done via /review-pr (local): fix correct; added 2 regression params to test_nemo_gym_utils (7/7 pass); renamed _content -> item_content. +- Report: reports/auto_research/lineage-ledger-0820/ledger-ab-report.md. +- 2026-08-20 11:29:36 r3 rerun 6365594 COMPLETED 5/5 (1:11:49): carve-out validated — valid rows 32/40 (was 24/40), zero failure-row rejections; residue 9x missing_terminal_row (2 first-call deaths + ~7 doomed-attempt terminal ids from harness). Dynamics unchanged (tmpe 1.0137-1.0171, gen_kl ~0.001). diff --git a/session/20260820_111240/handoff.md b/session/20260820_111240/handoff.md new file mode 100644 index 00000000000..69568d8d4f1 --- /dev/null +++ b/session/20260820_111240/handoff.md @@ -0,0 +1,66 @@ +# Handoff + +## Current Final State (2026-08-24 ~10:00 PDT) +Token-free ledger implementation committed (Gym `6061dadb`, NeMo-RL `6d12d9663`). +Gym unit tests 217/218 (1 pre-existing timeout). NeMo-RL hosting tests untestable +on login node (vllm/torch need GPU); changes correct by inspection. +Slurm smoke **6490178** (PENDING) submitted for end-to-end ledger row verification. +Chain-in-request validation campaign complete (rows 13–14). Sections below preserve history. + +## Resume From Here +Multi-worker ledger revalidation job **6368085** (het 6368085-6368086) submitted 2026-08-20 11:24, PENDING on batch/short. Branch `autoresearch/2026-08-20-lineage-ledger/mw-smoke-revalidation` (3 commits: launcher port 53713adeb, wrapper cbadca28b, ultra_launch fixes 02d700023). Mirrors failed job 6300221 with num_workers=16/4 intact. + +## Next Actions (2026-08-24) +- Poll: `squeue -j 6490178` (token-free ledger smoke); once done: + ```bash + OUTPUT_LOG=$(find /lustre/fsw/portfolios/llmservice/users/pthombre/sweRun/RL-pr3456-delta-smoke-workspace/results/swe-token-free-ledger-s43-0824 -path '*/wandb/run-*/files/output.log' -print | head -1) + grep -c '^step_metrics=' "$OUTPUT_LOG" # must be 5 + grep -nE 'chain_hash_mismatch|cumulative_hash_mismatch|invalid_worker_commit_coordinates|worker_capture_failed|409 Conflict|UnknownRolloutError' /lustre/fsw/.../ray_logs/swe-token-free-ledger-s43-0824/6490178-logs/ray-driver.log # must be 0 + ``` + JSONL custody row check (key new verification): + ```bash + python3 -c " + import json, glob + for f in glob.glob('/lustre/fsw/.../results/swe-token-free-ledger-s43-0824/*/gym_token_capture/lineage/*.lineage.jsonl'): + rows = [json.loads(l) for l in open(f) if l.strip() and '{' in l] + ext = [r for r in rows if r.get('staging_key')] + if ext: + print(f, len(ext), 'ext rows') + print(' cumulative_token_ids:', sum(1 for r in ext if r.get('cumulative_token_ids'))) + print(' chain_hash:', sum(1 for r in ext if r.get('chain_hash'))) + " + ``` + Expected: cumulative_token_ids count = 0, chain_hash count > 0 for every ledger file. +- Update experiments.tsv row 15 with result and mark keep/discard/crash. +- Update session/20260820_111240/token_free_ledger_progress.md with Slurm verdict. + +## OLD Next Actions (from lineage-ledger campaign — superseded) +- Poll: `squeue -j 6368085`; once running, watch + `/lustre/fsw/portfolios/llmservice/users/pthombre/sweRun/RL-pr3456-delta-smoke-workspace/ray_logs/nano35-rlvr-sc-tc-smoke-ledger/6368085-logs/ray-driver.log` +- Verdict greps on driver log: `grep -c '409 Conflict'` (must be 0), `grep -c UnknownRolloutError` (must be 0), `Collecting rollouts` reaches 16/16, >=2 steps + nccl_reshard sync, `capture_dir/lineage/*.lineage.jsonl` populated, no `manifest(...) fetch failed`. +- On verdict: add row to reports/auto_research/lineage-ledger-0820/experiments.tsv. + +## Watch Outs +- UV cache must stay /lustre/fsw/portfolios/llmservice/users/pthombre/uv — never /tmp, never UV_CACHE_DIR_OVERRIDE with prefetch containers. +- Do NOT pin num_workers=1 — 16/4 is the regression trigger under test. +- rollout_checkpointing.* overrides are ignored (extra-allow) in this tree — expected. +- Secrets in swe_nano.secrets.env (untracked, mode 600). + +## 2026-08-23 Chain Validation Resume Point +Run focused preflight checks for the uncommitted chain-in-request feature, then +commit it on a dedicated experiment branch and launch matched five-step +`swe_nano_sc_capture.sh` and `swe_nano_sc.sh` arms. The user explicitly wants a +real training smoke, not `test_chain_prefix_smoke.py`. Append every attempt and +failure/fix to `reports/auto_research/lineage-ledger-0820/experiments.tsv`. + +Jobs submitted: baseline `6465030`, capture `6465058`. Poll both with `squeue`. +Ray logs are under `.../ray_logs/token-chain-{baseline,capture}-s43-0823/-logs`. +Do not modify tracked source while these live-checkout jobs are pending/running. + +Final verdict at 04:44 PDT: baseline `6465030` (W&B `dyb1em3g`) and capture +`6465058` (W&B `wiyqqjau`) both passed 5/5 and exited 0:0. Capture produced +3,440 committed nodes across 44 ledgers, 98%+ token-in activity, chains up to +length 200, and zero chain/TQ/staging/commit integrity errors. It trained 31/40 +rows; nine max-context terminal calls were explicitly fail-closed. Experiment +rows 13–14 and the report addendum contain the full comparison. No relaunch or +code fix remains. diff --git a/session/20260820_111240/session_state.md b/session/20260820_111240/session_state.md new file mode 100644 index 00000000000..ac8a6ac8302 --- /dev/null +++ b/session/20260820_111240/session_state.md @@ -0,0 +1,41 @@ +# Session State + +- Session: 20260820_111240 +- Repo: /lustre/fs1/portfolios/llmservice/projects/llmservice_fm_text/users/pthombre/sweRun/RL-pr3456-delta-staging (fsw twin: /lustre/fsw/portfolios/llmservice/users/pthombre/sweRun/RL-pr3456-delta-staging) +- Branch: autoresearch/2026-08-20-lineage-ledger/mw-smoke-revalidation +- Started: 2026-08-20 11:12 +- Updated: 2026-08-20 11:25 + +## Goal +Rerun the failed nano-35-rlvr-sc-tc-smoke shape (job 6300221, amahishi) with identical params — critically policy_model num_workers=16 / reasoning_off 4 — on the lineage-ledger token-capture code, to prove the multi-worker gate failure (409 Conflict + UnknownRolloutError from the per-process LineageRegistry) cannot recur. + +## Current Subtask +Submit the smoke job and monitor to verdict. + +## Loaded Skills +- `launch-nemo-rl` — debugging playbook (k8s-focused; this run is Slurm). +- `nemo-rl-auto-research` — campaign workflow, TSV ledger, branching. +- `nemo-rl-session-memory` — this record. + +## Current Status +- Root cause confirmed: job 6300221's gate registry was per-uvicorn-process; register PUT hit 1 of 16 workers, data-plane calls load-balanced → 409 at prepare, UnknownRolloutError at ingest after fail_rollouts cleanup. Current code (Gym 1e1a16cb ledger) has no registration and uses FileLineageStore under a shared root with fcntl locks. +- Ported examples/nemo_gym/nemotron-3.5-nano/ (5 files) from nemo-rl-partial-rollout-recovery-refresh (commit 53713adeb). +- Wrote nano35_ledger_smoke.sh batch wrapper (commit cbadca28b): pthombre paths, zhiyul nightly-gym.2026-08-10 container, proven 0820 launch plumbing, UV_CACHE_DIR=/lustre/fsw/portfolios/llmservice/users/pthombre/uv (user directive: NEVER /tmp). +- Fixed examples/nemo_gym/nemotron-3-ultra/ultra_launch.sh (commit 02d700023): NRL_ENTRYPOINT, NRL_DRIVER_PIP_INSTALL/PYTHONPATH, NRL_DRIVER_UV_RUN_FLAGS, external UV_CACHE_DIR precedence. +- Dry run clean: SC driver + --locked --no-sync, lustre UV cache, num_workers 16/4 from student_rlvr1.yaml defaults, all model/data/judge paths readable. + +## Plan +- [x] Port launchers, write wrapper, dry-run +- [ ] DRY_RUN=0 submit; record job id +- [ ] Monitor ray-driver.log; verdict greps +- [ ] Record row in reports/auto_research/lineage-ledger-0820/experiments.tsv + +## Pass Criteria +0x "409 Conflict", 0x UnknownRolloutError in driver log; Collecting rollouts reaches 16/16; >=2 steps incl. nccl_reshard sync; lineage/*.lineage.jsonl populated; no manifest fetch failures. + +## Assumptions +- rollout_checkpointing.* overrides are extra-allowed no-ops in this tree (MasterConfig extra="allow"). +- Container deviation (nightly-gym.2026-08-10 vs amahishi's bake) is required for this branch's validated venv plumbing. + +## Blockers +- None known. diff --git a/session/20260820_111240/token_free_ledger_progress.md b/session/20260820_111240/token_free_ledger_progress.md new file mode 100644 index 00000000000..cb6b8ae8846 --- /dev/null +++ b/session/20260820_111240/token_free_ledger_progress.md @@ -0,0 +1,96 @@ +# Token-free ledger implementation — progress checkpoint (2026-08-24) + +Plan: `session/20260820_111240/token_free_ledger_plan.md` (all phases implemented and committed). + +## Source changes complete (all in Gym; NeMo-RL source untouched by design) + +- `staging/digest.py`: `_CHAIN_DIGEST_DOMAIN` + `compute_chain_hash(parent_chain_hash, token_ids_delta)`. +- `staging/records.py`: `CaptureAdmission.parent_chain_hash` (required for token_in, forbidden for text); + `CommitCoords` — `token_ids_delta` REMOVED (hard drop, no compat), `chain_hash`/`cumulative_hash` + required when staged, forbidden when capture_failed. +- `staging/capture.py complete_call`: computes both hashes, binds into staging digest + StagedCallRecord, + returns them on CommitCoords. +- `protocols.py`: `LineageMatch.chain_hash` added. +- `lineage.py`: custody columns + `_CUSTODY_FIELDS` gain chain_hash/cumulative_hash; `LineageNode.chain_hash`; + `RolloutLineage.record` takes explicit `cum_len` + `chain_hash`; custody rows written token-free in BOTH + stores (File omits the JSONL key; InMemory indexes empty cum_tokens); `_resolve` falls back to + `record.get("cumulative_token_ids") or ()`; manifest rows carry the hashes. +- `sink.py`: `CaptureContext.parent_chain_hash`; `resolve_parent` passes it into the admission; a resolved + external parent without chain_hash fails admission validation → poison row (fail closed). +- `responses_api_models/vllm_model/app.py`: commit hook no longer builds `cumulative` or calls + `compute_digest` (import removed); records `[]` tokens, digest = coords.cumulative_hash, passes both hashes. +- `staging/rebuild.py verify_and_linearize`: incremental chain-hash verification during the terminal-chain + walk (`chain_hash_mismatch`), terminal-only cumulative check (`cumulative_hash_mismatch`); absent hashes skip. +- `staging/__init__.py`: exports `compute_chain_hash`. + +**Gym commit**: `6061dadb feat(token-capture): token-free custody ledger via chained digest` + +## Test changes complete + +- Gym `test_token_capture_staging_worker.py`: `_child()` has parent_chain_hash; coords assertions moved to + hashes; delta assertions moved to sink records. +- Gym `test_token_capture_staging_core.py`: admission parent_chain_hash cases; coords built with hashes, + no delta; missing-chain_hash rejection. +- Gym `test_token_capture_staging_rebuild.py`: `_snapshot` takes chain/cumulative hashes; 4 new tests + (chained verifies, broken link, terminal cumulative mismatch, hash-free legacy). +- Gym `test_token_capture_ledger.py`: custody fixture carries hashes; token-free record calls; resolve + asserts empty tokens + prev_len + chain_hash; staging-chain growth test chains hashes; new legacy-row + test (hand-written JSONL row resolves with tokens, cannot anchor a chain → poison). +- NeMo-RL `tests/unit/data_plane/token_capture_test_fixtures.py`: `_record` computes real chain/cumulative + hashes; child chains from root. +- NeMo-RL `test_vllm_token_capture_hosting.py`: token_in ng_capture dicts carry parent_chain_hash; round-trip + asserts coords are token-free with hashes. + +**NeMo-RL commit**: `6d12d9663 test(token-capture): update fixtures and hosting test for token-free coords` + +## Test results + +- Gym suite: **217/218 passed** (1 pre-existing spawned-worker timeout, unchanged baseline). +- NeMo-RL hosting tests: not run on login node (vllm/torch need GPU). Changes are minimal, correct by + inspection: `token_ids_delta` not in coords (CommitCoords schema verified), `chain_hash`/`cumulative_hash` + present (validator enforces them), `parent_chain_hash` required for token_in (validator enforces it). + Previous session ran 13/13 on a GPU node; login-node environment hangs on `import vllm`. + +## Environment notes + +- Root `.venv` was missing `typing_extensions` → installed (`uv pip install -p .venv/bin/python typing_extensions`). +- Gym suites must run from `3rdparty/Gym-workspace/Gym` with `Gym/.venv` (namespace test spawns bare + subprocesses that import nemo_gym). +- Pre-fix baseline: 8 expected failures across core+worker suites; ledger/rebuild passed. +- Login-node constraint: `rl-pr3456-delta-tests` venv hangs on `import ray`; Gym venv lacks torch/transformers. + NeMo-RL hosting tests require GPU node (inside training container). + +## Slurm smoke — COMPLETE (2026-08-24) + +Job 6490451 (short QOS, 50:57 elapsed), W&B nmiv7zjc. + +- **5/5 step_metrics** — all steps completed, finite loss, 5–8 valid rows each step. +- **token_in_rate 0.968–0.986** — ≈1 as expected. +- **Zero integrity errors** — no chain_hash_mismatch, cumulative_hash_mismatch, + invalid_worker_commit_coordinates, worker_capture_failed, 409 Conflict, or UnknownRolloutError. +- **Custody rows token-free** — 45 ledgers, 3,081 external rows: + - `cumulative_token_ids` present: **0** (expected 0) ✓ + - `chain_hash` present: **3,081** (expected == total) ✓ + +Implementation is fully validated end-to-end. All source, test, and Slurm checks passed. + +## OLD REMAINING: Slurm smoke + +1. Submit: `swe_nano_sc_capture.sh` with SC_EXP_NAME=swe-token-free-ledger-s43-0824, + token_capture.staging_partition=rollout_staging_token_free_s43_0824, seed 43, 2 prompts/step, GBS 8, + max_num_steps=5. (Dry run in progress 2026-08-24 ~10:00 PDT.) +2. Verify: 5/5 step_metrics, token_in rate ≈1, zero chain/prefix/staging/coordinate errors. +3. **Key new check**: custody JSONL rows have NO `cumulative_token_ids` key but non-null + `chain_hash` and `cumulative_hash` values. + ```bash + grep -l '.' "$WORKSPACE/results/swe-token-free-ledger-s43-0824/*/gym_token_capture/lineage/*.lineage.jsonl" | + head -1 | xargs python3 -c " + import sys, json + rows = [json.loads(l) for l in open(sys.argv[1]) if l.strip()] + ext = [r for r in rows if r.get('staging_key')] + print(f'external rows: {len(ext)}') + print(f'with cumulative_token_ids: {sum(1 for r in ext if r.get(\"cumulative_token_ids\"))}') + print(f'with chain_hash: {sum(1 for r in ext if r.get(\"chain_hash\"))}') + " + ``` + Expected: `with cumulative_token_ids: 0`, `with chain_hash: N > 0`. diff --git a/swe_nano.env b/swe_nano.env new file mode 100644 index 00000000000..484439cf4d1 --- /dev/null +++ b/swe_nano.env @@ -0,0 +1,149 @@ +# ============================================================================= +# Nemotron 3 NANO v3 (30B-A3B) — SWE Teacher — MINIMAL 5-node smoke — GB200 +# ============================================================================= +# Small-scale SWE: swaps Ultra 550B for Nano v3 30B-A3B and shrinks to 6 nodes +# (train 4 + gen 2), GBS 64, 49k context. Shared by the nano SWE launchers; +# each one overrides CONFIG_PATH (and the entrypoint) for its variant: +# +# swe_nano_sc[_interactive].sh async GRPO via SingleController, TQ honoured (legacy arm) +# swe_nano_sc_capture[_interactive].sh same, with gate-authoritative token capture +# +# See docs/guides/nano-swe-transferqueue.md for which code paths actually +# honour the data plane and for the verified 5-step run. +# +# ----------------------------------------------------------------------------- +# READ vs WRITE directories — WHAT TO CHANGE WHEN YOU REUSE THIS RECIPE +# ----------------------------------------------------------------------------- +# The "shared read-only" block below (container images, SWE dataset, SIF images) +# is world-readable on Lustre — REUSE IT AS IS. Do not copy those multi-GB +# artifacts into your own tree. +# +# The "per-user write" block is owned by zhiyul and is NOT writable by you. +# Point every variable in that block at a directory you own before launching. +# ============================================================================= +export NRL_FORCE_REBUILD_VENVS=false + +# The 2026-07-26 ultra-recipes container's baked NemoGym venv has no usable +# nemo_gym install; rebuild only that one by default (identical on both A/B +# arms). The capture launcher extends this list. +export NRL_FORCE_REBUILD_VENVS_LIST="${NRL_FORCE_REBUILD_VENVS_LIST:-nemo_rl.environments.nemo_gym.NemoGym}" + +# Secrets (WANDB_API_KEY, HF_TOKEN) live in an untracked mode-600 file so they +# never enter git history. Create swe_nano.secrets.env next to this file. +_SECRETS="$(dirname "${BASH_SOURCE[0]}")/swe_nano.secrets.env" +# shellcheck disable=SC1090 +[ -f "${_SECRETS}" ] && source "${_SECRETS}" +export NCCL_MAX_NCHANNELS=1 +export NCCL_NVLS_ENABLE=0 + +# ----------------------------------------------------------------------------- +# SHARED READ-ONLY — reuse as is (world-readable; nothing to copy or rebuild) +# ----------------------------------------------------------------------------- +# Training container (GB200 aarch64, prefetched venvs) and the nemo-skills +# sandbox image. Both are plain squashfs files read by pyxis at job start. +# nightly-gym 08-10: vLLM 0.25.1 (the 07-26 bake's vLLM 0.20 routed-experts +# capturer overflows its slot buffer on hybrid Nemotron at scale). +export CONTAINER=/lustre/fsw/portfolios/llmservice/users/zhiyul/enroot-images/nvcr.io+nvidian+nemo-rl+nightly-gym.2026-08-10.squashfs +export SANDBOX_CONTAINER=/lustre/fsw/portfolios/coreai/users/cye/enroot/nemo-rl:skills-sandbox-latest.squashfs + +# SWE prompt set (7816 instances). Read-only; no need to duplicate. +export DATA_DIR=/lustre/fsw/portfolios/llmservice/users/pthombre/sweRun/RL-pr3456-delta-smoke-workspace/data + +# SWE rewards come from executing each instance in its apptainer .sif image +# (no GPU judges). Expects swerebench/ and swegym/ subdirectories. +export SIF_DIR=/lustre/fsw/portfolios/llmservice/users/sdevare/images + +# Nano v3 30B-A3B — HF repo id (Nemotron model → matches the nano_v3 reasoning +# parser in swe_teacher.yaml). Downloaded to HF_HOME on first run (~60 GB BF16); +# export HF_TOKEN before launching if the repo is gated (also makes the download +# much faster — the verified run downloaded unauthenticated and was rate limited). +export MODEL_PATH=nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16 + +# ----------------------------------------------------------------------------- +# PER-USER WRITE — CHANGE THESE. They are zhiyul-owned and not writable by you. +# Substitute your own Lustre paths (any writable location works): +# CODE_DIR your checkout of this branch; mounted into the container, +# so it must be the tree you are actually editing +# WORKSPACE_DIR results, ray logs, checkpoints (grows to tens of GB) +# HF_HOME HuggingFace cache — ~60 GB for the Nano BF16 checkpoint +# PERSISTENT_CACHE vLLM/Triton/Inductor compile caches reused across jobs +# ----------------------------------------------------------------------------- +# Code from this checkout (fsw path of the fs1 delta-staging tree). +export CODE_DIR=/lustre/fsw/portfolios/llmservice/users/pthombre/sweRun/RL-pr3456-delta-staging +export WORKSPACE_DIR=/lustre/fsw/portfolios/llmservice/users/pthombre/sweRun/RL-pr3456-delta-smoke-workspace + +export HF_HOME=/lustre/fsw/portfolios/llmservice/users/pthombre/hf_cache +export PERSISTENT_CACHE=/lustre/fsw/portfolios/llmservice/users/pthombre/persistent_cache +export NRL_MEGATRON_CHECKPOINT_DIR=${PERSISTENT_CACHE}/megatron_ckpt_cache + +# Slurm identity — change SLURM_ACCOUNT to an account you can charge. +export SLURM_PARTITION=batch +export SLURM_ACCOUNT=nemotron_sw_post +export GPUS_PER_NODE=4 # GB200 NVL72 + +# Nano SWE shape mirroring the Qwen3-30B-A3B MoE mesh (6 nodes total): +# Train 4 nodes (16 GPUs) = TP2 · PP2 · CP4 · DP1 (no EP) +# Gen 2 nodes ( 8 GPUs) = vLLM TP2 -> 4 engines (non-colocated) +# Gym 0 nodes = code-execution SIF rewards +export NUM_TRAIN_NODES=4 +export NUM_GEN_NODES=2 +export NUM_GYM_NODES=0 +# Total = 6; SEGMENT_SIZE must divide it (sbatch --segment). The nano .inc sets +# cluster.segment_size=null to disable the virtual-cluster NVLink segment filter. +export SEGMENT_SIZE=6 +# 3:59:00 is the max for the `batch` partition. 1h is NOT enough: the first run +# downloads ~60 GB from HF and converts to Megatron before the first rollout. +export WALLTIME=3:59:00 +# Deliberately NOT the `short` QOS. short gives higher priority but caps walltime +# at 2h and enforces a per-user node limit — a 6-node job of ours got held with +# Reason=QOSMaxNodePerUserLimit while a large job of the same user was running. +# ultra_launch.sh only auto-selects short when WALLTIME < 2h, so leaving this +# empty submits with no --qos. Set SLURM_QOS=short (and WALLTIME under 2h) if you +# want the priority boost and know your node budget is free. +export SLURM_QOS= +export NRL_MAX_STEPS="${NRL_MAX_STEPS:-5}" + +# Nano base checkpoint: MTP off, container's stock vLLM (no Ultra fork needed). +export ENABLE_MTP_INFERENCE=0 +export USE_CUSTOM_VLLM=0 + +# Mount RL-copy's own Gym (writable; SWE/OpenHands harness writes here at runtime). +export EXTRA_MOUNTS=/lustre:/lustre,${CODE_DIR}/3rdparty/Gym-workspace/Gym:/opt/nemo-rl/3rdparty/Gym-workspace/Gym +export WANDB_PROJ=${WANDB_PROJ:-nano-swe-smoke} + +# Start safe: inspect the resolved launch command before submission. +export DRY_RUN=1 +export USE_SNAPSHOT=0 + +# Default variant = async GRPO baseline. The tq/sc launchers override both. +# EXP_NAME namespaces results/, ray_logs/ and the W&B run — rename it per user. +export EXP_NAME=nano-swe-ledger-smoke-pthombre +export CONFIG_PATH=${CODE_DIR}/examples/configs/ultra/nano_swe_teacher_qwen3mesh.yaml +export TRAIN_PATH=${DATA_DIR}/swe.jsonl +export VAL_PATH=${DATA_DIR}/swe.jsonl + +export RESULTS_DIR=${WORKSPACE_DIR}/results/${EXP_NAME} +export BASE_LOG_DIR=${WORKSPACE_DIR}/ray_logs/${EXP_NAME} + +export NRL_WG_USE_RAY_REF=1 + +# nightly-gym 08-10 bakes TransferQueue 0.1.9 into the base env on every +# node, so the per-actor runtime_env pip pin (a compute-node GitHub fetch) +# is skipped. +export NRL_TQ_SKIP_RUNTIME_ENV_PIN=1 + +# Shared prewarmed uv cache (survives across jobs; removes compute-node +# GitHub dependence for venv rebuilds). Never /tmp: UV_CACHE_DIR rides into +# the driver's TRAIN_CMD (ultra_launch.sh). Do NOT set UV_CACHE_DIR_OVERRIDE +# with the prefetch-venvs containers — it bind-mounts over /root/.cache/uv +# and severs the baked venvs' hardlinked packages (job 6357295: +# ModuleNotFoundError urllib3.exceptions in /opt/nemo_rl_venv). +export UV_CACHE_DIR=/lustre/fsw/portfolios/llmservice/users/pthombre/uv + +# Forced venv rebuilds must install from the container's uv.lock, not +# re-resolve. +export NRL_VENV_SYNC_FROZEN=1 + +# A failed engine weight-sync must crash fast, not wedge the SC for the +# rest of the walltime. +export NRL_REFIT_ERRORS_FATAL=1 diff --git a/swe_nano_sc.sh b/swe_nano_sc.sh new file mode 100644 index 00000000000..8ab297cdce8 --- /dev/null +++ b/swe_nano_sc.sh @@ -0,0 +1,41 @@ +#!/bin/bash +# ============================================================================= +# swe_nano_sc.sh — BATCH launch of the nano SWE SingleController + TransferQueue +# recipe (ray.sub runs the driver directly; no interactive idle, no attach). +# +# Same 6-node shape, entrypoint and config as swe_nano_sc_interactive.sh — the +# driver command ray.sub runs is byte-for-byte the one the interactive path +# writes to -run-cmd.sh. Use this for an unattended reproduction; use the +# interactive script when you expect to iterate on the config, since a cold +# start pays the ~60 GB checkpoint download and Megatron conversion every time. +# +# Run from a NETWORKED shell at the repo root: +# bash swe_nano_sc.sh # DRY_RUN inherited from swe_nano.env (=1): inspect first +# DRY_RUN=0 bash swe_nano_sc.sh # submit +# DRY_RUN=0 bash swe_nano_sc.sh grpo.num_prompts_per_step=2 policy.train_global_batch_size=8 +# +# Logs land in ${WORKSPACE_DIR}/results/${EXP_NAME}/runs/latest/slurm/. +# ============================================================================= +set -euo pipefail +HERE="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" + +# Capture DRY_RUN passed on the command line BEFORE sourcing swe_nano.env, which +# exports DRY_RUN=1 and would otherwise clobber the caller's value. +_DRY_RUN_IN="${DRY_RUN:-}" +_USE_SNAPSHOT_IN="${USE_SNAPSHOT:-}" + +set -a +# shellcheck disable=SC1091 +source "${HERE}/swe_nano.env" + +# --- SingleController + TransferQueue overrides ------------------------------ +EXP_NAME="${SC_EXP_NAME:-nano-swe-sc-tq}" +NRL_ENTRYPOINT="${CODE_DIR}/examples/run_grpo_single_controller.py" +CONFIG_PATH="${CODE_DIR}/examples/configs/ultra/nano_swe_teacher_sc.yaml" +RESULTS_DIR="${WORKSPACE_DIR}/results/${EXP_NAME}" +BASE_LOG_DIR="${WORKSPACE_DIR}/ray_logs/${EXP_NAME}" +[ -n "${_DRY_RUN_IN}" ] && DRY_RUN="${_DRY_RUN_IN}" +[ -n "${_USE_SNAPSHOT_IN}" ] && USE_SNAPSHOT="${_USE_SNAPSHOT_IN}" +set +a + +bash "${HERE}/ultra_launch.sh" "$@" diff --git a/swe_nano_sc_capture.sh b/swe_nano_sc_capture.sh new file mode 100644 index 00000000000..e8f762551c6 --- /dev/null +++ b/swe_nano_sc_capture.sh @@ -0,0 +1,53 @@ +#!/bin/bash +# ============================================================================= +# swe_nano_sc_capture.sh — BATCH launch of the nano SWE SingleController + +# TransferQueue recipe with gate-authoritative token capture enabled. +# +# Batch counterpart of swe_nano_sc_capture_interactive.sh, exactly as +# swe_nano_sc.sh is the batch counterpart of swe_nano_sc_interactive.sh: +# same capture env block (driver PYTHONPATH + orjson, vllm worker venv +# rebuild) and the same two hydra appends; ray.sub runs the driver directly. +# +# Run from a NETWORKED shell at the repo root: +# DRY_RUN=0 SC_EXP_NAME= bash swe_nano_sc_capture.sh [extra overrides] +# Capture-canonical fingerprints: export NG_TIC_FP_CANONICAL=1 (recommended). +# ============================================================================= +set -euo pipefail +HERE="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" + +_DRY_RUN_IN="${DRY_RUN:-}" +_WALLTIME_IN="${WALLTIME:-}" +_USE_SNAPSHOT_IN="${USE_SNAPSHOT:-}" + +set -a +# shellcheck disable=SC1091 +source "${HERE}/swe_nano.env" + +EXP_NAME="${SC_EXP_NAME:-nano-swe-sc-tq-capture}" +NRL_ENTRYPOINT="${CODE_DIR}/examples/run_grpo_single_controller.py" +CONFIG_PATH="${CODE_DIR}/examples/configs/ultra/nano_swe_teacher_sc.yaml" +RESULTS_DIR="${WORKSPACE_DIR}/results/${EXP_NAME}" +BASE_LOG_DIR="${WORKSPACE_DIR}/ray_logs/${EXP_NAME}" + +NRL_FORCE_REBUILD_VENVS_LIST="nemo_rl.environments.nemo_gym.NemoGym,nemo_rl.models.generation.vllm.vllm_worker_async.VllmAsyncGenerationWorker" +NRL_DRIVER_PYTHONPATH="/opt/nemo-rl/3rdparty/Gym-workspace/Gym" +NRL_DRIVER_PIP_INSTALL="orjson" +# Ray is started from the container's prefetched environment. Keep the driver on +# that exact Python/Ray pair; only class-specific worker venvs are rebuilt below. +NRL_DRIVER_UV_RUN_FLAGS="--locked --no-sync" + +# --- Per-call latency breakdown (CALL_TIMING=0 to disable) -------------------- +if [ "${CALL_TIMING:-1}" = "1" ]; then + NRL_CALL_TIMING_DIR="${NRL_CALL_TIMING_DIR:-${WORKSPACE_DIR}/call_timing/${EXP_NAME}}" + NG_CALL_TIMING_DIR="${NG_CALL_TIMING_DIR:-${NRL_CALL_TIMING_DIR}}" + mkdir -p "${NRL_CALL_TIMING_DIR}" +fi +[ -n "${_DRY_RUN_IN}" ] && DRY_RUN="${_DRY_RUN_IN}" +[ -n "${_WALLTIME_IN}" ] && WALLTIME="${_WALLTIME_IN}" +[ -n "${_USE_SNAPSHOT_IN}" ] && USE_SNAPSHOT="${_USE_SNAPSHOT_IN}" +set +a + +bash "${HERE}/ultra_launch.sh" \ + token_capture.enabled=true \ + +env.nemo_gym.rollout_max_attempts_to_avoid_lp_nan=1 \ + "$@" diff --git a/swe_nano_sc_capture_interactive.sh b/swe_nano_sc_capture_interactive.sh new file mode 100755 index 00000000000..509f4a5602b --- /dev/null +++ b/swe_nano_sc_capture_interactive.sh @@ -0,0 +1,52 @@ +#!/bin/bash +# ============================================================================= +# swe_nano_sc_capture_interactive.sh — the CAPTURE arm of the nano SWE TQ A/B: +# swe_nano_sc_interactive.sh + gate-authoritative token capture enabled. +# +# Same 6-node SingleController shape as swe_nano_sc_interactive.sh; adds the +# posture the capture path needs (learned on job 5764598, 2026-08-01): +# token_capture.enabled=true the recipe yaml defines the block (enabled: false) +# defines the key (pydantic default only) +# +env.nemo_gym.rollout_max_attempts_to_avoid_lp_nan=1 +# capture hard-errors otherwise; pin on the +# legacy arm too when running an A/B pair +# NRL_DRIVER_PYTHONPATH driver imports nemo_gym staging records; +# the baked driver venv has no nemo_gym +# NRL_DRIVER_PIP_INSTALL=orjson Gym's token_id_capture/__init__ eagerly +# imports consumer->store->orjson (purity +# gap; PR #2278 feedback) +# VllmAsyncGenerationWorker in NRL_FORCE_REBUILD_VENVS_LIST +# the capture leg needs the VLLM_GYM venv +# (--extra nemo_gym); venv caching is not +# spec-aware (PR #3456 known issue), so a +# legacy-leg venv would be silently reused +# +# Usage (same contract as the other nano launchers): +# bash swe_nano_sc_capture_interactive.sh [hydra overrides...] +# ============================================================================= +set -euo pipefail +HERE="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" + +set -a +# shellcheck disable=SC1091 +source "${HERE}/swe_nano.env" + +EXP_NAME="${SC_EXP_NAME:-nano-swe-sc-tq-capture}" +NRL_ENTRYPOINT="${CODE_DIR}/examples/run_grpo_single_controller.py" +CONFIG_PATH="${CODE_DIR}/examples/configs/ultra/nano_swe_teacher_sc.yaml" +RESULTS_DIR="${WORKSPACE_DIR}/results/${EXP_NAME}" +BASE_LOG_DIR="${WORKSPACE_DIR}/ray_logs/${EXP_NAME}" + +NRL_FORCE_REBUILD_VENVS_LIST="nemo_rl.environments.nemo_gym.NemoGym,nemo_rl.models.generation.vllm.vllm_worker_async.VllmAsyncGenerationWorker" +NRL_DRIVER_PYTHONPATH="/opt/nemo-rl/3rdparty/Gym-workspace/Gym" +NRL_DRIVER_PIP_INSTALL="orjson" +# Ray is started from the container's prefetched environment. Keep the driver on +# that exact Python/Ray pair; only class-specific worker venvs are rebuilt below. +NRL_DRIVER_UV_RUN_FLAGS="--locked --no-sync" +set +a + +INTERACTIVE=1 DRY_RUN=0 INTERACTIVE_WAIT="${INTERACTIVE_WAIT:-1}" \ + bash "${HERE}/ultra_launch.sh" \ + token_capture.enabled=true \ + +env.nemo_gym.rollout_max_attempts_to_avoid_lp_nan=1 \ + "$@" diff --git a/swe_nano_sc_interactive.sh b/swe_nano_sc_interactive.sh new file mode 100644 index 00000000000..5233d56ddfc --- /dev/null +++ b/swe_nano_sc_interactive.sh @@ -0,0 +1,48 @@ +#!/bin/bash +# ============================================================================= +# swe_nano_sc_interactive.sh — nano SWE via SingleController async GRPO with a +# HONOURED TransferQueue data plane. This is the VERIFIED recipe (see +# docs/guides/nano-swe-transferqueue.md). +# +# 6-node nano shape (train 4 / gen 2): +# entrypoint examples/run_grpo_single_controller.py (NOT run_grpo_nemo_gym.py) +# config examples/configs/ultra/nano_swe_teacher_sc.yaml +# +# run_grpo_nemo_gym.py's async path (async_grpo_train) drives an in-memory +# ReplayBuffer and ignores data_plane; only grpo_train_sync and the +# SingleController path put rollouts through TransferQueue. +# +# The entrypoint is spawned by ABSOLUTE path from ${CODE_DIR}: the container +# only has nemo_rl/ and examples/configs mounted over it, so the baked +# examples/ has no run_grpo_single_controller.py. +# +# Run from a NETWORKED shell (the sandbox cannot reach slurmctld): +# bash swe_nano_sc_interactive.sh +# +# On submit it prints: +# bash -attach.sh # shell on the head node (Ray already up) +# source -run-cmd.sh # run the driver; edit + re-source to iterate +# Cancel with: scancel +# +# Extra hydra overrides pass through, e.g. a fast first training step: +# bash swe_nano_sc_interactive.sh grpo.num_prompts_per_step=2 policy.train_global_batch_size=8 +# Keep the invariant num_prompts_per_step × num_generations_per_prompt == +# train_global_batch_size — the SingleController split path enforces it. +# ============================================================================= +set -euo pipefail +HERE="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" + +set -a +# shellcheck disable=SC1091 +source "${HERE}/swe_nano.env" + +# --- override for the SingleController + TransferQueue variant --------------- +EXP_NAME=nano-swe-sc-tq-zhiyul +NRL_ENTRYPOINT="${CODE_DIR}/examples/run_grpo_single_controller.py" +CONFIG_PATH="${CODE_DIR}/examples/configs/ultra/nano_swe_teacher_sc.yaml" +RESULTS_DIR="${WORKSPACE_DIR}/results/${EXP_NAME}" +BASE_LOG_DIR="${WORKSPACE_DIR}/ray_logs/${EXP_NAME}" +set +a + +INTERACTIVE=1 DRY_RUN=0 INTERACTIVE_WAIT="${INTERACTIVE_WAIT:-1}" \ + bash "${HERE}/ultra_launch.sh" "$@" diff --git a/tests/functional/L1_Functional_Tests_SingleController.sh b/tests/functional/L1_Functional_Tests_SingleController.sh index 2a6fad63eea..201ac4cf02a 100755 --- a/tests/functional/L1_Functional_Tests_SingleController.sh +++ b/tests/functional/L1_Functional_Tests_SingleController.sh @@ -36,6 +36,9 @@ run_test() { run_test fast uv run --no-sync bash ./tests/functional/grpo_dp_single_controller.sh run_test fast uv run --no-sync bash ./tests/functional/grpo_async_gym_single_controller.sh +# Token-capture (gate-authoritative) path: same SC+Gym smoke with the gate +# custodying token lineage and the finalizer publishing training rows. +run_test uv run --no-sync bash ./tests/functional/grpo_async_gym_single_controller.sh ++token_capture.enabled=true cd ${PROJECT_ROOT}/tests if compgen -G ".coverage*" > /dev/null; then diff --git a/tests/unit/algorithms/test_advantage_validity.py b/tests/unit/algorithms/test_advantage_validity.py new file mode 100644 index 00000000000..64c1bc829b9 --- /dev/null +++ b/tests/unit/algorithms/test_advantage_validity.py @@ -0,0 +1,54 @@ +# Copyright (c) 2026, NVIDIA CORPORATION. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +"""S4: validity-aware GRPO baseline (token-capture placeholder rows).""" + +import torch + +from nemo_rl.algorithms.advantage_estimator import GRPOAdvantageEstimator + + +def _estimator(**overrides) -> GRPOAdvantageEstimator: + config = {"use_leave_one_out_baseline": False, "normalize_rewards": False} + config.update(overrides) + return GRPOAdvantageEstimator(config, loss_config=None) + + +def test_invalid_rows_do_not_bias_the_baseline(): + prompt_ids = torch.zeros( + 4, 3, dtype=torch.long + ) # one shared prompt (2D, as prompt_ids_for_adv) + # The last row is a token-capture placeholder: reward 0, sample_mask 0. + rewards = torch.tensor([1.0, 3.0, 2.0, 0.0]) + valid_mask = torch.tensor([1.0, 1.0, 1.0, 0.0]) + mask = torch.ones(4, 5) + + adv = _estimator().compute_advantage( + prompt_ids, rewards, mask, valid_mask=valid_mask + ) + # Baseline over valid rows only: mean(1,3,2) = 2 (placeholder's 0 excluded). + assert torch.allclose(adv[0], torch.full((5,), -1.0)) + assert torch.allclose(adv[1], torch.full((5,), 1.0)) + assert torch.allclose(adv[2], torch.full((5,), 0.0)) + + +def test_none_valid_mask_keeps_legacy_all_valid_behavior(): + prompt_ids = torch.zeros(2, 3, dtype=torch.long) + rewards = torch.tensor([1.0, 3.0]) + mask = torch.ones(2, 3) + legacy = _estimator().compute_advantage(prompt_ids, rewards, mask) + explicit = _estimator().compute_advantage( + prompt_ids, rewards, mask, valid_mask=torch.ones(2) + ) + assert torch.equal(legacy, explicit) diff --git a/tests/unit/data_plane/test_blackbox_finalizer.py b/tests/unit/data_plane/test_blackbox_finalizer.py new file mode 100644 index 00000000000..697c24921de --- /dev/null +++ b/tests/unit/data_plane/test_blackbox_finalizer.py @@ -0,0 +1,625 @@ +# Copyright (c) 2026, NVIDIA CORPORATION. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +"""S4: BlackboxFinalizer against a live TQ simple backend. + +Drives the S1 golden call sequences end to end: stage the fixture's delta +rows via TQTokenSink, hand the fixture receipt to the finalizer, and require +the published canonical rows to match the fixture's frozen training row. +Every rejection path (missing rows, digest corruption, poisoned receipts) +must yield a masked placeholder — always N rows — and the group publisher's +min/max weight versions and staging cleanup must hold. + +Marked nemo_gym (run with ``--nemo-gym-only``): the finalizer delegates +rebuild semantics to Gym's staging package. +""" + +from __future__ import annotations + +from collections import Counter +from dataclasses import replace + +import pytest +import torch + +nemo_gym = pytest.importorskip("nemo_gym.token_id_capture.staging") + +from nemo_gym.token_id_capture.staging.digest import ( # noqa: E402 + compute_extras_digest, + compute_staging_digest, +) +from nemo_gym.token_id_capture.staging.records import ( # noqa: E402 + StagedCallRecord, +) + +from nemo_rl.data_plane.tq_token_sink import ( # noqa: E402 + STAGING_FIELDS, + TQTokenSink, + TQTokenSource, +) +from nemo_rl.data_plane.schema import ( # noqa: E402 + ROUTE_PASSTHROUGH_FLAG, + ROUTE_PLAN_TAG, +) +from nemo_rl.data_plane.worker_mixin import TQWorkerMixin # noqa: E402 +from nemo_rl.experience.blackbox_finalizer import BlackboxFinalizer # noqa: E402 +from nemo_rl.experience.route_plan import decode_route_plan # noqa: E402 +from tests.unit.data_plane.token_capture_test_fixtures import ( # noqa: E402 + build_fixture_artifacts, + f32, +) + +pytestmark = pytest.mark.nemo_gym + +STAGING_PARTITION = "rollout_staging_fin_test" +CANONICAL_PARTITION = "rollout_data_fin_test" +PAD = 0 + + +@pytest.fixture() +def partitions(tq_client): + tq_client.register_partition( + partition_id=STAGING_PARTITION, + fields=list(STAGING_FIELDS), + num_samples=64, + consumer_tasks=["finalize"], + ) + tq_client.register_partition( + partition_id=CANONICAL_PARTITION, + fields=[ + "input_ids", + "input_lengths", + "generation_logprobs", + "token_mask", + "sample_mask", + "prompt_ids_for_adv", + "total_reward", + ], + num_samples=64, + consumer_tasks=["train"], + ) + yield + tq_client.clear_samples(sample_ids=None, partition_id=STAGING_PARTITION) + tq_client.clear_samples(sample_ids=None, partition_id=CANONICAL_PARTITION) + + +def _finalizer(tq_client, **overrides) -> BlackboxFinalizer: + kwargs = dict( + partition_id=CANONICAL_PARTITION, + staging_partition=STAGING_PARTITION, + pad_token_id=PAD, + mixed_weight_version_policy="allow", + min_valid_fraction_per_group=None, + ) + kwargs.update(overrides) + return BlackboxFinalizer(tq_client, **kwargs) + + +def _stage_fixture(tq_client, name: str, *, rollout_id: str | None = None): + """Stage one golden fixture's rows (optionally re-keyed to rollout_id) + and return (receipt_dict, expected LinearizedRow).""" + records, receipt, row = build_fixture_artifacts(name, rollout_id=rollout_id) + sink = TQTokenSink(tq_client, staging_partition=STAGING_PARTITION) + for record in records: + assert sink.stage(record).ok + return receipt.model_dump(), row + + +def test_finalize_rollout_reproduces_the_golden_row(tq_client, partitions): + receipt, expected = _stage_fixture(tq_client, "worked_example") + finalizer = _finalizer(tq_client) + row = finalizer.finalize_rollout("g7_r0", receipt, reward=1.0) + assert row.valid, row.rejection_reason + assert row.token_ids == expected.token_ids + assert row.token_mask == [f32(m) for m in expected.token_mask] + assert row.logprobs == [f32(p) for p in expected.logprobs] + assert row.prompt_len == expected.prompt_len + # The worked example spans a single weight version (wv 4 throughout). + assert (row.min_wv, row.max_wv) == (4, 4) + + +def test_finalize_rollout_rejections(tq_client, partitions): + finalizer = _finalizer(tq_client) + assert ( + finalizer.finalize_rollout("r", None, reward=0.0).rejection_reason + == "missing_receipt" + ) + + receipt, _ = _stage_fixture(tq_client, "single_call", rollout_id="rej_a") + poisoned = dict(receipt, capture_poisoned=True) + assert ( + finalizer.finalize_rollout("rej_a", poisoned, reward=0.0).rejection_reason + == "capture_poisoned" + ) + empty = dict(receipt, manifest=[], terminal_model_call_id=None) + assert ( + finalizer.finalize_rollout("rej_a", empty, reward=0.0).rejection_reason + == "empty_manifest" + ) + wrong_identity = finalizer.finalize_rollout("someone_else", receipt, reward=0.0) + assert (wrong_identity.rejection_reason or "").startswith("identity_mismatch") + + # A manifest naming rows that were never staged. + ghost = dict(receipt) + ghost["manifest"] = [ + {**entry, "staging_key": "ghost/row"} for entry in receipt["manifest"] + ] + missing = finalizer.finalize_rollout("rej_a", ghost, reward=0.0) + assert (missing.rejection_reason or "").startswith("missing_staging_row") + + # Digest corruption: break the manifest digest so recomputation misses. + corrupted = dict(receipt) + corrupted["manifest"] = [ + {**entry, "digest": "0" * 64} for entry in receipt["manifest"] + ] + bad = finalizer.finalize_rollout("rej_a", corrupted, reward=0.0) + assert (bad.rejection_reason or "").startswith("digest_mismatch") + + +def test_mixed_weight_version_policy_reject(tq_client, partitions): + receipt, _ = _stage_fixture(tq_client, "mixed_weight_versions", rollout_id="mix_r0") + receipt["rollout_id"] = "mix_r0" + allow_row = _finalizer(tq_client).finalize_rollout("mix_r0", receipt, reward=0.0) + assert allow_row.valid + assert allow_row.min_wv < allow_row.max_wv + reject_row = _finalizer( + tq_client, mixed_weight_version_policy="reject" + ).finalize_rollout("mix_r0", receipt, reward=0.0) + assert (reject_row.rejection_reason or "").startswith("mixed_weight_versions") + + +def _fetch_rows(tq_client, sample_ids): + return tq_client.get_samples( + sample_ids=sample_ids, + partition_id=CANONICAL_PARTITION, + select_fields=[ + "input_ids", + "input_lengths", + "generation_logprobs", + "token_mask", + "sample_mask", + "prompt_ids_for_adv", + "total_reward", + ], + ) + + +def test_finalize_group_publishes_n_rows_with_placeholder(tq_client, partitions): + group_id = "grp1" + receipt, expected = _stage_fixture( + tq_client, "worked_example", rollout_id=f"{group_id}_g0" + ) + receipt["rollout_id"] = f"{group_id}_g0" + # Mark this receipt's terminal as heuristically selected so the group + # metric sees a mixed declared/heuristic population. + receipt["terminal_selection"] = "heuristic" + rollout_ids = [f"{group_id}_g0", f"{group_id}_g1"] + + finalizer = _finalizer(tq_client) + finalized = finalizer.finalize_group( + group_id, + rollout_ids, + [receipt, None], # second rollout lost its receipt -> placeholder + [1.0, 0.0], + fallback_weight_version=9, + ) + assert not finalized.dropped + assert finalized.meta is not None + assert finalized.meta.sample_ids == rollout_ids + # Group staleness comes from the valid rollout's calls (wv 4), not the fallback. + assert (finalized.group_min_wv, finalized.group_max_wv) == (4, 4) + assert finalized.metrics["finalize/invalid_row_rate"] == 0.5 + assert finalized.metrics["finalize/terminal_selection_heuristic_count"] == 1.0 + assert finalized.metrics["finalize/terminal_selection_heuristic_fraction"] == 0.5 + assert finalized.metrics["finalize/terminal_selection_declared_count"] == 0.0 + assert finalized.metrics["finalize/terminal_witness_disagreement_count"] == 0.0 + + rows = _fetch_rows(tq_client, rollout_ids) + sample_mask = torch.as_tensor(rows["sample_mask"]).flatten() + assert sample_mask.tolist() == [1.0, 0.0] + valid_len = len(expected.token_ids) + input_ids = torch.as_tensor(rows["input_ids"][0]).flatten() + assert input_ids[:valid_len].tolist() == expected.token_ids + # Placeholder borrows the valid sibling's prompt for baseline grouping. + prompt = expected.token_ids[: expected.prompt_len] + adv_prompt_valid = torch.as_tensor(rows["prompt_ids_for_adv"][0]).flatten() + adv_prompt_placeholder = torch.as_tensor(rows["prompt_ids_for_adv"][1]).flatten() + assert adv_prompt_valid.tolist() == prompt + assert adv_prompt_placeholder.tolist() == prompt + placeholder_mask = torch.as_tensor(rows["token_mask"][1]).flatten() + assert placeholder_mask.sum().item() == 0.0 + rewards = torch.as_tensor(rows["total_reward"]).flatten() + assert rewards.tolist() == [1.0, 0.0] + + # The finalizer cleared its staged rows after publishing. + with pytest.raises(KeyError): + finalizer._source.fetch([receipt["manifest"][0]["staging_key"]]) + + +def test_finalize_group_min_valid_fraction_drops(tq_client, partitions): + group_id = "grp2" + rollout_ids = [f"{group_id}_g0", f"{group_id}_g1"] + finalizer = _finalizer(tq_client, min_valid_fraction_per_group=0.5) + finalized = finalizer.finalize_group( + group_id, + rollout_ids, + [None, None], + [0.0, 0.0], + fallback_weight_version=3, + ) + assert finalized.dropped + assert finalized.meta is None + assert (finalized.group_min_wv, finalized.group_max_wv) == (3, 3) + with pytest.raises((KeyError, RuntimeError, ValueError)): + rows = _fetch_rows(tq_client, rollout_ids) + assert not rows # nothing published + + +# --------------------------------------------------------------------------- +# Router replay (R3): routed_experts rebuilt from staged extras and published +# --------------------------------------------------------------------------- + +_R3_PARTITION = "rollout_data_fin_r3_test" +_R3_STAGING = "rollout_staging_fin_r3_test" + + +@pytest.fixture() +def r3_partitions(tq_client): + from nemo_rl.data_plane.tq_token_sink import ROUTED_EXPERTS_FIELD + + tq_client.register_partition( + partition_id=_R3_STAGING, + fields=list(STAGING_FIELDS) + [ROUTED_EXPERTS_FIELD], + num_samples=64, + consumer_tasks=["finalize"], + ) + tq_client.register_partition( + partition_id=_R3_PARTITION, + fields=[ + "input_ids", + "input_lengths", + "generation_logprobs", + "token_mask", + "sample_mask", + "prompt_ids_for_adv", + "total_reward", + "routed_experts", + ], + num_samples=64, + consumer_tasks=["train"], + ) + yield + tq_client.clear_samples(sample_ids=None, partition_id=_R3_STAGING) + tq_client.clear_samples(sample_ids=None, partition_id=_R3_PARTITION) + + +def _routes_for_delta(call_idx: int, n_tokens: int) -> list: + """[n][L=2][K=2] rows, value = call*1000 + pos (recognizable per token).""" + return [ + [[call_idx * 1000 + pos, call_idx * 1000 + pos + 500] for _ in range(2)] + for pos in range(n_tokens) + ] + + +def _record_with_routes(record: StagedCallRecord, routes: list) -> StagedCallRecord: + extras = {"routed_experts": routes} + extras_digest = compute_extras_digest(extras) + digest = compute_staging_digest( + schema_version=record.schema_version, + digest_version=record.digest_version, + extras_digest_version=record.extras_digest_version, + rollout_id=record.rollout_id, + model_call_id=record.model_call_id, + parent_call_id=record.parent_call_id, + mode=record.mode, + prev_len=record.prev_len, + delta_len=record.delta_len, + cum_len=record.cum_len, + weight_version=record.weight_version, + token_ids_delta=record.token_ids_delta, + token_mask_delta=record.token_mask_delta, + generation_log_probs_delta=record.generation_log_probs_delta, + extras_digest=extras_digest, + chain_hash=record.chain_hash, + cumulative_hash=record.cumulative_hash, + ) + return StagedCallRecord.model_validate( + record.model_dump() + | {"extras": extras, "extras_digest": extras_digest, "digest": digest} + ) + + +def _receipt_with_staged_records(receipt, records): + manifest_by_id = {record.model_call_id: record for record in receipt.manifest} + return receipt.model_copy( + update={ + "manifest": [ + manifest_by_id[record.model_call_id].model_copy( + update={ + "digest": record.digest, + "extras_digest": record.extras_digest, + } + ) + for record in records + ] + } + ) + + +def _stage_fixture_with_routes(tq_client, name: str, *, rollout_id: str): + """Stage the golden fixture with per-call routed_experts extras attached. + + Returns (receipt_dict, expected LinearizedRow, routes_by_call). + """ + records, receipt, row = build_fixture_artifacts(name, rollout_id=rollout_id) + sink = TQTokenSink(tq_client, staging_partition=_R3_STAGING) + routes_by_call = {} + staged_records = [] + for idx, record in enumerate(records): + routes = _routes_for_delta(idx, len(record.token_ids_delta)) + routes_by_call[record.model_call_id] = routes + staged = _record_with_routes(record, routes) + staged_records.append(staged) + assert sink.stage(staged).ok + receipt = _receipt_with_staged_records(receipt, staged_records) + return receipt.model_dump(), row, routes_by_call + + +def _gym_linearize_supports_routes() -> bool: + from nemo_gym.token_id_capture.staging.rebuild import LinearizedRow + + return "routed_experts" in getattr(LinearizedRow, "__dataclass_fields__", {}) + + +@pytest.mark.skipif( + not _gym_linearize_supports_routes(), + reason="Gym pin predates LinearizedRow.routed_experts (Gym PR #2278 R3 follow-up)", +) +def test_finalize_group_publishes_routed_experts(tq_client, r3_partitions): + group_id = "grpr3" + rollout_ids = [f"{group_id}_g0", f"{group_id}_g1"] + receipt, expected, routes_by_call = _stage_fixture_with_routes( + tq_client, "worked_example", rollout_id=rollout_ids[0] + ) + + finalizer = BlackboxFinalizer( + tq_client, + partition_id=_R3_PARTITION, + staging_partition=_R3_STAGING, + pad_token_id=PAD, + mixed_weight_version_policy="allow", + min_valid_fraction_per_group=None, + router_replay_enabled=True, + ) + finalized = finalizer.finalize_group( + group_id, + rollout_ids, + [receipt, None], # second rollout -> placeholder + [1.0, 0.0], + fallback_weight_version=9, + ) + assert not finalized.dropped + assert "routed_experts" in finalized.meta.fields + assert finalized.metrics["finalize/routed_experts_row_coverage"] == 1.0 + assert finalized.metrics["finalize/routed_experts_sentinel_token_fraction"] == 0.0 + + rows = tq_client.get_samples( + sample_ids=rollout_ids, + partition_id=_R3_PARTITION, + select_fields=["routed_experts", "input_lengths"], + ) + # Valid row: the delivered chain's staged extras, concatenated in chain + # order (the golden fixture is a single linear chain). + expected_routes = [ + row_routes + for call_id in expected.call_ids + for row_routes in routes_by_call[call_id] + ] + valid_len = len(expected.token_ids) + assert len(expected_routes) == valid_len + published = torch.as_tensor(rows["routed_experts"][0]).reshape(-1, 2, 2) + assert published[:valid_len].tolist() == expected_routes + # Placeholder row: all-sentinel (Megatron self-routes; sample_mask 0). + placeholder = torch.as_tensor(rows["routed_experts"][1]) + assert bool(placeholder.eq(-1).all().item()) + + +@pytest.mark.skipif( + not _gym_linearize_supports_routes(), + reason="Gym pin predates LinearizedRow.routed_experts (Gym PR #2278 R3 follow-up)", +) +def test_finalize_group_router_replay_without_routes_fails_loudly( + tq_client, r3_partitions +): + group_id = "grpr3b" + rollout_id = f"{group_id}_g0" + records, receipt, _ = build_fixture_artifacts( + "worked_example", rollout_id=rollout_id + ) + sink = TQTokenSink(tq_client, staging_partition=_R3_STAGING) + for record in records: + assert sink.stage(record).ok # no extras staged + + finalizer = BlackboxFinalizer( + tq_client, + partition_id=_R3_PARTITION, + staging_partition=_R3_STAGING, + pad_token_id=PAD, + mixed_weight_version_policy="allow", + min_valid_fraction_per_group=None, + router_replay_enabled=True, + ) + with pytest.raises(RuntimeError, match="routed_experts"): + finalizer.finalize_group( + group_id, + [rollout_id], + [receipt.model_dump()], + [1.0], + fallback_weight_version=9, + ) + + +# --------------------------------------------------------------------------- +# Deferred router replay: canonical small rows + strict plans, worker assembly +# --------------------------------------------------------------------------- + +_R3_DEFERRED_PARTITION = "rollout_data_fin_r3_deferred_test" +_R3_DEFERRED_STAGING = "rollout_staging_fin_r3_deferred_test" + + +@pytest.fixture() +def r3_deferred_partitions(tq_client): + from nemo_rl.data_plane.tq_token_sink import ROUTED_EXPERTS_FIELD + + tq_client.register_partition( + partition_id=_R3_DEFERRED_STAGING, + fields=list(STAGING_FIELDS) + [ROUTED_EXPERTS_FIELD], + num_samples=64, + consumer_tasks=["finalize", "prev_lp", "train"], + ) + tq_client.register_partition( + partition_id=_R3_DEFERRED_PARTITION, + fields=[ + "input_ids", + "input_lengths", + "generation_logprobs", + "token_mask", + "sample_mask", + "prompt_ids_for_adv", + "total_reward", + ], + num_samples=64, + consumer_tasks=["train"], + ) + yield + tq_client.clear_samples(sample_ids=None, partition_id=_R3_DEFERRED_STAGING) + tq_client.clear_samples(sample_ids=None, partition_id=_R3_DEFERRED_PARTITION) + + +def _stage_deferred_fixture(tq_client, *, rollout_id: str): + records, receipt, row = build_fixture_artifacts( + "worked_example", rollout_id=rollout_id + ) + sink = TQTokenSink(tq_client, staging_partition=_R3_DEFERRED_STAGING) + routes_by_call = {} + staged_records = [] + for idx, record in enumerate(records): + routes = _routes_for_delta(idx, len(record.token_ids_delta)) + routes_by_call[record.model_call_id] = routes + staged = _record_with_routes(record, routes) + staged_records.append(staged) + assert sink.stage(staged).ok + receipt = _receipt_with_staged_records(receipt, staged_records) + return receipt.model_dump(), row, routes_by_call + + +class _DeferredRouteWorker(TQWorkerMixin): + def __init__(self, client): + self._dp_client = client + self._route_fallback_counts = Counter() + + def _routed_experts_dimensions(self) -> tuple[int, int]: + return 2, 2 + + +def test_deferred_finalizer_publishes_plans_and_worker_replays_routes( + tq_client, r3_deferred_partitions +): + group_id = "grpr3deferred" + rollout_ids = [f"{group_id}_g0", f"{group_id}_g1"] + receipt, expected, routes_by_call = _stage_deferred_fixture( + tq_client, rollout_id=rollout_ids[0] + ) + finalizer = BlackboxFinalizer( + tq_client, + partition_id=_R3_DEFERRED_PARTITION, + staging_partition=_R3_DEFERRED_STAGING, + pad_token_id=PAD, + mixed_weight_version_policy="allow", + min_valid_fraction_per_group=None, + router_replay_enabled=True, + defer_routed_experts_to_policy=True, + ) + + finalized = finalizer.finalize_group( + group_id, + rollout_ids, + [receipt, None], + [1.0, 0.0], + fallback_weight_version=9, + ) + + assert finalized.meta is not None + assert "routed_experts" not in finalized.meta.fields + assert len(finalized.staging_keys) == len(receipt["manifest"]) + plans = [decode_route_plan(tag[ROUTE_PLAN_TAG]) for tag in finalized.meta.tags] + assert plans[0].expected_token_length == len(expected.token_ids) + assert set(plans[0].cleanup_staging_keys) == set(finalized.staging_keys) + assert not plans[1].spans + # Deferred finalization deliberately retains staging through consumption. + source = TQTokenSource(tq_client, staging_partition=_R3_DEFERRED_STAGING) + assert len(source.fetch_for_finalization(finalized.staging_keys)) == len( + finalized.staging_keys + ) + + worker_meta = replace( + finalized.meta, + extra_info={ROUTE_PASSTHROUGH_FLAG: True}, + task_name="train", + ) + materialized = _DeferredRouteWorker(tq_client)._fetch( + worker_meta, + dp_aligned_seq_len=False, + ) + expected_routes = [ + route for call_id in expected.call_ids for route in routes_by_call[call_id] + ] + valid_len = len(expected.token_ids) + assert materialized["routed_experts"][0, :valid_len].tolist() == expected_routes + assert bool(materialized["routed_experts"][1].eq(-1).all()) + + +@pytest.mark.parametrize("bad_routed_len", [-1, 999]) +def test_deferred_finalizer_rejects_invalid_routed_len( + tq_client, r3_deferred_partitions, bad_routed_len +): + from dataclasses import replace as dataclass_replace + + rollout_id = "bad_route_len_g0" + receipt, _, _ = _stage_deferred_fixture(tq_client, rollout_id=rollout_id) + finalizer = BlackboxFinalizer( + tq_client, + partition_id=_R3_DEFERRED_PARTITION, + staging_partition=_R3_DEFERRED_STAGING, + pad_token_id=PAD, + mixed_weight_version_policy="allow", + min_valid_fraction_per_group=None, + router_replay_enabled=True, + defer_routed_experts_to_policy=True, + ) + fetched = finalizer._source.fetch_for_finalization( + [record["staging_key"] for record in receipt["manifest"]] + ) + fetched[0] = dataclass_replace(fetched[0], routed_len=bad_routed_len) + + class _InjectedSource: + def fetch_for_finalization(self, staging_keys): + del staging_keys + return fetched + + finalizer._source = _InjectedSource() + row = finalizer.finalize_rollout(rollout_id, receipt, reward=0.0) + + assert not row.valid + assert (row.rejection_reason or "").startswith("routed_len_mismatch") diff --git a/tests/unit/data_plane/test_preshard_extras.py b/tests/unit/data_plane/test_preshard_extras.py index 0c5b9e0d62f..8d5f6aca149 100644 --- a/tests/unit/data_plane/test_preshard_extras.py +++ b/tests/unit/data_plane/test_preshard_extras.py @@ -133,6 +133,17 @@ def test_shard_meta_for_dp_preserves_partition_id(): assert all(m.partition_id == "train" for m in metas) +def test_shard_meta_for_dp_preserves_tag_row_identity(): + meta = _meta(8) + meta.tags = [{"row": sample_id} for sample_id in meta.sample_ids] + + metas, _ = shard_meta_for_dp(meta, dp_world=4, batch_size=8) + + for shard in metas: + assert shard.tags is not None + assert [tag["row"] for tag in shard.tags] == shard.sample_ids + + def test_shard_meta_for_dp_unsorted_round_trip(): """unsorted_indices must reconstruct the input order from DP-rank concat.""" n, dp = 8, 4 diff --git a/tests/unit/data_plane/test_tq_policy_routes.py b/tests/unit/data_plane/test_tq_policy_routes.py new file mode 100644 index 00000000000..00cd8a805dc --- /dev/null +++ b/tests/unit/data_plane/test_tq_policy_routes.py @@ -0,0 +1,92 @@ +# Copyright (c) 2026, NVIDIA CORPORATION. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +"""TQPolicy direct/deferred route field selection tests.""" + +from __future__ import annotations + +import pytest + +from nemo_rl.data_plane import KVBatchMeta +from nemo_rl.data_plane.schema import ( + ROUTE_PASSTHROUGH_FLAG, + ROUTE_PLAN_TAG, + ROUTED_EXPERTS_FIELD, +) +from nemo_rl.models.policy.tq_policy import TQPolicy + + +def _policy() -> TQPolicy: + policy = object.__new__(TQPolicy) + policy._router_replay_enabled = True + return policy + + +def _meta(tags) -> KVBatchMeta: + return KVBatchMeta( + partition_id="canonical", + task_name="train", + sample_ids=[f"row{i}" for i in range(len(tags))], + fields=["input_ids"], + sequence_lengths=[2] * len(tags), + tags=tags, + ) + + +def test_deferred_prev_lp_and_train_omit_canonical_route_field() -> None: + meta = _meta([{ROUTE_PLAN_TAG: {"plan": 0}}, {ROUTE_PLAN_TAG: {"plan": 1}}]) + + result = _policy()._with_route_fields( + meta, + ("input_ids",), + task_name="prev_lp", + want_routes=True, + ) + + assert ROUTED_EXPERTS_FIELD not in result.fields + assert result.extra_info[ROUTE_PASSTHROUGH_FLAG] is True + + +def test_reference_lp_never_requests_or_materializes_routes() -> None: + meta = _meta([{ROUTE_PLAN_TAG: {"plan": 0}}]) + + result = _policy()._with_route_fields( + meta, + ("input_ids",), + task_name="ref_lp", + want_routes=False, + ) + + assert ROUTED_EXPERTS_FIELD not in result.fields + assert ROUTE_PASSTHROUGH_FLAG not in result.extra_info + + +def test_direct_storage_keeps_canonical_route_field() -> None: + result = _policy()._with_route_fields( + _meta([{"weight_version": 1}]), + ("input_ids",), + task_name="train", + want_routes=True, + ) + assert ROUTED_EXPERTS_FIELD in result.fields + assert ROUTE_PASSTHROUGH_FLAG not in result.extra_info + + +def test_mixed_direct_and_deferred_rows_are_rejected() -> None: + with pytest.raises(RuntimeError, match="mixed direct/deferred"): + _policy()._with_route_fields( + _meta([{ROUTE_PLAN_TAG: {}}, {"weight_version": 1}]), + ("input_ids",), + task_name="train", + want_routes=True, + ) diff --git a/tests/unit/data_plane/test_tq_token_sink.py b/tests/unit/data_plane/test_tq_token_sink.py new file mode 100644 index 00000000000..f712eceabc0 --- /dev/null +++ b/tests/unit/data_plane/test_tq_token_sink.py @@ -0,0 +1,192 @@ +# Copyright (c) 2026, NVIDIA CORPORATION. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +"""TQTokenSink / TQTokenSource against a live TQ backend. + +Runs NeMo-Gym's installable conformance kit (golden call sequences → +byte-exact digests, manifests, and linearized rows) over the TransferQueue +implementations — the framework-CI half of the § 3.0 contract — plus the +protocol edges the kit does not cover (missing keys, stage failure shape). +""" + +from __future__ import annotations + +import pytest + +nemo_gym = pytest.importorskip("nemo_gym.token_id_capture.staging") + +from nemo_gym.token_id_capture.staging.protocols import ( # noqa: E402 + StagingSink as TokenSinkProtocol, +) +from nemo_gym.token_id_capture.staging.protocols import ( # noqa: E402 + StagingSource as TokenSourceProtocol, +) + +from nemo_rl.data_plane.tq_token_sink import ( # noqa: E402 + STAGING_FIELDS, + TQTokenSink, + TQTokenSource, +) +from tests.unit.data_plane.token_capture_test_fixtures import ( # noqa: E402 + build_fixture_artifacts, + fixture_names, +) + +STAGING_PARTITION = "rollout_staging_test" + +pytestmark = pytest.mark.nemo_gym + + +@pytest.fixture() +def staging_partition(tq_client): + tq_client.register_partition( + partition_id=STAGING_PARTITION, + fields=list(STAGING_FIELDS), + num_samples=64, + consumer_tasks=["finalize"], + ) + yield STAGING_PARTITION + tq_client.clear_samples(sample_ids=None, partition_id=STAGING_PARTITION) + + +def test_implementations_satisfy_protocols(tq_client, staging_partition): + sink = TQTokenSink(tq_client, staging_partition=staging_partition) + source = TQTokenSource(tq_client, staging_partition=staging_partition) + assert isinstance(sink, TokenSinkProtocol) + assert isinstance(source, TokenSourceProtocol) + + +@pytest.mark.parametrize( + "fixture_name", ["worked_example", "single_call", "mixed_weight_versions"] +) +def test_tq_sink_source_passes_conformance(tq_client, staging_partition, fixture_name): + assert fixture_name in fixture_names() + sink = TQTokenSink(tq_client, staging_partition=staging_partition) + source = TQTokenSource(tq_client, staging_partition=staging_partition) + records, _, _ = build_fixture_artifacts(fixture_name) + for record in records: + assert sink.stage(record).ok + snapshots = source.fetch([record.staging_key for record in records]) + assert [snapshot.model_dump() for snapshot in snapshots] == [ + record.model_dump() for record in records + ] + + +def test_fetch_missing_key_raises_keyerror(tq_client, staging_partition): + source = TQTokenSource(tq_client, staging_partition=staging_partition) + with pytest.raises(KeyError): + source.fetch(["ghost_rollout/ghost_call"]) + + +def test_fetch_for_finalization_is_small_typed_and_identity_preserving( + tq_client, staging_partition +): + class RecordingClient: + def __init__(self, client): + self.client = client + self.select_fields = None + + def get_samples(self, **kwargs): + self.select_fields = list(kwargs["select_fields"]) + return self.client.get_samples(**kwargs) + + sink = TQTokenSink(tq_client, staging_partition=staging_partition) + records, _, _ = build_fixture_artifacts("single_call") + assert sink.stage(records[0]).ok + recording_client = RecordingClient(tq_client) + source = TQTokenSource(recording_client, staging_partition=staging_partition) + + fetched = source.fetch_for_finalization([records[0].staging_key]) + + assert recording_client.select_fields == STAGING_FIELDS + assert "routed_experts" not in recording_client.select_fields + assert len(fetched) == 1 + assert fetched[0].staging_key == records[0].staging_key + assert fetched[0].snapshot.model_call_id == records[0].model_call_id + assert fetched[0].routed_len == 0 + + +def test_fetch_for_finalization_rejects_duplicate_request_keys( + tq_client, staging_partition +): + source = TQTokenSource(tq_client, staging_partition=staging_partition) + with pytest.raises(KeyError, match="duplicate keys"): + source.fetch_for_finalization(["r/c", "r/c"]) + + +def test_stage_failure_reports_not_raises(staging_partition): + class ExplodingClient: + def put_samples(self, **kwargs): + raise RuntimeError("controller down") + + sink = TQTokenSink(ExplodingClient(), staging_partition=staging_partition) + records, _, _ = build_fixture_artifacts("single_call") + result = sink.stage(records[0]) + assert not result.ok + assert result.staging_key == records[0].staging_key + assert "controller down" in (result.error or "") + + +def test_sink_clear_drops_rows(tq_client, staging_partition): + sink = TQTokenSink(tq_client, staging_partition=staging_partition) + source = TQTokenSource(tq_client, staging_partition=staging_partition) + records, _, _ = build_fixture_artifacts("single_call") + for record in records: + assert sink.stage(record).ok + keys = [record.staging_key for record in records] + assert len(source.fetch(keys)) == len(keys) + sink.clear(keys) + with pytest.raises(KeyError): + source.fetch(keys) + + +def test_fetch_prefix_token_ids_empty(tq_client, staging_partition): + source = TQTokenSource(tq_client, staging_partition=staging_partition) + assert source.fetch_prefix_token_ids([]) == [] + + +def test_fetch_prefix_token_ids_single_key(tq_client, staging_partition): + sink = TQTokenSink(tq_client, staging_partition=staging_partition) + source = TQTokenSource(tq_client, staging_partition=staging_partition) + records, _, _ = build_fixture_artifacts("single_call") + record = records[0] + assert sink.stage(record).ok + result = source.fetch_prefix_token_ids([record.staging_key]) + assert result == record.token_ids_delta + + +def test_fetch_prefix_token_ids_three_keys_concatenates(tq_client, staging_partition): + sink = TQTokenSink(tq_client, staging_partition=staging_partition) + source = TQTokenSource(tq_client, staging_partition=staging_partition) + records, _, _ = build_fixture_artifacts("worked_example") + for record in records: + assert sink.stage(record).ok + keys = [record.staging_key for record in records] + result = source.fetch_prefix_token_ids(keys) + expected = [t for record in records for t in record.token_ids_delta] + assert result == expected + + +def test_fetch_prefix_token_ids_missing_key_raises_keyerror( + tq_client, staging_partition +): + source = TQTokenSource(tq_client, staging_partition=staging_partition) + with pytest.raises(KeyError): + source.fetch_prefix_token_ids(["ghost_rollout/ghost_call"]) + + +def test_fetch_prefix_token_ids_rejects_duplicates(tq_client, staging_partition): + source = TQTokenSource(tq_client, staging_partition=staging_partition) + with pytest.raises(KeyError, match="duplicates"): + source.fetch_prefix_token_ids(["r/c", "r/c"]) diff --git a/tests/unit/data_plane/test_worker_route_assembly.py b/tests/unit/data_plane/test_worker_route_assembly.py new file mode 100644 index 00000000000..015f618f1d9 --- /dev/null +++ b/tests/unit/data_plane/test_worker_route_assembly.py @@ -0,0 +1,205 @@ +# Copyright (c) 2026, NVIDIA CORPORATION. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +"""Policy-worker deferred route assembly tests.""" + +from __future__ import annotations + +from collections import Counter + +import torch +from nemo_gym.token_id_capture.staging.digest import compute_extras_digest + +from nemo_rl.data_plane import KVBatchMeta +from nemo_rl.data_plane.schema import ( + ROUTE_PASSTHROUGH_FLAG, + ROUTE_PLAN_TAG, + ROUTED_EXPERTS_ENCODING_FIELD, + ROUTED_EXPERTS_FIELD, + ROUTED_EXTRAS_METADATA_FIELD, +) +from nemo_rl.data_plane.worker_mixin import TQWorkerMixin +from nemo_rl.distributed.batched_data_dict import BatchedDataDict +from nemo_rl.experience.route_plan import ( + ROUTE_PLAN_SCHEMA_VERSION, + RouteAssemblyPlan, + RouteSpan, + encode_route_plan, +) +from nemo_rl.utils.routed_experts_codec import encode_routed_experts + + +class _Rows(dict): + def __init__(self, routed: list[torch.Tensor]) -> None: + super().__init__( + { + ROUTED_EXPERTS_FIELD: routed, + ROUTED_EXPERTS_ENCODING_FIELD: [ + torch.tensor([1], dtype=torch.int64) for _ in routed + ], + ROUTED_EXTRAS_METADATA_FIELD: [ + torch.tensor(list(b"{}"), dtype=torch.uint8) for _ in routed + ], + } + ) + self.batch_size = (len(routed),) + + +class _RouteClient: + def __init__(self, fragments: dict[str, torch.Tensor]) -> None: + self.fragments = fragments + self.calls: list[list[str]] = [] + + def get_samples(self, *, sample_ids, partition_id, select_fields): + assert partition_id == "staging" + assert select_fields == [ + ROUTED_EXPERTS_FIELD, + ROUTED_EXPERTS_ENCODING_FIELD, + ROUTED_EXTRAS_METADATA_FIELD, + ] + self.calls.append(list(sample_ids)) + return _Rows([self.fragments[key] for key in sample_ids]) + + +class _Worker(TQWorkerMixin): + def __init__(self, client: _RouteClient) -> None: + self._dp_client = client + self._route_fallback_counts = Counter() + + def _routed_experts_dimensions(self) -> tuple[int, int]: + return 1, 2 + + +def _plan( + spans: tuple[RouteSpan, ...], *, expected: int, cleanup: tuple[str, ...] +) -> dict: + return encode_route_plan( + RouteAssemblyPlan( + schema_version=ROUTE_PLAN_SCHEMA_VERSION, + staging_partition="staging", + spans=spans, + cleanup_staging_keys=cleanup, + expected_token_length=expected, + ) + ) + + +def _span( + client: _RouteClient, + staging_key: str, + carry_len: int, + generation_len: int, + staged_route_len: int, +) -> RouteSpan: + routes = client.fragments[staging_key] + extras_digest = compute_extras_digest( + {ROUTED_EXPERTS_FIELD: encode_routed_experts(routes)} + ) + return RouteSpan( + staging_key, + carry_len, + generation_len, + staged_route_len, + extras_digest_version=1, + extras_digest=extras_digest, + ) + + +def _meta(plans: list[dict], lengths: list[int]) -> tuple[KVBatchMeta, BatchedDataDict]: + meta = KVBatchMeta( + partition_id="canonical", + task_name="train", + sample_ids=[f"row{i}" for i in range(len(plans))], + fields=["input_ids", "input_lengths"], + sequence_lengths=lengths, + tags=[{ROUTE_PLAN_TAG: plan} for plan in plans], + extra_info={ROUTE_PASSTHROUGH_FLAG: True}, + ) + data = BatchedDataDict( + { + "input_ids": torch.zeros((len(plans), max(lengths) + 1), dtype=torch.long), + "input_lengths": torch.tensor(lengths, dtype=torch.long), + } + ) + return meta, data + + +def test_worker_coalesces_keys_and_replays_full_tail_and_placeholder() -> None: + fragments = { + "r/c0": torch.tensor([[[10, 11]], [[12, 13]]], dtype=torch.int16), + "r/c1": torch.tensor([[[22, 23]]], dtype=torch.int16), + } + client = _RouteClient(fragments) + worker = _Worker(client) + plans = [ + _plan( + ( + _span(client, "r/c0", 0, 2, 2), + _span(client, "r/c1", 1, 1, 1), + ), + expected=4, + cleanup=("r/c0", "r/c1", "r/off_chain"), + ), + _plan((), expected=1, cleanup=("r/rejected",)), + ] + meta, data = _meta(plans, [4, 1]) + + result = worker._maybe_assemble_routed_experts(meta, data) + + assert client.calls == [["r/c0", "r/c1"]] + routed = result[ROUTED_EXPERTS_FIELD] + assert routed.dtype == torch.int16 + assert routed.shape == (2, 5, 1, 2) + assert routed[0, :4, 0].tolist() == [ + [10, 11], + [12, 13], + [-1, -1], + [22, 23], + ] + assert bool(routed[0, 4].eq(-1).all()) + assert bool(routed[1].eq(-1).all()) + assert not worker._route_fallback_counts + + +def test_wrong_model_shape_falls_back_for_entire_rollout() -> None: + client = _RouteClient({"r/c0": torch.tensor([[[10]], [[11]]], dtype=torch.int16)}) + worker = _Worker(client) + plan = _plan( + (_span(client, "r/c0", 0, 2, 2),), + expected=2, + cleanup=("r/c0",), + ) + meta, data = _meta([plan], [2]) + + routed = worker._maybe_assemble_routed_experts(meta, data)[ROUTED_EXPERTS_FIELD] + + assert bool(routed.eq(-1).all()) + assert worker._route_fallback_counts == Counter({"fragment_model_shape": 1}) + + +def test_tampered_fragment_falls_back_for_entire_rollout() -> None: + client = _RouteClient( + {"r/c0": torch.tensor([[[10, 11]], [[12, 13]]], dtype=torch.int16)} + ) + worker = _Worker(client) + span = _span(client, "r/c0", 0, 2, 2) + client.fragments["r/c0"][0, 0, 0] = 999 + meta, data = _meta( + [_plan((span,), expected=2, cleanup=("r/c0",))], + [2], + ) + + routed = worker._maybe_assemble_routed_experts(meta, data)[ROUTED_EXPERTS_FIELD] + + assert bool(routed.eq(-1).all()) + assert worker._route_fallback_counts == Counter({"fragment_integrity": 1}) diff --git a/tests/unit/data_plane/token_capture_test_fixtures.py b/tests/unit/data_plane/token_capture_test_fixtures.py new file mode 100644 index 00000000000..7fc92162c1c --- /dev/null +++ b/tests/unit/data_plane/token_capture_test_fixtures.py @@ -0,0 +1,147 @@ +# Copyright (c) 2026, NVIDIA CORPORATION. All rights reserved. +# SPDX-License-Identifier: Apache-2.0 + +"""Small v2 Gym staging fixtures shared by the RL data-plane tests.""" + +from __future__ import annotations + +from nemo_gym.token_id_capture.staging.digest import ( + compute_chain_hash, + compute_extras_digest, + compute_staging_digest, + hash_token_ids, +) +from nemo_gym.token_id_capture.staging.rebuild import verify_and_linearize +from nemo_gym.token_id_capture.staging.records import ( + CallRecord, + RolloutReceipt, + StagedCallRecord, + StagedCallSnapshot, +) + + +def f32(value: float) -> float: + """Round one value through the float32 wire representation.""" + import struct + + return struct.unpack(">f", struct.pack(">f", value))[0] + + +def _record( + *, + rollout_id: str, + model_call_id: str, + parent_call_id: str | None, + prev_len: int, + token_ids: list[int], + token_mask: list[float], + logprobs: list[float], + weight_version: int, + parent_chain_hash: str | None = None, + cumulative_prefix: list[int] | None = None, +) -> StagedCallRecord: + token_mask = [f32(value) for value in token_mask] + logprobs = [f32(value) for value in logprobs] + mode = "text" if parent_call_id is None else "token_in" + delta_len = len(token_ids) + cum_len = prev_len + delta_len + extras_digest = compute_extras_digest(None) + values = { + "rollout_id": rollout_id, + "model_call_id": model_call_id, + "parent_call_id": parent_call_id, + "mode": mode, + "prev_len": prev_len, + "delta_len": delta_len, + "cum_len": cum_len, + "weight_version": weight_version, + "token_ids_delta": token_ids, + "token_mask_delta": token_mask, + "generation_log_probs_delta": logprobs, + "extras": None, + "extras_digest": extras_digest, + "chain_hash": compute_chain_hash(parent_chain_hash, token_ids), + "cumulative_hash": hash_token_ids(list(cumulative_prefix or []) + token_ids), + } + return StagedCallRecord( + **values, + digest=compute_staging_digest( + schema_version=2, + digest_version=2, + extras_digest_version=1, + **{key: value for key, value in values.items() if key != "extras"}, + ), + ) + + +def _manifest(record: StagedCallRecord) -> CallRecord: + return CallRecord( + model_call_id=record.model_call_id, + parent_call_id=record.parent_call_id, + mode=record.mode, + prev_len=record.prev_len, + delta_len=record.delta_len, + cum_len=record.cum_len, + weight_version=record.weight_version, + digest=record.digest, + extras_digest=record.extras_digest, + staging_key=record.staging_key, + chain_hash=record.chain_hash, + cumulative_hash=record.cumulative_hash, + response_id=f"chatcmpl-{record.model_call_id}", + ) + + +def fixture_names() -> tuple[str, ...]: + return ("worked_example", "single_call", "mixed_weight_versions") + + +def build_fixture_artifacts( + name: str, *, rollout_id: str | None = None +) -> tuple[list[StagedCallRecord], RolloutReceipt, object]: + if name not in fixture_names(): + raise KeyError(name) + rollout_id = ( + rollout_id + or { + "worked_example": "g7_r0", + "single_call": "single_r0", + "mixed_weight_versions": "mixed_r0", + }[name] + ) + root = _record( + rollout_id=rollout_id, + model_call_id="c1", + parent_call_id=None, + prev_len=0, + token_ids=[10, 11, 12, 13], + token_mask=[0.0, 0.0, 1.0, 1.0], + logprobs=[0.0, 0.0, -0.1, -0.2], + weight_version=4, + ) + records = [root] + if name != "single_call": + records.append( + _record( + rollout_id=rollout_id, + model_call_id="c2", + parent_call_id="c1", + prev_len=root.cum_len, + token_ids=[20, 21, 22], + token_mask=[0.0, 1.0, 1.0], + logprobs=[0.0, -0.3, -0.4], + weight_version=5 if name == "mixed_weight_versions" else 4, + parent_chain_hash=root.chain_hash, + cumulative_prefix=root.token_ids_delta, + ) + ) + receipt = RolloutReceipt( + rollout_id=rollout_id, + terminal_model_call_id=records[-1].model_call_id, + manifest=[_manifest(record) for record in records], + terminal_selection="declared", + ) + snapshots = [ + StagedCallSnapshot.model_validate(record.model_dump()) for record in records + ] + return records, receipt, verify_and_linearize(receipt, snapshots) diff --git a/tests/unit/environments/test_nemo_gym_token_capture.py b/tests/unit/environments/test_nemo_gym_token_capture.py new file mode 100644 index 00000000000..e6cc771ba22 --- /dev/null +++ b/tests/unit/environments/test_nemo_gym_token_capture.py @@ -0,0 +1,377 @@ +# Copyright (c) 2026, NVIDIA CORPORATION. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +from __future__ import annotations + +import asyncio +from unittest.mock import AsyncMock + +from nemo_rl.environments.nemo_gym import NemoGym + + +def _capture_env() -> NemoGym: + env_cls = NemoGym.__ray_metadata__.modified_class + return object.__new__(env_cls) + + +def _manifest_record( + call_id: str, + *, + response_id: str | None = None, + parent: str | None = None, + cumulative_hash: str | None = None, +) -> dict: + prev_len = 0 if parent is None else 900 + return { + "model_call_id": call_id, + "parent_call_id": parent, + "prev_len": prev_len, + "delta_len": 100, + "cum_len": prev_len + 100, + "weight_version": 3, + "digest": "a" * 64, + "extras_digest": "b" * 64, + "staging_key": f"r0/{call_id}", + "mode": "text" if parent is None else "token_in", + "response_id": response_id or f"resp-{call_id}", + "cumulative_hash": cumulative_hash, + } + + +def test_receipt_postprocess_without_a_terminal_logical_id_uses_the_heuristic() -> None: + env = _capture_env() + records = [ + _manifest_record("c1"), + _manifest_record("c2", parent="c1"), + ] + env._control = AsyncMock( + return_value={"rollout_id": "r0", "records": records, "failures": []} + ) + + result = asyncio.run( + env._postprocess_receipt_mode( + {"_ng_rollout_id": "r0"}, + {"reward": 1.0}, + ) + ) + + env._control.assert_awaited() + receipt = result["receipt"] + assert receipt["terminal_model_call_id"] == "c2" + assert receipt["terminal_selection"] == "heuristic" + assert receipt["capture_poisoned"] is False + + +def test_receipt_postprocess_fetches_manifest_and_selects_terminal_row() -> None: + env = _capture_env() + records = [ + _manifest_record("c1"), + _manifest_record("c2", parent="c1"), + ] + env._control = AsyncMock( + return_value={"rollout_id": "r0", "records": records, "failures": []} + ) + + result = asyncio.run( + env._postprocess_receipt_mode( + {"_ng_rollout_id": "r0"}, + {"reward": 1.0, "terminal_logical_request_id": "resp-c2"}, + ) + ) + + call = env._control.await_args + assert call.args == ( + "GET", + "/training-token-capture/control/rollouts/r0/manifest", + ) + receipt = result["receipt"] + assert receipt["rollout_id"] == "r0" + assert receipt["terminal_model_call_id"] == "c2" + assert receipt["terminal_selection"] == "declared" + assert receipt["capture_poisoned"] is False + assert receipt["failure_reason"] is None + assert receipt["reward"] == 1.0 + assert [r["model_call_id"] for r in receipt["manifest"]] == ["c1", "c2"] + + +def test_receipt_assembly_poisons_on_failure_rows() -> None: + env = _capture_env() + manifest = { + "rollout_id": "r0", + "records": [_manifest_record("c1")], + "failures": [{"model_call_id": "c2", "reason": "worker_capture_failed"}], + } + receipt = env._assemble_receipt( + "r0", manifest, terminal_logical_request_id="resp-c1", reward=0.0 + ) + assert receipt["capture_poisoned"] is True + assert receipt["failure_reason"] == "worker_capture_failed" + + +def test_receipt_assembly_ignores_uncommitted_call_failures_off_the_terminal_chain() -> None: + """A call that died without coordinates never served a completion and can + never be a lineage parent (no committed row to resolve against), so it is + structurally off-chain — e.g. the doomed final call of a rollout that + exhausted the context window. It must not poison the verified chain.""" + env = _capture_env() + manifest = { + "rollout_id": "r0", + "records": [ + _manifest_record("c1"), + _manifest_record("c2", parent="c1"), + ], + "failures": [ + { + "model_call_id": "c3", + "reason": "request_finished_without_staged_coordinates", + } + ], + } + receipt = env._assemble_receipt( + "r0", manifest, terminal_logical_request_id="resp-c2", reward=1.0 + ) + assert receipt["capture_poisoned"] is False + assert receipt["failure_reason"] is None + assert receipt["terminal_model_call_id"] == "c2" + + +def test_receipt_assembly_still_poisons_when_the_terminal_call_died_uncommitted() -> None: + """If the reported terminal request itself died without coordinates there + is no terminal row — the missing-terminal check must mask the rollout.""" + env = _capture_env() + manifest = { + "rollout_id": "r0", + "records": [_manifest_record("c1")], + "failures": [ + { + "model_call_id": "c2", + "reason": "request_finished_without_staged_coordinates", + } + ], + } + receipt = env._assemble_receipt( + "r0", manifest, terminal_logical_request_id="resp-c2", reward=0.0 + ) + assert receipt["capture_poisoned"] is True + assert receipt["failure_reason"] == "missing_terminal_row" + + +def test_receipt_assembly_poisons_when_the_terminal_row_is_missing() -> None: + env = _capture_env() + manifest = { + "rollout_id": "r0", + "records": [_manifest_record("c1")], + "failures": [], + } + receipt = env._assemble_receipt( + "r0", manifest, terminal_logical_request_id="resp-lost", reward=0.0 + ) + assert receipt["capture_poisoned"] is True + assert receipt["failure_reason"] == "missing_terminal_row" + assert receipt["terminal_model_call_id"] is None + # A declared id is authoritative: a miss never falls back to the heuristic + # even when the manifest holds an unambiguous chain. + assert receipt["terminal_selection"] == "declared" + + +def test_receipt_assembly_heuristic_eliminates_abandoned_retry() -> None: + env = _capture_env() + records = [ + _manifest_record("c1"), + _manifest_record("c2", parent="c1"), + _manifest_record("c2r", parent="c1"), + _manifest_record("c3", parent="c2"), + ] + manifest = {"rollout_id": "r0", "records": records, "failures": []} + receipt = env._assemble_receipt( + "r0", manifest, terminal_logical_request_id=None, reward=1.0 + ) + assert receipt["terminal_model_call_id"] == "c3" + assert receipt["terminal_selection"] == "heuristic" + assert receipt["capture_poisoned"] is False + + +def test_receipt_assembly_heuristic_masks_a_final_call_retry() -> None: + env = _capture_env() + records = [ + _manifest_record("c1"), + _manifest_record("c2", parent="c1"), + _manifest_record("c2r", parent="c1"), + ] + manifest = {"rollout_id": "r0", "records": records, "failures": []} + receipt = env._assemble_receipt( + "r0", manifest, terminal_logical_request_id=None, reward=0.0 + ) + assert receipt["terminal_model_call_id"] is None + assert receipt["capture_poisoned"] is True + assert receipt["failure_reason"] == "ambiguous_terminal" + + +def test_receipt_assembly_heuristic_masks_an_empty_manifest() -> None: + env = _capture_env() + manifest = {"rollout_id": "r0", "records": [], "failures": []} + receipt = env._assemble_receipt( + "r0", manifest, terminal_logical_request_id=None, reward=0.0 + ) + assert receipt["terminal_model_call_id"] is None + assert receipt["capture_poisoned"] is True + assert receipt["failure_reason"] == "no_records" + + +def test_receipt_assembly_heuristic_masks_invalid_manifest_rows() -> None: + env = _capture_env() + bad = _manifest_record("c1") + bad["delta_len"] = 0 # violates the CallRecord length contract + manifest = {"rollout_id": "r0", "records": [bad], "failures": []} + receipt = env._assemble_receipt( + "r0", manifest, terminal_logical_request_id=None, reward=0.0 + ) + assert receipt["terminal_model_call_id"] is None + assert receipt["capture_poisoned"] is True + assert receipt["failure_reason"] == "invalid_manifest_row" + + +def test_receipt_assembly_keeps_dead_branch_siblings_in_the_manifest() -> None: + """A retry sibling stays enumerable (its staged row must be cleaned) but + never becomes the terminal call.""" + env = _capture_env() + records = [ + _manifest_record("c1"), + _manifest_record("c2", parent="c1"), + _manifest_record("c2r", parent="c1"), + ] + manifest = {"rollout_id": "r0", "records": records, "failures": []} + receipt = env._assemble_receipt( + "r0", manifest, terminal_logical_request_id="resp-c2r", reward=1.0 + ) + assert receipt["terminal_model_call_id"] == "c2r" + assert receipt["capture_poisoned"] is False + assert {r["model_call_id"] for r in receipt["manifest"]} == {"c1", "c2", "c2r"} + + +def test_receipt_postprocess_returns_placeholder_on_fetch_failure() -> None: + env = _capture_env() + env._control = AsyncMock(side_effect=RuntimeError("control plane down")) + + result = asyncio.run( + env._postprocess_receipt_mode( + {"_ng_rollout_id": "r0"}, + {"reward": 1.0, "terminal_logical_request_id": "resp-c1"}, + ) + ) + assert result["receipt"] is None + + +def test_response_id_witness_resolves_a_final_call_retry() -> None: + """The heuristic masks a retried final call; the scored response's served + envelope id names the sibling the harness kept, recovering the rollout.""" + env = _capture_env() + records = [ + _manifest_record("c1"), + _manifest_record("c2", parent="c1", cumulative_hash="a" * 64), + _manifest_record("c2r", parent="c1", cumulative_hash="c" * 64), + ] + manifest = {"rollout_id": "r0", "records": records, "failures": []} + receipt = env._assemble_receipt( + "r0", + manifest, + terminal_logical_request_id=None, + scored_response={"id": "resp-c2r", "output": []}, + reward=1.0, + ) + assert receipt["terminal_model_call_id"] == "c2r" + assert receipt["terminal_selection"] == "response_id" + assert receipt["capture_poisoned"] is False + + +def test_unattributed_scored_response_falls_back_to_the_heuristic() -> None: + env = _capture_env() + records = [ + _manifest_record("c1"), + _manifest_record("c2", parent="c1"), + ] + manifest = {"rollout_id": "r0", "records": records, "failures": []} + receipt = env._assemble_receipt( + "r0", + manifest, + terminal_logical_request_id=None, + scored_response={"id": "resp-unknown", "output": []}, + reward=1.0, + ) + assert receipt["terminal_model_call_id"] == "c2" + assert receipt["terminal_selection"] == "heuristic" + assert "response_id_no_match" in (receipt["terminal_attribution_reason"] or "") + + +def test_declared_and_response_id_witnesses_corroborate() -> None: + env = _capture_env() + records = [ + _manifest_record("c1"), + _manifest_record("c2", parent="c1"), + ] + manifest = {"rollout_id": "r0", "records": records, "failures": []} + receipt = env._assemble_receipt( + "r0", + manifest, + terminal_logical_request_id="resp-c2", + scored_response={"id": "resp-c2", "output": []}, + reward=1.0, + ) + assert receipt["terminal_model_call_id"] == "c2" + assert receipt["terminal_selection"] == "declared" + assert "corroborated_by=response_id" in (receipt["terminal_attribution_reason"] or "") + + +def test_witness_disagreement_masks_a_retry_instead_of_guessing() -> None: + """A declaration naming one retry sibling while the scored response's id + names the other is a contradiction: attribution abstains, the declared + path stays authoritative, and the rollout masks.""" + env = _capture_env() + records = [ + _manifest_record("c1"), + _manifest_record("c2", parent="c1", cumulative_hash="a" * 64), + _manifest_record("c2r", parent="c1", cumulative_hash="c" * 64), + ] + manifest = {"rollout_id": "r0", "records": records, "failures": []} + receipt = env._assemble_receipt( + "r0", + manifest, + terminal_logical_request_id="resp-c2", + scored_response={"id": "resp-c2r", "output": []}, + reward=1.0, + ) + assert receipt["terminal_model_call_id"] is None + assert receipt["capture_poisoned"] is True + assert "witness_disagreement[" in (receipt["terminal_attribution_reason"] or "") + + +def test_postprocess_passes_the_scored_response_to_attribution() -> None: + env = _capture_env() + records = [ + _manifest_record("c1"), + _manifest_record("c2", parent="c1", cumulative_hash="a" * 64), + _manifest_record("c2r", parent="c1", cumulative_hash="c" * 64), + ] + env._control = AsyncMock( + return_value={"rollout_id": "r0", "records": records, "failures": []} + ) + result = asyncio.run( + env._postprocess_receipt_mode( + {"_ng_rollout_id": "r0"}, + {"reward": 1.0, "response": {"id": "resp-c2", "output": []}}, + ) + ) + receipt = result["receipt"] + assert receipt["terminal_model_call_id"] == "c2" + assert receipt["terminal_selection"] == "response_id" diff --git a/tests/unit/environments/test_nemo_gym_utils.py b/tests/unit/environments/test_nemo_gym_utils.py index fa60df05fb1..904f3ec155f 100644 --- a/tests/unit/environments/test_nemo_gym_utils.py +++ b/tests/unit/environments/test_nemo_gym_utils.py @@ -55,6 +55,18 @@ True, False, ), + # A tool-call-only assistant item carries content: None — a structured + # (executed) call, never a penalty (regression: jobs 6342333/6358268). + ( + {"content": None, "tool_calls": [{"function": {"name": "bash"}}]}, + False, + False, + ), + ( + {}, + False, + False, + ), ], ) def test_detect_invalid_tool_call_and_malformed_thinking( diff --git a/tests/unit/experience/test_finalizer_actor.py b/tests/unit/experience/test_finalizer_actor.py new file mode 100644 index 00000000000..cf03c09c725 --- /dev/null +++ b/tests/unit/experience/test_finalizer_actor.py @@ -0,0 +1,79 @@ +# Copyright (c) 2026, NVIDIA CORPORATION. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +"""Metadata-only finalizer actor boundary tests.""" + +from __future__ import annotations + +import pytest +import torch + +from nemo_rl.data_plane import KVBatchMeta +from nemo_rl.experience.blackbox_finalizer import FinalizedGroup +from nemo_rl.experience.finalizer_actor import ( + FinalizationRequest, + assert_metadata_only, +) + + +def _request() -> FinalizationRequest: + return FinalizationRequest( + group_id="group", + rollout_ids=("group_g0",), + receipts=( + { + "rollout_id": "group_g0", + "manifest": [ + { + "call_id": "call", + "staging_key": "group_g0/call", + "delta_len": 2, + } + ], + }, + ), + rewards=(1.0,), + fallback_weight_version=4, + ) + + +def test_finalizer_request_and_result_are_metadata_only() -> None: + assert_metadata_only(_request()) + result = FinalizedGroup( + meta=KVBatchMeta( + partition_id="canonical", + task_name="train", + sample_ids=["group_g0"], + fields=["input_ids"], + sequence_lengths=[3], + tags=[{"weight_version": 4}], + ), + group_min_wv=4, + group_max_wv=4, + staging_keys=["group_g0/call"], + metrics={"finalize/total_ms": 1.0}, + ) + assert_metadata_only(result) + + +@pytest.mark.parametrize( + "payload", + [ + torch.ones(2), + {"input_ids": [1, 2]}, + {"routed_experts": [[[[1, 2]]]]}, + ], +) +def test_metadata_guard_rejects_tensor_and_heavy_row_payloads(payload) -> None: + with pytest.raises(TypeError): + assert_metadata_only(payload) diff --git a/tests/unit/experience/test_rollout_manager.py b/tests/unit/experience/test_rollout_manager.py index 30909aa856b..3bd614edbf4 100644 --- a/tests/unit/experience/test_rollout_manager.py +++ b/tests/unit/experience/test_rollout_manager.py @@ -77,6 +77,7 @@ def __init__(self) -> None: self.reserve_calls: list[int] = [] # weight_versions passed to reserve self.commit_calls: list[tuple[str, object, int, int]] = [] self.remove_calls: list[str] = [] + self.abort_calls: list[str] = [] # reserve(weight_version=X) -> group_id; commit fills the slot. self._slots: list[str] = [] @@ -86,13 +87,22 @@ def reserve( weight_version: int, target_step: int | None = None, group_id: str | None = None, + rollout_ids: list[str] | None = None, ) -> str: + del target_step, rollout_ids if group_id is None: group_id = str(uuid.uuid4()) self.reserve_calls.append(weight_version) self._slots.append(group_id) return group_id + def abort(self, group_id: str) -> bool: + self.abort_calls.append(group_id) + if group_id in self._slots: + self._slots.remove(group_id) + return True + return False + async def commit( self, group_id: str, @@ -132,6 +142,7 @@ def _make_manager(buffer: _FakeBuffer, impl: _FakeImpl) -> RolloutManager: mgr._tokenizer = None mgr._num_generations_per_prompt = 1 mgr._tq_buffer = buffer + mgr._env_handles = {} mgr._weight_version = 0 return mgr @@ -294,6 +305,7 @@ async def _second_run(_sample): second_mgr._tokenizer = None second_mgr._num_generations_per_prompt = 1 second_mgr._tq_buffer = buf + second_mgr._env_handles = {} second_mgr._weight_version = 0 async def _drive(): @@ -319,6 +331,39 @@ def test_requires_tq_buffer(self): with pytest.raises(AssertionError, match="tq_buffer"): _run(mgr.generate_and_push({"prompt": "p"})) + def test_failed_rollout_aborts_reserved_slot(self): + """A dispatch that raises must not leave a phantom unready slot.""" + + async def _boom(_input_sample): + raise RuntimeError("rollout exploded") + + buf = _FakeBuffer() + mgr = _make_manager(buf, _FakeImpl(on_run=_boom)) + + with pytest.raises(RuntimeError, match="rollout exploded"): + _run(mgr.generate_and_push({"prompt": "p"})) + + assert len(buf.reserve_calls) == 1 + assert buf.commit_calls == [] + assert len(buf.remove_calls) == 1 + assert buf._slots == [] # the reserved slot was dropped + + def test_failed_commit_aborts_reserved_slot(self): + """Commit failures (e.g. evicted slot) also abort the reservation.""" + + class _CommitBoomBuffer(_FakeBuffer): + async def commit( + self, group_id, record, start_weight_version, end_weight_version + ): + raise ValueError("no live slot") + + buf = _CommitBoomBuffer() + mgr = _make_manager(buf, _FakeImpl()) + + with pytest.raises(ValueError, match="no live slot"): + _run(mgr.generate_and_push({"prompt": "p"})) + assert len(buf.remove_calls) == 1 + # --------------------------------------------------------------------------- # Tests for RolloutManager @@ -942,3 +987,100 @@ def _last_assistant_token_ids(msg_log): assert orig_val == pytest.approx(new_val), ( f"rollout_metrics[{key!r}] mismatch — original {orig_val}, manager {new_val}" ) + + +class _FakeCaptureBuffer(_FakeBuffer): + def __init__(self): + super().__init__() + self.reserve_rollout_ids: list[list[str] | None] = [] + + def reserve( + self, *, weight_version, target_step=None, group_id=None, rollout_ids=None + ): + self.reserve_rollout_ids.append(rollout_ids) + return super().reserve( + weight_version=weight_version, + target_step=target_step, + group_id=group_id, + rollout_ids=rollout_ids, + ) + + +def _receipt_record(rollout_ids, receipts): + completions = [ + Completion( + message_log=[], + env_extras={"reward": 0.5, "ng_receipt": receipt, "ng_rollout_id": rid}, + truncated=False, + reward=0.5, + ) + for rid, receipt in zip(rollout_ids, receipts) + ] + return PromptGroupRecord( + prompt_idx=0, + prompt=[], + extra_env_info={}, + metadata={"task_name": "nemo_gym"}, + completions=completions, + rollout_metrics={}, + ) + + +def _make_capture_manager(buf, *, on_run=None, num_generations=2): + mgr = object.__new__(RolloutManager) + mgr._tokenizer = None + mgr._num_generations_per_prompt = num_generations + mgr._tq_buffer = buf + mgr._env_handles = {} + mgr._weight_version = 7 + + class _CaptureImpl: + def __init__(self): + self.seen_rollout_ids = None + + async def run_rollout(self, _sample, *, rollout_ids=None): + self.seen_rollout_ids = rollout_ids + if on_run is not None: + await on_run(_sample) + return _receipt_record( + rollout_ids, [{"rollout_id": rid} for rid in rollout_ids] + ) + + mgr._impl = _CaptureImpl() + return mgr + + +class TestGenerateForFinalizationFlow: + def test_mints_ids_and_returns_metadata_request(self): + buf = _FakeCaptureBuffer() + mgr = _make_capture_manager(buf) + + request = _run(mgr.generate_for_finalization({"prompt": "p"}, target_step=5)) + + # Rollout ids were minted from the reserved group id and threaded + # end to end: reserve -> impl -> metadata-only actor request. + (group_id,) = buf._slots + expected_ids = [f"{group_id}_g0", f"{group_id}_g1"] + assert buf.reserve_rollout_ids == [expected_ids] + assert mgr._impl.seen_rollout_ids == expected_ids + assert request.group_id == group_id + assert request.rollout_ids == tuple(expected_ids) + assert [r["rollout_id"] for r in request.receipts] == expected_ids + assert request.rewards == (0.5, 0.5) + assert request.fallback_weight_version == 7 + # Finalization and commit are exclusively owned by the controller's + # actor-pool path; the manager leaves the reservation unready. + assert buf.commit_calls == [] + + def test_failed_dispatch_aborts_the_reservation(self): + buf = _FakeCaptureBuffer() + + async def _boom(_sample): + raise RuntimeError("rollout exploded") + + mgr = _make_capture_manager(buf, on_run=_boom) + with pytest.raises(RuntimeError, match="rollout exploded"): + _run(mgr.generate_for_finalization({"prompt": "p"})) + # The slot is released; abandoned staged rows are swept with the + # staging partition at run end (no per-rollout control-plane call). + assert len(buf.abort_calls) == 1 diff --git a/tests/unit/experience/test_route_plan.py b/tests/unit/experience/test_route_plan.py new file mode 100644 index 00000000000..af292e0cd5d --- /dev/null +++ b/tests/unit/experience/test_route_plan.py @@ -0,0 +1,120 @@ +# Copyright (c) 2026, NVIDIA CORPORATION. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +"""Strict deferred-route plan contract tests.""" + +from __future__ import annotations + +import pytest + +from nemo_rl.experience.route_plan import ( + ROUTE_PLAN_SCHEMA_VERSION, + RouteAssemblyPlan, + RouteSpan, + classify_route_span, + decode_route_plan, + encode_route_plan, +) + +_DIGEST = "0" * 64 + + +def _span( + staging_key: str, + carry_len: int, + generation_len: int, + staged_route_len: int, +) -> RouteSpan: + return RouteSpan( + staging_key, + carry_len, + generation_len, + staged_route_len, + extras_digest_version=1, + extras_digest=_DIGEST, + ) + + +def _plan() -> RouteAssemblyPlan: + return RouteAssemblyPlan( + schema_version=ROUTE_PLAN_SCHEMA_VERSION, + staging_partition="staging", + spans=( + _span("r/c0", carry_len=2, generation_len=2, staged_route_len=4), + _span("r/c1", carry_len=3, generation_len=1, staged_route_len=1), + _span("r/c2", carry_len=0, generation_len=0, staged_route_len=0), + _span("r/c3", carry_len=1, generation_len=1, staged_route_len=0), + ), + cleanup_staging_keys=("r/c0", "r/c1", "r/c2", "r/c3", "r/off_chain"), + expected_token_length=10, + ) + + +def test_route_plan_round_trip_is_strict_and_lossless() -> None: + plan = _plan() + assert decode_route_plan(encode_route_plan(plan)) == plan + + +def test_route_span_classification_uses_shared_length_table() -> None: + spans = _plan().spans + assert [classify_route_span(span) for span in spans] == [ + "full", + "tail", + "sentinel", + "sentinel", + ] + + +@pytest.mark.parametrize( + ("mutation", "match"), + [ + ({"schema_version": 999}, "unsupported route plan schema"), + ({"expected_token_length": -1}, "must be non-negative"), + ({"expected_token_length": 9}, "spans contribute 10 tokens"), + ], +) +def test_route_plan_rejects_invalid_top_level_values(mutation, match) -> None: + encoded = encode_route_plan(_plan()) + encoded.update(mutation) + with pytest.raises(ValueError, match=match): + decode_route_plan(encoded) + + +def test_route_plan_rejects_unknown_or_missing_fields() -> None: + encoded = encode_route_plan(_plan()) + encoded["compat_guess"] = True + with pytest.raises(ValueError, match="fields must be exactly"): + decode_route_plan(encoded) + + del encoded["compat_guess"] + del encoded["spans"][0]["staged_route_len"] + with pytest.raises(ValueError, match="fields must be exactly"): + decode_route_plan(encoded) + + +def test_route_plan_rejects_keys_outside_full_cleanup_manifest() -> None: + encoded = encode_route_plan(_plan()) + encoded["cleanup_staging_keys"].remove("r/c1") + with pytest.raises(ValueError, match="outside cleanup_staging_keys"): + decode_route_plan(encoded) + + +def test_placeholder_plan_can_carry_length_without_route_reads() -> None: + placeholder = RouteAssemblyPlan( + schema_version=ROUTE_PLAN_SCHEMA_VERSION, + staging_partition="staging", + spans=(), + cleanup_staging_keys=("r/rejected",), + expected_token_length=7, + ) + assert decode_route_plan(encode_route_plan(placeholder)) == placeholder diff --git a/tests/unit/models/generation/test_vllm_token_capture_hosting.py b/tests/unit/models/generation/test_vllm_token_capture_hosting.py new file mode 100644 index 00000000000..87336f2c80b --- /dev/null +++ b/tests/unit/models/generation/test_vllm_token_capture_hosting.py @@ -0,0 +1,408 @@ +# Copyright (c) 2026, NVIDIA CORPORATION. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +"""S2 worker hosting: install_capture wiring, fan-outs, version stamping. + +Marked nemo_gym (run with ``--nemo-gym-only``): the hosting seam imports +Gym's capture core. No engine or GPU is needed — the worker methods are +driven unbound against light fakes, and the VllmGeneration fan-outs against +a mock worker group. +""" + +from __future__ import annotations + +import asyncio +from types import SimpleNamespace +from unittest.mock import MagicMock + +import pytest + +nemo_gym = pytest.importorskip("nemo_gym.token_id_capture.staging") + +from nemo_gym.token_id_capture.staging.capture import ( # noqa: E402 + RolloutTokenCapture, +) +from nemo_gym.token_id_capture.staging.records import ( # noqa: E402 + CaptureAdmission, + StagedCallRecord, + StageResult, +) + +from nemo_rl.models.generation.vllm.vllm_generation import VllmGeneration # noqa: E402 +from nemo_rl.models.generation.vllm.vllm_worker_async import ( # noqa: E402 + VllmAsyncGenerationWorkerImpl, +) + +pytestmark = pytest.mark.nemo_gym + + +class _MemorySink: + def __init__(self) -> None: + self.records: list[StagedCallRecord] = [] + + def stage(self, record: StagedCallRecord) -> StageResult: + self.records.append(record) + return StageResult(ok=True, staging_key=record.staging_key) + + +def _fake_worker(*, is_model_owner: bool = True) -> SimpleNamespace: + """The attribute surface setup_token_capture touches, minus the engine.""" + worker = SimpleNamespace( + is_model_owner=is_model_owner, + token_capture=None, + _rollout_weight_version=0, + _staging_source=None, + _prefix_cache={}, + ) + worker.install_token_capture = lambda capture: setattr( + worker, "token_capture", capture + ) + return worker + + +def test_setup_token_capture_installs_capture_with_vllm_adapter(monkeypatch): + sink = _MemorySink() + monkeypatch.setattr( + "nemo_rl.data_plane.build_data_plane_client", + lambda dp_cfg, bootstrap: MagicMock(name="dp_client"), + ) + monkeypatch.setattr( + "nemo_rl.data_plane.tq_token_sink.TQTokenSink", + lambda dp_client, *, staging_partition: sink, + ) + worker = _fake_worker() + + installed = asyncio.run( + VllmAsyncGenerationWorkerImpl.setup_token_capture( + worker, dp_cfg={"backend": "simple"}, staging_partition="rollout_staging" + ) + ) + + assert installed is True + assert isinstance(worker.token_capture, RolloutTokenCapture) + assert worker.token_capture.adapter is not None + # The adapter is the vLLM one (prefix ids enter via the worker's field). + payload = worker.token_capture.adapter.enter_prefix({}, [1, 2]) + assert payload["required_prefix_token_ids"] == [1, 2] + + +def test_setup_token_capture_skips_non_model_owners(monkeypatch): + worker = _fake_worker(is_model_owner=False) + installed = asyncio.run( + VllmAsyncGenerationWorkerImpl.setup_token_capture( + worker, dp_cfg={}, staging_partition="rollout_staging" + ) + ) + assert installed is False + assert worker.token_capture is None + + +def test_weight_version_is_stamped_from_worker_state(monkeypatch): + """The install closure reads _rollout_weight_version live: a + set_rollout_weight_version between calls changes the stamp.""" + sink = _MemorySink() + monkeypatch.setattr( + "nemo_rl.data_plane.build_data_plane_client", + lambda dp_cfg, bootstrap: MagicMock(), + ) + monkeypatch.setattr( + "nemo_rl.data_plane.tq_token_sink.TQTokenSink", + lambda dp_client, *, staging_partition: sink, + ) + worker = _fake_worker() + asyncio.run( + VllmAsyncGenerationWorkerImpl.setup_token_capture( + worker, dp_cfg={}, staging_partition="rollout_staging" + ) + ) + + asyncio.run(VllmAsyncGenerationWorkerImpl.set_rollout_weight_version(worker, 4)) + first = worker.token_capture.begin_call( + CaptureAdmission(rollout_id="r", model_call_id="c1", mode="text") + ) + asyncio.run(VllmAsyncGenerationWorkerImpl.set_rollout_weight_version(worker, 5)) + second = worker.token_capture.begin_call( + CaptureAdmission(rollout_id="r", model_call_id="c2", mode="text") + ) + + assert (first.weight_version, second.weight_version) == (4, 5) + + coords = worker.token_capture.complete_call( + first, prompt_token_ids=[1], generated_token_ids=[2], generated_logprobs=[-0.1] + ) + assert coords.weight_version == 4 + assert sink.records[0].weight_version == 4 + + +def _generation_with_mock_group(*, async_engine: bool = True) -> VllmGeneration: + gen = object.__new__(VllmGeneration) + gen.cfg = {"vllm_cfg": {"async_engine": async_engine}} + gen.worker_group = MagicMock() + gen.worker_group.run_all_workers_single_data.return_value = [] + return gen + + +def test_generation_setup_token_capture_fans_out(monkeypatch): + gen = _generation_with_mock_group() + monkeypatch.setattr( + "nemo_rl.models.generation.vllm.vllm_generation.ray.get", + lambda futures: futures, + ) + gen.setup_token_capture({"backend": "simple"}, "rollout_staging") + gen.worker_group.run_all_workers_single_data.assert_called_once_with( + "setup_token_capture", + dp_cfg={"backend": "simple"}, + staging_partition="rollout_staging", + run_rank_0_only_axes=["tensor_parallel", "pipeline_parallel"], + ) + + +def test_generation_setup_token_capture_requires_async_engine(): + gen = _generation_with_mock_group(async_engine=False) + with pytest.raises(AssertionError, match="async vLLM engine"): + gen.setup_token_capture({}, "rollout_staging") + + +def test_generation_set_rollout_weight_version_fans_out(monkeypatch): + gen = _generation_with_mock_group() + monkeypatch.setattr( + "nemo_rl.models.generation.vllm.vllm_generation.ray.get", + lambda futures: futures, + ) + gen.set_rollout_weight_version(7) + gen.worker_group.run_all_workers_single_data.assert_called_once_with( + "set_rollout_weight_version", + version=7, + run_rank_0_only_axes=["tensor_parallel", "pipeline_parallel"], + ) + + +# --------------------------------------------------------------------------- +# S4: the request-path hookup (begin -> finish/abort around a served call) +# --------------------------------------------------------------------------- + + +class _FakeRequest(SimpleNamespace): + pass + + +def _worker_with_capture(sink: _MemorySink): + from nemo_gym.token_id_capture.adapters.vllm import VLLMCaptureAdapter + + worker = _fake_worker() + worker._capture_calls = {} + worker._prefix_cache = {} + worker._staging_source = None + worker._delta_align_routed_experts = ( + VllmAsyncGenerationWorkerImpl._delta_align_routed_experts + ) + worker._fetch_chain_prefix = lambda staging_chain: ( + VllmAsyncGenerationWorkerImpl._fetch_chain_prefix(worker, staging_chain) + ) + worker.token_capture = RolloutTokenCapture( + sink=sink, + weight_version_fn=lambda: worker._rollout_weight_version, + adapter=VLLMCaptureAdapter(), + ) + return worker + + +class _MemoryPrefixSource: + def __init__(self, deltas: dict[str, list[int]]) -> None: + self.deltas = deltas + self.calls: list[list[str]] = [] + + def fetch_prefix_token_ids(self, staging_keys: list[str]) -> list[int]: + self.calls.append(list(staging_keys)) + return [token for key in staging_keys for token in self.deltas[key]] + + +def _served_content(gen_ids, logprobs): + return { + "choices": [ + { + "index": 0, + "message": {"role": "assistant", "content": "x"}, + "logprobs": { + "content": [ + {"token": f"token_id:{t}", "logprob": lp} + for t, lp in zip(gen_ids, logprobs) + ] + }, + } + ] + } + + +def test_request_capture_round_trip_stages_and_rides_coords(): + sink = _MemorySink() + worker = _worker_with_capture(sink) + request = _FakeRequest( + ng_capture={ + "rollout_id": "r0", + "model_call_id": "c1", + "parent_call_id": None, + "prev_len": 0, + "mode": "text", + }, + stream=False, + ) + VllmAsyncGenerationWorkerImpl._begin_request_capture(worker, request, [10, 11, 12]) + content = _served_content([13, 14], [-0.1, -0.2]) + content = VllmAsyncGenerationWorkerImpl._finish_request_capture( + worker, request, content + ) + # Bytes were staged before the coords existed (fail-closed ordering). + assert len(sink.records) == 1 + assert sink.records[0].token_ids_delta == [10, 11, 12, 13, 14] + coords = content["ng_commit_coords"] + assert coords["disposition"] == "staged" + assert (coords["delta_len"], coords["cum_len"]) == (5, 5) + # Coords are token-free: hashes ride the wire, deltas stay in the sink. + assert "token_ids_delta" not in coords + assert coords["chain_hash"] == sink.records[0].chain_hash + assert coords["cumulative_hash"] == sink.records[0].cumulative_hash + # Logprobs never transit worker -> gate; state map is drained. + assert ( + "logprobs" not in content["choices"][0] + or content["choices"][0]["logprobs"] is None + ) + assert worker._capture_calls == {} + + +def test_request_capture_token_in_prev_len_chains(): + sink = _MemorySink() + worker = _worker_with_capture(sink) + request = _FakeRequest( + ng_capture={ + "rollout_id": "r0", + "model_call_id": "c2", + "parent_call_id": "c1", + "prev_len": 3, + "mode": "token_in", + "required_prefix_token_ids": [10, 11, 12], + "parent_chain_hash": "1" * 64, + }, + stream=False, + ) + spliced_prompt = [10, 11, 12, 20, 21] # exact prefix + fresh suffix + VllmAsyncGenerationWorkerImpl._begin_request_capture( + worker, request, spliced_prompt + ) + content = VllmAsyncGenerationWorkerImpl._finish_request_capture( + worker, request, _served_content([22], [-0.5]) + ) + coords = content["ng_commit_coords"] + assert coords["parent_call_id"] == "c1" + assert (coords["delta_len"], coords["cum_len"]) == (3, 6) + assert sink.records[0].token_ids_delta == [20, 21, 22] + + +def test_staging_chain_fetches_patches_and_begins_capture(): + sink = _MemorySink() + worker = _worker_with_capture(sink) + source = _MemoryPrefixSource({"r0/c1": [10, 11], "r0/c2": [12]}) + worker._staging_source = source + request = _FakeRequest( + ng_capture={ + "rollout_id": "r0", + "model_call_id": "c3", + "parent_call_id": "c2", + "prev_len": 3, + "mode": "token_in", + "staging_chain": ["r0/c1", "r0/c2"], + "parent_chain_hash": "2" * 64, + }, + stream=False, + ) + + prefix = VllmAsyncGenerationWorkerImpl._patch_chain_prefix( + worker, request.ng_capture + ) + VllmAsyncGenerationWorkerImpl._begin_request_capture(worker, request, prefix + [20]) + + assert prefix == [10, 11, 12] + assert source.calls == [["r0/c1", "r0/c2"]] + assert request.ng_capture["required_prefix_token_ids"] == prefix + call, prompt = worker._capture_calls[id(request)] + assert call.admission.required_prefix_token_ids == prefix + assert prompt == [10, 11, 12, 20] + + +def test_staging_chain_cache_fetches_only_uncached_suffix(): + worker = _worker_with_capture(_MemorySink()) + source = _MemoryPrefixSource({"r0/c1": [10, 11], "r0/c2": [12]}) + worker._staging_source = source + + first = VllmAsyncGenerationWorkerImpl._fetch_chain_prefix(worker, ["r0/c1"]) + second = VllmAsyncGenerationWorkerImpl._fetch_chain_prefix( + worker, ["r0/c1", "r0/c2"] + ) + + assert first == [10, 11] + assert second == [10, 11, 12] + assert source.calls == [["r0/c1"], ["r0/c2"]] + + +def test_staging_chain_rejects_fetched_length_mismatch(): + worker = _worker_with_capture(_MemorySink()) + worker._staging_source = _MemoryPrefixSource({"r0/c1": [10, 11]}) + admission = {"prev_len": 3, "staging_chain": ["r0/c1"]} + + with pytest.raises(ValueError, match="expected 3, fetched 2"): + VllmAsyncGenerationWorkerImpl._patch_chain_prefix(worker, admission) + + assert "required_prefix_token_ids" not in admission + + +def test_request_capture_is_a_noop_without_context_or_capture(): + sink = _MemorySink() + worker = _worker_with_capture(sink) + plain = _FakeRequest(stream=False) # no ng_capture attribute + VllmAsyncGenerationWorkerImpl._begin_request_capture(worker, plain, [1, 2]) + content = { + "choices": [{"message": {"role": "assistant"}, "logprobs": {"content": []}}] + } + out = VllmAsyncGenerationWorkerImpl._finish_request_capture( + worker, plain, dict(content) + ) + assert "ng_commit_coords" not in out + assert out["choices"][0]["logprobs"] is not None # untouched off the capture path + assert sink.records == [] + + +def test_request_capture_abort_fails_the_call_and_drains_state(): + sink = _MemorySink() + worker = _worker_with_capture(sink) + request = _FakeRequest( + ng_capture={ + "rollout_id": "r0", + "model_call_id": "c1", + "parent_call_id": None, + "prev_len": 0, + "mode": "text", + }, + stream=False, + ) + VllmAsyncGenerationWorkerImpl._begin_request_capture(worker, request, [1, 2]) + VllmAsyncGenerationWorkerImpl._abort_request_capture( + worker, request, reason="engine_error" + ) + assert worker._capture_calls == {} + assert sink.records == [] + # A late finish after abort is a no-op (state already drained). + out = VllmAsyncGenerationWorkerImpl._finish_request_capture( + worker, request, _served_content([3], [-0.1]) + ) + assert "ng_commit_coords" not in out diff --git a/tests/unit/single_controller/test_finalizer_lifecycle.py b/tests/unit/single_controller/test_finalizer_lifecycle.py new file mode 100644 index 00000000000..5f061fec181 --- /dev/null +++ b/tests/unit/single_controller/test_finalizer_lifecycle.py @@ -0,0 +1,248 @@ +# Copyright (c) 2026, NVIDIA CORPORATION. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +"""Lifecycle tests for actor-pool finalization and deferred-route ownership.""" + +from __future__ import annotations + +import asyncio +from types import SimpleNamespace +from typing import Any +from unittest.mock import AsyncMock, MagicMock + +import pytest + +from nemo_rl.algorithms.single_controller import SingleControllerActor +from nemo_rl.data_plane import KVBatchMeta +from nemo_rl.data_plane.schema import ROUTE_PLAN_TAG +from nemo_rl.experience.blackbox_finalizer import FinalizedGroup +from nemo_rl.experience.finalizer_actor import FinalizationRequest +from nemo_rl.experience.route_plan import ( + ROUTE_PLAN_SCHEMA_VERSION, + RouteAssemblyPlan, + RouteSpan, + encode_route_plan, +) + + +class _RemoteFinalize: + def __init__( + self, + *, + result: FinalizedGroup | None = None, + error: BaseException | None = None, + ) -> None: + self._result = result + self._error = error + self.calls: list[FinalizationRequest] = [] + + def remote(self, request: FinalizationRequest) -> Any: + self.calls.append(request) + + async def _result(): + if self._error is not None: + raise self._error + assert self._result is not None + return self._result + + return _result() + + +class _DataPlaneClient: + def __init__(self) -> None: + self.clear_calls: list[dict[str, Any]] = [] + + async def clear_samples(self, *, sample_ids: list[str], partition_id: str) -> None: + self.clear_calls.append( + {"sample_ids": list(sample_ids), "partition_id": partition_id} + ) + + +def _request() -> FinalizationRequest: + return FinalizationRequest( + group_id="group", + rollout_ids=("group_g0",), + receipts=( + { + "manifest": [ + {"staging_key": "group_g0/call"}, + {"staging_key": "group_g0/call"}, + ] + }, + ), + rewards=(1.0,), + fallback_weight_version=3, + ) + + +def _controller(actor: object) -> Any: + controller_cls = SingleControllerActor.__ray_metadata__.modified_class + ctrl = object.__new__(controller_cls) + ctrl._available_finalizers = asyncio.Queue() + ctrl._available_finalizers.put_nowait(actor) + ctrl._active_finalizers = 0 + ctrl._finalizer_waiters = 0 + ctrl._finalizer_unknown_outcomes = 0 + ctrl._finalizer_metrics_by_group = {} + ctrl._buffer = MagicMock() + ctrl._buffer.commit_finalized = AsyncMock() + ctrl._dp_client = _DataPlaneClient() + ctrl._partition_id = "canonical" + ctrl._master_config = SimpleNamespace( + token_capture=SimpleNamespace(staging_partition="staging") + ) + return ctrl + + +def test_successful_actor_finalization_returns_actor_and_transfers_ownership() -> None: + meta = KVBatchMeta( + partition_id="canonical", + task_name="train", + sample_ids=["group_g0"], + fields=["input_ids"], + sequence_lengths=[3], + tags=[{"weight_version": 3}], + ) + result = FinalizedGroup( + meta=meta, + group_min_wv=3, + group_max_wv=3, + staging_keys=["group_g0/call"], + metrics={"finalize/group_ms": 1.0}, + ) + finalize = _RemoteFinalize(result=result) + actor = SimpleNamespace(finalize=finalize) + ctrl = _controller(actor) + request = _request() + + asyncio.run(ctrl._finalize_with_actor(request)) + + assert finalize.calls == [request] + assert ctrl._available_finalizers.get_nowait() is actor + assert ctrl._active_finalizers == 0 + assert ctrl._finalizer_unknown_outcomes == 0 + ctrl._buffer.commit_finalized.assert_awaited_once_with( + "group", + meta, + 3, + 3, + staging_keys=["group_g0/call"], + ) + assert ctrl._finalizer_metrics_by_group["group"]["finalize/group_ms"] == 1.0 + + +def test_actor_rpc_failure_is_fatal_and_does_not_retry_or_requeue_actor() -> None: + finalize = _RemoteFinalize(error=RuntimeError("actor died after submission")) + actor = SimpleNamespace(finalize=finalize) + ctrl = _controller(actor) + request = _request() + + with pytest.raises(RuntimeError, match="actor died after submission"): + asyncio.run(ctrl._finalize_with_actor(request)) + + assert finalize.calls == [request] + assert ctrl._available_finalizers.empty() + assert ctrl._active_finalizers == 0 + assert ctrl._finalizer_unknown_outcomes == 1 + ctrl._buffer.commit_finalized.assert_not_awaited() + ctrl._buffer.abort.assert_not_called() + assert ctrl._dp_client.clear_calls == [] + + +def test_missing_actor_metadata_cleans_known_canonical_and_staging_ownership() -> None: + result = FinalizedGroup( + meta=None, + group_min_wv=3, + group_max_wv=3, + staging_keys=["group_g0/call"], + metrics={}, + ) + actor = SimpleNamespace(finalize=_RemoteFinalize(result=result)) + ctrl = _controller(actor) + + with pytest.raises(RuntimeError, match="no metadata for non-dropped group"): + asyncio.run(ctrl._finalize_with_actor(_request())) + + assert ctrl._dp_client.clear_calls == [ + {"sample_ids": ["group_g0"], "partition_id": "canonical"}, + {"sample_ids": ["group_g0/call"], "partition_id": "staging"}, + ] + ctrl._buffer.abort.assert_called_once_with("group") + assert ctrl._available_finalizers.get_nowait() is actor + + +def test_dropped_actor_group_cleans_known_canonical_and_staging_ownership() -> None: + result = FinalizedGroup( + meta=None, + group_min_wv=3, + group_max_wv=3, + staging_keys=["group_g0/call"], + metrics={}, + dropped=True, + ) + actor = SimpleNamespace(finalize=_RemoteFinalize(result=result)) + ctrl = _controller(actor) + + with pytest.raises(RuntimeError, match="group group dropped"): + asyncio.run(ctrl._finalize_with_actor(_request())) + + assert ctrl._dp_client.clear_calls == [ + {"sample_ids": ["group_g0"], "partition_id": "canonical"}, + {"sample_ids": ["group_g0/call"], "partition_id": "staging"}, + ] + ctrl._buffer.abort.assert_called_once_with("group") + assert ctrl._available_finalizers.get_nowait() is actor + + +def test_post_train_cleanup_clears_canonical_rows_and_route_plan_staging_keys() -> None: + plan = RouteAssemblyPlan( + schema_version=ROUTE_PLAN_SCHEMA_VERSION, + staging_partition="staging", + spans=( + RouteSpan( + staging_key="group_g0/call", + carry_len=2, + generation_len=1, + staged_route_len=3, + extras_digest_version=1, + extras_digest="0" * 64, + ), + ), + cleanup_staging_keys=("group_g0/call", "group_g0/fork"), + expected_token_length=3, + ) + meta = KVBatchMeta( + partition_id="canonical", + task_name="train", + sample_ids=["group_g0", "group_g1"], + fields=["input_ids"], + tags=[ + {ROUTE_PLAN_TAG: encode_route_plan(plan)}, + {ROUTE_PLAN_TAG: encode_route_plan(plan)}, + ], + ) + ctrl = _controller(SimpleNamespace()) + + asyncio.run(ctrl._cleanup_consumed_metas([meta])) + + assert ctrl._dp_client.clear_calls == [ + { + "sample_ids": ["group_g0", "group_g1"], + "partition_id": "canonical", + }, + { + "sample_ids": ["group_g0/call", "group_g0/fork"], + "partition_id": "staging", + }, + ] diff --git a/tests/unit/single_controller/test_rollout_pump.py b/tests/unit/single_controller/test_rollout_pump.py index 19c6d2a98a7..ff78ce1caea 100644 --- a/tests/unit/single_controller/test_rollout_pump.py +++ b/tests/unit/single_controller/test_rollout_pump.py @@ -107,6 +107,7 @@ async def generate_and_push( grpo=GRPOConfig.model_construct(max_num_epochs=1) ) ctrl._rollout_manager = _RecordingRolloutManager(buffer) + ctrl._finalizer_actors = [] # The sampler owns admission + target_step stamping (the dispatch counter # lives on the sampler, not the actor). ctrl._sampler = make_sampler(buffer) @@ -221,6 +222,7 @@ async def _main() -> None: grpo=GRPOConfig.model_construct(max_num_epochs=1) ) ctrl._rollout_manager = manager + ctrl._finalizer_actors = [] # Over-sampled windowed policy: admit never gates (buffer unused here). ctrl._sampler = WindowedSampler(None, max_staleness_versions=1) ctrl._dataloader = [ @@ -307,6 +309,7 @@ async def _main() -> None: grpo=GRPOConfig.model_construct(max_num_epochs=1) ) ctrl._rollout_manager = _NeverCalledRolloutManager() + ctrl._finalizer_actors = [] # Over-sampled windowed policy: admit never gates (buffer unused here). ctrl._sampler = WindowedSampler(None, max_staleness_versions=1) ctrl._dataloader = [ @@ -334,6 +337,81 @@ async def _main() -> None: asyncio.run(_main()) +def test_actor_path_releases_generation_permit_before_finalization() -> None: + class _SplitRolloutManager: + def __init__(self) -> None: + self.generated = 0 + self.two_generated = asyncio.Event() + + async def generate_for_finalization( + self, + prompt: Any, + *, + target_step: int | None = None, + inflight_registry: dict[str, tuple[asyncio.Task[None], int]] | None = None, + ) -> Any: + del prompt, target_step, inflight_registry + self.generated += 1 + if self.generated == 2: + self.two_generated.set() + return SimpleNamespace(group_id=f"g{self.generated}") + + async def _main() -> None: + manager = _SplitRolloutManager() + release_finalizers = asyncio.Event() + finalizers_started = 0 + + async def _delayed_finalize(request: Any) -> None: + nonlocal finalizers_started + del request + finalizers_started += 1 + await release_finalizers.wait() + + controller_cls = SingleControllerActor.__ray_metadata__.modified_class + ctrl = object.__new__(controller_cls) + ctrl._async_cfg = SimpleNamespace(max_inflight_prompts=1, diagnostics=False) + ctrl._master_config = SimpleNamespace( + grpo=GRPOConfig.model_construct(max_num_epochs=1) + ) + ctrl._rollout_manager = manager + ctrl._finalizer_actors = [object()] + ctrl._finalize_with_actor = _delayed_finalize + ctrl._sampler = WindowedSampler(None, max_staleness_versions=1) + ctrl._dataloader = [ + BatchedDataDict( + { + "message_log": [ + [{"role": "user", "content": "first"}], + [{"role": "user", "content": "second"}], + ] + } + ) + ] + ctrl._rollout_permitted = asyncio.Event() + ctrl._rollout_permitted.set() + ctrl._rollout_exhausted = asyncio.Event() + ctrl._buffer_capacity = asyncio.Semaphore(2) + ctrl._inflight_rollouts = 0 + ctrl._inflight_by_group_id = {} + ctrl._dispatched_rollouts = set() + ctrl._trainer_version = 0 + ctrl._current_epoch = 0 + + pump = asyncio.create_task(ctrl._rollout_pump()) + await asyncio.wait_for(manager.two_generated.wait(), timeout=1.0) + await asyncio.sleep(0) + assert finalizers_started == 2 + assert ctrl._inflight_rollouts == 0 + release_finalizers.set() + await asyncio.wait_for(pump, timeout=1.0) + + # Successful commits transfer both buffer permits to the train pump. + assert ctrl._buffer_capacity._value == 0 + assert ctrl._rollout_exhausted.is_set() + + asyncio.run(_main()) + + @pytest.mark.vllm def test_rollout_pump_writes_expected_tq_data( multi_step_setup_vllm_async, # noqa: F811 @@ -416,6 +494,7 @@ def test_rollout_pump_writes_expected_tq_data( rollout_manager=rollout_manager, tq_buffer=tq_buffer, partition_id=_PARTITION_ID, + finalizer_actors=[], ) ctrl = SingleControllerActor.remote( master_config=master_config, diff --git a/tests/unit/single_controller/test_single_controller.py b/tests/unit/single_controller/test_single_controller.py index 91eeb884105..343f18a1baa 100644 --- a/tests/unit/single_controller/test_single_controller.py +++ b/tests/unit/single_controller/test_single_controller.py @@ -200,7 +200,9 @@ def test_sync_weights_honors_recompute_kv_cache_config( ctrl._inflight_by_group_id = {} # env={} -> _should_use_nemo_gym is False, so _sync_weights takes the native # abort path (empty registry -> no-op) instead of the gym gate. - ctrl._master_config = SimpleNamespace(env={}) + ctrl._master_config = SimpleNamespace( + env={}, token_capture=SimpleNamespace(enabled=False) + ) asyncio.run(ctrl._sync_weights()) @@ -229,7 +231,9 @@ def test_sync_weights_calibrates_and_forwards_fp8_kv_scales() -> None: ctrl._inflight_by_group_id = {} # env={} -> _should_use_nemo_gym is False, so _sync_weights takes the native # abort path (empty registry -> no-op) instead of the gym gate. - ctrl._master_config = SimpleNamespace(env={}) + ctrl._master_config = SimpleNamespace( + env={}, token_capture=SimpleNamespace(enabled=False) + ) calibration_data = BatchedDataDict( { "input_ids": torch.tensor([[1, 2]]), diff --git a/tests/unit/single_controller/test_single_controller_setup.py b/tests/unit/single_controller/test_single_controller_setup.py index 1fb7bef41f6..9baf13f77ad 100644 --- a/tests/unit/single_controller/test_single_controller_setup.py +++ b/tests/unit/single_controller/test_single_controller_setup.py @@ -224,6 +224,10 @@ def test_multiple_dataloader_not_supported(self): ("min_groups", "must be >="), ("global_batch_size", "must equal policy.train_global_batch_size"), ("buffer_capacity", "required capacity"), + ( + "deferred_routes_without_capture", + "defer_routed_experts_to_policy requires", + ), ], ) def test_invalid_config_fails_before_setup_factories( @@ -239,6 +243,8 @@ def test_invalid_config_fails_before_setup_factories( mc.policy["train_global_batch_size"] = 7 elif invalid_case == "buffer_capacity": mc.async_rl.max_buffered_rollouts = 7 + elif invalid_case == "deferred_routes_without_capture": + mc.token_capture.defer_routed_experts_to_policy = True else: # pragma: no cover raise AssertionError(f"unknown test case {invalid_case}") @@ -284,6 +290,7 @@ def test_returns_actor_args(self, patched_factories): assert actor_args.partition_id == "rollout_data" assert actor_args.tq_buffer._partition_id == "rollout_data" assert actor_args.tq_buffer._require_routed_experts is False + assert actor_args.finalizer_actors == [] def test_router_replay_requires_routes_in_tq_buffer(self, patched_factories): mc = _make_master_config(colocated=True) @@ -427,9 +434,54 @@ def test_nemo_gym_wires_env_handle(self, patched_factories): enable_router_replay=False, routed_experts_dtype="int16", use_fastokens=False, + token_capture=None, ) assert actor_args.env_handles["nemo_gym"] is fake_gym_actor + def test_token_capture_always_creates_finalizer_actor_pool(self, patched_factories): + mc = _make_master_config(colocated=True, backend="vllm") + mc.policy["generation"].update( + { + "model_name": "test-model", + "stop_strings": None, + "stop_token_ids": None, + "top_k": None, + "vllm_cfg": {"async_engine": True}, + } + ) + mc.logger = {"log_dir": "/tmp/test-token-capture"} + mc.token_capture.enabled = True + mc.token_capture.num_finalizer_workers = 3 + patched_factories["setup_response_data"].return_value = ( + list(range(8)), + None, + ) + fake_actors = [MagicMock(name=f"finalizer_{index}") for index in range(3)] + + with ( + patch.object(sc_setup_mod, "_should_use_nemo_gym", return_value=True), + patch.object( + sc_setup_mod, "spinup_nemo_gym_actor", return_value=MagicMock() + ), + patch.object(sc_setup_mod, "router_replay_enabled", return_value=False), + patch( + "nemo_rl.experience.finalizer_actor.create_finalizer_actors", + return_value=fake_actors, + ) as mock_create_finalizer_actors, + ): + actor_args, _ = setup_single_controller(mc, MagicMock(pad_token_id=9)) + + (actor_dp_config, actor_config), actor_kwargs = ( + mock_create_finalizer_actors.call_args + ) + assert actor_dp_config == mc.data_plane + assert actor_config.partition_id == "rollout_data" + assert actor_config.staging_partition == mc.token_capture.staging_partition + assert actor_config.pad_token_id == 9 + assert actor_kwargs == {"num_workers": 3} + assert actor_args.finalizer_actors == fake_actors + assert not hasattr(actor_args.rollout_manager, "_finalizer") + def test_setup_timing_populated_for_colocated_vllm(self, patched_factories): """Colocated vLLM records gen+policy+collective+total+worker fields.""" mc = _make_master_config(colocated=True, backend="vllm") diff --git a/tests/unit/single_controller/test_tq_replay_buffer.py b/tests/unit/single_controller/test_tq_replay_buffer.py index 3cc29fd2540..9e48b346296 100644 --- a/tests/unit/single_controller/test_tq_replay_buffer.py +++ b/tests/unit/single_controller/test_tq_replay_buffer.py @@ -25,8 +25,15 @@ import nemo_rl.algorithms.async_utils.replay_buffer as _replay_buffer_module from nemo_rl.algorithms.async_utils.replay_buffer import TQReplayBuffer from nemo_rl.data_plane import KVBatchMeta +from nemo_rl.data_plane.schema import ROUTE_PLAN_TAG from nemo_rl.distributed.batched_data_dict import BatchedDataDict from nemo_rl.experience.interfaces import PromptGroupRecord +from nemo_rl.experience.route_plan import ( + ROUTE_PLAN_SCHEMA_VERSION, + RouteAssemblyPlan, + RouteSpan, + encode_route_plan, +) # Each record yields _N_GENS training rows. _N_GENS = 2 @@ -401,3 +408,276 @@ def test_size_and_len(self): _run(buf.remove([0], remove_in_dp=True)) assert buf.size() == 1 assert len(buf) == 1 + + +class MultiPartitionFakeDataPlaneClient(FakeDataPlaneClient): + """Fake DP client that tracks rows per partition (token-capture mode).""" + + def __init__(self) -> None: + super().__init__(partition_id="rollout_data") + self.rows_by_partition: dict[str, dict[str, Any]] = {} + self.clear_calls_by_partition: list[tuple[str, list[str]]] = [] + + def put_samples(self, sample_ids, partition_id, fields=None, tags=None): + bucket = self.rows_by_partition.setdefault(partition_id, {}) + for i, sid in enumerate(sample_ids): + bucket[sid] = {"tag": dict(tags[i]) if tags is not None else {}} + return KVBatchMeta( + partition_id=partition_id, + task_name=None, + sample_ids=list(sample_ids), + fields=None, + tags=[dict(t) for t in tags] if tags is not None else None, + ) + + def clear_samples(self, sample_ids, partition_id): + ids = list(sample_ids) if sample_ids is not None else [] + self.clear_calls_by_partition.append((partition_id, ids)) + bucket = self.rows_by_partition.setdefault(partition_id, {}) + for sid in ids: + bucket.pop(sid, None) + + +class TestTQReplayBufferTokenCaptureMode: + """commit_finalized / abort / rollout_ids / staging-aware remove. + + All of these are uncalled on the legacy (token_capture.enabled=false) + path; the existing test classes above are the legacy-invariance guard. + """ + + def _make_capture_buffer(self, dp) -> TQReplayBuffer: + return TQReplayBuffer( + dp, + partition_id="rollout_data", + pad_value_dict={"token_ids": 0}, + staging_partition_id="rollout_staging", + ) + + def test_reserve_records_rollout_ids(self): + buf = self._make_capture_buffer(MultiPartitionFakeDataPlaneClient()) + buf.reserve(weight_version=1, rollout_ids=["g0_g0", "g0_g1"]) + assert buf._rollout_ids_list == [["g0_g0", "g0_g1"]] + # Legacy reserve records None. + buf.reserve(weight_version=1) + assert buf._rollout_ids_list[1] is None + + def test_commit_finalized_fills_slot_with_group_min_wv(self): + dp = MultiPartitionFakeDataPlaneClient() + buf = self._make_capture_buffer(dp) + group_id = buf.reserve(weight_version=4, rollout_ids=["r0", "r1"]) + # The finalizer published its own rows; commit_finalized only fills the slot. + meta = KVBatchMeta( + partition_id="rollout_data", + task_name=None, + sample_ids=[f"{group_id}_g0", f"{group_id}_g1"], + fields=None, + ) + _run( + buf.commit_finalized( + group_id, + meta, + group_min_wv=3, + group_max_wv=5, + staging_keys=["r0/c1", "r0/c2", "r1/c1"], + ) + ) + assert buf.ready_list == [True] + assert buf.start_weight_list == [3] # oldest call version, not reserve-time 4 + assert buf.end_weight_list == [5] + assert buf.meta_list[0] is meta + assert buf._staging_keys_list == [["r0/c1", "r0/c2", "r1/c1"]] + # No tensorize/put happened here. + assert dp.rows_by_partition.get("rollout_data") is None + + def test_commit_finalized_raises_for_evicted_slot(self): + buf = self._make_capture_buffer(MultiPartitionFakeDataPlaneClient()) + meta = KVBatchMeta( + partition_id="rollout_data", task_name=None, sample_ids=[], fields=None + ) + with pytest.raises(ValueError, match="no live slot"): + _run(buf.commit_finalized("ghost", meta, group_min_wv=0, group_max_wv=0)) + + def test_commit_finalized_verifies_full_plan_manifest_ownership(self): + dp = MultiPartitionFakeDataPlaneClient() + buf = self._make_capture_buffer(dp) + group_id = buf.reserve(weight_version=1, rollout_ids=["r0"]) + plan = encode_route_plan( + RouteAssemblyPlan( + schema_version=ROUTE_PLAN_SCHEMA_VERSION, + staging_partition="rollout_staging", + spans=(RouteSpan("r0/on_chain", 0, 2, 2, 1, "0" * 64),), + cleanup_staging_keys=("r0/on_chain", "r0/off_chain"), + expected_token_length=2, + ) + ) + meta = KVBatchMeta( + partition_id="rollout_data", + task_name="train", + sample_ids=["r0"], + fields=["input_ids"], + tags=[{ROUTE_PLAN_TAG: plan}], + ) + + with pytest.raises(ValueError, match="ownership does not match"): + _run( + buf.commit_finalized( + group_id, + meta, + group_min_wv=1, + group_max_wv=1, + staging_keys=["r0/on_chain"], + ) + ) + + assert buf.size() == 1 + assert buf.ready_list == [False] + + def test_abort_drops_unready_slot_only(self): + dp = MultiPartitionFakeDataPlaneClient() + buf = self._make_capture_buffer(dp) + gid_unready = buf.reserve(weight_version=1) + gid_ready = buf.reserve(weight_version=1) + _run( + buf.commit( + gid_ready, + _make_record(), + start_weight_version=1, + end_weight_version=1, + ) + ) + assert buf.abort(gid_unready) is True + assert buf.size() == 1 + # Ready slots and unknown ids are not abortable. + assert buf.abort(gid_ready) is False + assert buf.abort("ghost") is False + assert buf.size() == 1 + + def test_remove_clears_staging_rows_alongside_canonical(self): + dp = MultiPartitionFakeDataPlaneClient() + buf = self._make_capture_buffer(dp) + group_id = buf.reserve(weight_version=1, rollout_ids=["r0"]) + meta = KVBatchMeta( + partition_id="rollout_data", + task_name=None, + sample_ids=[f"{group_id}_g0"], + fields=None, + ) + _run( + buf.commit_finalized( + group_id, + meta, + group_min_wv=1, + group_max_wv=1, + staging_keys=["r0/c1", "r0/c2"], + ) + ) + n = _run(buf.remove([0], remove_in_dp=True)) + assert n == 1 + assert ("rollout_data", [f"{group_id}_g0"]) in dp.clear_calls_by_partition + assert ("rollout_staging", ["r0/c1", "r0/c2"]) in dp.clear_calls_by_partition + + def test_remove_without_staging_partition_skips_staging_clear(self): + dp = MultiPartitionFakeDataPlaneClient() + buf = TQReplayBuffer( + dp, partition_id="rollout_data", pad_value_dict={"token_ids": 0} + ) + group_id = buf.reserve(weight_version=1) + meta = KVBatchMeta( + partition_id="rollout_data", + task_name=None, + sample_ids=[f"{group_id}_g0"], + fields=None, + ) + _run(buf.commit_finalized(group_id, meta, group_min_wv=1, group_max_wv=1)) + _run(buf.remove([0], remove_in_dp=True)) + partitions_cleared = {p for p, _ in dp.clear_calls_by_partition} + assert partitions_cleared == {"rollout_data"} + + def test_cleanup_failure_retains_buffer_ownership(self): + class FailingStagingCleanupClient(MultiPartitionFakeDataPlaneClient): + def clear_samples(self, sample_ids, partition_id): + if partition_id == "rollout_staging": + raise RuntimeError("injected staging cleanup failure") + return super().clear_samples(sample_ids, partition_id) + + dp = FailingStagingCleanupClient() + buf = self._make_capture_buffer(dp) + group_id = buf.reserve(weight_version=1, rollout_ids=["r0"]) + meta = KVBatchMeta( + partition_id="rollout_data", + task_name="train", + sample_ids=["r0"], + fields=None, + ) + _run( + buf.commit_finalized( + group_id, + meta, + group_min_wv=1, + group_max_wv=1, + staging_keys=["r0/c1"], + ) + ) + + with pytest.raises(RuntimeError, match="retained replay-buffer ownership"): + _run(buf.remove([0], remove_in_dp=True)) + + assert buf.size() == 1 + assert buf._staging_keys_list == [["r0/c1"]] + + +class TestTQReplayBufferEvictedCommit: + def test_commit_on_evicted_slot_writes_nothing(self): + """The pre-write check: an evicted group must not orphan rows.""" + dp = FakeDataPlaneClient() + buf = _make_buffer(dp) + group_id = buf.reserve(weight_version=1) + _run(buf.remove([0], remove_in_dp=False)) + + with pytest.raises(ValueError, match="no live slot"): + _run( + buf.commit( + group_id, + _make_record(), + start_weight_version=1, + end_weight_version=1, + ) + ) + assert dp.put_calls == [] + assert dp.depth() == 0 + + def test_commit_evicted_during_write_unwrites_rows(self): + """Eviction interleaving with the awaited put must clear the rows.""" + + class EvictDuringPut(FakeDataPlaneClient): + def __init__(self): + super().__init__() + self.buf: TQReplayBuffer | None = None + + async def put_samples( + self, sample_ids, partition_id, fields=None, tags=None + ): + result = FakeDataPlaneClient.put_samples( + self, sample_ids, partition_id, fields=fields, tags=tags + ) + # Simulate the sampler evicting the slot mid-write. + await self.buf.remove([0], remove_in_dp=False) + return result + + dp = EvictDuringPut() + buf = _make_buffer(dp) + dp.buf = buf + group_id = buf.reserve(weight_version=1) + + with pytest.raises(ValueError, match="evicted during"): + _run( + buf.commit( + group_id, + _make_record(), + start_weight_version=1, + end_weight_version=1, + ) + ) + # The written rows were un-written. + assert dp.depth() == 0 + assert len(dp.clear_calls) == 1 diff --git a/tests/unit/single_controller/test_train_pump.py b/tests/unit/single_controller/test_train_pump.py index 8c2eb928c2f..4fa887aea63 100644 --- a/tests/unit/single_controller/test_train_pump.py +++ b/tests/unit/single_controller/test_train_pump.py @@ -342,6 +342,7 @@ def test_train_pump_drives_mcore_training_step( rollout_manager=rollout_manager, tq_buffer=tq_buffer, partition_id=_PARTITION_ID, + finalizer_actors=[], ) ctrl = _RecordingSingleControllerActor.remote( metric_log_handle=log, diff --git a/token-capture-external-sink-recommendation.md b/token-capture-external-sink-recommendation.md new file mode 100644 index 00000000000..bb4ea7baefd --- /dev/null +++ b/token-capture-external-sink-recommendation.md @@ -0,0 +1,183 @@ +# PR 2278 Review: External Token Staging with `LineageStore` + +## Recommendation + +Use the existing `LineageStore` directly for parent resolution and cumulative +token tracking. Supporting an external staging sink does not require a separate +`RolloutCaptureGate` or `GateStateStore`. + +The current gate does not provide a second lineage algorithm. It resolves the +parent through `LineageStore`, duplicates the resulting call state in the gate, +and publishes successful commits back to `LineageStore`. This creates two +sources of truth for the same rollout ancestry. + +The recommended initial implementation is framework-authoritative: + +- Gym owns request correlation and lineage resolution. +- The inference worker owns durable staging through `StagingSink`. +- The external framework owns rollout completeness, terminal selection, + finalization, and cleanup. +- The finalizer verifies the staged records with `verify_and_linearize()` before + publishing a training row. + +## Why `LineageStore` Is Sufficient for Lineage + +`LineageStore` already provides the two operations needed by the serving path: + +1. `resolve(rollout_id, request_items)` identifies the verified parent and + returns its cumulative token IDs. +2. `record(...)` publishes a completed call so a later request, including one + handled by another model-server worker, can continue it. + +`FileLineageStore` supplies cross-process read-after-write visibility for +multi-worker model servers. The gate currently delegates lineage publication +back to this interface after it receives worker commit coordinates. + +Removing the gate does not change the lineage algorithm. It removes the +duplicated lifecycle database around it. + +## Simplified External-Sink Flow + +1. Gym assigns `rollout_id` and `model_call_id` in the request-scoped capture + context. +2. Gym calls `LineageStore.resolve()` before dispatch. +3. Gym sends the call identity, parent identity, and required prefix to the + inference worker. +4. The worker creates a `StagedCallRecord` and makes it durable through the + configured `StagingSink` before acknowledging success. +5. After successful staging, Gym computes the cumulative token sequence and + calls `LineageStore.record()`. +6. Gym removes token IDs, log probabilities, routed-expert values, and internal + coordinates before returning the model response to the agent. +7. The external framework reads staged records through `StagingSource`, selects + the terminal ancestry, and calls `verify_and_linearize()`. + +The ordering requirement is important: a call must not become a lineage parent +until its external staged record is durable. + +## What the Gate Adds + +The gate adds lifecycle and authorization behavior, not lineage behavior: + +- rollout registration and per-rollout data capabilities; +- admitted-but-not-committed call tracking; +- owner-bound and operation-bound retries; +- logical-request-to-model-call indexing; +- seal and failure transitions; +- receipt construction; +- TTL expiry, tombstones, cleanup queues, and metrics. + +These behaviors may be useful, but they should not be treated as prerequisites +for an external sink. If NeMo RL already owns rollout lifecycle and finalization, +duplicating those responsibilities inside Gym increases coupling without +improving lineage resolution. + +## Important Caveat: Root Versus Unresolved Lineage + +The existing lineage result uses `None` for several different cases: + +- the request is a genuine first call with no assistant-authored history; +- the assistant fingerprint is ambiguous; +- the request context digest no longer matches; +- a prior call cannot be found. + +The local capture builder can sometimes recover a missing parent through strict +token-prefix matching. The external finalizer follows explicit parent links and +cannot rely on that recovery. + +The external path should therefore distinguish: + +```text +ROOT -> admit as a text-mode root +MATCH -> admit as a token-in child +UNRESOLVED -> reject or poison token capture +``` + +Silently converting `UNRESOLVED` into a new root can turn earlier +policy-generated tokens into prompt tokens with mask zero and produce an +incorrect training trajectory. + +This is a lineage API correction and remains necessary whether or not a gate is +present. + +## Responsibilities Outside `LineageStore` + +`LineageStore` as currently defined does not store: + +- external staging keys; +- weight versions and complete commit manifests; +- admitted calls that never committed; +- terminal model-call identity; +- failed or sealed rollout state. + +For the framework-authoritative design, those responsibilities remain with the +external sink, source, controller, and finalizer. The staged record already +contains the parent link, lengths, weight version, and integrity digests needed +for final verification. + +If Gym must eventually own fail-closed receipt construction, extend the lineage +store into a per-rollout append-only capture ledger. Do not introduce a separate +global gate state file. A ledger could add `CallStarted`, `CallCommitted`, +`CallFailed`, and `RolloutSealed` events alongside the existing lineage data. + +## External Sink Contract Recommendations + +Keep the public integration surface small: + +- `StagingSink.stage(record) -> StageResult` +- `StagingSource.fetch(staging_keys) -> snapshots` +- versioned `StagedCallRecord`, `CallRecord`, and receipt types where needed; +- digest and conformance helpers; +- `verify_and_linearize()` as the final trust boundary; +- engine-specific capture adapters behind the generic capture protocol. + +For cleanup after a lost worker acknowledgment, prefer a deterministic staging +key derived from `rollout_id` and `model_call_id`, or allocate the key before +dispatch. An opaque key returned only after staging cannot be recovered when the +acknowledgment is lost. + +Authorization, if required for untrusted agent traffic, can be implemented as a +small stateless signed capability check. It does not require a complete rollout +state machine. + +## Suggested PR Scope + +The core Gym PR should contain: + +- staging protocols and versioned records; +- integrity and conformance helpers; +- the worker capture hook; +- the generic lineage integration; +- the external finalization verifier; +- a vLLM adapter behind the engine-neutral interface. + +Move framework- or harness-specific behavior into companion changes: + +- NeMo RL TransferQueue sink/source and finalizer wiring; +- SWE-specific credential-file routing and terminal discovery; +- rollout-controller policy, cleanup, and metrics; +- unrelated packaging exclusions or runtime-directory cleanup. + +## Decision Summary + +Both behaviors are required for a fully fail-closed system, but both current +components are not: + +| Concern | Recommended owner | +| --- | --- | +| Parent resolution and exact prefix | Existing `LineageStore` | +| Durable token/logprob/route staging | External `StagingSink` | +| Snapshot retrieval | External `StagingSource` | +| Terminal selection and rollout completeness | External framework | +| Final integrity verification | `verify_and_linearize()` | +| Separate Gym gate database | Remove from the initial external-sink path | + +The smallest useful change is therefore to keep `LineageStore` and the staging +contract, remove the independent gate state system, and leave lifecycle policy +with the framework that already owns the rollout. + +## Reviewed Change + +This recommendation is based on the five-commit delta in +[NVIDIA-NeMo/Gym PR 2278](https://github.com/NVIDIA-NeMo/Gym/pull/2278), from +base `d2123272` through head `1b342084`. diff --git a/token-capture-lineage-ledger-approach.md b/token-capture-lineage-ledger-approach.md new file mode 100644 index 00000000000..451b13898c1 --- /dev/null +++ b/token-capture-lineage-ledger-approach.md @@ -0,0 +1,443 @@ +# Token Capture Lineage Ledger — Approach Plan + +This plan replaces the `RolloutCaptureGate` / `GateStateStore` pair in NeMo Gym's +token-capture path with a small extension to the existing `LineageStore`, turning it +into a per-rollout append-only capture ledger. The external staging contract +(`StagingSink` / `StagingSource`), the worker capture path, and the +`verify_and_linearize()` trust boundary are unchanged. + +**Scope note:** when this change lands, the gate is **removed completely from the Gym +codebase** — `gate.py`, `gate_store.py`, the gate control routes, gate configuration, +and gate tests are deleted in the same change. The gate is not retained behind a +config flag and no backward-compatibility path is kept. Both components ship in the +same PR (NVIDIA-NeMo/Gym PR 2278), so there are no external deployments to migrate. + +## Motivation + +The gate does not provide a second lineage algorithm. Parent resolution runs upstream +through `LineageStore.resolve()`; the gate cross-checks that result against its own +copy of the call state and republishes successful commits back to +`LineageStore.record()`. This creates two sources of truth for the same ancestry: +each call's cumulative token IDs are stored both in `GateCallState` and in the +lineage JSONL. + +The gate's state store is also a scalability liability. `FileGateStateStore` +serializes the **entire global gate state** — every live rollout's cumulative token +arrays and stored request items — and atomically rewrites it under one exclusive +file lock. `commit_coords()` performs three such transactions per model call. For +long-context rollouts this is megabytes of fsynced rewrite per call, serialized +across all serving workers. + +What the gate legitimately provides — admission, retry idempotency, rollout +completeness, terminal selection, cleanup — is either a pure function of the lineage +result or belongs to the framework that already owns the rollout. This design moves +each responsibility to its natural owner and deletes the redundant state machine. + +## Design + +### 1. `LineageStore` becomes the capture ledger + +`FileLineageStore` already writes one append-only JSONL row per committed call +(`model_call_id`, fingerprint, context digest, cumulative token IDs, digest) with +per-rollout locked appends and cross-worker read-after-write visibility. Three +additions make it the single record of rollout capture state: + +1. **`record(...)` gains the token-free `CallRecord` fields.** The commit hook + already holds the worker's `CommitCoords`, so each row additionally stores + `parent_call_id`, `staging_key`, `weight_version`, `prev_len` / `delta_len` / + `cum_len`, `extras_digest`, `mode`, and `logical_request_id` (the request header + when present, else the response id — the same binding the gate's commit uses + today). Additive change; existing fields are untouched. +2. **`record_failure(rollout_id, model_call_id, reason)`** appends a failure row when + a call's coordinates come back `capture_failed` or the request dies after + admission. Failure rows carry no fingerprint, so `resolve()` never returns them + as parents — no filtering logic is needed. +3. **`manifest(rollout_id) -> list[CallRecord]`** reads the rows back token-free + (cumulative token IDs are stripped; token arrays stay off Gym's HTTP surface). + +The only duplication that remains between ledger rows and staged records (digest, +lengths, parent link) is intentional cross-attestation — it is exactly what +`verify_and_linearize()` checks one against the other. + +### 2. Gate-free admission in `resolve_parent()` + +When external staging is enabled, the model server builds the `CaptureAdmission` +directly from the lineage result. Admission is a pure function; no shared state is +required. The lineage outcome is a strict tri-state: + +| Lineage outcome | Admission | +| --- | --- | +| `ROOT` — empty assistant fingerprint, or unmatched fingerprint on a rollout with no ledger rows (seeded assistant history in the task prompt) | `text` mode, no parent | +| `MATCH` — unique fingerprint match with verified context digest | `token_in` mode, `required_prefix_token_ids` = parent's cumulative tokens | +| `UNRESOLVED` — non-empty fingerprint with no match, ambiguity, or digest mismatch | no admission; `record_failure()` poisons the call | + +`UNRESOLVED` is never silently converted into a new root: doing so would turn earlier +policy-generated tokens into mask-zero prompt tokens and corrupt the training row. +The seeded-history carve-out (unmatched fingerprint on an empty rollout is a `ROOT`) +requires a cheap "has any rows" check on the ledger. + +### 3. Gate-free commit + +Where the model server today hands `ng_commit_coords` to `gate.commit_coords()`, +it instead: + +- on `disposition == "staged"`: computes + `cumulative = context.parent_tokens + coords.token_ids_delta` and calls + `lineage_store.record(...)` with the extended row. This is the lineage-publication + block currently inside the gate, relocated — minus the state machine around it. +- on `disposition == "capture_failed"`, missing coordinates, or a request error after + admission: calls `record_failure(...)`. + +The ordering invariant the external sink requires — *a call must not become a +lineage parent until its staged record is durable* — holds structurally: the worker +stages through `StagingSink.stage()` before acknowledging, coordinates exist only +after the bytes are durable, and the ledger row (which is what makes a call +resolvable as a parent) is written only after the coordinates arrive. + +### 4. One read-only control route + +The register / seal / fail control routes are deleted (see removal section). They +are replaced by a single stateless read: + +``` +GET /training-token-capture/rollouts/{rollout_id}/manifest +``` + +which returns `manifest(rollout_id)`. The framework must not read the ledger JSONL +files directly — the route keeps `LineageStore` implementations swappable. + +### 5. Retry handling (fail-closed now, idempotent as a follow-up) + +For the initial change, `model_call_id` stays a per-request `uuid4` minted by the +capture middleware, and a ledger row's `logical_request_id` field is filled the way +the gate's commit binding fills it today: the client header when present, else the +vLLM response id. A retried HTTP request (Gym's `ServerClient` retries with backoff) +therefore lands as a *distinct* call. The ledger handles this fail-closed rather +than idempotently: + +- a retry whose first attempt never committed leaves a failure row (the request + died after admission) → the rollout is poisoned; +- a retry whose first attempt committed becomes a sibling row. If the regenerated + text differs, later calls resolve uniquely to the survivor and the losing sibling + is a dead branch in the manifest (never on the terminal chain); if the text is + identical, resolution is ambiguous → `UNRESOLVED` → poisoned. Terminal selection + still works because the agent reports the response id of the response it actually + kept, which matches exactly one row's `logical_request_id`. + +No retry outcome is silently wrong — the current gate's worst case (ambiguous +resolution silently falling back to a text-mode root) is closed by the `UNRESOLVED` +rule. What is deferred is *recovery*: making identical retries collapse into the +same row instead of poisoning. That follow-up is two touch points — the harness +mints a per-call `x-nemo-gym-logical-request-id` (reused verbatim on retry), and the +middleware derives `model_call_id` deterministically from it — after which +`record()`'s existing semantics absorb retries: an identical payload is a no-op, a +conflicting payload raises and the call is poisoned, and staging keys +(`{rollout_id}/{model_call_id}`) overwrite/no-op rather than orphaning a record. + +### 6. Framework-owned receipt, lifecycle, and cleanup + +NeMo RL (the rollout owner) assembles the `RolloutReceipt` locally at rollout end: + +- `manifest` = the fetched `CallRecord` list, deduped by `model_call_id`; +- `terminal_model_call_id` = the row whose `logical_request_id` matches the + rollout's terminal logical request; +- `capture_poisoned` = any failure row present, or no row for the terminal request. + +`verify_and_linearize(receipt, snapshots)` runs unchanged — it consumes a plain +value and does not care who built it. Cleanup is framework-driven: on publish or on +rollout failure/abandonment, NeMo RL prefix-clears `{rollout_id}/` in the staging +backend and drops the ledger file. No registration TTLs, tombstones, or +cleanup-manifest queues. + +## Complete gate removal (no backward compatibility) + +The same change that introduces the ledger **deletes the gate entirely** from Gym. +Nothing is kept behind a flag, deprecated, or retained for compatibility: + +- `nemo_gym/token_id_capture/gate.py` — deleted. +- `nemo_gym/token_id_capture/gate_store.py` (including `FileGateStateStore`, + `InMemoryGateStateStore`, `SharedGateState`, `GateRolloutState`, `GateCallState`, + `GateTombstone`, `CleanupManifest`, `RolloutRegistration`) — deleted. + `CallRecord` and `RolloutReceipt` survive: they live in `staging/records.py` and + are consumed by `verify_and_linearize()`. +- Control routes `PUT /rollouts/{id}`, `POST /rollouts/{id}/seal`, + `POST /rollouts/{id}/fail`, and the gate metrics route — deleted, replaced by the + single manifest route. +- `token_id_capture.gate.*` configuration (`enabled`, `state_store_path`, + `registration_ttl_s`, `tombstone_ttl_s`) — deleted, not deprecated. A config that + still sets these keys fails validation loudly rather than silently ignoring them. +- Gate wiring in `base_responses_api_model.py` (`RolloutCaptureGate` construction, + `_fail_uncommitted_gate_call`, `_reject_gate_streaming`, capability header + handling) — deleted. Streaming rejection is retained where external capture + requires it, keyed off the capture context rather than the gate. +- `CaptureContext.staging_gate` and `data_capability` fields — deleted. +- Gate unit tests (`test_token_capture_gate.py`, + `test_token_capture_gate_multiworker.py`) — deleted; their invariants that still + apply (admission tri-state, commit ordering, retry idempotency, poisoning) are + re-expressed as ledger and admission tests. + +Rationale for not keeping a compatibility path: the gate ships in the same unmerged +PR as the external sink, so there is no deployed consumer of the gate API. Retaining +it would preserve the dual-source-of-truth problem this design exists to remove and +double the surface that must be tested. + +Data-capability authorization is dropped with the gate. The NeMo RL deployment is a +trusted, framework-owned serving path. If untrusted agent traffic later needs +per-rollout authorization, a stateless signed capability check can be added without +any state store. + +## NeMo RL companion changes + +- `nemo_rl/environments/nemo_gym.py`: remove the register / seal / fail control-route + calls and capability plumbing; fetch the manifest at rollout end; keep configuring + `FileLineageStore` for the policy model server. +- `nemo_rl/experience/blackbox_finalizer.py`: build the `RolloutReceipt` from the + fetched manifest as described above; verification and linearization are unchanged. +- Rollout failure paths (`rollout_manager.py`): prefix-clear staged rows for the + rollout instead of draining gate cleanup manifests. +- `TQTokenSink` / `TQTokenSource` and the vLLM worker capture path + (`vllm_worker_async.py`): unchanged. + +## Example walkthrough + +A SWE-agent rollout `r42_g0` with three model calls, served by a Gym model server +with the ledger and a vLLM worker staging into TransferQueue. + +**Call 1 (root).** The agent server sends the task prompt; the server mints +`model_call_id=c1`, and commit binds `logical_request_id=lr-1` (the response id) +onto the ledger row. The request has no +assistant-authored turns → `ROOT` → `text` admission. The worker generates +(prompt 700 tokens, generation 200), stages +`StagedCallRecord(staging_key="r42_g0/c1", prev_len=0, delta_len=900, cum_len=900, +weight_version=17, ...)` durably into TransferQueue, then returns token-free +coordinates. The server appends ledger row 1 (lineage fields + `CallRecord` fields + +`lr-1`). The agent receives text only. + +**Call 2 (continuation).** The agent appends a 150-token tool result and sends +`lr-2`. The fingerprint uniquely matches row 1 and the context digest verifies → +`MATCH` → `token_in` admission with the 900-token required prefix. The worker +asserts the prompt begins with exactly that prefix, generates 120 tokens, stages +`r42_g0/c2` (`prev_len=900`, `delta_len=270` — 150 carry tokens mask 0 + 120 +generated mask 1, `cum_len=1170`). Ledger row 2 records `parent_call_id=c1`. + +**Call 3 (terminal).** Same shape via `lr-3` → `c3`, `cum_len=1290`, ledger row 3. +The rollout result returns to NeMo RL with the reward and +`terminal_logical_request_id=lr-3`. + +**Finalization.** +1. `GET .../rollouts/r42_g0/manifest` → three `CallRecord`s. +2. Build the receipt: terminal row is `lr-3` → `terminal_model_call_id=c3`; no + failure rows → not poisoned. +3. `StagingSource.fetch(["r42_g0/c1", "r42_g0/c2", "r42_g0/c3"])`, then + `verify_and_linearize(receipt, snapshots)`: digest recomputation, parent-graph + walk `c3 → c2 → c1 → root`, length chaining `900 + 270 = 1170`, + `1170 + 120 = 1290`, mask-order checks. Output: one contiguous 1290-token + training row, mask 1 over the 320 policy tokens. +4. Publish the row with the reward; prefix-clear `r42_g0/*` from TransferQueue and + drop the ledger file. + +## Failure semantics + +- **Capture fails mid-rollout** (e.g. call 2): the model call still succeeds for the + agent, but a failure row is written instead of a lineage row. Call 3 then misses + resolution with a non-empty assistant history → `UNRESOLVED` → failure row. + Finalization sees failure rows → poisoned → masked placeholder row (the group + still publishes exactly N rows); staging is prefix-cleared. +- **Terminal response lost, harness retries:** the retry is admitted as a new call + (uuid identity) and becomes a sibling of the lost attempt. The harness keeps the + retry's response, so its response id is reported as the terminal logical request + and receipt assembly selects the retry's row; the lost attempt is a dead branch + that never joins the terminal chain and is prefix-cleared with the rollout. Two + versions of the same call can never *both* train, and an ambiguous mid-rollout + sibling (identical regenerated text) poisons via `UNRESOLVED` instead of silently + becoming a root. (The deterministic-identity follow-up upgrades this to full + idempotent collapse.) +- **Crash after staging but before the ledger append:** the staged record exists but + the ledger has no row. If the call has descendants, they resolve `UNRESOLVED` and + poison; if it was terminal, receipt assembly finds no terminal row and poisons. + The orphaned staged record is removed by the prefix-clear. Fail-closed with zero + admitted-call bookkeeping. +- **Abandoned rollout:** NeMo RL's failure path prefix-clears TransferQueue and + drops the ledger file. No TTL machinery. + +## Constraints and caveats + +- **`InMemoryLineageStore` cannot serve the ledger role.** Its backing index evicts + rollouts under memory bounds, which is acceptable for a resolution cache but not + for a completeness record. External-sink mode requires a non-evicting store + (`FileLineageStore` or a framework-provided implementation); multi-worker serving + already requires this. The in-memory store remains for single-worker development + and unit tests only. +- **Receipt independence is reduced.** The manifest is derived from the same commit + stream that produced the staged records rather than from an independent state + machine. Per-record integrity is unaffected (digests are recomputed at + finalization); completeness rests on lineage poisoning plus the terminal-row + check, which the failure analysis above shows is fail-closed at the same points + the gate was. +- **Ledger row growth.** Rows store full cumulative token ID arrays (as the lineage + JSONL does today), so a rollout's ledger grows quadratically with call count. + Acceptable at current rollout lengths; if it becomes a problem, rows can store a + prefix reference instead of the full array without changing the contract. + +## PR scope + +Gym (PR 2278): + +- `LineageStore` protocol + `FileLineageStore`: extended `record()`, + `record_failure()`, `manifest()`, has-rows check. +- `sink.py`: gate-free admission tri-state in `resolve_parent()`. +- Commit hook: relocated lineage publication + failure recording. +- New manifest control route; `logical_request_id` stored on ledger rows (header or + response-id fallback). Deterministic `model_call_id` derivation is **out of scope** + (follow-up; see Implementation plan). +- **Full deletion of the gate** (files, routes, config, wiring, tests) as itemized + above. + +NeMo RL (companion): + +- Receipt assembly in the finalizer; manifest fetch in `nemo_gym.py`; removal of + register/seal/fail calls; prefix-clear cleanup on failure paths. + +## Implementation plan + +File-by-file changes, in landing order. Retry *idempotency* (client-minted logical +ids + deterministic `model_call_id`) is explicitly deferred — see the follow-up +section at the end. Gym paths are relative to the `3rdparty/Gym-workspace/Gym` +checkout; NeMo RL paths are relative to the repo root. Line references are against +the current state of both trees. + +### Gym (PR 2278) + +**1. `nemo_gym/token_id_capture/protocols.py` — widen the `LineageStore` protocol.** +`record()` (`protocols.py:74-82`) gains the token-free `CallRecord` fields: +`parent_call_id`, `staging_key`, `weight_version`, `prev_len`, `delta_len`, +`cum_len`, `extras_digest`, `mode`, `logical_request_id`. New protocol methods: +`record_failure(rollout_id, model_call_id, reason)`, +`manifest(rollout_id) -> list[CallRecord]`, and `has_rows(rollout_id) -> bool`. +Additive, so it lands standalone. + +**2. `nemo_gym/token_id_capture/lineage.py` — implement in both stores.** +- `FileLineageStore._record` (`lineage.py:520-527`) extends the six-key JSONL row + with the new fields. The existing same-id idempotency scan (`lineage.py:528-535` + — identical payload no-op, conflicting payload `ValueError`) is kept unchanged. +- `record_failure()` appends a row with `reason` and **no `fingerprint`**; since + `_resolve` (`lineage.py:472-490`) filters by fingerprint, failure rows can never + be returned as parents — no filtering logic needed. +- `manifest()` reads rows under the per-rollout lock, strips + `cumulative_token_ids`, returns validated `CallRecord`s. `has_rows()` is a locked + existence check. +- `InMemoryLineageStore` implements the same methods but is rejected in + external-sink mode (its `LineageIndex` evicts rollouts, `lineage.py:296-344`, + which breaks completeness); it remains for unit tests and single-worker dev. + +**3. `nemo_gym/base_responses_api_model.py` — de-wiring (no identity change).** +- `model_call_id = uuid4().hex` (`:1254`) **stays as-is** for now. +- Delete gate construction and control-route install (`:1481-1498`), the + `GateError` handler (`:1500-1510`), and capability-header handling in the + middleware (`:1263-1278`). +- Replace `_fail_uncommitted_gate_call` (`:1161-1176`, `finally` call site `:1286`) + with a `record_failure(reason="request_finished_without_staged_coordinates")` in + the same `finally` — a request that dies after admission must still poison. +- `CaptureContext` (`token_id_capture/sink.py:53-87`) drops `staging_gate` and + `data_capability`. +- `_reject_gate_streaming` (`:104-111`) is re-keyed off the capture context instead + of the gate; its three dialect call sites (`:235`, `:280`, `:322`) stay. + +**4. `nemo_gym/token_id_capture/sink.py` — tri-state admission in +`resolve_parent()`.** Replace the gate admission step (`sink.py:153-159`) with the +pure function of the lineage result described in Design §2: unique verified match → +`token_in`; empty fingerprint or unmatched fingerprint with `has_rows() == False` +(seeded history) → `text` root; anything else → no admission + +`record_failure(reason="unresolved_parent")`. This closes the gate's silent +root-fallback hole and is load-bearing for the deferred-retry story. + +**5. `responses_api_models/vllm_model/app.py` — gate-free commit hook.** +`_finalize_gate_capture` (`app.py:946-987`) keeps its shape; the gate call +(`:962-966`) becomes: +- `disposition == "staged"` → `cumulative = context.parent_tokens + + coords.token_ids_delta`, then the extended `lineage_store.record(...)` — the + publication block relocated from `gate.py:356-384`; +- `capture_failed`, missing coords, or any exception (including `record()`'s + conflict `ValueError`) → `record_failure(...)`. +The `logical_request_id` binding keeps today's fallback: +`context.logical_request_id or str(payload["id"])`. The existing catch-all that +never turns a valid completion into a harness failure (`:972-985`) stays. + +**6. `nemo_gym/token_id_capture/control_routes.py` — one read-only route.** +Delete `PUT /rollouts/{id}`, `POST .../seal`, `POST .../fail`, `GET /cleanup`, +`GET /metrics`, and the TTL sweeper task (`control_routes.py:74-164`). Add +`GET /training-token-capture/rollouts/{rollout_id}/manifest` returning +`manifest(rollout_id)`. `RolloutControlClient` (`:167-266`) shrinks to `manifest()`. + +**7. Deletions.** `gate.py`, `gate_store.py`, +`test_token_capture_gate*.py`; `token_id_capture.gate.*` config keys removed with +**loud validation failure** on leftovers; the `rebuild_response`-with-gate guard +(`token_id_capture/config.py:162-166`) re-keyed to external capture. +`CallRecord` / `RolloutReceipt` / `CommitCoords` stay in `staging/records.py`; +`verify_and_linearize()` is untouched. Gate test invariants that still apply +(tri-state admission, commit ordering, same-call commit idempotency, poisoning) are +re-expressed as ledger and admission tests. + +### NeMo RL (companion) + +**8. `nemo_rl/environments/nemo_gym.py`.** Delete `register_rollouts` +(`:600-618`), `fail_rollouts` (`:620-637`), `gate_metrics` (`:639-640`), capability +stamping in `run_rollouts` (`:664-672`), and the `gate:` config block (`:521-527`), +keeping the `FileLineageStore` configuration (`:519-520`). +`_postprocess_receipt_mode()` (`:753-812`) replaces the seal call (`:779-789`) with +the manifest `GET`, then builds the `RolloutReceipt` locally: terminal row = the +manifest row whose `logical_request_id` equals the reported +`terminal_logical_request_id` (a response id, unchanged agent behavior); +`capture_poisoned` = any failure row or no terminal row. **Open check:** retry +duplicates now appear as dead-branch rows in the manifest (distinct call ids, same +parent); receipt assembly must either prune to the terminal parent-chain or we must +confirm `_validate_manifest_graph` (`staging/rebuild.py:159`) tolerates rows +unreferenced by the terminal chain. + +**9. `nemo_rl/experience/blackbox_finalizer.py`.** Near-zero delta: the receipt +already arrives by value into `finalize_rollout` (`:241`); every downstream check, +`verify_and_linearize` (`:411-424`), placeholder masking (`:566-590`), and +`_clear_staging` (`:776-786`) are unchanged. Dead-branch rows are in the manifest, +so their staging keys are enumerated and cleaned — no new orphan class. + +**10. `nemo_rl/experience/rollout_manager.py`.** The abandonment path (`:929-935`) +drops `fail_rollouts.remote(...)`; cleanup becomes staging clear + ledger drop. The +`gate_metrics` proxy (`:799-811`) is removed or repointed at ledger-derived counters. + +### Landing order + +1→2 (additive ledger, standalone-testable) → 3→7 as one Gym change → 8→10 as the +NeMo RL companion → end-to-end via `docs/guides/nano-swe-token-capture.md`. + +### Open item to settle before step 10 + +**No prefix-clear primitive exists.** `TQTokenSink.clear()` is explicit-keys-only +(`nemo_rl/data_plane/tq_token_sink.py:280-290`); the layer below +(`data_plane/interfaces.py:415-440`) only offers a producer-local clear-all that may +silently no-op for non-producers. Manifest-enumerated keys cover every row the +ledger knows about, including dead branches. The remaining gap is the +crash-between-stage-and-ledger-append orphan (a staged row no manifest names). +Options: (a) add a prefix/scan delete to the TransferQueue data plane, (b) accept +the leak until partition teardown, (c) have the worker report the staging key in +failure coords so the orphan lands in a failure row and stays enumerable. + +### Follow-up (explicitly out of scope for this change) + +**Retry idempotency via deterministic identity.** The harness mints a unique +`x-nemo-gym-logical-request-id` per model call before dispatch, reuses it verbatim +on retry, and reports the terminal call's logical id (replacing the post-hoc +response-id derivation at `responses_api_agents/swe_agents/app.py:3119`); the +middleware then derives `model_call_id` deterministically from it (`:1254`). +Identical retries collapse into the same row and staging key instead of poisoning +or dead-branching; divergent retries poison via `record()`'s conflict rule. +Optionally layered on top: store the response payload on the ledger row and replay +it on retry (idempotency cache), eliminating divergent regeneration entirely. The +deferred surface is exactly two touch points, so nothing in this plan is redone. + +## Related documents + +- `token-capture-external-sink-recommendation.md` (repo root) — the review that + motivated this design; this document promotes its "per-rollout append-only capture + ledger" option from a future alternative to the recommended mechanism. +- `docs/guides/nano-swe-token-capture.md` — the recipe exercising this path end to + end. diff --git a/ultra_launch.sh b/ultra_launch.sh new file mode 100755 index 00000000000..a313cdbfbde --- /dev/null +++ b/ultra_launch.sh @@ -0,0 +1,957 @@ +#!/bin/bash +set -euo pipefail + +# ============================================================================= +# ultra_launch.sh +# +# Public launcher for Nemotron 3 Ultra post-training stages on a SLURM cluster. +# +# Each training stage (Student RLVR, teacher RLVR/RLHF stages, MOPD) has a +# matching YAML config under examples/configs/ultra/. The stage-specific +# hyperparameters (batch size, advantage clip, MoE parallelism, etc.) live +# in the YAML; this launcher only handles orchestration: SLURM submission, +# code snapshotting, persistent cache management, container mounts, and the +# Hydra overrides that vary per run (data paths, model checkpoint, judge +# endpoints, log directories). +# +# Usage: +# +# EXP_NAME=ultra-student-rlvr-001 \ +# CONFIG_PATH=examples/configs/ultra/student_rlvr.yaml \ +# MODEL_PATH=/path/to/sft_checkpoint \ +# TRAIN_PATH=/path/to/train.jsonl \ +# VAL_PATH=/path/to/val.jsonl \ +# CONTAINER=nvcr.io/nvidia/nemo-rl: \ +# SANDBOX_CONTAINER=/path/to/nemo-skills-sandbox.sqsh \ +# PERSISTENT_CACHE=/path/to/persistent/cache \ +# SLURM_PARTITION=batch \ +# SLURM_ACCOUNT=your_account \ +# GENRM_MODEL= # Required for local and remote GenRM +# NL2BASH_JUDGE_MODEL=Qwen/Qwen3-235B-A22B-Instruct-2507-FP8 \ +# SAFETY_JUDGE_MODEL=/path/to/safety_checkpoint \ +# bash ultra_launch.sh +# +# Optional knobs: +# WALLTIME=4:00:00 Slurm --time +# SLURM_QOS= Slurm --qos; defaults to short when +# WALLTIME is under two hours +# SLURM_RESERVATION= Slurm --reservation +# SLURM_DEPENDENCY= Extra Slurm dependency, merged with +# singleton (e.g. afterany:) +# GENRM_BASE_URL= Remote GenRM OpenAI base URL +# GENRM_BASE_URL_FILE= Shared file containing a remote GenRM +# host:port or OpenAI base URL. Resolved +# after the job allocation starts. +# GENRM_WAIT_TIMEOUT_SECONDS=3600 Endpoint-file readiness timeout +# EXCLUDE_NODES= Slurm --exclude +# NUM_TRAIN_NODES=64 Training (Megatron) nodes +# NUM_GEN_NODES=172 vLLM generation nodes +# NUM_GYM_NODES=20 NeMo Gym (judge) nodes +# ENABLE_MTP_INFERENCE=0 1 to enable MTP speculative decoding +# NUM_SPECULATIVE_TOKENS=5 MTP speculative tokens +# MAX_NUM_BATCHED_TOKENS=8480 vLLM max batched tokens (MTP) +# NRL_MAX_STEPS= Override grpo.max_num_steps +# EXTRA_MOUNTS= Comma-separated host:container pairs +# USE_SNAPSHOT=1 Snapshot source tree at submission +# USE_CUSTOM_VLLM=1 1 to require the Ultra vLLM fork; +# 0 to use the container's regular vLLM +# DRY_RUN=0 1 to print TRAIN_CMD and exit +# INTERACTIVE=0 1 to bring up Ray and idle for attach +# (no training driver) for debugging +# INTERACTIVE_WAIT=1 0 to submit and return immediately +# INTERACTIVE_WALLTIME= override WALLTIME for the interactive alloc +# HF_HOME= HuggingFace cache root (recommended) +# HF_TOKEN= HuggingFace API token +# NRL_DRIVER_UV_RUN_FLAGS= Extra flags for the driver-only uv run +# NEMO_RL_VENV_DIR= Worker venv root forwarded to the driver +# NEMO_GYM_VENV_DIR= Gym service venv root forwarded to the driver +# UV_PYTHON= Python request used for uv-managed worker venvs +# UV_PYTHON_INSTALL_DIR= Managed Python root forwarded to the driver +# NRL_UV_BIN_DIR= Directory containing the uv binary used by workers +# WANDB_API_KEY= Weights & Biases API key +# WANDB_PROJ=nemotron-3-ultra W&B project +# WANDB_ENTITY= W&B entity +# +# Hydra overrides are forwarded verbatim as positional arguments: +# bash ultra_launch.sh policy.megatron_cfg.optimizer.lr=1e-6 grpo.val_period=50 +# +# GB200 NVL72 nodes have 4 GPUs each. SLURM total = NUM_TRAIN + NUM_GEN + NUM_GYM +# and must be a multiple of SEGMENT_SIZE (default 16, one NVLink domain group). +# ============================================================================= + +# ============================================================================= +# Required environment +# ============================================================================= +: "${EXP_NAME:?EXP_NAME is required (used for job name, W&B run, checkpoint/log dirs)}" +: "${CONFIG_PATH:?CONFIG_PATH is required (e.g. examples/configs/ultra/student_rlvr.yaml)}" +: "${MODEL_PATH:?MODEL_PATH is required (initial policy checkpoint, HF repo id or local path)}" +: "${TRAIN_PATH:?TRAIN_PATH is required (training data jsonl path)}" +: "${VAL_PATH:?VAL_PATH is required (validation data jsonl path)}" +: "${CONTAINER:?CONTAINER is required (NGC image URI or .sqsh path)}" +: "${SANDBOX_CONTAINER:?SANDBOX_CONTAINER is required (nemo-skills sandbox image)}" +: "${PERSISTENT_CACHE:?PERSISTENT_CACHE is required (Lustre dir for vLLM/Triton/Inductor caches)}" +: "${SLURM_PARTITION:?SLURM_PARTITION is required}" +: "${SLURM_ACCOUNT:?SLURM_ACCOUNT is required}" +# Judge models are recipe-specific. Most teachers (student RLVR, IFBench, RLHF, +# Reasoning) need all three (GenRM, NL2Bash, Safety). The SWE teacher uses +# code-execution rewards and needs none of them. Set per recipe; unset vars +# skip the corresponding override. +NL2BASH_JUDGE_MODEL="${NL2BASH_JUDGE_MODEL:-}" +SAFETY_JUDGE_MODEL="${SAFETY_JUDGE_MODEL:-}" +GENRM_BASE_URL="${GENRM_BASE_URL:-}" +GENRM_BASE_URL_FILE="${GENRM_BASE_URL_FILE:-}" +GENRM_WAIT_TIMEOUT_SECONDS="${GENRM_WAIT_TIMEOUT_SECONDS:-3600}" +GENRM_MODEL="${GENRM_MODEL:-}" +GENRM_OVERRIDE="" +GENRM_RUNTIME_SETUP="" +if [[ -n "${GENRM_BASE_URL}" && -n "${GENRM_BASE_URL_FILE}" ]]; then + echo "ERROR: set only one of GENRM_BASE_URL or GENRM_BASE_URL_FILE" >&2 + exit 1 +fi +if [[ -n "${GENRM_BASE_URL}" || -n "${GENRM_BASE_URL_FILE}" ]]; then + if [[ -z "${GENRM_MODEL}" ]]; then + echo "ERROR: GENRM_MODEL is required when using a remote GenRM endpoint" >&2 + exit 1 + fi +fi +if [[ ! "${GENRM_WAIT_TIMEOUT_SECONDS}" =~ ^[1-9][0-9]*$ ]]; then + echo "ERROR: GENRM_WAIT_TIMEOUT_SECONDS must be a positive integer" >&2 + exit 1 +fi +if [[ -n "${GENRM_BASE_URL_FILE}" ]]; then + GENRM_RUNTIME_SETUP="\ +GENRM_BASE_URL_FILE='${GENRM_BASE_URL_FILE}' ; \ +GENRM_WAIT_TIMEOUT_SECONDS=${GENRM_WAIT_TIMEOUT_SECONDS} ; \ +GENRM_WAIT_DEADLINE=\$((SECONDS + GENRM_WAIT_TIMEOUT_SECONDS)) ; \ +echo \"Waiting for GenRM endpoint file: \${GENRM_BASE_URL_FILE}\" ; \ +while true; do \ + if [[ -s \"\${GENRM_BASE_URL_FILE}\" ]]; then \ + GENRM_ENDPOINT=\$(tr -d '[:space:]' < \"\${GENRM_BASE_URL_FILE}\") ; \ + case \"\${GENRM_ENDPOINT}\" in \ + http://*|https://*) GENRM_RUNTIME_BASE_URL=\"\${GENRM_ENDPOINT%/}\" ;; \ + *) GENRM_RUNTIME_BASE_URL=\"http://\${GENRM_ENDPOINT%/}\" ;; \ + esac ; \ + case \"\${GENRM_RUNTIME_BASE_URL}\" in \ + */v1) ;; \ + *) GENRM_RUNTIME_BASE_URL=\"\${GENRM_RUNTIME_BASE_URL}/v1\" ;; \ + esac ; \ + if curl --fail --silent --show-error --connect-timeout 5 --max-time 10 \ + \"\${GENRM_RUNTIME_BASE_URL}/models\" >/dev/null; then \ + echo \"GenRM is ready at \${GENRM_RUNTIME_BASE_URL}\" ; \ + break ; \ + fi ; \ + fi ; \ + if (( SECONDS >= GENRM_WAIT_DEADLINE )); then \ + echo \"ERROR: timed out waiting for GenRM via \${GENRM_BASE_URL_FILE}\" >&2 ; \ + exit 1 ; \ + fi ; \ + sleep 10 ; \ +done ; " + GENRM_OVERRIDE="env.nemo_gym.genrm_model.responses_api_models.genrm_model.base_url=\${GENRM_RUNTIME_BASE_URL} env.nemo_gym.genrm_model.responses_api_models.genrm_model.model=${GENRM_MODEL}" +elif [[ -n "${GENRM_BASE_URL}" ]]; then + GENRM_BASE_URL="${GENRM_BASE_URL%/}" + [[ "${GENRM_BASE_URL}" == */v1 ]] || GENRM_BASE_URL="${GENRM_BASE_URL}/v1" + GENRM_OVERRIDE="env.nemo_gym.genrm_model.responses_api_models.genrm_model.base_url=${GENRM_BASE_URL} env.nemo_gym.genrm_model.responses_api_models.genrm_model.model=${GENRM_MODEL}" +elif [[ -n "${GENRM_MODEL}" ]]; then + GENRM_OVERRIDE="env.nemo_gym.genrm_model.responses_api_models.genrm_model.model=${GENRM_MODEL}" +fi + +# SIF_DIR: for the SWE teacher recipe — directory containing apptainer .sif +# images for SWE-Bench / SWE-Gym / R2E-Gym instances. The yaml's +# container_formatter uses `${sif_dir}/...` paths. Unset for non-SWE recipes. +SIF_DIR="${SIF_DIR:-}" + +if [[ ! -f "${CONFIG_PATH}" ]]; then + echo "ERROR: CONFIG_PATH does not exist: ${CONFIG_PATH}" >&2 + exit 1 +fi + +# ============================================================================= +# Project root and code root +# ============================================================================= +PROJECT_ROOT=$(realpath "$PWD") +cd "${PROJECT_ROOT}" + +# ============================================================================= +# Job identity — fixed name for singleton. +# Slurm --dependency=singleton serialises queued submissions with the same name +# so a resubmission after preemption resumes from the latest checkpoint instead +# of running in parallel. +# ============================================================================= +JOB_NAME="${EXP_NAME}" + +# ============================================================================= +# Output directories +# ============================================================================= +RESULTS_DIR="${RESULTS_DIR:-results/${EXP_NAME}}" +CHECKPOINT_DIR="${CHECKPOINT_DIR:-${RESULTS_DIR}/checkpoints}" + +# Per-submission dirs for logs and Slurm output (timestamped for history). +RUN_DIR="${RESULTS_DIR}/runs/$(date +%Y%m%d-%H%M)" +LOG_DIR="${RUN_DIR}/logs" +SLURM_LOG_DIR="${RUN_DIR}/slurm" +mkdir -p "${CHECKPOINT_DIR}" "${LOG_DIR}" "${SLURM_LOG_DIR}" +ln -sfn "$(realpath "${RUN_DIR}")" "${RESULTS_DIR}/runs/latest" + +# ray.sub reads BASE_LOG_DIR and creates $BASE_LOG_DIR/$SLURM_JOB_ID-logs/ for +# ray infrastructure logs (ray-head.log, ray-driver.log, ray-worker-*.log, +# topology probes, attach scripts, etc.). +export BASE_LOG_DIR="${BASE_LOG_DIR:-${RESULTS_DIR}/ray_logs}" + +# ============================================================================= +# SLURM configuration +# ============================================================================= +WALLTIME="${WALLTIME:-4:00:00}" +SLURM_QOS="${SLURM_QOS:-}" +SLURM_RESERVATION="${SLURM_RESERVATION:-}" +EXCLUDE_NODES="${EXCLUDE_NODES:-}" + +slurm_walltime_seconds() { + local value="$1" + local days=0 + local -a fields + + if [[ "${value}" == *-* ]]; then + days="${value%%-*}" + value="${value#*-}" + fi + [[ "${days}" =~ ^[0-9]+$ ]] || return 1 + + IFS=: read -r -a fields <<< "${value}" + for field in "${fields[@]}"; do + [[ "${field}" =~ ^[0-9]+$ ]] || return 1 + done + + case "${#fields[@]}" in + 1) + if (( days > 0 )); then + echo $((10#${days} * 86400 + 10#${fields[0]} * 3600)) + else + echo $((10#${fields[0]} * 60)) + fi + ;; + 2) + if (( days > 0 )); then + echo $((10#${days} * 86400 + 10#${fields[0]} * 3600 + 10#${fields[1]} * 60)) + else + echo $((10#${fields[0]} * 60 + 10#${fields[1]})) + fi + ;; + 3) + echo $((10#${days} * 86400 + 10#${fields[0]} * 3600 + 10#${fields[1]} * 60 + 10#${fields[2]})) + ;; + *) return 1 ;; + esac +} + +if [[ -z "${SLURM_QOS}" ]]; then + if WALLTIME_SECONDS="$(slurm_walltime_seconds "${WALLTIME}")"; then + if (( WALLTIME_SECONDS < 2 * 60 * 60 )); then + SLURM_QOS=short + fi + else + echo "[WARN] Could not parse WALLTIME=${WALLTIME}; leaving SLURM_QOS unset." >&2 + fi +fi +# INTERACTIVE=1 brings up the Ray cluster and idles for attachment (no training +# driver), so you can run/debug the recipe by hand. INTERACTIVE_WAIT=1 (default) +# blocks until Ray is ready; INTERACTIVE_WALLTIME overrides WALLTIME for the alloc. +INTERACTIVE="${INTERACTIVE:-0}" +INTERACTIVE_WAIT="${INTERACTIVE_WAIT:-1}" +# If set (format DD:HH:MM:SS), training stops early to reserve time for a final +# checkpoint save before walltime. Unset to use the YAML's default and let +# slurm walltime end the job naturally — fine when each step checkpoints. +CHECKPOINTING_SAVE_BY="${CHECKPOINTING_SAVE_BY:-}" + +# ============================================================================= +# Container & mounts +# ============================================================================= +export CONTAINER +MOUNTS="${MOUNTS:-}" + +# GB200 NVL72 defaults to 4 GPUs/node. Allow H100 smoke configs to request +# their native 8-GPU node shape through the launch environment. +export GPUS_PER_NODE="${GPUS_PER_NODE:-4}" +export CPUS_PER_WORKER="${CPUS_PER_WORKER:-144}" + +# ============================================================================= +# HuggingFace configuration +# ============================================================================= +if [[ -n "${HF_HOME:-}" ]]; then + export HF_HOME + export HF_HUB_CACHE="${HF_HUB_CACHE:-${HF_HOME}/hub}" + export HF_DATASETS_CACHE="${HF_DATASETS_CACHE:-${HF_HOME}/hub}" +else + echo "[WARN] HF_HOME is not set — HuggingFace will use the default cache (~/.cache/huggingface) per-node." >&2 +fi + +# ============================================================================= +# W&B configuration +# ============================================================================= +WANDB_PROJ="${WANDB_PROJ:-nemotron-3-ultra}" +WANDB_NAME="${EXP_NAME}" +WANDB_ENABLED=False +if [[ -n "${WANDB_API_KEY:-}" ]]; then + export WANDB_API_KEY + WANDB_ENABLED=True + if [[ -n "${WANDB_ENTITY:-}" ]]; then + export WANDB_ENTITY + fi +else + echo "[WARN] WANDB_API_KEY is not set — W&B logging will be disabled." >&2 +fi + +# ============================================================================= +# Training overrides +# ============================================================================= +NRL_MAX_STEPS="${NRL_MAX_STEPS:-}" + +# ============================================================================= +# MTP speculative decoding (optional) +# ============================================================================= +ENABLE_MTP_INFERENCE="${ENABLE_MTP_INFERENCE:-0}" +NUM_SPECULATIVE_TOKENS="${NUM_SPECULATIVE_TOKENS:-5}" +MAX_NUM_BATCHED_TOKENS="${MAX_NUM_BATCHED_TOKENS:-8480}" +MTP_EXTRA_ARGS="" +if [[ "${ENABLE_MTP_INFERENCE}" == "1" ]]; then + MTP_EXTRA_ARGS="\ +++policy.generation.vllm_cfg.enable_prefix_caching=true \ +++policy.generation.vllm_kwargs.enable_chunked_prefill=true \ +++policy.generation.vllm_kwargs.max_num_batched_tokens=${MAX_NUM_BATCHED_TOKENS} \ +++policy.generation.vllm_kwargs.mamba_cache_mode=align \ +~policy.generation.vllm_kwargs.compilation_config.cudagraph_capture_sizes \ +++policy.generation.vllm_kwargs.speculative_config.num_speculative_tokens=${NUM_SPECULATIVE_TOKENS} \ +++policy.generation.vllm_kwargs.speculative_config.method=mtp" + echo "MTP speculative decoding ENABLED (num_speculative_tokens=${NUM_SPECULATIVE_TOKENS})" +fi + +# ============================================================================= +# Job shape — defaults match the 256-node student_rlvr.yaml +# +# Training: 64 nodes ( 256 GPUs) — Megatron training backend +# vLLM: 172 nodes ( 688 GPUs) — async generation, EP=8 instances at TP=8 +# Gym: 20 nodes ( 80 GPUs) — judges (GenRM, NL2Bash, Safety) +# +# Override via NUM_TRAIN_NODES / NUM_GEN_NODES / NUM_GYM_NODES. +# +# For STAGE_TYPE=mopd, additional teacher nodes are allocated for the +# non-colocated teacher panel: NUM_UNIQUE_TEACHERS × NUM_NODES_PER_TEACHER. +# ============================================================================= +NUM_TRAIN_NODES="${NUM_TRAIN_NODES:-64}" +NUM_GEN_NODES="${NUM_GEN_NODES:-172}" +NUM_GYM_NODES="${NUM_GYM_NODES:-20}" + +STAGE_TYPE="${STAGE_TYPE:-grpo}" +NUM_TEACHER_NODES=0 +MOPD_OVERRIDES="" +if [[ "${STAGE_TYPE}" == "mopd" ]]; then + : "${NRL_GENERAL_TEACHER_PATH:?NRL_GENERAL_TEACHER_PATH is required for STAGE_TYPE=mopd (path to the Student RLVR output checkpoint)}" + NUM_UNIQUE_TEACHERS="${NUM_UNIQUE_TEACHERS:-5}" + NUM_NODES_PER_TEACHER="${NUM_NODES_PER_TEACHER:-4}" + NUM_TEACHER_NODES=$((NUM_UNIQUE_TEACHERS * NUM_NODES_PER_TEACHER)) + TEACHER_TP="${TEACHER_TP:-8}" + TEACHER_CP="${TEACHER_CP:-2}" + TEACHER_PP="${TEACHER_PP:-1}" + TEACHER_EP="${TEACHER_EP:-16}" + + # _teachers.general is required; other slots fall back via the YAML's + # interpolation. Pass only the slots the user explicitly set. + MOPD_OVERRIDES="_teachers.general=${NRL_GENERAL_TEACHER_PATH}" + for _slot in RLHF IFBENCH REASONING SWE; do + _var="NRL_${_slot}_TEACHER_PATH" + _val="${!_var:-}" + if [[ -n "${_val}" ]]; then + MOPD_OVERRIDES="${MOPD_OVERRIDES} _teachers.$(echo ${_slot} | tr A-Z a-z)=${_val}" + fi + done + + # Teacher parallelism + per-teacher node count + MOPD_OVERRIDES="${MOPD_OVERRIDES} \ +on_policy_distillation.non_colocated_teachers.default_teacher_cfg.tensor_model_parallel_size=${TEACHER_TP} \ +on_policy_distillation.non_colocated_teachers.default_teacher_cfg.context_parallel_size=${TEACHER_CP} \ +on_policy_distillation.non_colocated_teachers.default_teacher_cfg.pipeline_model_parallel_size=${TEACHER_PP} \ +on_policy_distillation.non_colocated_teachers.default_teacher_cfg.expert_model_parallel_size=${TEACHER_EP} \ +on_policy_distillation.non_colocated_teachers.default_teacher_cfg.num_nodes=${NUM_NODES_PER_TEACHER}" + + echo "MOPD: ${NUM_UNIQUE_TEACHERS} teacher pools × ${NUM_NODES_PER_TEACHER} nodes = ${NUM_TEACHER_NODES} teacher nodes" +fi + +NUM_ACTOR_NODES=$((NUM_TRAIN_NODES + NUM_GEN_NODES + NUM_TEACHER_NODES)) +NUM_TOTAL_NODES=$((NUM_ACTOR_NODES + NUM_GYM_NODES)) + +if (( NUM_TRAIN_NODES <= 0 )); then + echo "ERROR: NUM_TRAIN_NODES must be > 0 (got ${NUM_TRAIN_NODES})" >&2; exit 1 +fi +if (( NUM_GEN_NODES <= 0 )); then + echo "ERROR: NUM_GEN_NODES must be > 0 (got ${NUM_GEN_NODES})" >&2; exit 1 +fi +if (( NUM_GYM_NODES < 0 )); then + echo "ERROR: NUM_GYM_NODES must be >= 0 (got ${NUM_GYM_NODES})" >&2; exit 1 +fi + +# GB200 NVL72 topology: 18 nodes per NVLink domain, allocate in groups of 16. +SEGMENT_SIZE="${SEGMENT_SIZE:-16}" +if (( NUM_TOTAL_NODES < SEGMENT_SIZE )); then + echo "ERROR: NUM_TOTAL_NODES=${NUM_TOTAL_NODES} < SEGMENT_SIZE=${SEGMENT_SIZE}" >&2 + exit 1 +fi +if (( NUM_TOTAL_NODES % SEGMENT_SIZE != 0 )); then + echo "ERROR: NUM_TOTAL_NODES=${NUM_TOTAL_NODES} is not divisible by SEGMENT_SIZE=${SEGMENT_SIZE}." >&2 + echo " Training=${NUM_TRAIN_NODES} + Generation=${NUM_GEN_NODES} + Gym=${NUM_GYM_NODES} + Teachers=${NUM_TEACHER_NODES} = ${NUM_TOTAL_NODES}" >&2 + echo " Adjust node counts so the total is a multiple of ${SEGMENT_SIZE}." >&2 + exit 1 +fi + +# ============================================================================= +# NeMo Skills sandbox (for math_formal_lean, ns_tools, etc.) +# ============================================================================= +export SANDBOX_CONTAINER +export SANDBOX_COMMAND="${SANDBOX_COMMAND:-/start-with-nginx.sh}" +export NEMO_SKILLS_SANDBOX_PORT="${NEMO_SKILLS_SANDBOX_PORT:-6000}" + +# ============================================================================= +# Ray log sync +# ============================================================================= +export RAY_LOG_SYNC_FREQUENCY="${RAY_LOG_SYNC_FREQUENCY:-60}" + +CODE_ROOT="/opt/nemo-rl" +USE_CUSTOM_VLLM="${USE_CUSTOM_VLLM:-1}" +case "${USE_CUSTOM_VLLM}" in + 1) + VLLM_ENV_SOURCE="source /opt/nemo-rl/3rdparty/vllm/nemo-rl.env && " + ;; + 0) + VLLM_ENV_SOURCE="" + ;; + *) + echo "ERROR: USE_CUSTOM_VLLM must be 0 or 1, got: ${USE_CUSTOM_VLLM}" >&2 + exit 1 + ;; +esac + +# ============================================================================= +# Persistent cache directories +# ============================================================================= +# Lustre holds the warm persistent cache. At job start, SETUP_COMMAND clears +# stale /tmp caches then seeds node-local /tmp from Lustre. JIT writes go to +# /tmp to avoid Lustre metadata contention from parallel compilation. +_vllm_cache_precision="bf16" +CACHE_READ_DIR="${PERSISTENT_CACHE}/cache_read" +CACHE_WRITE_DIR="${PERSISTENT_CACHE}/cache_write" +LUSTRE_VLLM_CACHE="${CACHE_WRITE_DIR}/vllm_compile_cache_${_vllm_cache_precision}" +LUSTRE_FLASHINFER_CUBIN_CACHE="${PERSISTENT_CACHE}/flashinfer_cubins" +FLASHINFER_CUBIN_CACHE="/tmp/nemo_rl_flashinfer_cubins" +FLASHINFER_WS_BASE="${PERSISTENT_CACHE}/flashinfer_workspace" +LUSTRE_INDUCTOR_CACHE="${PERSISTENT_CACHE}/inductor_cache" +LUSTRE_TRITON_CACHE="${PERSISTENT_CACHE}/triton_cache" +NRL_VLLM_LOCAL_CACHE_DIR="/tmp/nemo_rl_vllm_cache" +NRL_VLLM_CACHE_SEED_DIR="/tmp/nemo_rl_vllm_cache_warm" +INDUCTOR_CACHE_DIR="/tmp/nemo_rl_inductor_cache" +TRITON_CACHE_DIR="/tmp/nemo_rl_triton_cache" +CACHE_SYNC_FREQUENCY="${CACHE_SYNC_FREQUENCY:-0}" + +export LUSTRE_VLLM_CACHE +export LUSTRE_INDUCTOR_CACHE +export LUSTRE_TRITON_CACHE +export CACHE_READ_DIR +export CACHE_WRITE_DIR +export NRL_VLLM_LOCAL_CACHE_DIR +export INDUCTOR_CACHE_DIR +export TRITON_CACHE_DIR +export CACHE_SYNC_FREQUENCY + +mkdir -p "${LUSTRE_FLASHINFER_CUBIN_CACHE}" "${FLASHINFER_WS_BASE}" \ + "${LUSTRE_INDUCTOR_CACHE}" "${LUSTRE_TRITON_CACHE}" \ + "${CACHE_READ_DIR}" "${CACHE_WRITE_DIR}" + +# Read path : cache_read/*.tar.zst — compute nodes extract tarballs (hundreds of concurrent reads) +# Write path : cache_write/*/ — sidecar rsyncs individual files (one sequential writer) +# Splitting reads (tarball) from writes (directory) avoids Lustre MDT invalidation storms +# and lets rsync accumulate the union of all roles' kernels across jobs. +for _name in inductor_cache triton_cache; do + _write_dir="${CACHE_WRITE_DIR}/${_name}" + _old_dir="${PERSISTENT_CACHE}/${_name}" + + # One-time migration: move legacy dir → cache_write/ (instant rename, same FS) + if ([ ! -d "$_write_dir" ] || [ -z "$(ls -A "$_write_dir" 2>/dev/null)" ]) \ + && [ -d "$_old_dir" ] && [ -n "$(ls -A "$_old_dir" 2>/dev/null)" ]; then + [ -d "$_write_dir" ] && rmdir "$_write_dir" 2>/dev/null + mv "$_old_dir" "$_write_dir" 2>/dev/null \ + && echo "[CACHE] Moved legacy ${_name}/ → cache_write/${_name}/" \ + || echo "[CACHE] Failed to move legacy ${_name}/" + fi +done + +# vLLM: migrate the most recent legacy seed dir → cache_write/ (one-time, instant rename) +_vllm_write="${CACHE_WRITE_DIR}/vllm_compile_cache_${_vllm_cache_precision}" +_vllm_read_tar="${CACHE_READ_DIR}/vllm_compile_cache_${_vllm_cache_precision}.tar.zst" + +if [ ! -d "$_vllm_write" ] || [ -z "$(ls -A "$_vllm_write" 2>/dev/null)" ]; then + _best="$(ls -1dt \ + "${PERSISTENT_CACHE}/vllm_compile_cache_${_vllm_cache_precision}" \ + "${PERSISTENT_CACHE}/vllm_compile_cache_${_vllm_cache_precision}_"* \ + 2>/dev/null \ + | while IFS= read -r d; do + [ -d "$d" ] && [ -n "$(ls -A "$d" 2>/dev/null)" ] && echo "$d" && break + done + )" || true + if [ -n "$_best" ]; then + [ -d "$_vllm_write" ] && rmdir "$_vllm_write" 2>/dev/null || true + mv "$_best" "$_vllm_write" 2>/dev/null \ + && echo "[CACHE] Moved $(basename "$_best") → cache_write/vllm_compile_cache_${_vllm_cache_precision}/" \ + || echo "[CACHE] Failed to move vLLM cache" + fi +fi + +# Purge redundant legacy vLLM cache directories. +# The old sidecar wrote every vLLM seed as a separate directory on Lustre +# (e.g. vllm_compile_cache_bf16_2058, _3072, ...). With cache_write/ + tarball, +# only cache_write/vllm_compile_cache_{precision}/ matters. All seed copies are +# content-addressed duplicates — safe to remove after migration. +_purge_count=0 +for _d in "${PERSISTENT_CACHE}/vllm_compile_cache_${_vllm_cache_precision}" \ + "${PERSISTENT_CACHE}/vllm_compile_cache_${_vllm_cache_precision}_"*; do + [ -d "$_d" ] || continue + rm -rf "$_d" 2>/dev/null && (( _purge_count++ )) || true +done +for _d in "${PERSISTENT_CACHE}"/vllm_compile_cache_[0-9]*/; do + [ -d "$_d" ] || continue + rm -rf "$_d" 2>/dev/null && (( _purge_count++ )) || true +done +for _d in "${PERSISTENT_CACHE}/vllm_compile_cache" \ + "${PERSISTENT_CACHE}/vllm_compile_cache_warm"; do + [ -d "$_d" ] || continue + rm -rf "$_d" 2>/dev/null && (( _purge_count++ )) || true +done +if (( _purge_count > 0 )); then + echo "[CACHE] Purged ${_purge_count} redundant legacy vLLM cache directories from ${PERSISTENT_CACHE}/" +fi + +# ============================================================================= +# Code snapshot +# ============================================================================= +# Snapshot the git-tracked source tree so the code is frozen at submission time. +# This guarantees we know exactly which code was used for a given experiment. +# Set USE_SNAPSHOT=0 to skip (runs from container built-in or live checkout). +# Interactive mode defaults to the live checkout for fast iteration; batch snapshots. +if [[ "${INTERACTIVE}" == "1" ]]; then + USE_SNAPSHOT="${USE_SNAPSHOT:-0}" +else + USE_SNAPSHOT="${USE_SNAPSHOT:-1}" +fi + +if [[ "${USE_SNAPSHOT}" == "1" ]]; then + if [[ ! -f "${PROJECT_ROOT}/tools/code_snapshot.sh" ]]; then + echo "ERROR: tools/code_snapshot.sh not found at ${PROJECT_ROOT}/tools/code_snapshot.sh" >&2 + echo " Set USE_SNAPSHOT=0 to run from the live checkout instead." >&2 + exit 1 + fi + SNAPSHOT_DIR=$(bash "${PROJECT_ROOT}/tools/code_snapshot.sh" "${JOB_NAME}") + + if [[ -d "${PROJECT_ROOT}/3rdparty/vllm" ]] && [[ ! -e "${SNAPSHOT_DIR}/3rdparty/vllm" ]]; then + mkdir -p "${SNAPSHOT_DIR}/3rdparty" + ln -s "${PROJECT_ROOT}/3rdparty/vllm" "${SNAPSHOT_DIR}/3rdparty/vllm" + fi + + echo "Code snapshot: ${SNAPSHOT_DIR}" + OVERLAY_SOURCE="${SNAPSHOT_DIR}" +else + OVERLAY_SOURCE="${PROJECT_ROOT}" +fi + +# ============================================================================= +# Container mounts +# ============================================================================= +# By default, the project metadata/lock, nemo_rl (Python package), +# examples/configs (YAML configs), and local uv sources/workspace members from +# the code snapshot are overlaid into the container. Keeping the project files +# and their local sources together is required for `uv sync --locked`. +# +# To overlay additional components (e.g. a local Megatron-LM checkout), pass +# EXTRA_MOUNTS as a comma-separated list of host:container pairs: +# +# EXTRA_MOUNTS="/path/to/Megatron-LM:/opt/nemo-rl/3rdparty/Megatron-LM-workspace/Megatron-LM" bash ultra_launch.sh +# +# Container paths for reference: +# /opt/nemo-rl/nemo_rl — Python package +# /opt/nemo-rl/examples/configs — YAML configs +# /opt/nemo-rl/3rdparty/Megatron-LM-workspace/Megatron-LM — Megatron-LM +# /opt/nemo-rl/3rdparty/Megatron-Bridge-workspace/Megatron-Bridge — Megatron-Bridge +# /opt/nemo-rl/3rdparty/Gym-workspace/Gym — NeMo-Gym +# /opt/nemo-rl/3rdparty/vllm — vLLM +# ============================================================================= +_append_mount() { + if [[ -z "${MOUNTS}" ]]; then + MOUNTS="$1" + else + MOUNTS="${MOUNTS},$1" + fi +} + +if [[ -d "${OVERLAY_SOURCE}/nemo_rl" ]]; then + _append_mount "${OVERLAY_SOURCE}/nemo_rl:/opt/nemo-rl/nemo_rl" + echo " Mount: nemo_rl → /opt/nemo-rl/nemo_rl" +fi +for _project_file in pyproject.toml uv.lock .python-version; do + if [[ -f "${OVERLAY_SOURCE}/${_project_file}" ]]; then + _append_mount "${OVERLAY_SOURCE}/${_project_file}:/opt/nemo-rl/${_project_file}" + echo " Mount: ${_project_file} → /opt/nemo-rl/${_project_file}" + fi +done +if [[ -d "${OVERLAY_SOURCE}/examples/configs" ]]; then + _append_mount "${OVERLAY_SOURCE}/examples/configs:/opt/nemo-rl/examples/configs" + echo " Mount: configs → /opt/nemo-rl/examples/configs" +fi +_local_project_paths=( + "3rdparty/TensorRT-LLM-workspace" + "3rdparty/Automodel-workspace/Automodel" + "3rdparty/Megatron-Bridge-workspace/Megatron-Bridge" + "3rdparty/Gym-workspace/Gym" + "research/template_project" +) +for _local_project_path in "${_local_project_paths[@]}"; do + if [[ -d "${OVERLAY_SOURCE}/${_local_project_path}" ]]; then + _append_mount "${OVERLAY_SOURCE}/${_local_project_path}:/opt/nemo-rl/${_local_project_path}" + echo " Mount: ${_local_project_path} → /opt/nemo-rl/${_local_project_path}" + fi +done + +if [[ "${USE_SNAPSHOT}" == "1" ]]; then + _append_mount "${SNAPSHOT_DIR}:${SNAPSHOT_DIR}" +fi + +if [[ -n "${EXTRA_MOUNTS:-}" ]]; then + _append_mount "${EXTRA_MOUNTS}" + echo " Extra mounts: ${EXTRA_MOUNTS}" +fi + +export MOUNTS + +# ============================================================================= +# Resolve ray.sub +# ============================================================================= +RAY_SUB="${RAY_SUB:-${PROJECT_ROOT}/ray.sub}" +if [[ ! -f "${RAY_SUB}" ]]; then + echo "ERROR: ray.sub not found at ${RAY_SUB}" >&2 + exit 1 +fi + +# ============================================================================= +# Per-node cache seeding (SETUP_COMMAND) +# ============================================================================= +# Triton, Inductor, and FlashInfer cubins compile/download to node-local /tmp to +# avoid Lustre race conditions and file lock contention during concurrent JIT +# compilation. To avoid cold-start penalties, we seed /tmp from a warm Lustre +# cache before Ray starts. +# +# IMPORTANT: Stale /tmp caches from previous jobs can cause hangs (e.g. the +# Triton bundler skipping non-empty temp dirs). We rm -rf /tmp caches first, +# then seed fresh from Lustre. +# ============================================================================= +read -r -d '' SETUP_COMMAND </dev/null 2>&1 || { apt-get update -qq && apt-get install -y -qq zstd; } 2>/dev/null || true +echo "[CACHE SEED] Clearing stale /tmp caches and seeding from Lustre..." +WARM_SEED="${NRL_VLLM_CACHE_SEED_DIR}" +LOCAL_IND="${INDUCTOR_CACHE_DIR}" +LOCAL_TRI="${TRITON_CACHE_DIR}" +CACHE_READ="${CACHE_READ_DIR}" + +# vLLM caches are per-instance (VLLM_CACHE_ROOT_{seed}). Clear ALL from prior jobs. +rm -rf /tmp/nemo_rl_vllm_cache /tmp/nemo_rl_vllm_cache_* +rm -rf "\$LOCAL_IND" "\$LOCAL_TRI" +mkdir -p "\$LOCAL_IND" "\$LOCAL_TRI" + +_seed_cache() { + local tarball="\$1" local_dir="\$2" name="\$3" + if [ -f "\$tarball" ]; then + tar --zstd -xf "\$tarball" -C "\$local_dir" \ + && echo "[CACHE SEED] \$name: seeded from tarball (\$(du -sh "\$local_dir" 2>/dev/null | cut -f1))" \ + || echo "[CACHE SEED] \$name: tarball extract failed (non-fatal)" + else + echo "[CACHE SEED] \$name: no warm cache on Lustre yet" + fi +} + +# Seed vLLM compile cache from cache_read/ tarball (one per precision). +rm -rf "\$WARM_SEED" +_vllm_tar="\$CACHE_READ/vllm_compile_cache_${_vllm_cache_precision}.tar.zst" +if [ -f "\$_vllm_tar" ]; then + mkdir -p "\$WARM_SEED" + tar --zstd -xf "\$_vllm_tar" -C "\$WARM_SEED" \ + && echo "[CACHE SEED] vLLM (${_vllm_cache_precision}): seeded from tarball (\$(du -sh "\$WARM_SEED" 2>/dev/null | cut -f1))" \ + || echo "[CACHE SEED] vLLM: tarball extract failed (non-fatal)" +else + echo "[CACHE SEED] vLLM: no warm cache on Lustre yet" +fi + +_seed_cache "\$CACHE_READ/inductor_cache.tar.zst" "\$LOCAL_IND" "Inductor" +_seed_cache "\$CACHE_READ/triton_cache.tar.zst" "\$LOCAL_TRI" "Triton" + +echo "[CACHE SEED] Done." +SETUPEOF +export SETUP_COMMAND + +# ============================================================================= +# Build the training command +# ============================================================================= +# Stage-specific hyperparameters (batch sizes, advantage clip, MoE parallelism, +# learning rate, etc.) live in CONFIG_PATH. The launcher only passes the +# per-run overrides: cluster shape, paths, judge endpoints, logging. +# ============================================================================= +if [[ -n "${UV_CACHE_DIR:-}" ]]; then + TRAIN_UV_CACHE_DIR="${UV_CACHE_DIR}" +else + TRAIN_UV_CACHE_DIR='/tmp/nemo-gym-uv-cache-${SLURM_JOB_ID:-default}' +fi +TRAIN_NEMO_RL_VENV_DIR="${NEMO_RL_VENV_DIR:-}" +TRAIN_NEMO_GYM_VENV_DIR="${NEMO_GYM_VENV_DIR:-}" +TRAIN_UV_PYTHON="${UV_PYTHON:-}" +TRAIN_UV_PYTHON_INSTALL_DIR="${UV_PYTHON_INSTALL_DIR:-}" +if [[ -n "${NRL_UV_BIN_DIR:-}" ]]; then + TRAIN_NRL_PATH="${NRL_UV_BIN_DIR}:\${PATH}" +else + TRAIN_NRL_PATH="" +fi + +TRAIN_CMD="cd ${CODE_ROOT} && date ; \ +${NRL_DRIVER_PIP_INSTALL:+uv pip install --python /opt/nemo_rl_venv/bin/python ${NRL_DRIVER_PIP_INSTALL} ; }\ +${GENRM_RUNTIME_SETUP}\ +${VLLM_ENV_SOURCE}\ +${NRL_DRIVER_PYTHONPATH:+PYTHONPATH=${NRL_DRIVER_PYTHONPATH} }\ +OMP_NUM_THREADS=16 \ +RAY_DEDUP_LOGS=1 \ +WANDB_INIT_TIMEOUT=300 \ +VLLM_CACHE_ROOT=${NRL_VLLM_LOCAL_CACHE_DIR} \ +NRL_VLLM_CACHE_SEED_DIR=${NRL_VLLM_CACHE_SEED_DIR} \ +DG_JIT_CACHE_DIR=${NRL_VLLM_LOCAL_CACHE_DIR}/deep_gemm \ +TORCHINDUCTOR_CACHE_DIR=${INDUCTOR_CACHE_DIR} \ +TRITON_CACHE_DIR=${TRITON_CACHE_DIR} \ +UV_CACHE_DIR=${TRAIN_UV_CACHE_DIR} \ +${TRAIN_NEMO_RL_VENV_DIR:+NEMO_RL_VENV_DIR=${TRAIN_NEMO_RL_VENV_DIR} }\ +${TRAIN_NEMO_GYM_VENV_DIR:+NEMO_GYM_VENV_DIR=${TRAIN_NEMO_GYM_VENV_DIR} }\ +${TRAIN_UV_PYTHON:+UV_PYTHON=${TRAIN_UV_PYTHON} }\ +${TRAIN_UV_PYTHON_INSTALL_DIR:+UV_PYTHON_INSTALL_DIR=${TRAIN_UV_PYTHON_INSTALL_DIR} }\ +${TRAIN_NRL_PATH:+PATH=${TRAIN_NRL_PATH} }\ +UV_LOCK_TIMEOUT=1800 \ +RAY_ENABLE_UV_RUN_RUNTIME_ENV=0 \ +UV_HTTP_TIMEOUT=10 \ +VLLM_USE_FLASHINFER_MOE_FP8=1 \ +VLLM_FLASHINFER_MOE_BACKEND=latency \ +NRL_VLLM_ASYNC_TIMEOUT_SECONDS=1800 \ +NRL_WG_USE_RAY_REF=1 \ +HF_HOME=${HF_HOME:-} \ +NRL_USE_FASTOKENS=${NRL_USE_FASTOKENS:-1} \ +uv run ${NRL_DRIVER_UV_RUN_FLAGS:-} ${NRL_ENTRYPOINT:-./examples/nemo_gym/run_grpo_nemo_gym.py} \ +--config ${CONFIG_PATH} \ +policy.model_name=${MODEL_PATH} \ +cluster.num_nodes=${NUM_ACTOR_NODES} \ +policy.generation.colocated.resources.num_nodes=${NUM_GEN_NODES} \ +env.nemo_gym.num_gpu_nodes=${NUM_GYM_NODES} \ +checkpointing.checkpoint_dir=${CHECKPOINT_DIR} \ +${CHECKPOINTING_SAVE_BY:+checkpointing.checkpoint_must_save_by=${CHECKPOINTING_SAVE_BY}} \ +data.train.data_path=${TRAIN_PATH} \ +data.validation.data_path=${VAL_PATH} \ +${GENRM_OVERRIDE:+${GENRM_OVERRIDE}} \ +${NL2BASH_JUDGE_MODEL:+env.nemo_gym.nl2bash_judge_model.responses_api_models.local_vllm_model.model=${NL2BASH_JUDGE_MODEL}} \ +${SAFETY_JUDGE_MODEL:+env.nemo_gym.safety_judge_model.responses_api_models.local_vllm_model.model=${SAFETY_JUDGE_MODEL}} \ +${SIF_DIR:+sif_dir=${SIF_DIR}} \ +env.nemo_gym.nemo_gym_log_dir=${LOG_DIR}/nemo_gym \ +logger.log_dir=${LOG_DIR} \ +logger.wandb_enabled=${WANDB_ENABLED} \ +logger.wandb.name=${WANDB_NAME} \ +logger.wandb.project=${WANDB_PROJ} \ +${NRL_MAX_STEPS:+grpo.max_num_steps=${NRL_MAX_STEPS}} \ +${MTP_EXTRA_ARGS} \ +${MOPD_OVERRIDES} \ +${*}" + +export COMMAND="${TRAIN_CMD}" + +# ============================================================================= +# Summary +# ============================================================================= +echo "" +echo "================================================================" +echo " Nemotron 3 Ultra — ${EXP_NAME} (${NUM_TOTAL_NODES}-node)" +echo "================================================================" +echo " Job name: ${JOB_NAME} (singleton — only one runs at a time)" +echo " Config: ${CONFIG_PATH}" +echo " Nodes: ${NUM_TOTAL_NODES} total (segment=${SEGMENT_SIZE})" +echo " Training: ${NUM_TRAIN_NODES} ($((NUM_TRAIN_NODES * GPUS_PER_NODE)) GPUs)" +echo " vLLM gen: ${NUM_GEN_NODES} ($((NUM_GEN_NODES * GPUS_PER_NODE)) GPUs)" +echo " Gym: ${NUM_GYM_NODES} ($((NUM_GYM_NODES * GPUS_PER_NODE)) GPUs)" +if (( NUM_TEACHER_NODES > 0 )); then +echo " Teachers: ${NUM_TEACHER_NODES} ($((NUM_TEACHER_NODES * GPUS_PER_NODE)) GPUs)" +fi +echo " Walltime: ${WALLTIME}" +echo "" +echo " Checkpoints: ${CHECKPOINT_DIR} (stable — auto-resumes across jobs)" +echo " Run dir: ${RUN_DIR}" +echo " Logs: ${LOG_DIR}" +echo " Slurm logs: ${SLURM_LOG_DIR}" +echo " W&B: ${WANDB_PROJ} / ${WANDB_NAME} (enabled=${WANDB_ENABLED})" +echo "" +echo " Model: ${MODEL_PATH}" +echo " Train data: ${TRAIN_PATH}" +echo " Val data: ${VAL_PATH}" +echo " Container: ${CONTAINER}" +echo " Custom vLLM: ${USE_CUSTOM_VLLM}" +echo " Sandbox: ${SANDBOX_CONTAINER}" +if [[ -n "${GENRM_BASE_URL_FILE}" ]]; then + echo " GenRM: runtime endpoint from ${GENRM_BASE_URL_FILE}" +elif [[ -n "${GENRM_BASE_URL}" ]]; then + echo " GenRM: remote endpoint ${GENRM_BASE_URL}" +elif [[ -n "${GENRM_MODEL}" ]]; then + echo " GenRM: local model ${GENRM_MODEL}" +fi +if [[ "${USE_SNAPSHOT}" == "1" ]]; then +echo " Snapshot: ${SNAPSHOT_DIR}" +fi +echo "" +echo " Monitor: squeue -u \$USER -n ${JOB_NAME}" +echo " Logs: tail -f ${SLURM_LOG_DIR}/*.out" +echo " Latest: ls -la ${RESULTS_DIR}/runs/latest" +echo "" +echo "================================================================" +echo "" + +# ============================================================================= +# Record code provenance in the run directory +# ============================================================================= +TRAIN_CMD_REDACTED="$( + printf '%s\n' "${TRAIN_CMD}" \ + | sed -E 's/(HF_TOKEN|WANDB_API_KEY|NGC_API_KEY)=[^[:space:]]+/\1=/g' +)" +{ + echo "timestamp: $(date -Iseconds)" + echo "branch: $(git -C "${PROJECT_ROOT}" rev-parse --abbrev-ref HEAD 2>/dev/null || echo unknown)" + echo "commit: $(git -C "${PROJECT_ROOT}" rev-parse HEAD 2>/dev/null || echo unknown)" + echo "dirty: $(git -C "${PROJECT_ROOT}" status --porcelain 2>/dev/null | head -20)" + echo "snapshot: ${USE_SNAPSHOT}" + if [[ "${USE_SNAPSHOT}" == "1" ]]; then + echo "snapshot_dir: ${SNAPSHOT_DIR}" + fi + echo "container: ${CONTAINER}" + echo "config: ${CONFIG_PATH}" + echo "genrm_base_url_file: ${GENRM_BASE_URL_FILE}" + echo "command: ${TRAIN_CMD_REDACTED}" +} > "${RUN_DIR}/provenance.txt" + +# ============================================================================= +# Dry-run mode: print everything, don't submit +# ============================================================================= +DRY_RUN="${DRY_RUN:-0}" +if [[ "${DRY_RUN}" == "1" ]]; then + echo "DRY_RUN=1 — printing TRAIN_CMD and exiting without submission." + echo "" + echo "--- TRAIN_CMD ---" + echo "${TRAIN_CMD_REDACTED}" + echo "--- end ---" + exit 0 +fi + +# ============================================================================= +# Interactive mode: bring up Ray and idle for attachment (no training driver) +# ============================================================================= +# With COMMAND empty, ray.sub starts the Ray cluster, writes -attach.sh, +# then idles. We save the driver command to -run-cmd.sh so you can attach +# and run it by hand, edit it, and re-run without requeueing. +if [[ "${INTERACTIVE}" == "1" ]]; then + unset COMMAND 2>/dev/null || true # empty COMMAND -> ray.sub idle/interactive mode + WALLTIME="${INTERACTIVE_WALLTIME:-${WALLTIME}}" + + echo "" + echo "================================================================" + echo " INTERACTIVE MODE — ${NUM_TOTAL_NODES}-node allocation (walltime ${WALLTIME})" + echo " Ray will start and idle until you attach." + echo "================================================================" + + SBATCH_OUTPUT=$(sbatch \ + --nodes="${NUM_TOTAL_NODES}" \ + --account="${SLURM_ACCOUNT}" \ + --job-name="interactive-${JOB_NAME}" \ + --partition="${SLURM_PARTITION}" \ + --time="${WALLTIME}" \ + --gres=gpu:${GPUS_PER_NODE} \ + --exclusive \ + --mem=0 \ + --segment="${SEGMENT_SIZE}" \ + --output="${SLURM_LOG_DIR}/%j.out" \ + --error="${SLURM_LOG_DIR}/%j.err" \ + ${SLURM_QOS:+--qos="${SLURM_QOS}"} \ + ${EXCLUDE_NODES:+--exclude="${EXCLUDE_NODES}"} \ + ${SLURM_RESERVATION:+--reservation="${SLURM_RESERVATION}"} \ + --comment='{"OccupiedIdleGPUsJobReaper":{"exemptIdleTimeMins":"60","reason":"interactive","description":"interactive debugging"}}' \ + "${RAY_SUB}") + echo "${SBATCH_OUTPUT}" + JOB_ID=$(echo "${SBATCH_OUTPUT}" | grep -oP '\d+$') + [[ -z "${JOB_ID}" ]] && { echo "ERROR: could not parse job ID from sbatch output." >&2; exit 1; } + + LAUNCH_DIR="$(pwd)" + ATTACH_SCRIPT="${LAUNCH_DIR}/${JOB_ID}-attach.sh" + CMD_FILE="${LAUNCH_DIR}/${JOB_ID}-run-cmd.sh" + cat > "${CMD_FILE}" </dev/null || true) + [[ -z "${state}" ]] && { echo " Job ${JOB_ID} left the queue. Check: sacct -j ${JOB_ID}"; exit 1; } + [[ "${state}" != "${prev_state}" ]] && { echo " [$(date +%H:%M:%S)] state: ${state}"; prev_state="${state}"; } + sleep 15 + done + echo "" + echo " Ray is ready — attach: bash ${ATTACH_SCRIPT}" + fi + exit 0 +fi + +# ============================================================================= +# Submit +# ============================================================================= +# Always serialise same-name submissions via singleton; optionally chain after +# another job with SLURM_DEPENDENCY (e.g. "afterany:3044848" or "afterok:JOBID"). +SLURM_DEPENDENCY="${SLURM_DEPENDENCY:-}" +DEPENDENCY="singleton" +[[ -n "${SLURM_DEPENDENCY}" ]] && DEPENDENCY="singleton,${SLURM_DEPENDENCY}" + +SBATCH_OUTPUT=$(sbatch \ + --nodes="${NUM_TOTAL_NODES}" \ + --account="${SLURM_ACCOUNT}" \ + --job-name="${JOB_NAME}" \ + --partition="${SLURM_PARTITION}" \ + --time="${WALLTIME}" \ + --gres=gpu:${GPUS_PER_NODE} \ + --exclusive \ + --mem=0 \ + --dependency="${DEPENDENCY}" \ + --segment="${SEGMENT_SIZE}" \ + --output="${SLURM_LOG_DIR}/%j.out" \ + --error="${SLURM_LOG_DIR}/%j.err" \ + ${SLURM_QOS:+--qos="${SLURM_QOS}"} \ + ${EXCLUDE_NODES:+--exclude="${EXCLUDE_NODES}"} \ + ${SLURM_RESERVATION:+--reservation="${SLURM_RESERVATION}"} \ + --comment='{"OccupiedIdleGPUsJobReaper":{"exemptIdleTimeMins":"60","reason":"interactive","description":"interactive debugging"}}' \ + "${RAY_SUB}") + +echo "${SBATCH_OUTPUT}" +JOB_ID=$(echo "${SBATCH_OUTPUT}" | grep -oP '\d+$') + +if [[ -n "${JOB_ID}" ]]; then + echo "" + echo " Ray logs: ${BASE_LOG_DIR}/${JOB_ID}-logs/" + echo "" +fi diff --git a/uv.lock b/uv.lock index 988cb862fb0..bf86d3de96c 100644 --- a/uv.lock +++ b/uv.lock @@ -1,5 +1,5 @@ version = 1 -revision = 3 +revision = 4 requires-python = ">=3.13.14, <3.14" resolution-markers = [ "platform_machine == 'x86_64' and sys_platform == 'linux'", @@ -10,21 +10,6 @@ supported-markers = [ "platform_machine == 'aarch64' and sys_platform == 'linux'", ] conflicts = [[ - { package = "nemo-gym", extra = "vllm" }, - { package = "nemo-rl", extra = "automodel" }, -], [ - { package = "nemo-gym", extra = "vllm" }, - { package = "nemo-rl", extra = "vllm" }, -], [ - { package = "nemo-gym", extra = "vllm" }, - { package = "nemo-rl", extra = "sglang" }, -], [ - { package = "nemo-gym", extra = "vllm" }, - { package = "nemo-rl", extra = "mcore" }, -], [ - { package = "nemo-gym", extra = "vllm" }, - { package = "nemo-rl", extra = "trtllm" }, -], [ { package = "nemo-rl", extra = "fsdp" }, { package = "nemo-rl", extra = "sglang" }, ], [ @@ -64,7 +49,6 @@ conflicts = [[ [manifest] members = [ - "nemo-gym", "nemo-rl", "template-project", ] @@ -315,18 +299,6 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/98/3b/40a68de458904bcc143622015fff2352b6461cd92fd66d3527bf1c6f5716/aiohttp_cors-0.8.1-py3-none-any.whl", hash = "sha256:3180cf304c5c712d626b9162b195b1db7ddf976a2a25172b35bb2448b890a80d", size = 25231, upload-time = "2025-03-31T14:16:18.478Z" }, ] -[[package]] -name = "aiohttp-retry" -version = "2.9.1" -source = { registry = "https://pypi.org/simple" } -dependencies = [ - { name = "aiohttp" }, -] -sdist = { url = "https://files.pythonhosted.org/packages/9d/61/ebda4d8e3d8cfa1fd3db0fb428db2dd7461d5742cea35178277ad180b033/aiohttp_retry-2.9.1.tar.gz", hash = "sha256:8eb75e904ed4ee5c2ec242fefe85bf04240f685391c4879d8f541d6028ff01f1", size = 13608, upload-time = "2024-11-06T10:44:54.574Z" } -wheels = [ - { url = "https://files.pythonhosted.org/packages/1a/99/84ba7273339d0f3dfa57901b846489d2e5c2cd731470167757f1935fffbd/aiohttp_retry-2.9.1-py3-none-any.whl", hash = "sha256:66d2759d1921838256a05a3f80ad7e724936f083e35be5abb5e16eed6be6dc54", size = 9981, upload-time = "2024-11-06T10:44:52.917Z" }, -] - [[package]] name = "aioitertools" version = "0.13.0" @@ -440,7 +412,7 @@ name = "apache-tvm-ffi" version = "0.1.9" source = { registry = "https://pypi.org/simple" } dependencies = [ - { name = "typing-extensions", marker = "platform_machine == 'x86_64' or extra != 'extra-7-nemo-rl-automodel' or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-8-nemo-gym-vllm')" }, + { name = "typing-extensions", marker = "platform_machine == 'x86_64' or extra != 'extra-7-nemo-rl-automodel' or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-vllm')" }, ] sdist = { url = "https://files.pythonhosted.org/packages/6f/60/1e787a0b5ebf318483235be2a689ee367173983067e441b8379564f667c0/apache_tvm_ffi-0.1.9.tar.gz", hash = "sha256:d2d402587e8906de0a07f4746aa78f3d452c7efe3625d4bb39ac2ad693bce530", size = 2513731, upload-time = "2026-02-27T19:28:06.602Z" } wheels = [ @@ -450,19 +422,6 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/70/ef/5402da5d37f5270fd88ea0348acca78dba9be8bdbf6c2bcae0935eb03ef1/apache_tvm_ffi-0.1.9-cp312-abi3-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:f45eb43499acac45ff6c93564f0ff2d3ca27b69656d540fd56ce59d51c0b4c65", size = 2278991, upload-time = "2026-02-27T19:27:30.729Z" }, ] -[[package]] -name = "ast-serialize" -version = "0.6.0" -source = { registry = "https://pypi.org/simple" } -sdist = { url = "https://files.pythonhosted.org/packages/58/ad/0d70a3a2d6e01968d985415259e8ec7ad3f777903f9b1c1f3c8c44642c60/ast_serialize-0.6.0.tar.gz", hash = "sha256:aadd3ffcf4858c9726bf3515f7b199c7eadbe504f96028e4a87172c0da65a8fe", size = 61489, upload-time = "2026-06-30T20:02:55.555Z" } -wheels = [ - { url = "https://files.pythonhosted.org/packages/6f/02/613a7534a41d0122f37d1e0c64aa8ac78bfb831f8c92f6db057a311abb3c/ast_serialize-0.6.0-cp39-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:305802f2ce2a7c4e87835078ea85c58b586ddda8095b92fe2ead9364ae19c80a", size = 1238620, upload-time = "2026-06-30T20:02:33.664Z" }, - { url = "https://files.pythonhosted.org/packages/ca/87/b4d6c38e0ccd5e85dc54cecdf933a152c60b28fe5d993a6d8a72fa6d5896/ast_serialize-0.6.0-cp39-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:dcbed41e9386059fc0261d602445ede0976c2ecec2939688bcbcb9ed0b6f28b7", size = 1261693, upload-time = "2026-06-30T20:02:39.123Z" }, - { url = "https://files.pythonhosted.org/packages/b1/f2/13736d920ab3d49bbee80ef1a277dd7b7aaf3b3545efd9d2a8114fe05525/ast_serialize-0.6.0-cp39-abi3-musllinux_1_2_aarch64.whl", hash = "sha256:c4af9a1386166e40ed01464991806f89038a2d89782576c7774876fa77034e32", size = 1413698, upload-time = "2026-06-30T20:02:44.179Z" }, - { url = "https://files.pythonhosted.org/packages/95/93/5524a3dc6c3f593de3228ed9cbef73afa047625b7000ec21b7f58e6eb4d4/ast_serialize-0.6.0-cp39-abi3-musllinux_1_2_x86_64.whl", hash = "sha256:4ed29121da8b3fdc291002801a1de0f76248fa07dce89157a5f277842cf6126e", size = 1457164, upload-time = "2026-06-30T20:02:48.294Z" }, - { url = "https://files.pythonhosted.org/packages/4f/c0/36a6ffb4d653cf621427b4c4928671f53ad800c453474de2b82564a44ad9/ast_serialize-0.6.0-cp39-abi3-pyemscripten_2026_0_wasm32.whl", hash = "sha256:b1dac4e09d341c1300ba69cdcbe62867b32a8c75d90db9bf4d083bec3b039f0b", size = 863014, upload-time = "2026-06-30T20:02:49.742Z" }, -] - [[package]] name = "astor" version = "0.8.1" @@ -580,15 +539,6 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/88/c6/92fcd42f1ba33e1184263f25bfabf3d27c383410470f169e4b8163bf9c17/beautifulsoup4-4.15.0-py3-none-any.whl", hash = "sha256:d6f88de62e1d4e38ecb1077eb9724cd0eff29d2a08ca16a401e9b9e93f117cf9", size = 109924, upload-time = "2026-06-07T16:44:21.566Z" }, ] -[[package]] -name = "bidict" -version = "0.23.1" -source = { registry = "https://pypi.org/simple" } -sdist = { url = "https://files.pythonhosted.org/packages/9a/6e/026678aa5a830e07cd9498a05d3e7e650a4f56a42f267a53d22bcda1bdc9/bidict-0.23.1.tar.gz", hash = "sha256:03069d763bc387bbd20e7d49914e75fc4132a41937fa3405417e1a5a2d006d71", size = 29093, upload-time = "2024-02-18T19:09:05.748Z" } -wheels = [ - { url = "https://files.pythonhosted.org/packages/99/37/e8730c3587a65eb5645d4aba2d27aae48e8003614d6aaf15dda67f702f1f/bidict-0.23.1-py3-none-any.whl", hash = "sha256:5dae8d4d79b552a71cbabc7deb25dfe8ce710b17ff41711e13010ead2abfc3e5", size = 32764, upload-time = "2024-02-18T19:09:04.156Z" }, -] - [[package]] name = "blake3" version = "1.0.9" @@ -713,9 +663,9 @@ name = "causal-conv1d" version = "1.5.4" source = { git = "https://github.com/Dao-AILab/causal-conv1d?rev=4f6ae4e26ae5fe8af9372f8d312ab25cc4595223#4f6ae4e26ae5fe8af9372f8d312ab25cc4595223" } dependencies = [ - { name = "ninja", marker = "extra == 'extra-7-nemo-rl-automodel' or extra == 'extra-7-nemo-rl-fsdp' or extra == 'extra-7-nemo-rl-mcore' or extra != 'extra-7-nemo-rl-sglang' or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-8-nemo-gym-vllm')" }, - { name = "packaging", marker = "extra == 'extra-7-nemo-rl-automodel' or extra == 'extra-7-nemo-rl-fsdp' or extra == 'extra-7-nemo-rl-mcore' or extra != 'extra-7-nemo-rl-sglang' or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-8-nemo-gym-vllm')" }, - { name = "torch", marker = "extra == 'extra-7-nemo-rl-automodel' or extra == 'extra-7-nemo-rl-fsdp' or extra == 'extra-7-nemo-rl-mcore' or extra != 'extra-7-nemo-rl-sglang' or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-8-nemo-gym-vllm')" }, + { name = "ninja", marker = "extra == 'extra-7-nemo-rl-automodel' or extra == 'extra-7-nemo-rl-fsdp' or extra == 'extra-7-nemo-rl-mcore' or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm')" }, + { name = "packaging", marker = "extra == 'extra-7-nemo-rl-automodel' or extra == 'extra-7-nemo-rl-fsdp' or extra == 'extra-7-nemo-rl-mcore' or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm')" }, + { name = "torch", marker = "extra == 'extra-7-nemo-rl-automodel' or extra == 'extra-7-nemo-rl-fsdp' or extra == 'extra-7-nemo-rl-mcore' or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm')" }, ] [[package]] @@ -744,7 +694,7 @@ name = "cffi" version = "2.1.0" source = { registry = "https://pypi.org/simple" } dependencies = [ - { name = "pycparser", marker = "implementation_name != 'PyPy' or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-vllm' and extra == 'extra-8-nemo-gym-vllm')" }, + { name = "pycparser", marker = "implementation_name != 'PyPy' or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm')" }, ] sdist = { url = "https://files.pythonhosted.org/packages/57/5f/ff100cae70ebe9d8df1c01a00e510e45d9adb5c1fdda84791b199141de97/cffi-2.1.0.tar.gz", hash = "sha256:efc1cdd798b1aaf39b4610bba7aad28c9bea9b910f25c784ccf9ec1fa719d1f9", size = 531036, upload-time = "2026-07-06T21:34:30.382Z" } wheels = [ @@ -841,7 +791,7 @@ version = "3.58.4" source = { registry = "https://pypi.org/simple" } dependencies = [ { name = "dulwich" }, - { name = "everett", extra = ["ini"], marker = "extra == 'extra-7-nemo-rl-mcore' or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-vllm' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-mcore' and extra == 'extra-8-nemo-gym-vllm')" }, + { name = "everett", extra = ["ini"], marker = "extra == 'extra-7-nemo-rl-mcore' or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm')" }, { name = "jsonschema" }, { name = "psutil" }, { name = "python-box" }, @@ -945,7 +895,7 @@ name = "cryptography" version = "48.0.1" source = { registry = "https://pypi.org/simple" } dependencies = [ - { name = "cffi", marker = "platform_python_implementation != 'PyPy' or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-vllm' and extra == 'extra-8-nemo-gym-vllm')" }, + { name = "cffi", marker = "platform_python_implementation != 'PyPy' or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm')" }, ] sdist = { url = "https://files.pythonhosted.org/packages/12/45/870e7f4bef50e5f53b9f51d4428aee5290eedf58ba443f16b1ebb7ab8e66/cryptography-48.0.1.tar.gz", hash = "sha256:266f4ee051abb2f725b74ef8072b521ce1feacf685a3364fa6a6b45548db791a", size = 832989, upload-time = "2026-06-09T22:32:31.8Z" } wheels = [ @@ -1161,134 +1111,6 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/05/66/73034ad30b59f13439b75e620989dacba4c047256e358ba7c2e9ec98ea22/datasets-5.0.0-py3-none-any.whl", hash = "sha256:7dd34927a0fd7046e98aad5cb9430e699c373238a15befa7b9bf22b991a7fee6", size = 555084, upload-time = "2026-06-05T13:18:24.435Z" }, ] -[[package]] -name = "daytona" -version = "0.200.1" -source = { registry = "https://pypi.org/simple" } -dependencies = [ - { name = "aiofiles" }, - { name = "aiohttp" }, - { name = "daytona-analytics-api-client" }, - { name = "daytona-analytics-api-client-async" }, - { name = "daytona-api-client" }, - { name = "daytona-api-client-async" }, - { name = "daytona-toolbox-api-client" }, - { name = "daytona-toolbox-api-client-async" }, - { name = "deprecated" }, - { name = "httpx" }, - { name = "httpx-ws" }, - { name = "obstore" }, - { name = "opentelemetry-api" }, - { name = "opentelemetry-exporter-otlp-proto-http" }, - { name = "opentelemetry-instrumentation-aiohttp-client" }, - { name = "opentelemetry-sdk" }, - { name = "pydantic" }, - { name = "python-dotenv" }, - { name = "python-multipart" }, - { name = "python-socketio", extra = ["asyncio-client", "client"] }, - { name = "toml" }, - { name = "typing-extensions" }, - { name = "urllib3" }, - { name = "wsproto" }, -] -sdist = { url = "https://files.pythonhosted.org/packages/47/eb/2d2db2068f23697964b2aab383bb8391b618462df1ae6dca1df464f27259/daytona-0.200.1.tar.gz", hash = "sha256:02fc040cdbb54417ba7c1dc6bc1aa7489543bd15fc3fe9aea6968b0516f7ea4b", size = 177604, upload-time = "2026-07-22T12:23:22.305Z" } -wheels = [ - { url = "https://files.pythonhosted.org/packages/c1/32/87982084d380e6a34aff1b2fac8eb6f8649f831c4f8f39c8d0eb90a0749f/daytona-0.200.1-py3-none-any.whl", hash = "sha256:68029ff267499a8dcc9d946fc71e79b9ea456170c77fb836bf4b0cd3c1e59959", size = 213381, upload-time = "2026-07-22T12:23:23.43Z" }, -] - -[[package]] -name = "daytona-analytics-api-client" -version = "0.200.1" -source = { registry = "https://pypi.org/simple" } -dependencies = [ - { name = "pydantic" }, - { name = "python-dateutil" }, - { name = "typing-extensions" }, - { name = "urllib3" }, -] -sdist = { url = "https://files.pythonhosted.org/packages/f3/de/9e95c70c792f20ef0c43e157a5a8f56d71d57b716335da2dc7d29ddb082a/daytona_analytics_api_client-0.200.1.tar.gz", hash = "sha256:1624d437cbc50c91f921c4b46567f55f8ddf89894fa80840acf01788daf85a4b", size = 30058, upload-time = "2026-07-22T12:22:47.817Z" } -wheels = [ - { url = "https://files.pythonhosted.org/packages/a7/34/e17297664a6e8a0a921ad971683e220bce69e57a3c97f37c2815a3f70991/daytona_analytics_api_client-0.200.1-py3-none-any.whl", hash = "sha256:9f8ad5648f0da73b34ce9b13ce3c60173562d48404ddc7f6331eeec4beb2c772", size = 45010, upload-time = "2026-07-22T12:22:48.643Z" }, -] - -[[package]] -name = "daytona-analytics-api-client-async" -version = "0.200.1" -source = { registry = "https://pypi.org/simple" } -dependencies = [ - { name = "aiohttp" }, - { name = "aiohttp-retry" }, - { name = "pydantic" }, - { name = "python-dateutil" }, - { name = "typing-extensions" }, -] -sdist = { url = "https://files.pythonhosted.org/packages/77/ec/e9bd405a74382f97b1a16376686bf90ed9c3b183718897d519eb8ced73d9/daytona_analytics_api_client_async-0.200.1.tar.gz", hash = "sha256:f5edc0b1a6590ae802490d97789f21f94d3bf73c53a4d984c980d89cc0eca132", size = 30071, upload-time = "2026-07-22T12:22:39.053Z" } -wheels = [ - { url = "https://files.pythonhosted.org/packages/d5/c5/2979f438def31f26dc98b3e23a9e8787570d2bb95019f85596fa7eafb8a6/daytona_analytics_api_client_async-0.200.1-py3-none-any.whl", hash = "sha256:637106455f9498d6785efb51be9d589a2ecd3f257e2a3d9f1c7b6bd8c461721a", size = 45283, upload-time = "2026-07-22T12:22:39.882Z" }, -] - -[[package]] -name = "daytona-api-client" -version = "0.200.1" -source = { registry = "https://pypi.org/simple" } -dependencies = [ - { name = "pydantic" }, - { name = "python-dateutil" }, - { name = "typing-extensions" }, - { name = "urllib3" }, -] -sdist = { url = "https://files.pythonhosted.org/packages/ef/be/9e604a59ea3b3dc5a120b9dd216b1db12f4da2e6b8601e7d4f49b1874c96/daytona_api_client-0.200.1.tar.gz", hash = "sha256:fcf17e051eb7e0a9b8a38d02ef74f7ed67f84c3b4d150c8dd76c8c942f521fa9", size = 124815, upload-time = "2026-07-22T12:22:56.725Z" } -wheels = [ - { url = "https://files.pythonhosted.org/packages/43/1a/094725c4f495eba4411666a12307f885cead3aa29917eccf72f3630f36ff/daytona_api_client-0.200.1-py3-none-any.whl", hash = "sha256:7d7c41d7c5b601d9df908bd0ab0f0f567ade2913edf2eb087f65cfc6fcbe73a0", size = 316076, upload-time = "2026-07-22T12:22:57.798Z" }, -] - -[[package]] -name = "daytona-api-client-async" -version = "0.200.1" -source = { registry = "https://pypi.org/simple" } -dependencies = [ - { name = "aiohttp" }, - { name = "aiohttp-retry" }, - { name = "pydantic" }, - { name = "python-dateutil" }, - { name = "typing-extensions" }, -] -sdist = { url = "https://files.pythonhosted.org/packages/5e/dc/132b3ac1f44434ef4115abe99761542da9ac025a20f011fc081638148245/daytona_api_client_async-0.200.1.tar.gz", hash = "sha256:269874af7f97368531a08762639590a59f0ece95e194fa6ebe998fbf8258d4dd", size = 125346, upload-time = "2026-07-22T12:22:39.687Z" } -wheels = [ - { url = "https://files.pythonhosted.org/packages/2a/47/ea80967ccdcc4fbabbf628a1b162f0f500edd75eebffc55fea88ada41ea8/daytona_api_client_async-0.200.1-py3-none-any.whl", hash = "sha256:df80d0aeb52c82a4660d746464842bdb6c8a2e0203b0cdc6374037b164c47824", size = 318668, upload-time = "2026-07-22T12:22:41.49Z" }, -] - -[[package]] -name = "daytona-toolbox-api-client" -version = "0.200.1" -source = { registry = "https://pypi.org/simple" } -dependencies = [ - { name = "pydantic" }, - { name = "python-dateutil" }, - { name = "typing-extensions" }, - { name = "urllib3" }, -] -sdist = { url = "https://files.pythonhosted.org/packages/0e/d8/650c9b70734931ac369dc5369d4168bc81e65fca244d19afb6d77b496bd9/daytona_toolbox_api_client-0.200.1.tar.gz", hash = "sha256:c5c74b390c20396494a3a70e843a5da70187ba5d197b26fe98be366e2a1a81da", size = 86312, upload-time = "2026-07-22T12:22:49.038Z" } -wheels = [ - { url = "https://files.pythonhosted.org/packages/7d/a2/329d03f4898e3b768bf285b3353f3db347e6946f8f7a3b7587325f7e6468/daytona_toolbox_api_client-0.200.1-py3-none-any.whl", hash = "sha256:7baaebf323d663e669db6dbbba816fccd7facca101d6cffa86959dee06593ce3", size = 247064, upload-time = "2026-07-22T12:22:50.356Z" }, -] - -[[package]] -name = "daytona-toolbox-api-client-async" -version = "0.200.1" -source = { registry = "https://pypi.org/simple" } -dependencies = [ - { name = "aiohttp" }, - { name = "aiohttp-retry" }, - { name = "pydantic" }, - { name = "python-dateutil" }, - { name = "typing-extensions" }, -] -sdist = { url = "https://files.pythonhosted.org/packages/8f/ad/92aa88099f210e2ff73cebfa0ca08538f05068e6c5c25a3e565eb9438d75/daytona_toolbox_api_client_async-0.200.1.tar.gz", hash = "sha256:ffe6c6bd9289b56a448a7e0f1f36a295fda431e0d73fc74c53cfb9492f590174", size = 80178, upload-time = "2026-07-22T12:22:39.642Z" } -wheels = [ - { url = "https://files.pythonhosted.org/packages/b6/00/b3f5b3e40a37af8b883590d78a7d6935a464e74364129732daa322c4bd37/daytona_toolbox_api_client_async-0.200.1-py3-none-any.whl", hash = "sha256:fc953a1ee1d9967900b21c5eb6188c54617b1d66d5ad6b737e457a9e76a4fb4a", size = 245534, upload-time = "2026-07-22T12:22:40.771Z" }, -] - [[package]] name = "debugpy" version = "1.8.21" @@ -1353,18 +1175,6 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/07/6c/aa3f2f849e01cb6a001cd8554a88d4c77c5c1a31c95bdf1cf9301e6d9ef4/defusedxml-0.7.1-py2.py3-none-any.whl", hash = "sha256:a352e7e428770286cc899e2542b6cdaedb2b4953ff269a210103ec58f6198a61", size = 25604, upload-time = "2021-03-08T10:59:24.45Z" }, ] -[[package]] -name = "deprecated" -version = "1.3.1" -source = { registry = "https://pypi.org/simple" } -dependencies = [ - { name = "wrapt" }, -] -sdist = { url = "https://files.pythonhosted.org/packages/49/85/12f0a49a7c4ffb70572b6c2ef13c90c88fd190debda93b23f026b25f9634/deprecated-1.3.1.tar.gz", hash = "sha256:b1b50e0ff0c1fddaa5708a2c6b0a6588bb09b892825ab2b214ac9ea9d92a5223", size = 2932523, upload-time = "2025-10-30T08:19:02.757Z" } -wheels = [ - { url = "https://files.pythonhosted.org/packages/84/d0/205d54408c08b13550c733c4b85429e7ead111c7f0014309637425520a9a/deprecated-1.3.1-py2.py3-none-any.whl", hash = "sha256:597bfef186b6f60181535a29fbe44865ce137a5079f295b479886c82729d5f3f", size = 11298, upload-time = "2025-10-30T08:19:00.758Z" }, -] - [[package]] name = "deprecation" version = "2.1.0" @@ -1595,15 +1405,6 @@ ini = [ { name = "configobj" }, ] -[[package]] -name = "execnet" -version = "2.1.2" -source = { registry = "https://pypi.org/simple" } -sdist = { url = "https://files.pythonhosted.org/packages/bf/89/780e11f9588d9e7128a3f87788354c7946a9cbb1401ad38a48c4db9a4f07/execnet-2.1.2.tar.gz", hash = "sha256:63d83bfdd9a23e35b9c6a3261412324f964c2ec8dcd8d3c6916ee9373e0befcd", size = 166622, upload-time = "2025-11-12T09:56:37.75Z" } -wheels = [ - { url = "https://files.pythonhosted.org/packages/ab/84/02fc1827e8cdded4aa65baef11296a9bbe595c474f0d6d758af082d849fd/execnet-2.1.2-py3-none-any.whl", hash = "sha256:67fba928dd5a544b783f6056f449e5e3931a5c378b128bc18501f7ea79e296ec", size = 40708, upload-time = "2025-11-12T09:56:36.333Z" }, -] - [[package]] name = "executing" version = "2.2.1" @@ -1632,11 +1433,11 @@ resolution-markers = [ "platform_machine == 'aarch64' and sys_platform == 'linux'", ] dependencies = [ - { name = "annotated-doc", marker = "(extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-8-nemo-gym-vllm') or (extra != 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-vllm') or (extra != 'extra-7-nemo-rl-automodel' and extra == 'extra-8-nemo-gym-vllm') or (extra != 'extra-7-nemo-rl-fsdp' and extra != 'extra-7-nemo-rl-mcore' and extra != 'extra-7-nemo-rl-sglang' and extra != 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra != 'extra-7-nemo-rl-fsdp' and extra != 'extra-7-nemo-rl-mcore' and extra != 'extra-7-nemo-rl-sglang' and extra != 'extra-7-nemo-rl-trtllm' and extra == 'extra-8-nemo-gym-vllm')" }, - { name = "pydantic", marker = "(extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-8-nemo-gym-vllm') or (extra != 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-vllm') or (extra != 'extra-7-nemo-rl-automodel' and extra == 'extra-8-nemo-gym-vllm') or (extra != 'extra-7-nemo-rl-fsdp' and extra != 'extra-7-nemo-rl-mcore' and extra != 'extra-7-nemo-rl-sglang' and extra != 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra != 'extra-7-nemo-rl-fsdp' and extra != 'extra-7-nemo-rl-mcore' and extra != 'extra-7-nemo-rl-sglang' and extra != 'extra-7-nemo-rl-trtllm' and extra == 'extra-8-nemo-gym-vllm')" }, - { name = "starlette", marker = "(extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-8-nemo-gym-vllm') or (extra != 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-vllm') or (extra != 'extra-7-nemo-rl-automodel' and extra == 'extra-8-nemo-gym-vllm') or (extra != 'extra-7-nemo-rl-fsdp' and extra != 'extra-7-nemo-rl-mcore' and extra != 'extra-7-nemo-rl-sglang' and extra != 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra != 'extra-7-nemo-rl-fsdp' and extra != 'extra-7-nemo-rl-mcore' and extra != 'extra-7-nemo-rl-sglang' and extra != 'extra-7-nemo-rl-trtllm' and extra == 'extra-8-nemo-gym-vllm')" }, - { name = "typing-extensions", marker = "(extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-8-nemo-gym-vllm') or (extra != 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-vllm') or (extra != 'extra-7-nemo-rl-automodel' and extra == 'extra-8-nemo-gym-vllm') or (extra != 'extra-7-nemo-rl-fsdp' and extra != 'extra-7-nemo-rl-mcore' and extra != 'extra-7-nemo-rl-sglang' and extra != 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra != 'extra-7-nemo-rl-fsdp' and extra != 'extra-7-nemo-rl-mcore' and extra != 'extra-7-nemo-rl-sglang' and extra != 'extra-7-nemo-rl-trtllm' and extra == 'extra-8-nemo-gym-vllm')" }, - { name = "typing-inspection", marker = "(extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-8-nemo-gym-vllm') or (extra != 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-vllm') or (extra != 'extra-7-nemo-rl-automodel' and extra == 'extra-8-nemo-gym-vllm') or (extra != 'extra-7-nemo-rl-fsdp' and extra != 'extra-7-nemo-rl-mcore' and extra != 'extra-7-nemo-rl-sglang' and extra != 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra != 'extra-7-nemo-rl-fsdp' and extra != 'extra-7-nemo-rl-mcore' and extra != 'extra-7-nemo-rl-sglang' and extra != 'extra-7-nemo-rl-trtllm' and extra == 'extra-8-nemo-gym-vllm')" }, + { name = "annotated-doc", marker = "(extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra != 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-vllm') or (extra != 'extra-7-nemo-rl-fsdp' and extra != 'extra-7-nemo-rl-mcore' and extra != 'extra-7-nemo-rl-sglang' and extra != 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm')" }, + { name = "pydantic", marker = "(extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra != 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-vllm') or (extra != 'extra-7-nemo-rl-fsdp' and extra != 'extra-7-nemo-rl-mcore' and extra != 'extra-7-nemo-rl-sglang' and extra != 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm')" }, + { name = "starlette", marker = "(extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra != 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-vllm') or (extra != 'extra-7-nemo-rl-fsdp' and extra != 'extra-7-nemo-rl-mcore' and extra != 'extra-7-nemo-rl-sglang' and extra != 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm')" }, + { name = "typing-extensions", marker = "(extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra != 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-vllm') or (extra != 'extra-7-nemo-rl-fsdp' and extra != 'extra-7-nemo-rl-mcore' and extra != 'extra-7-nemo-rl-sglang' and extra != 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm')" }, + { name = "typing-inspection", marker = "(extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra != 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-vllm') or (extra != 'extra-7-nemo-rl-fsdp' and extra != 'extra-7-nemo-rl-mcore' and extra != 'extra-7-nemo-rl-sglang' and extra != 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm')" }, ] sdist = { url = "https://files.pythonhosted.org/packages/81/2d/ff8d91d7b564d464629a0fd50a4489c97fcb836ac230bf3a7269232a9b1f/fastapi-0.136.3.tar.gz", hash = "sha256:e487fae93ad408e6f47641ee4dfe389864fd7bec92e547ea8498fc13f43e83ab", size = 396410, upload-time = "2026-05-23T18:53:15.192Z" } wheels = [ @@ -1646,14 +1447,14 @@ wheels = [ [package.optional-dependencies] standard = [ { name = "email-validator" }, - { name = "fastapi-cli", extra = ["standard"], marker = "(extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-8-nemo-gym-vllm') or (extra != 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-vllm') or (extra != 'extra-7-nemo-rl-automodel' and extra == 'extra-8-nemo-gym-vllm') or (extra != 'extra-7-nemo-rl-fsdp' and extra != 'extra-7-nemo-rl-mcore' and extra != 'extra-7-nemo-rl-sglang' and extra != 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra != 'extra-7-nemo-rl-fsdp' and extra != 'extra-7-nemo-rl-mcore' and extra != 'extra-7-nemo-rl-sglang' and extra != 'extra-7-nemo-rl-trtllm' and extra == 'extra-8-nemo-gym-vllm')" }, + { name = "fastapi-cli", extra = ["standard"], marker = "(extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra != 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-vllm') or (extra != 'extra-7-nemo-rl-fsdp' and extra != 'extra-7-nemo-rl-mcore' and extra != 'extra-7-nemo-rl-sglang' and extra != 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm')" }, { name = "fastar" }, { name = "httpx" }, { name = "jinja2" }, { name = "pydantic-extra-types" }, { name = "pydantic-settings" }, { name = "python-multipart" }, - { name = "uvicorn", extra = ["standard"], marker = "(extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-8-nemo-gym-vllm') or (extra != 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-vllm') or (extra != 'extra-7-nemo-rl-automodel' and extra == 'extra-8-nemo-gym-vllm') or (extra != 'extra-7-nemo-rl-fsdp' and extra != 'extra-7-nemo-rl-mcore' and extra != 'extra-7-nemo-rl-sglang' and extra != 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra != 'extra-7-nemo-rl-fsdp' and extra != 'extra-7-nemo-rl-mcore' and extra != 'extra-7-nemo-rl-sglang' and extra != 'extra-7-nemo-rl-trtllm' and extra == 'extra-8-nemo-gym-vllm')" }, + { name = "uvicorn", extra = ["standard"], marker = "(extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra != 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-vllm') or (extra != 'extra-7-nemo-rl-fsdp' and extra != 'extra-7-nemo-rl-mcore' and extra != 'extra-7-nemo-rl-sglang' and extra != 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm')" }, ] [[package]] @@ -1665,11 +1466,11 @@ resolution-markers = [ "platform_machine == 'aarch64' and sys_platform == 'linux'", ] dependencies = [ - { name = "annotated-doc", marker = "extra == 'extra-7-nemo-rl-automodel' or extra == 'extra-7-nemo-rl-mcore' or extra == 'extra-7-nemo-rl-sglang' or extra == 'extra-7-nemo-rl-trtllm' or (extra == 'extra-7-nemo-rl-vllm' and extra == 'extra-8-nemo-gym-vllm') or (extra != 'extra-7-nemo-rl-vllm' and extra != 'extra-8-nemo-gym-vllm')" }, - { name = "pydantic", marker = "extra == 'extra-7-nemo-rl-automodel' or extra == 'extra-7-nemo-rl-mcore' or extra == 'extra-7-nemo-rl-sglang' or extra == 'extra-7-nemo-rl-trtllm' or (extra == 'extra-7-nemo-rl-vllm' and extra == 'extra-8-nemo-gym-vllm') or (extra != 'extra-7-nemo-rl-vllm' and extra != 'extra-8-nemo-gym-vllm')" }, - { name = "starlette", marker = "extra == 'extra-7-nemo-rl-automodel' or extra == 'extra-7-nemo-rl-mcore' or extra == 'extra-7-nemo-rl-sglang' or extra == 'extra-7-nemo-rl-trtllm' or (extra == 'extra-7-nemo-rl-vllm' and extra == 'extra-8-nemo-gym-vllm') or (extra != 'extra-7-nemo-rl-vllm' and extra != 'extra-8-nemo-gym-vllm')" }, - { name = "typing-extensions", marker = "extra == 'extra-7-nemo-rl-automodel' or extra == 'extra-7-nemo-rl-mcore' or extra == 'extra-7-nemo-rl-sglang' or extra == 'extra-7-nemo-rl-trtllm' or (extra == 'extra-7-nemo-rl-vllm' and extra == 'extra-8-nemo-gym-vllm') or (extra != 'extra-7-nemo-rl-vllm' and extra != 'extra-8-nemo-gym-vllm')" }, - { name = "typing-inspection", marker = "extra == 'extra-7-nemo-rl-automodel' or extra == 'extra-7-nemo-rl-mcore' or extra == 'extra-7-nemo-rl-sglang' or extra == 'extra-7-nemo-rl-trtllm' or (extra == 'extra-7-nemo-rl-vllm' and extra == 'extra-8-nemo-gym-vllm') or (extra != 'extra-7-nemo-rl-vllm' and extra != 'extra-8-nemo-gym-vllm')" }, + { name = "annotated-doc", marker = "extra == 'extra-7-nemo-rl-automodel' or extra == 'extra-7-nemo-rl-mcore' or extra == 'extra-7-nemo-rl-sglang' or extra == 'extra-7-nemo-rl-trtllm' or extra != 'extra-7-nemo-rl-vllm'" }, + { name = "pydantic", marker = "extra == 'extra-7-nemo-rl-automodel' or extra == 'extra-7-nemo-rl-mcore' or extra == 'extra-7-nemo-rl-sglang' or extra == 'extra-7-nemo-rl-trtllm' or extra != 'extra-7-nemo-rl-vllm'" }, + { name = "starlette", marker = "extra == 'extra-7-nemo-rl-automodel' or extra == 'extra-7-nemo-rl-mcore' or extra == 'extra-7-nemo-rl-sglang' or extra == 'extra-7-nemo-rl-trtllm' or extra != 'extra-7-nemo-rl-vllm'" }, + { name = "typing-extensions", marker = "extra == 'extra-7-nemo-rl-automodel' or extra == 'extra-7-nemo-rl-mcore' or extra == 'extra-7-nemo-rl-sglang' or extra == 'extra-7-nemo-rl-trtllm' or extra != 'extra-7-nemo-rl-vllm'" }, + { name = "typing-inspection", marker = "extra == 'extra-7-nemo-rl-automodel' or extra == 'extra-7-nemo-rl-mcore' or extra == 'extra-7-nemo-rl-sglang' or extra == 'extra-7-nemo-rl-trtllm' or extra != 'extra-7-nemo-rl-vllm'" }, ] sdist = { url = "https://files.pythonhosted.org/packages/cd/95/d3f0ae10836324a2eab98a52b61210ac609f08200bf4bb0dc8132d32f78a/fastapi-0.139.2.tar.gz", hash = "sha256:333145a6891e9b5b3cfceb69baf817e8240cde4d4588ae5a10bf56ffacb6255e", size = 423428, upload-time = "2026-07-16T15:06:17.912Z" } wheels = [ @@ -1683,7 +1484,7 @@ source = { registry = "https://pypi.org/simple" } dependencies = [ { name = "rich-toolkit" }, { name = "typer" }, - { name = "uvicorn", extra = ["standard"], marker = "(extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-8-nemo-gym-vllm') or (extra != 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-vllm') or (extra != 'extra-7-nemo-rl-automodel' and extra == 'extra-8-nemo-gym-vllm') or (extra != 'extra-7-nemo-rl-fsdp' and extra != 'extra-7-nemo-rl-mcore' and extra != 'extra-7-nemo-rl-sglang' and extra != 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra != 'extra-7-nemo-rl-fsdp' and extra != 'extra-7-nemo-rl-mcore' and extra != 'extra-7-nemo-rl-sglang' and extra != 'extra-7-nemo-rl-trtllm' and extra == 'extra-8-nemo-gym-vllm')" }, + { name = "uvicorn", extra = ["standard"], marker = "(extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra != 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-vllm') or (extra != 'extra-7-nemo-rl-fsdp' and extra != 'extra-7-nemo-rl-mcore' and extra != 'extra-7-nemo-rl-sglang' and extra != 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm')" }, ] sdist = { url = "https://files.pythonhosted.org/packages/33/eb/3b534c6f8e157f9ddbf2a153512307c886cad0b258739c200dd8ff8c4452/fastapi_cli-0.0.32.tar.gz", hash = "sha256:38024d2345275e1b37ce8848727a580d84901b570e96b3256d9d36a9a5039424", size = 26636, upload-time = "2026-07-16T12:16:58.678Z" } wheels = [ @@ -1693,7 +1494,7 @@ wheels = [ [package.optional-dependencies] standard = [ { name = "fastapi-cloud-cli" }, - { name = "uvicorn", extra = ["standard"], marker = "(extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-8-nemo-gym-vllm') or (extra != 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-vllm') or (extra != 'extra-7-nemo-rl-automodel' and extra == 'extra-8-nemo-gym-vllm') or (extra != 'extra-7-nemo-rl-fsdp' and extra != 'extra-7-nemo-rl-mcore' and extra != 'extra-7-nemo-rl-sglang' and extra != 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra != 'extra-7-nemo-rl-fsdp' and extra != 'extra-7-nemo-rl-mcore' and extra != 'extra-7-nemo-rl-sglang' and extra != 'extra-7-nemo-rl-trtllm' and extra == 'extra-8-nemo-gym-vllm')" }, + { name = "uvicorn", extra = ["standard"], marker = "(extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra != 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-vllm') or (extra != 'extra-7-nemo-rl-fsdp' and extra != 'extra-7-nemo-rl-mcore' and extra != 'extra-7-nemo-rl-sglang' and extra != 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm')" }, ] [[package]] @@ -1704,12 +1505,12 @@ dependencies = [ { name = "detect-installer" }, { name = "fastar" }, { name = "httpx" }, - { name = "pydantic", extra = ["email"], marker = "(extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-8-nemo-gym-vllm') or (extra != 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-vllm') or (extra != 'extra-7-nemo-rl-automodel' and extra == 'extra-8-nemo-gym-vllm') or (extra != 'extra-7-nemo-rl-fsdp' and extra != 'extra-7-nemo-rl-mcore' and extra != 'extra-7-nemo-rl-sglang' and extra != 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra != 'extra-7-nemo-rl-fsdp' and extra != 'extra-7-nemo-rl-mcore' and extra != 'extra-7-nemo-rl-sglang' and extra != 'extra-7-nemo-rl-trtllm' and extra == 'extra-8-nemo-gym-vllm')" }, + { name = "pydantic", extra = ["email"], marker = "(extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra != 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-vllm') or (extra != 'extra-7-nemo-rl-fsdp' and extra != 'extra-7-nemo-rl-mcore' and extra != 'extra-7-nemo-rl-sglang' and extra != 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm')" }, { name = "rich-toolkit" }, { name = "rignore" }, { name = "sentry-sdk" }, { name = "typer" }, - { name = "uvicorn", extra = ["standard"], marker = "(extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-8-nemo-gym-vllm') or (extra != 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-vllm') or (extra != 'extra-7-nemo-rl-automodel' and extra == 'extra-8-nemo-gym-vllm') or (extra != 'extra-7-nemo-rl-fsdp' and extra != 'extra-7-nemo-rl-mcore' and extra != 'extra-7-nemo-rl-sglang' and extra != 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra != 'extra-7-nemo-rl-fsdp' and extra != 'extra-7-nemo-rl-mcore' and extra != 'extra-7-nemo-rl-sglang' and extra != 'extra-7-nemo-rl-trtllm' and extra == 'extra-8-nemo-gym-vllm')" }, + { name = "uvicorn", extra = ["standard"], marker = "(extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra != 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-vllm') or (extra != 'extra-7-nemo-rl-fsdp' and extra != 'extra-7-nemo-rl-mcore' and extra != 'extra-7-nemo-rl-sglang' and extra != 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm')" }, ] sdist = { url = "https://files.pythonhosted.org/packages/25/f2/36bfe990baa656de89a2b98a77a15dcd018474f7245c8e4a10cada0553c5/fastapi_cloud_cli-0.22.2.tar.gz", hash = "sha256:7ec78c1fed58f578af5eb1fb54ec4b456eba4dc1eaca1c3a93cf499a7cbc7ab3", size = 94480, upload-time = "2026-07-14T09:27:01.349Z" } wheels = [ @@ -1815,25 +1616,16 @@ resolution-markers = [ "platform_machine == 'aarch64' and sys_platform == 'linux'", ] dependencies = [ - { name = "einops", marker = "(platform_machine != 'x86_64' and extra == 'extra-7-nemo-rl-automodel') or (platform_machine != 'x86_64' and extra == 'extra-7-nemo-rl-fsdp') or (platform_machine != 'x86_64' and extra == 'extra-7-nemo-rl-mcore') or (platform_machine != 'x86_64' and extra != 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-vllm' and extra == 'extra-8-nemo-gym-vllm')" }, - { name = "ninja", marker = "(platform_machine != 'x86_64' and extra == 'extra-7-nemo-rl-automodel') or (platform_machine != 'x86_64' and extra == 'extra-7-nemo-rl-fsdp') or (platform_machine != 'x86_64' and extra == 'extra-7-nemo-rl-mcore') or (platform_machine != 'x86_64' and extra != 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-vllm' and extra == 'extra-8-nemo-gym-vllm')" }, - { name = "psutil", marker = "(platform_machine != 'x86_64' and extra == 'extra-7-nemo-rl-automodel') or (platform_machine != 'x86_64' and extra == 'extra-7-nemo-rl-fsdp') or (platform_machine != 'x86_64' and extra == 'extra-7-nemo-rl-mcore') or (platform_machine != 'x86_64' and extra != 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-vllm' and extra == 'extra-8-nemo-gym-vllm')" }, - { name = "setuptools", marker = "(platform_machine != 'x86_64' and extra == 'extra-7-nemo-rl-automodel') or (platform_machine != 'x86_64' and extra == 'extra-7-nemo-rl-fsdp') or (platform_machine != 'x86_64' and extra == 'extra-7-nemo-rl-mcore') or (platform_machine != 'x86_64' and extra != 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-vllm' and extra == 'extra-8-nemo-gym-vllm')" }, - { name = "torch", marker = "(platform_machine != 'x86_64' and extra == 'extra-7-nemo-rl-automodel') or (platform_machine != 'x86_64' and extra == 'extra-7-nemo-rl-fsdp') or (platform_machine != 'x86_64' and extra == 'extra-7-nemo-rl-mcore') or (platform_machine != 'x86_64' and extra != 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-vllm' and extra == 'extra-8-nemo-gym-vllm')" }, + { name = "einops", marker = "(platform_machine != 'x86_64' and extra == 'extra-7-nemo-rl-automodel') or (platform_machine != 'x86_64' and extra == 'extra-7-nemo-rl-fsdp') or (platform_machine != 'x86_64' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm')" }, + { name = "ninja", marker = "(platform_machine != 'x86_64' and extra == 'extra-7-nemo-rl-automodel') or (platform_machine != 'x86_64' and extra == 'extra-7-nemo-rl-fsdp') or (platform_machine != 'x86_64' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm')" }, + { name = "psutil", marker = "(platform_machine != 'x86_64' and extra == 'extra-7-nemo-rl-automodel') or (platform_machine != 'x86_64' and extra == 'extra-7-nemo-rl-fsdp') or (platform_machine != 'x86_64' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm')" }, + { name = "setuptools", marker = "(platform_machine != 'x86_64' and extra == 'extra-7-nemo-rl-automodel') or (platform_machine != 'x86_64' and extra == 'extra-7-nemo-rl-fsdp') or (platform_machine != 'x86_64' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm')" }, + { name = "torch", marker = "(platform_machine != 'x86_64' and extra == 'extra-7-nemo-rl-automodel') or (platform_machine != 'x86_64' and extra == 'extra-7-nemo-rl-fsdp') or (platform_machine != 'x86_64' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm')" }, ] wheels = [ { url = "https://github.com/Dao-AILab/flash-attention/releases/download/v2.8.1/flash_attn-2.8.1+cu13torch2.10cxx11abiTRUE-cp313-cp313-linux_aarch64.whl", hash = "sha256:9fdfbdf2d4ca984ffa34518a4054c62c295fa06e184d7a233848dc06be8f1ec2" }, ] -[package.metadata] -requires-dist = [ - { name = "einops" }, - { name = "ninja" }, - { name = "psutil" }, - { name = "setuptools" }, - { name = "torch" }, -] - [[package]] name = "flash-attn" version = "2.8.1+cu13torch2.10cxx11abitrue" @@ -1842,25 +1634,16 @@ resolution-markers = [ "platform_machine == 'x86_64' and sys_platform == 'linux'", ] dependencies = [ - { name = "einops", marker = "(platform_machine == 'x86_64' and extra == 'extra-7-nemo-rl-automodel') or (platform_machine == 'x86_64' and extra == 'extra-7-nemo-rl-fsdp') or (platform_machine == 'x86_64' and extra == 'extra-7-nemo-rl-mcore') or (platform_machine == 'x86_64' and extra != 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-vllm' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-8-nemo-gym-vllm')" }, - { name = "ninja", marker = "(platform_machine == 'x86_64' and extra == 'extra-7-nemo-rl-automodel') or (platform_machine == 'x86_64' and extra == 'extra-7-nemo-rl-fsdp') or (platform_machine == 'x86_64' and extra == 'extra-7-nemo-rl-mcore') or (platform_machine == 'x86_64' and extra != 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-vllm' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-8-nemo-gym-vllm')" }, - { name = "psutil", marker = "(platform_machine == 'x86_64' and extra == 'extra-7-nemo-rl-automodel') or (platform_machine == 'x86_64' and extra == 'extra-7-nemo-rl-fsdp') or (platform_machine == 'x86_64' and extra == 'extra-7-nemo-rl-mcore') or (platform_machine == 'x86_64' and extra != 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-vllm' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-8-nemo-gym-vllm')" }, - { name = "setuptools", marker = "(platform_machine == 'x86_64' and extra == 'extra-7-nemo-rl-automodel') or (platform_machine == 'x86_64' and extra == 'extra-7-nemo-rl-fsdp') or (platform_machine == 'x86_64' and extra == 'extra-7-nemo-rl-mcore') or (platform_machine == 'x86_64' and extra != 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-vllm' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-8-nemo-gym-vllm')" }, - { name = "torch", marker = "(platform_machine == 'x86_64' and extra == 'extra-7-nemo-rl-automodel') or (platform_machine == 'x86_64' and extra == 'extra-7-nemo-rl-fsdp') or (platform_machine == 'x86_64' and extra == 'extra-7-nemo-rl-mcore') or (platform_machine == 'x86_64' and extra != 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-vllm' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-8-nemo-gym-vllm')" }, + { name = "einops", marker = "(platform_machine == 'x86_64' and extra == 'extra-7-nemo-rl-automodel') or (platform_machine == 'x86_64' and extra == 'extra-7-nemo-rl-fsdp') or (platform_machine == 'x86_64' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm')" }, + { name = "ninja", marker = "(platform_machine == 'x86_64' and extra == 'extra-7-nemo-rl-automodel') or (platform_machine == 'x86_64' and extra == 'extra-7-nemo-rl-fsdp') or (platform_machine == 'x86_64' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm')" }, + { name = "psutil", marker = "(platform_machine == 'x86_64' and extra == 'extra-7-nemo-rl-automodel') or (platform_machine == 'x86_64' and extra == 'extra-7-nemo-rl-fsdp') or (platform_machine == 'x86_64' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm')" }, + { name = "setuptools", marker = "(platform_machine == 'x86_64' and extra == 'extra-7-nemo-rl-automodel') or (platform_machine == 'x86_64' and extra == 'extra-7-nemo-rl-fsdp') or (platform_machine == 'x86_64' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm')" }, + { name = "torch", marker = "(platform_machine == 'x86_64' and extra == 'extra-7-nemo-rl-automodel') or (platform_machine == 'x86_64' and extra == 'extra-7-nemo-rl-fsdp') or (platform_machine == 'x86_64' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm')" }, ] wheels = [ { url = "https://github.com/Dao-AILab/flash-attention/releases/download/v2.8.1/flash_attn-2.8.1+cu13torch2.10cxx11abiTRUE-cp313-cp313-linux_x86_64.whl", hash = "sha256:a14fea157eb61dfee56f605a24a7a6150704517759125b3942dd04bffa05d6cf" }, ] -[package.metadata] -requires-dist = [ - { name = "einops" }, - { name = "ninja" }, - { name = "psutil" }, - { name = "setuptools" }, - { name = "torch" }, -] - [[package]] name = "flash-attn-4" version = "4.0.0b11" @@ -1963,18 +1746,6 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/56/0c/80dad211d424c3f25199ccd9bb1913c3e2d7378b5cd3dbcd2f75a635b6dd/flashinfer_cubin-0.6.11.post1-py3-none-any.whl", hash = "sha256:1eb801fc80b5576760d356e31eb452d05ab240f44185f539331bd5f2236e504a", size = 360908523, upload-time = "2026-05-13T01:31:43.386Z" }, ] -[[package]] -name = "flashinfer-cubin" -version = "0.6.12" -source = { registry = "https://pypi.org/simple" } -resolution-markers = [ - "platform_machine == 'x86_64' and sys_platform == 'linux'", - "platform_machine == 'aarch64' and sys_platform == 'linux'", -] -wheels = [ - { url = "https://files.pythonhosted.org/packages/7d/c6/63b1bb7b1a7ae612ecf53c0e568312c3d004f9f7558b0ab5edcf7900c360/flashinfer_cubin-0.6.12-py3-none-any.whl", hash = "sha256:01de132c493bb21d5df42ebe6890966cf83b40aa970dae06b2a3c0bed85f13ec", size = 447533460, upload-time = "2026-05-29T23:45:27.579Z" }, -] - [[package]] name = "flashinfer-cubin" version = "0.6.13" @@ -2095,13 +1866,12 @@ resolution-markers = [ dependencies = [ { name = "apache-tvm-ffi" }, { name = "click" }, - { name = "cuda-tile", version = "1.1.0", source = { registry = "https://pypi.org/simple" }, marker = "extra == 'extra-7-nemo-rl-trtllm' or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-vllm' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-mcore' and extra != 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-mcore' and extra != 'extra-7-nemo-rl-trtllm' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra != 'extra-7-nemo-rl-mcore' and extra != 'extra-7-nemo-rl-trtllm' and extra != 'extra-7-nemo-rl-vllm') or (extra != 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-mcore' and extra != 'extra-7-nemo-rl-sglang' and extra != 'extra-7-nemo-rl-vllm') or (extra != 'extra-7-nemo-rl-fsdp' and extra != 'extra-7-nemo-rl-mcore' and extra != 'extra-7-nemo-rl-sglang' and extra != 'extra-7-nemo-rl-vllm' and extra == 'extra-8-nemo-gym-vllm')" }, + { name = "cuda-tile", version = "1.1.0", source = { registry = "https://pypi.org/simple" }, marker = "extra == 'extra-7-nemo-rl-trtllm' or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-mcore' and extra != 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra != 'extra-7-nemo-rl-mcore' and extra != 'extra-7-nemo-rl-trtllm' and extra != 'extra-7-nemo-rl-vllm') or (extra != 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-mcore' and extra != 'extra-7-nemo-rl-sglang' and extra != 'extra-7-nemo-rl-vllm')" }, { name = "einops" }, { name = "ninja" }, { name = "numpy" }, { name = "nvidia-cudnn-frontend" }, - { name = "nvidia-cutlass-dsl", version = "4.5.0", source = { registry = "https://pypi.org/simple" }, marker = "extra == 'extra-7-nemo-rl-trtllm' or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-vllm' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-trtllm' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-mcore' and extra != 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-mcore' and extra != 'extra-7-nemo-rl-trtllm' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra != 'extra-7-nemo-rl-mcore' and extra != 'extra-7-nemo-rl-trtllm' and extra != 'extra-7-nemo-rl-vllm' and extra != 'extra-8-nemo-gym-vllm') or (extra != 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-mcore' and extra != 'extra-7-nemo-rl-sglang' and extra != 'extra-7-nemo-rl-vllm' and extra != 'extra-8-nemo-gym-vllm')" }, - { name = "nvidia-cutlass-dsl", version = "4.5.2", source = { registry = "https://pypi.org/simple" }, marker = "(extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-8-nemo-gym-vllm') or (extra != 'extra-7-nemo-rl-automodel' and extra == 'extra-8-nemo-gym-vllm') or (extra != 'extra-7-nemo-rl-fsdp' and extra != 'extra-7-nemo-rl-mcore' and extra != 'extra-7-nemo-rl-sglang' and extra != 'extra-7-nemo-rl-trtllm' and extra != 'extra-7-nemo-rl-vllm' and extra == 'extra-8-nemo-gym-vllm')" }, + { name = "nvidia-cutlass-dsl", version = "4.5.0", source = { registry = "https://pypi.org/simple" } }, { name = "nvidia-ml-py" }, { name = "packaging" }, { name = "requests" }, @@ -2125,7 +1895,7 @@ resolution-markers = [ dependencies = [ { name = "apache-tvm-ffi" }, { name = "click" }, - { name = "cuda-tile", version = "1.1.0", source = { registry = "https://pypi.org/simple" }, marker = "extra == 'extra-7-nemo-rl-vllm' or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-vllm' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-mcore' and extra != 'extra-7-nemo-rl-vllm' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra != 'extra-7-nemo-rl-vllm' and extra == 'extra-8-nemo-gym-vllm')" }, + { name = "cuda-tile", version = "1.1.0", source = { registry = "https://pypi.org/simple" }, marker = "(extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra != 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-vllm') or (extra != 'extra-7-nemo-rl-fsdp' and extra != 'extra-7-nemo-rl-mcore' and extra != 'extra-7-nemo-rl-sglang' and extra != 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm')" }, { name = "einops" }, { name = "ninja" }, { name = "numpy" }, @@ -2346,15 +2116,6 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/e7/c8/e2645aa8ed02fd4c7a2f59d68783b65b1f3cbdfe39a6308e156509d1fee8/googleapis_common_protos-1.75.0-py3-none-any.whl", hash = "sha256:961ed60399c457ceb0ee8f285a84c870aabc9c6a832b9d37bb281b5bebde43ed", size = 300631, upload-time = "2026-05-07T08:03:30.345Z" }, ] -[[package]] -name = "gprof2dot" -version = "2025.4.14" -source = { registry = "https://pypi.org/simple" } -sdist = { url = "https://files.pythonhosted.org/packages/bb/fd/cad13fa1f7a463a607176432c4affa33ea162f02f58cc36de1d40d3e6b48/gprof2dot-2025.4.14.tar.gz", hash = "sha256:35743e2d2ca027bf48fa7cba37021aaf4a27beeae1ae8e05a50b55f1f921a6ce", size = 39536, upload-time = "2025-04-14T07:21:45.76Z" } -wheels = [ - { url = "https://files.pythonhosted.org/packages/71/ed/89d760cb25279109b89eb52975a7b5479700d3114a2421ce735bfb2e7513/gprof2dot-2025.4.14-py3-none-any.whl", hash = "sha256:0742e4c0b4409a5e8777e739388a11e1ed3750be86895655312ea7c20bd0090e", size = 37555, upload-time = "2025-04-14T07:21:43.319Z" }, -] - [[package]] name = "graphene" version = "3.4.3" @@ -2606,21 +2367,6 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/d2/fd/6668e5aec43ab844de6fc74927e155a3b37bf40d7c3790e49fc0406b6578/httpx_sse-0.4.3-py3-none-any.whl", hash = "sha256:0ac1c9fe3c0afad2e0ebb25a934a59f4c7823b60792691f779fad2c5568830fc", size = 8960, upload-time = "2025-10-10T21:48:21.158Z" }, ] -[[package]] -name = "httpx-ws" -version = "0.9.0" -source = { registry = "https://pypi.org/simple" } -dependencies = [ - { name = "anyio" }, - { name = "httpcore" }, - { name = "httpx" }, - { name = "wsproto" }, -] -sdist = { url = "https://files.pythonhosted.org/packages/cd/cd/ca91a07ae446451f7476bf3fcc909e98cb942ff032ebfda0e3fe449aca7b/httpx_ws-0.9.0.tar.gz", hash = "sha256:797373326f70eec1ae96f6e43ae9f12002fd7d73aee139a4985eaab964338a08", size = 107105, upload-time = "2026-03-28T14:11:10.781Z" } -wheels = [ - { url = "https://files.pythonhosted.org/packages/98/f8/a6bc80313a9e93c888fa10534dfce2ad76ff86911b6f485777ce6de6a073/httpx_ws-0.9.0-py3-none-any.whl", hash = "sha256:71640d2fb1bf9a225775015b33cd755cfd4c5f7e21c885192fe3adc4c387b248", size = 15759, upload-time = "2026-03-28T14:11:11.887Z" }, -] - [[package]] name = "huey" version = "3.2.1" @@ -2650,47 +2396,10 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/5f/c3/aeaaf3911d2529614be18d1c8b5496afc185560e76568063d517283318af/huggingface_hub-1.24.0-py3-none-any.whl", hash = "sha256:6ed4120a84a6beec900640aa7e346bd766a6b7341e41526fef5dc8bd81fb7d59", size = 771904, upload-time = "2026-07-17T09:53:59.106Z" }, ] -[[package]] -name = "humming-kernels" -version = "0.1.6" -source = { registry = "https://pypi.org/simple" } -resolution-markers = [ - "platform_machine == 'x86_64' and sys_platform == 'linux'", - "platform_machine == 'aarch64' and sys_platform == 'linux'", -] -dependencies = [ - { name = "cuda-bindings" }, - { name = "jinja2" }, - { name = "numpy" }, - { name = "nvidia-ml-py" }, - { name = "pyelftools" }, - { name = "safetensors" }, - { name = "tabulate" }, - { name = "torch" }, - { name = "tqdm" }, - { name = "triton" }, -] -sdist = { url = "https://files.pythonhosted.org/packages/29/5a/fbf574dcd83e9fea6aa3fa96b37bbdec40b8672407b1ed9679efa31fff1d/humming_kernels-0.1.6.tar.gz", hash = "sha256:882b9f382a010165a7cf8eecbad943bfe8d6b17566328fb57611c9a34bdccc9a", size = 214408, upload-time = "2026-06-20T04:04:43.221Z" } -wheels = [ - { url = "https://files.pythonhosted.org/packages/25/85/490681b9ba24531da91d0bae801d2b26850e5a80bbd02c2efc500756e36b/humming_kernels-0.1.6-py3-none-any.whl", hash = "sha256:e64c0883fca930074bf920f4ba47cbf3acd244d7352f6c74c8d2182439770d8f", size = 178759, upload-time = "2026-06-20T04:04:41.66Z" }, -] - -[package.optional-dependencies] -cu13 = [ - { name = "nvidia-cuda-cccl" }, - { name = "nvidia-cuda-nvcc" }, - { name = "nvidia-cuda-nvrtc" }, - { name = "nvidia-cuda-runtime" }, -] - [[package]] name = "humming-kernels" version = "0.1.10" source = { registry = "https://pypi.org/simple" } -resolution-markers = [ - "platform_machine == 'x86_64' and sys_platform == 'linux'", - "platform_machine == 'aarch64' and sys_platform == 'linux'", -] dependencies = [ { name = "cuda-bindings" }, { name = "jinja2" }, @@ -2725,9 +2434,9 @@ resolution-markers = [ "platform_machine == 'aarch64' and sys_platform == 'linux'", ] dependencies = [ - { name = "antlr4-python3-runtime", marker = "extra == 'extra-7-nemo-rl-mcore' or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-vllm' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-mcore' and extra == 'extra-8-nemo-gym-vllm')" }, - { name = "omegaconf", marker = "extra == 'extra-7-nemo-rl-mcore' or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-vllm' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-mcore' and extra == 'extra-8-nemo-gym-vllm')" }, - { name = "packaging", marker = "extra == 'extra-7-nemo-rl-mcore' or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-vllm' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-mcore' and extra == 'extra-8-nemo-gym-vllm')" }, + { name = "antlr4-python3-runtime", marker = "extra == 'extra-7-nemo-rl-mcore' or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm')" }, + { name = "omegaconf", marker = "extra == 'extra-7-nemo-rl-mcore' or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm')" }, + { name = "packaging", marker = "extra == 'extra-7-nemo-rl-mcore' or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm')" }, ] sdist = { url = "https://files.pythonhosted.org/packages/6d/8e/07e42bc434a847154083b315779b0a81d567154504624e181caf2c71cd98/hydra-core-1.3.2.tar.gz", hash = "sha256:8a878ed67216997c3e9d88a8e72e7b4767e81af37afb4ea3334b269a4390a824", size = 3263494, upload-time = "2023-02-23T18:33:43.03Z" } wheels = [ @@ -2743,9 +2452,9 @@ resolution-markers = [ "platform_machine == 'aarch64' and sys_platform == 'linux'", ] dependencies = [ - { name = "antlr4-python3-runtime", marker = "extra == 'extra-7-nemo-rl-automodel' or extra != 'extra-7-nemo-rl-mcore' or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-8-nemo-gym-vllm')" }, - { name = "omegaconf", marker = "extra == 'extra-7-nemo-rl-automodel' or extra != 'extra-7-nemo-rl-mcore' or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-8-nemo-gym-vllm')" }, - { name = "packaging", marker = "extra == 'extra-7-nemo-rl-automodel' or extra != 'extra-7-nemo-rl-mcore' or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-8-nemo-gym-vllm')" }, + { name = "antlr4-python3-runtime", marker = "extra == 'extra-7-nemo-rl-automodel' or extra != 'extra-7-nemo-rl-mcore' or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm')" }, + { name = "omegaconf", marker = "extra == 'extra-7-nemo-rl-automodel' or extra != 'extra-7-nemo-rl-mcore' or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm')" }, + { name = "packaging", marker = "extra == 'extra-7-nemo-rl-automodel' or extra != 'extra-7-nemo-rl-mcore' or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm')" }, ] sdist = { url = "https://files.pythonhosted.org/packages/10/dd/220f0e91743136725352497e98540772a01fc7c3ab96ff16c3c74424e984/hydra_core-1.3.4.tar.gz", hash = "sha256:ad0f7b05a0242255a8984d5a4ed2f6847f7b783ed727368a2c0155ec52d6c34c", size = 3263348, upload-time = "2026-07-04T16:25:38.891Z" } wheels = [ @@ -3076,19 +2785,6 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/b5/ba/c63c5786dfee4c3417094c4b00966e61e4a63efecee22cb7b4c0387dda83/librosa-0.11.0-py3-none-any.whl", hash = "sha256:0b6415c4fd68bff4c29288abe67c6d80b587e0e1e2cfb0aad23e4559504a7fa1", size = 260749, upload-time = "2025-03-11T15:09:52.982Z" }, ] -[[package]] -name = "librt" -version = "0.13.0" -source = { registry = "https://pypi.org/simple" } -sdist = { url = "https://files.pythonhosted.org/packages/dc/2f/3908645ddddab7120b46295e541ead308109fa48dbec7d67d7a778870d60/librt-0.13.0.tar.gz", hash = "sha256:1d2a610c14ac0d0750ee0a3ab8548e83155258387891caaca04def4bf7289781", size = 211402, upload-time = "2026-07-08T12:26:29.834Z" } -wheels = [ - { url = "https://files.pythonhosted.org/packages/0e/16/52b1c99bf19057a062aac39c900cbb81499f6f75d6c537c14463d247ba78/librt-0.13.0-cp313-cp313-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:32c26893cd085c1efe83219e78d866da23fb20a066101b8f68210004361d224c", size = 502499, upload-time = "2026-07-08T12:25:22.055Z" }, - { url = "https://files.pythonhosted.org/packages/8f/f8/094d6b2bd93f3fdaa54db54cc788c4a365333bddad65ab02e04da0b1d004/librt-0.13.0-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:94b85d664d777bab6c0d709416cb42938251fda9e221b79e3a2215d85df5f4f9", size = 531576, upload-time = "2026-07-08T12:25:24.648Z" }, - { url = "https://files.pythonhosted.org/packages/c6/b5/255673cfdbf5ba663339d36cd863c897289ab4337577e19f9405ce059f36/librt-0.13.0-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:109b84a9edf69ad89dc1f66358659e14a031baca95e3e5b0060bd903ede8efd6", size = 543053, upload-time = "2026-07-08T12:25:27.436Z" }, - { url = "https://files.pythonhosted.org/packages/5a/76/927e267a6daa290174ac281b23c9804c8829b042ade9c6f24a065f540958/librt-0.13.0-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:9f836c37478f167a81200d8c8b2c920a22224564bed2c23d7aeec760965c367a", size = 573582, upload-time = "2026-07-08T12:25:31.507Z" }, - { url = "https://files.pythonhosted.org/packages/10/24/b6c5213efe39c19f9e13605644d0cf063b4ddaa33ac2e45b088e23a70e2e/librt-0.13.0-cp313-cp313-pyemscripten_2025_0_wasm32.whl", hash = "sha256:4000d961ff9598ac6ea603c6c836a5ed49bc205ade5fc378b998dfe1e2c36628", size = 82189, upload-time = "2026-07-08T12:25:32.675Z" }, -] - [[package]] name = "llguidance" version = "1.3.0" @@ -3170,10 +2866,10 @@ name = "mamba-ssm" version = "2.2.6.post3" source = { git = "https://github.com/state-spaces/mamba.git?rev=a14b1dff0454a3bc27d9eb31355dc01e4b2490ec#a14b1dff0454a3bc27d9eb31355dc01e4b2490ec" } dependencies = [ - { name = "causal-conv1d", marker = "extra == 'extra-7-nemo-rl-automodel' or extra == 'extra-7-nemo-rl-fsdp' or extra == 'extra-7-nemo-rl-mcore' or extra != 'extra-7-nemo-rl-sglang' or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-8-nemo-gym-vllm')" }, - { name = "ninja", marker = "extra == 'extra-7-nemo-rl-automodel' or extra == 'extra-7-nemo-rl-fsdp' or extra == 'extra-7-nemo-rl-mcore' or extra != 'extra-7-nemo-rl-sglang' or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-8-nemo-gym-vllm')" }, - { name = "packaging", marker = "extra == 'extra-7-nemo-rl-automodel' or extra == 'extra-7-nemo-rl-fsdp' or extra == 'extra-7-nemo-rl-mcore' or extra != 'extra-7-nemo-rl-sglang' or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-8-nemo-gym-vllm')" }, - { name = "torch", marker = "extra == 'extra-7-nemo-rl-automodel' or extra == 'extra-7-nemo-rl-fsdp' or extra == 'extra-7-nemo-rl-mcore' or extra != 'extra-7-nemo-rl-sglang' or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-8-nemo-gym-vllm')" }, + { name = "causal-conv1d", marker = "extra == 'extra-7-nemo-rl-automodel' or extra == 'extra-7-nemo-rl-fsdp' or extra == 'extra-7-nemo-rl-mcore' or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm')" }, + { name = "ninja", marker = "extra == 'extra-7-nemo-rl-automodel' or extra == 'extra-7-nemo-rl-fsdp' or extra == 'extra-7-nemo-rl-mcore' or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm')" }, + { name = "packaging", marker = "extra == 'extra-7-nemo-rl-automodel' or extra == 'extra-7-nemo-rl-fsdp' or extra == 'extra-7-nemo-rl-mcore' or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm')" }, + { name = "torch", marker = "extra == 'extra-7-nemo-rl-automodel' or extra == 'extra-7-nemo-rl-fsdp' or extra == 'extra-7-nemo-rl-mcore' or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm')" }, ] [[package]] @@ -3320,7 +3016,7 @@ dependencies = [ { name = "flashinfer-cubin", version = "0.6.8.post1", source = { registry = "https://pypi.org/simple" } }, { name = "flashinfer-python", version = "0.6.8.post1", source = { registry = "https://pypi.org/simple" } }, { name = "hydra-core", version = "1.3.2", source = { registry = "https://pypi.org/simple" } }, - { name = "megatron-core", extra = ["dev", "mlm"], marker = "extra == 'extra-7-nemo-rl-mcore' or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-vllm' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-mcore' and extra == 'extra-8-nemo-gym-vllm')" }, + { name = "megatron-core", extra = ["dev", "mlm"], marker = "extra == 'extra-7-nemo-rl-mcore' or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm')" }, { name = "mistral-common" }, { name = "mlflow" }, { name = "nvidia-resiliency-ext" }, @@ -3347,91 +3043,7 @@ ssm = [ { name = "mamba-ssm" }, ] te = [ - { name = "transformer-engine", marker = "extra == 'extra-7-nemo-rl-mcore' or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-vllm' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-mcore' and extra == 'extra-8-nemo-gym-vllm')" }, -] - -[package.metadata] -requires-dist = [ - { name = "accelerate" }, - { name = "causal-conv1d", marker = "extra == 'ssm'" }, - { name = "comet-ml", specifier = ">=3.50.0" }, - { name = "datasets", specifier = ">=2.20.0" }, - { name = "diffusers", specifier = ">=0.36.0" }, - { name = "einops" }, - { name = "flash-linear-attention" }, - { name = "flashinfer-cubin", specifier = "==0.6.8.post1" }, - { name = "flashinfer-python", specifier = "==0.6.8.post1" }, - { name = "hydra-core", specifier = ">1.3,<=1.3.2" }, - { name = "librosa", marker = "extra == 'audio'" }, - { name = "mamba-ssm", marker = "extra == 'ssm'" }, - { name = "megatron-core", extras = ["dev", "mlm"], editable = "3rdparty/Megatron-Bridge-workspace/Megatron-Bridge/3rdparty/Megatron-LM" }, - { name = "mistral-common", specifier = ">=1.10.0" }, - { name = "mlflow", specifier = ">=3.15.1" }, - { name = "nemo-run", marker = "extra == 'recipes'" }, - { name = "nvdlfw-inspect", marker = "extra == 'tensor-inspect'", specifier = "==0.2.1" }, - { name = "nvidia-resiliency-ext" }, - { name = "omegaconf", specifier = ">=2.3.0" }, - { name = "open-clip-torch", specifier = ">=3.2.0" }, - { name = "peft", specifier = ">=0.18.0" }, - { name = "peft", specifier = ">=0.18.1" }, - { name = "pyarrow", marker = "extra == 'parquet'", specifier = ">=14.0.0" }, - { name = "pyyaml", specifier = ">=6.0.2" }, - { name = "qwen-vl-utils" }, - { name = "regex", specifier = ">=2024.11.6" }, - { name = "rich" }, - { name = "six", specifier = ">=1.17.0" }, - { name = "tensorboard", specifier = ">=2.19.0" }, - { name = "timm" }, - { name = "torch", specifier = ">=2.6.0" }, - { name = "tqdm", specifier = ">=4.67.1" }, - { name = "transformer-engine", extras = ["core-cu13", "pytorch"], marker = "extra == 'te'" }, - { name = "transformers", specifier = ">=5.8,<=5.12.1" }, - { name = "typing-extensions" }, - { name = "wandb", specifier = ">=0.25.0" }, -] -provides-extras = ["recipes", "parquet", "tensor-inspect", "te", "ssm", "audio"] - -[package.metadata.requires-dev] -build = [ - { name = "cython", specifier = ">=3.0.0" }, - { name = "ninja" }, - { name = "numpy" }, - { name = "nvidia-cudnn-frontend", specifier = ">=1.25.0" }, - { name = "nvidia-mathdx" }, - { name = "pybind11" }, - { name = "setuptools" }, - { name = "torch" }, -] -dev = [ - { name = "mypy", specifier = ">=1.8.0" }, - { name = "pre-commit", specifier = ">=3.6.0" }, - { name = "ruff", specifier = ">=0.9.9" }, -] -diffusion = [ - { name = "av" }, - { name = "imageio" }, - { name = "imageio-ffmpeg" }, -] -docs = [ - { name = "myst-parser", specifier = ">=4.0.1" }, - { name = "nvidia-sphinx-theme", specifier = ">=0.0.8" }, - { name = "sphinx", specifier = ">=8.1.3" }, - { name = "sphinx-autobuild", specifier = ">=2024.10.3" }, - { name = "sphinx-autodoc2", specifier = ">=0.5.0" }, - { name = "sphinx-copybutton", specifier = ">=0.5.2" }, - { name = "sphinxcontrib-mermaid" }, -] -no-pypi-wheels = [{ name = "flash-mla", git = "https://github.com/deepseek-ai/FlashMLA?rev=nv_dev" }] -test = [ - { name = "click" }, - { name = "coverage", specifier = ">=7.8.1" }, - { name = "flake8", specifier = ">=7.2.0" }, - { name = "pygithub" }, - { name = "pylint", specifier = ">=3.3.7" }, - { name = "pytest", specifier = ">=8.3.5" }, - { name = "pytest-mock", specifier = ">=3.14.0" }, - { name = "pytest-runner", specifier = ">=6.0.1" }, - { name = "pytest-timeout", specifier = ">=2.4.0" }, + { name = "transformer-engine", marker = "extra == 'extra-7-nemo-rl-mcore' or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm')" }, ] [[package]] @@ -3453,13 +3065,13 @@ dev = [ { name = "flash-linear-attention", version = "0.4.2", source = { registry = "https://pypi.org/simple" } }, { name = "flashinfer-python", version = "0.6.8.post1", source = { registry = "https://pypi.org/simple" } }, { name = "hypercorn" }, - { name = "megatron-energon", marker = "extra == 'extra-7-nemo-rl-mcore' or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-vllm' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-mcore' and extra == 'extra-8-nemo-gym-vllm')" }, + { name = "megatron-energon", marker = "extra == 'extra-7-nemo-rl-mcore' or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm')" }, { name = "multi-storage-client" }, { name = "nvidia-cudnn-frontend" }, - { name = "nvidia-modelopt", marker = "extra == 'extra-7-nemo-rl-mcore' or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-vllm' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-mcore' and extra == 'extra-8-nemo-gym-vllm')" }, + { name = "nvidia-modelopt", marker = "extra == 'extra-7-nemo-rl-mcore' or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm')" }, { name = "nvidia-resiliency-ext" }, { name = "onnxscript" }, - { name = "openai", extra = ["aiohttp"], marker = "extra == 'extra-7-nemo-rl-mcore' or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-vllm' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-mcore' and extra == 'extra-8-nemo-gym-vllm')" }, + { name = "openai", extra = ["aiohttp"], marker = "extra == 'extra-7-nemo-rl-mcore' or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm')" }, { name = "opentelemetry-api" }, { name = "orjson" }, { name = "quart" }, @@ -3478,126 +3090,24 @@ mlm = [ { name = "wandb" }, ] -[package.metadata] -requires-dist = [ - { name = "accelerate", marker = "extra == 'mlm'" }, - { name = "accelerate", marker = "extra == 'training'" }, - { name = "av", marker = "extra == 'dev'" }, - { name = "causal-conv1d", marker = "extra == 'ssm'", specifier = "~=1.5" }, - { name = "datasets", marker = "extra == 'dev'" }, - { name = "einops", marker = "extra == 'dev'", specifier = "~=0.8" }, - { name = "emerging-optimizers", marker = "extra == 'dev'", git = "https://github.com/NVIDIA-NeMo/Emerging-Optimizers.git?rev=v0.2.0" }, - { name = "fast-hadamard-transform", marker = "extra == 'dev'", git = "https://github.com/Dao-AILab/fast-hadamard-transform.git?rev=f134af63deb2df17e1171a9ec1ea4a7d8604d5ca" }, - { name = "fastapi", marker = "extra == 'dev'", specifier = "~=0.50" }, - { name = "flash-linear-attention", marker = "extra == 'dev'", specifier = "~=0.4.0" }, - { name = "flashinfer-python", marker = "extra == 'dev'", specifier = ">=0.5.0,<0.7.0" }, - { name = "flask-restful", marker = "extra == 'mlm'" }, - { name = "flask-restful", marker = "extra == 'training'" }, - { name = "hypercorn", marker = "extra == 'dev'" }, - { name = "mamba-ssm", marker = "extra == 'ssm'", git = "https://github.com/state-spaces/mamba.git?rev=0048fbf2e7b2f214dcbe703ea3dec2b9647595e1" }, - { name = "megatron-energon", extras = ["av-decode"], marker = "extra == 'dev'", specifier = "~=7.0" }, - { name = "multi-storage-client", marker = "extra == 'dev'", specifier = "~=0.50" }, +[[package]] +name = "megatron-energon" +version = "7.4.0" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "braceexpand" }, + { name = "click" }, + { name = "filetype" }, + { name = "mfusepy" }, + { name = "multi-storage-client" }, { name = "numpy" }, - { name = "nvidia-cudnn-frontend", extras = ["cutedsl"], marker = "extra == 'dev'", specifier = "==1.26.0" }, - { name = "nvidia-modelopt", extras = ["torch"], marker = "sys_platform != 'darwin' and extra == 'dev'", specifier = ">=0.44" }, - { name = "nvidia-resiliency-ext", marker = "extra == 'dev'", specifier = "==0.6.0" }, - { name = "omegaconf", marker = "extra == 'mlm'" }, - { name = "omegaconf", marker = "extra == 'training'" }, - { name = "onnxscript", marker = "extra == 'dev'" }, - { name = "openai", extras = ["aiohttp"], marker = "extra == 'dev'" }, - { name = "opentelemetry-api", marker = "extra == 'dev'", specifier = "~=1.33.1" }, - { name = "orjson", marker = "extra == 'dev'" }, - { name = "packaging", specifier = ">=24.2" }, - { name = "quart", marker = "extra == 'dev'" }, - { name = "sentencepiece", marker = "extra == 'mlm'" }, - { name = "sentencepiece", marker = "extra == 'training'" }, - { name = "tensorstore", marker = "extra == 'dev'", specifier = "~=0.1,!=0.1.46,!=0.1.72" }, - { name = "tiktoken", marker = "extra == 'mlm'" }, - { name = "tiktoken", marker = "extra == 'training'" }, - { name = "torch", specifier = ">=2.6.0" }, - { name = "tqdm", marker = "extra == 'dev'" }, - { name = "transformer-engine", extras = ["core-cu13", "pytorch"], marker = "extra == 'te'", git = "https://github.com/NVIDIA/TransformerEngine.git?rev=e7c550c5f80636cf841a8204b1d6f85a5f3f28b7" }, - { name = "transformers", marker = "extra == 'mlm'" }, - { name = "transformers", marker = "extra == 'training'" }, - { name = "wandb", marker = "extra == 'mlm'" }, - { name = "wandb", marker = "extra == 'training'" }, - { name = "wget", marker = "extra == 'dev'" }, - { name = "zstandard", marker = "extra == 'dev'" }, -] -provides-extras = ["training", "mlm", "dev", "lts", "te", "ssm"] - -[package.metadata.requires-dev] -batch-invariant = [{ name = "deep-gemm", git = "https://github.com/deepseek-ai/DeepGEMM.git?rev=714dd1a4a980f7937a74343d19a8eba4fe321480" }] -build = [ - { name = "cython", specifier = ">=3.0.0" }, - { name = "hatchling" }, - { name = "nvidia-cudnn-frontend", specifier = ">=1.25.0" }, - { name = "nvidia-mathdx" }, - { name = "packaging", specifier = ">=24.2" }, - { name = "pybind11" }, - { name = "setuptools", specifier = ">=80" }, + { name = "pillow" }, + { name = "pyyaml" }, + { name = "rapidyaml" }, + { name = "s3fs" }, { name = "torch" }, -] -ci = [ - { name = "pandas" }, - { name = "python-gitlab" }, - { name = "slack-sdk" }, -] -docs = [ - { name = "myst-parser" }, - { name = "nvidia-sphinx-theme" }, - { name = "sphinx" }, - { name = "sphinx-autobuild" }, - { name = "sphinx-autodoc2" }, - { name = "sphinx-copybutton" }, -] -linting = [ - { name = "black", specifier = "==26.3.0" }, - { name = "flake8", specifier = "==7.1.0" }, - { name = "isort", specifier = "==5.13.2" }, - { name = "pylint", specifier = "==3.2.6" }, - { name = "ruff", specifier = "~=0.9.0" }, -] -no-pypi-wheels = [ - { name = "deep-gemm", git = "https://github.com/deepseek-ai/DeepGEMM.git?rev=714dd1a4a980f7937a74343d19a8eba4fe321480" }, - { name = "emerging-optimizers", git = "https://github.com/NVIDIA-NeMo/Emerging-Optimizers.git?rev=v0.2.0" }, - { name = "flash-mla", git = "https://github.com/deepseek-ai/FlashMLA?rev=nv_dev" }, -] -test = [ - { name = "coverage" }, - { name = "mock" }, - { name = "nemo-run", git = "https://github.com/NVIDIA-NeMo/Run.git?rev=e3935393a290aed1822af52139b4b8ee270fed1f" }, - { name = "nltk" }, - { name = "pydantic" }, - { name = "pygithub" }, - { name = "pytest", specifier = "==9.1.1" }, - { name = "pytest-asyncio" }, - { name = "pytest-cov" }, - { name = "pytest-mock" }, - { name = "pytest-random-order" }, - { name = "pyyaml" }, - { name = "tensorboard" }, - { name = "wrapt" }, -] - -[[package]] -name = "megatron-energon" -version = "7.4.0" -source = { registry = "https://pypi.org/simple" } -dependencies = [ - { name = "braceexpand" }, - { name = "click" }, - { name = "filetype" }, - { name = "mfusepy" }, - { name = "multi-storage-client" }, - { name = "numpy" }, - { name = "pillow" }, - { name = "pyyaml" }, - { name = "rapidyaml" }, - { name = "s3fs" }, - { name = "torch" }, - { name = "tqdm" }, - { name = "webdataset" }, + { name = "tqdm" }, + { name = "webdataset" }, ] sdist = { url = "https://files.pythonhosted.org/packages/7b/8a/690e08320622954d347c57270be852cad053798559a8e54810f28f8f6949/megatron_energon-7.4.0.tar.gz", hash = "sha256:df78a42d56dd443e9e1961d899698be2b737010c10ae8747a9428308a6b8e3b1", size = 211145, upload-time = "2026-06-17T10:18:23.881Z" } wheels = [ @@ -3712,8 +3222,8 @@ dependencies = [ { name = "click" }, { name = "cloudpickle" }, { name = "databricks-sdk" }, - { name = "fastapi", version = "0.136.3", source = { registry = "https://pypi.org/simple" }, marker = "(extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-8-nemo-gym-vllm') or (extra != 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-vllm') or (extra != 'extra-7-nemo-rl-automodel' and extra == 'extra-8-nemo-gym-vllm') or (extra != 'extra-7-nemo-rl-fsdp' and extra != 'extra-7-nemo-rl-mcore' and extra != 'extra-7-nemo-rl-sglang' and extra != 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra != 'extra-7-nemo-rl-fsdp' and extra != 'extra-7-nemo-rl-mcore' and extra != 'extra-7-nemo-rl-sglang' and extra != 'extra-7-nemo-rl-trtllm' and extra == 'extra-8-nemo-gym-vllm')" }, - { name = "fastapi", version = "0.139.2", source = { registry = "https://pypi.org/simple" }, marker = "extra == 'extra-7-nemo-rl-automodel' or extra == 'extra-7-nemo-rl-mcore' or extra == 'extra-7-nemo-rl-sglang' or extra == 'extra-7-nemo-rl-trtllm' or (extra == 'extra-7-nemo-rl-vllm' and extra == 'extra-8-nemo-gym-vllm') or (extra != 'extra-7-nemo-rl-vllm' and extra != 'extra-8-nemo-gym-vllm')" }, + { name = "fastapi", version = "0.136.3", source = { registry = "https://pypi.org/simple" }, marker = "(extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra != 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-vllm') or (extra != 'extra-7-nemo-rl-fsdp' and extra != 'extra-7-nemo-rl-mcore' and extra != 'extra-7-nemo-rl-sglang' and extra != 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm')" }, + { name = "fastapi", version = "0.139.2", source = { registry = "https://pypi.org/simple" }, marker = "extra == 'extra-7-nemo-rl-automodel' or extra == 'extra-7-nemo-rl-mcore' or extra == 'extra-7-nemo-rl-sglang' or extra == 'extra-7-nemo-rl-trtllm' or extra != 'extra-7-nemo-rl-vllm'" }, { name = "gitpython" }, { name = "importlib-metadata" }, { name = "opentelemetry-api" }, @@ -3922,34 +3432,6 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/7e/82/69e539c4c2027f1e1697e09aaa2449243085a0edf81ae2c6341e84d769b6/multiprocess-0.70.19-py39-none-any.whl", hash = "sha256:0d4b4397ed669d371c81dcd1ef33fd384a44d6c3de1bd0ca7ac06d837720d3c5", size = 133477, upload-time = "2026-01-19T06:47:38.619Z" }, ] -[[package]] -name = "mypy" -version = "2.3.0" -source = { registry = "https://pypi.org/simple" } -dependencies = [ - { name = "ast-serialize" }, - { name = "librt", marker = "platform_python_implementation != 'PyPy' or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-vllm' and extra == 'extra-8-nemo-gym-vllm')" }, - { name = "mypy-extensions" }, - { name = "pathspec" }, - { name = "typing-extensions" }, -] -sdist = { url = "https://files.pythonhosted.org/packages/12/af/4e516a05d3ca2eb9283e9ec45b2c02225c1514dd6da49fd3c9eaa6639370/mypy-2.3.0.tar.gz", hash = "sha256:465965d41cd9a2726694e983e8ce7113259327bec798115d1e1dfa2a52fb666e", size = 3988104, upload-time = "2026-07-13T11:34:53.387Z" } -wheels = [ - { url = "https://files.pythonhosted.org/packages/d9/fb/43c031f0190513d1ec248ed037eceb742ddd2a4d74bbf406658a28173837/mypy-2.3.0-cp313-cp313-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:de6d2c484742a4d7b0ed6d07b143375624d3b899c5749c7b3c947f56261f48a6", size = 14151586, upload-time = "2026-07-13T11:29:18.615Z" }, - { url = "https://files.pythonhosted.org/packages/ec/c3/f8b2ffc60883084da91be51af58e88a7ffd4ff9795acb7d902ff88d31eb1/mypy-2.3.0-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:7da939dd335cfd2ad788bdfd081c9f4e47634ab995e5a45eb15fd1e5bc052f8b", size = 15227411, upload-time = "2026-07-13T11:30:29.904Z" }, - { url = "https://files.pythonhosted.org/packages/83/2e/16b917fc7adcf03f1aadddfc93aab804ffb234b1ab09c0ffd6d92a5d34a2/mypy-2.3.0-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:7247eb2824f996722a949530183394921ca71deb9680052a338cf53cff7925c2", size = 15478790, upload-time = "2026-07-13T11:33:14.686Z" }, - { url = "https://files.pythonhosted.org/packages/2c/fa/fdc54fe583ba3cafbcedfb70eeeaf03849f75b1827a07096c7bd996f582d/mypy-2.3.0-py3-none-any.whl", hash = "sha256:6b1cdb579446b60432432b2b2403a6201b4b475a004d7f488511c9ba177c9e88", size = 2753292, upload-time = "2026-07-13T11:33:18.48Z" }, -] - -[[package]] -name = "mypy-extensions" -version = "1.1.0" -source = { registry = "https://pypi.org/simple" } -sdist = { url = "https://files.pythonhosted.org/packages/a2/6e/371856a3fb9d31ca8dac321cda606860fa4548858c0cc45d9d1d4ca2628b/mypy_extensions-1.1.0.tar.gz", hash = "sha256:52e68efc3284861e772bbcd66823fde5ae21fd2fdb51c62a211403730b916558", size = 6343, upload-time = "2025-04-22T14:54:24.164Z" } -wheels = [ - { url = "https://files.pythonhosted.org/packages/79/7b/2c79738432f5c924bef5071f933bcc9efd0473bac3b4aa584a6f7c1c8df8/mypy_extensions-1.1.0-py3-none-any.whl", hash = "sha256:1be4cccdb0f2482337c4743e60421de3a356cd97508abadd57d47403e94f5505", size = 4963, upload-time = "2025-04-22T14:54:22.983Z" }, -] - [[package]] name = "myst-parser" version = "5.1.0" @@ -3998,7 +3480,7 @@ dependencies = [ { name = "datasets" }, { name = "flashoptim" }, { name = "megatron-fsdp" }, - { name = "mistral-common", extra = ["audio", "hf-hub", "sentencepiece"], marker = "extra == 'extra-7-nemo-rl-automodel' or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-vllm' and extra == 'extra-8-nemo-gym-vllm')" }, + { name = "mistral-common", extra = ["audio", "hf-hub", "sentencepiece"], marker = "extra == 'extra-7-nemo-rl-automodel' or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm')" }, { name = "mlflow" }, { name = "pybind11" }, { name = "pyyaml" }, @@ -4013,113 +3495,13 @@ dependencies = [ [package.optional-dependencies] moe = [ { name = "causal-conv1d" }, - { name = "deep-ep", version = "1.2.1+29d31c0", source = { git = "https://github.com/deepseek-ai/DeepEP.git?rev=29d31c095796f3c8ece47ee9cdcc167051bbeed9#29d31c095796f3c8ece47ee9cdcc167051bbeed9" }, marker = "(platform_machine == 'x86_64' and extra == 'extra-7-nemo-rl-automodel') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-vllm' and extra == 'extra-8-nemo-gym-vllm')" }, - { name = "deep-ep", version = "1.2.1+29d31c0", source = { git = "https://github.com/deepseek-ai/DeepEP.git?rev=a48493600c4886c1b297aaa78db0e1ebc2d8dd6c#a48493600c4886c1b297aaa78db0e1ebc2d8dd6c" }, marker = "(platform_machine != 'x86_64' and extra == 'extra-7-nemo-rl-automodel') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-vllm' and extra == 'extra-8-nemo-gym-vllm')" }, + { name = "deep-ep", version = "1.2.1+29d31c0", source = { git = "https://github.com/deepseek-ai/DeepEP.git?rev=29d31c095796f3c8ece47ee9cdcc167051bbeed9#29d31c095796f3c8ece47ee9cdcc167051bbeed9" }, marker = "(platform_machine == 'x86_64' and extra == 'extra-7-nemo-rl-automodel') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm')" }, + { name = "deep-ep", version = "1.2.1+29d31c0", source = { git = "https://github.com/deepseek-ai/DeepEP.git?rev=a48493600c4886c1b297aaa78db0e1ebc2d8dd6c#a48493600c4886c1b297aaa78db0e1ebc2d8dd6c" }, marker = "(platform_machine != 'x86_64' and extra == 'extra-7-nemo-rl-automodel') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm')" }, { name = "flash-linear-attention", version = "0.5.1", source = { registry = "https://pypi.org/simple" } }, { name = "mamba-ssm" }, { name = "nv-grouped-gemm" }, { name = "onnxscript" }, - { name = "transformer-engine", marker = "extra == 'extra-7-nemo-rl-automodel' or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-vllm' and extra == 'extra-8-nemo-gym-vllm')" }, -] - -[package.metadata] -requires-dist = [ - { name = "albumentations", marker = "extra == 'vlm'" }, - { name = "backoff", marker = "extra == 'vlm'" }, - { name = "bitsandbytes", marker = "extra == 'cuda-source'" }, - { name = "causal-conv1d", marker = "extra == 'cuda'" }, - { name = "databricks-sql-connector", marker = "extra == 'delta-databricks'", specifier = ">=3.0.0" }, - { name = "datasets", specifier = ">=4.0.0" }, - { name = "deep-ep", marker = "extra == 'moe'", git = "https://github.com/deepseek-ai/DeepEP.git?rev=7febc6e25660af0f54d95dd781ecdcd62265ecca" }, - { name = "deltalake", marker = "extra == 'delta-databricks'", specifier = ">=1.0.0" }, - { name = "diffusers", marker = "extra == 'diffusion'", specifier = ">=0.37.0" }, - { name = "flash-attn", marker = "extra == 'fa'", specifier = "<=2.8.3" }, - { name = "flash-linear-attention", marker = "extra == 'fla'", specifier = ">=0.4.2" }, - { name = "flashoptim", specifier = ">=0.1.3" }, - { name = "ftfy", marker = "extra == 'diffusion'" }, - { name = "imageio", marker = "extra == 'diffusion'" }, - { name = "imageio-ffmpeg", marker = "extra == 'diffusion'" }, - { name = "kernels", marker = "extra == 'diffusion-kernels'" }, - { name = "mamba-ssm", marker = "extra == 'cuda'" }, - { name = "megatron-fsdp", specifier = ">=0.2.3" }, - { name = "mistral-common", extras = ["audio", "hf-hub", "image", "sentencepiece"] }, - { name = "mistral-common", extras = ["opencv"], marker = "extra == 'vlm'", specifier = ">=1.11.0" }, - { name = "mlflow" }, - { name = "nemo-automodel", extras = ["cuda"], marker = "extra == 'all'" }, - { name = "nemo-automodel", extras = ["cuda"], marker = "extra == 'moe'" }, - { name = "nemo-automodel", extras = ["delta-databricks"], marker = "extra == 'all'" }, - { name = "nemo-automodel", extras = ["diffusion"], marker = "extra == 'all'" }, - { name = "nemo-automodel", extras = ["diffusion"], marker = "extra == 'diffusion-kernels'" }, - { name = "nemo-automodel", extras = ["extra"], marker = "extra == 'all'" }, - { name = "nemo-automodel", extras = ["fla"], marker = "extra == 'all'" }, - { name = "nemo-automodel", extras = ["fla"], marker = "extra == 'moe'" }, - { name = "nemo-automodel", extras = ["vlm"], marker = "extra == 'all'" }, - { name = "numba", marker = "extra == 'vlm'" }, - { name = "numpy", marker = "extra == 'vlm'" }, - { name = "nv-grouped-gemm", marker = "extra == 'cuda'" }, - { name = "onnxscript", marker = "extra == 'cuda'", specifier = ">=0.5.6" }, - { name = "open-clip-torch", marker = "extra == 'vlm'" }, - { name = "opencv-python-headless", specifier = "==4.10.0.84" }, - { name = "opencv-python-headless", marker = "extra == 'diffusion'" }, - { name = "perceptron", marker = "extra == 'extra'" }, - { name = "pillow", marker = "extra == 'vlm'" }, - { name = "pybind11" }, - { name = "pyyaml" }, - { name = "pyyaml", marker = "extra == 'cli'" }, - { name = "qwen-omni-utils", marker = "extra == 'vlm'" }, - { name = "qwen-vl-utils", extras = ["decord"], marker = "platform_machine == 'x86_64' and sys_platform != 'darwin' and extra == 'vlm'" }, - { name = "sentencepiece", marker = "extra == 'extra'" }, - { name = "tiktoken" }, - { name = "timm", marker = "extra == 'vlm'", specifier = "<=1.0.22" }, - { name = "torch", marker = "sys_platform != 'darwin' and sys_platform != 'linux'", specifier = ">=2.6.0", index = "https://download.pytorch.org/whl/cpu" }, - { name = "torch", marker = "sys_platform == 'darwin'", specifier = ">=2.6.0", index = "https://pypi.org/simple" }, - { name = "torch", marker = "sys_platform == 'linux'", specifier = ">=2.6.0", index = "https://download.pytorch.org/whl/cu129" }, - { name = "torchao" }, - { name = "torchcodec", marker = "platform_machine == 'x86_64' and sys_platform != 'darwin' and extra == 'vlm'" }, - { name = "torchdata" }, - { name = "torchvision", marker = "sys_platform == 'darwin' and extra == 'diffusion'", index = "https://pypi.org/simple" }, - { name = "torchvision", marker = "sys_platform != 'darwin' and sys_platform != 'linux' and extra == 'diffusion'", index = "https://download.pytorch.org/whl/cpu" }, - { name = "torchvision", marker = "sys_platform == 'linux' and extra == 'diffusion'", index = "https://download.pytorch.org/whl/cu129" }, - { name = "transformer-engine", extras = ["pytorch"], marker = "extra == 'cuda'", specifier = "<=2.11.0" }, - { name = "transformers", specifier = "==5.5.0" }, - { name = "wandb" }, -] -provides-extras = ["diffusion", "diffusion-kernels", "cuda", "cuda-source", "extra", "fa", "fla", "delta-databricks", "moe", "vlm", "cli", "all"] - -[package.metadata.requires-dev] -build = [ - { name = "packaging" }, - { name = "psutil" }, - { name = "setuptools" }, - { name = "torch", marker = "sys_platform != 'darwin' and sys_platform != 'linux'", specifier = "<=2.10.0", index = "https://download.pytorch.org/whl/cpu" }, - { name = "torch", marker = "sys_platform == 'darwin'", specifier = "<=2.10.0", index = "https://pypi.org/simple" }, - { name = "torch", marker = "sys_platform == 'linux'", specifier = "<=2.10.0", index = "https://download.pytorch.org/whl/cu129" }, -] -dev = [ - { name = "cut-cross-entropy", git = "https://github.com/apple/ml-cross-entropy.git?rev=87a86aba72cfd2f0d8abecaf81c13c4528ea07d8" }, - { name = "dion", marker = "platform_machine == 'x86_64' and sys_platform != 'darwin'", git = "https://github.com/microsoft/dion.git" }, - { name = "liger-kernel", marker = "platform_machine == 'x86_64' and sys_platform != 'darwin'", git = "https://github.com/linkedin/Liger-Kernel.git?rev=1f1a4b8d6a6c3c1ddb3573a46480c41cef25bde6" }, -] -docs = [ - { name = "myst-parser" }, - { name = "nvidia-sphinx-theme" }, - { name = "sphinx" }, - { name = "sphinx-autobuild" }, - { name = "sphinx-autodoc2" }, - { name = "sphinx-copybutton" }, - { name = "sphinx-design" }, -] -linting = [ - { name = "import-linter", specifier = "~=2.4" }, - { name = "pre-commit", specifier = ">=4.2.0" }, - { name = "ruff", specifier = "~=0.9.0" }, - { name = "ty", specifier = ">=0.0.20" }, -] -test = [ - { name = "coverage" }, - { name = "onnxruntime-gpu", marker = "platform_machine == 'x86_64' and sys_platform != 'darwin'", index = "https://aiinfra.pkgs.visualstudio.com/PublicPackages/_packaging/onnxruntime-cuda-13/pypi/simple/" }, - { name = "peft", specifier = ">=0.18.1" }, - { name = "pytest" }, + { name = "transformer-engine", marker = "extra == 'extra-7-nemo-rl-automodel' or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm')" }, ] [[package]] @@ -4130,13 +3512,12 @@ dependencies = [ { name = "anthropic" }, { name = "datasets" }, { name = "devtools" }, - { name = "fastapi", version = "0.136.3", source = { registry = "https://pypi.org/simple" }, marker = "(extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-8-nemo-gym-vllm') or (extra != 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-vllm') or (extra != 'extra-7-nemo-rl-automodel' and extra == 'extra-8-nemo-gym-vllm') or (extra != 'extra-7-nemo-rl-fsdp' and extra != 'extra-7-nemo-rl-mcore' and extra != 'extra-7-nemo-rl-sglang' and extra != 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra != 'extra-7-nemo-rl-fsdp' and extra != 'extra-7-nemo-rl-mcore' and extra != 'extra-7-nemo-rl-sglang' and extra != 'extra-7-nemo-rl-trtllm' and extra == 'extra-8-nemo-gym-vllm')" }, - { name = "fastapi", version = "0.139.2", source = { registry = "https://pypi.org/simple" }, marker = "extra == 'extra-7-nemo-rl-automodel' or extra == 'extra-7-nemo-rl-mcore' or extra == 'extra-7-nemo-rl-sglang' or extra == 'extra-7-nemo-rl-trtllm' or (extra == 'extra-7-nemo-rl-vllm' and extra == 'extra-8-nemo-gym-vllm') or (extra != 'extra-7-nemo-rl-vllm' and extra != 'extra-8-nemo-gym-vllm')" }, + { name = "fastapi", version = "0.136.3", source = { registry = "https://pypi.org/simple" }, marker = "(extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra != 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-vllm') or (extra != 'extra-7-nemo-rl-fsdp' and extra != 'extra-7-nemo-rl-mcore' and extra != 'extra-7-nemo-rl-sglang' and extra != 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm')" }, + { name = "fastapi", version = "0.139.2", source = { registry = "https://pypi.org/simple" }, marker = "extra == 'extra-7-nemo-rl-automodel' or extra == 'extra-7-nemo-rl-mcore' or extra == 'extra-7-nemo-rl-sglang' or extra == 'extra-7-nemo-rl-trtllm' or extra != 'extra-7-nemo-rl-vllm'" }, { name = "fonttools" }, { name = "gitpython" }, - { name = "gprof2dot" }, - { name = "hydra-core", version = "1.3.2", source = { registry = "https://pypi.org/simple" }, marker = "extra == 'extra-7-nemo-rl-mcore' or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-vllm' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-mcore' and extra == 'extra-8-nemo-gym-vllm')" }, - { name = "hydra-core", version = "1.3.4", source = { registry = "https://pypi.org/simple" }, marker = "extra == 'extra-7-nemo-rl-automodel' or extra != 'extra-7-nemo-rl-mcore' or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-8-nemo-gym-vllm')" }, + { name = "hydra-core", version = "1.3.2", source = { registry = "https://pypi.org/simple" }, marker = "extra == 'extra-7-nemo-rl-mcore' or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm')" }, + { name = "hydra-core", version = "1.3.4", source = { registry = "https://pypi.org/simple" }, marker = "extra == 'extra-7-nemo-rl-automodel' or extra != 'extra-7-nemo-rl-mcore' or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm')" }, { name = "itsdangerous" }, { name = "mcp" }, { name = "mlflow" }, @@ -4144,6 +3525,7 @@ dependencies = [ { name = "omegaconf" }, { name = "openai" }, { name = "orjson" }, + { name = "packaging" }, { name = "psutil" }, { name = "pyarrow" }, { name = "pyasn1" }, @@ -4161,99 +3543,6 @@ dependencies = [ { name = "yappi" }, ] -[package.optional-dependencies] -all = [ - { name = "boto3" }, - { name = "coverage" }, - { name = "daytona" }, - { name = "mypy" }, - { name = "opensandbox" }, - { name = "openshell" }, - { name = "pre-commit" }, - { name = "pytest" }, - { name = "pytest-asyncio" }, - { name = "pytest-cov" }, - { name = "pytest-xdist" }, - { name = "requests-mock" }, - { name = "ruff" }, - { name = "tenacity" }, -] -dev = [ - { name = "coverage" }, - { name = "mypy" }, - { name = "pre-commit" }, - { name = "pytest" }, - { name = "pytest-asyncio" }, - { name = "pytest-cov" }, - { name = "pytest-xdist" }, - { name = "requests-mock" }, - { name = "ruff" }, -] -sandbox = [ - { name = "boto3" }, - { name = "daytona" }, - { name = "opensandbox" }, - { name = "openshell" }, - { name = "tenacity" }, -] -vllm = [ - { name = "flashinfer-python", version = "0.6.12", source = { registry = "https://pypi.org/simple" } }, - { name = "vllm", version = "0.24.0", source = { registry = "https://pypi.org/simple" } }, -] - -[package.metadata] -requires-dist = [ - { name = "aiohttp", specifier = ">=3.14.1" }, - { name = "anthropic", specifier = "<=0.109.2" }, - { name = "boto3", marker = "extra == 'sandbox'", specifier = ">=1.34" }, - { name = "coverage", extras = ["toml"], marker = "extra == 'dev'" }, - { name = "datasets" }, - { name = "daytona", marker = "extra == 'sandbox'", specifier = ">=0.179.0" }, - { name = "devtools" }, - { name = "fastapi" }, - { name = "flashinfer-python", marker = "extra == 'vllm'", specifier = "==0.6.12" }, - { name = "fonttools", specifier = ">=4.60.2" }, - { name = "gitpython", specifier = ">=3.1.57" }, - { name = "gprof2dot" }, - { name = "hydra-core" }, - { name = "itsdangerous" }, - { name = "mcp", specifier = ">=1.28.1,<2" }, - { name = "mlflow", specifier = ">=3.15.1" }, - { name = "mlflow-skinny", specifier = ">=3.15.1" }, - { name = "mypy", marker = "extra == 'dev'", specifier = ">=1.8.0" }, - { name = "nemo-gym", extras = ["dev", "sandbox"], marker = "extra == 'all'", editable = "3rdparty/Gym-workspace/Gym" }, - { name = "omegaconf" }, - { name = "openai", specifier = "<=2.7.2" }, - { name = "opensandbox", marker = "extra == 'sandbox'", specifier = ">=0.1.15" }, - { name = "openshell", marker = "extra == 'sandbox'", specifier = ">=0.0.92,<0.1" }, - { name = "orjson" }, - { name = "pre-commit", marker = "extra == 'dev'", specifier = ">=3.6.0" }, - { name = "psutil" }, - { name = "pyarrow", specifier = ">=23.0.1" }, - { name = "pyasn1", specifier = ">=0.6.4" }, - { name = "pydantic" }, - { name = "pydantic-core" }, - { name = "pydot" }, - { name = "pytest", marker = "extra == 'dev'" }, - { name = "pytest-asyncio", marker = "extra == 'dev'" }, - { name = "pytest-cov", marker = "extra == 'dev'" }, - { name = "pytest-xdist", marker = "extra == 'dev'" }, - { name = "python-multipart", specifier = ">=0.0.22" }, - { name = "ray", extras = ["default"], specifier = ">=2.56.1" }, - { name = "requests-mock", marker = "extra == 'dev'" }, - { name = "rich" }, - { name = "ruff", marker = "extra == 'dev'" }, - { name = "tenacity", marker = "extra == 'sandbox'", specifier = ">=9.1.4" }, - { name = "tqdm" }, - { name = "urllib3", specifier = ">=2.7.0" }, - { name = "uvicorn" }, - { name = "uvloop" }, - { name = "vllm", marker = "extra == 'vllm'", specifier = "==0.24.0" }, - { name = "wandb" }, - { name = "yappi" }, -] -provides-extras = ["all", "vllm", "sandbox", "dev"] - [[package]] name = "nemo-rl" source = { editable = "." } @@ -4266,8 +3555,8 @@ dependencies = [ { name = "datasets" }, { name = "debugpy" }, { name = "fastokens-b10" }, - { name = "hydra-core", version = "1.3.2", source = { registry = "https://pypi.org/simple" }, marker = "extra == 'extra-7-nemo-rl-mcore' or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-vllm' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-mcore' and extra == 'extra-8-nemo-gym-vllm')" }, - { name = "hydra-core", version = "1.3.4", source = { registry = "https://pypi.org/simple" }, marker = "extra == 'extra-7-nemo-rl-automodel' or extra != 'extra-7-nemo-rl-mcore' or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-8-nemo-gym-vllm')" }, + { name = "hydra-core", version = "1.3.2", source = { registry = "https://pypi.org/simple" }, marker = "extra == 'extra-7-nemo-rl-mcore' or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm')" }, + { name = "hydra-core", version = "1.3.4", source = { registry = "https://pypi.org/simple" }, marker = "extra == 'extra-7-nemo-rl-automodel' or extra != 'extra-7-nemo-rl-mcore' or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm')" }, { name = "math-verify" }, { name = "matplotlib" }, { name = "mlflow" }, @@ -4298,17 +3587,17 @@ dependencies = [ { name = "tensorboard" }, { name = "tensordict" }, { name = "tiktoken" }, - { name = "tilelang", version = "0.1.8", source = { registry = "https://pypi.org/simple" }, marker = "(platform_machine == 'x86_64' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-vllm' and extra == 'extra-8-nemo-gym-vllm')" }, - { name = "tilelang", version = "0.1.9", source = { registry = "https://pypi.org/simple" }, marker = "(platform_machine == 'x86_64' and extra == 'extra-7-nemo-rl-automodel') or (platform_machine == 'x86_64' and extra == 'extra-7-nemo-rl-fsdp') or (platform_machine == 'x86_64' and extra == 'extra-7-nemo-rl-mcore') or (platform_machine == 'x86_64' and extra != 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-vllm' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-8-nemo-gym-vllm')" }, + { name = "tilelang", version = "0.1.8", source = { registry = "https://pypi.org/simple" }, marker = "(platform_machine == 'x86_64' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm')" }, + { name = "tilelang", version = "0.1.9", source = { registry = "https://pypi.org/simple" }, marker = "(platform_machine == 'x86_64' and extra == 'extra-7-nemo-rl-automodel') or (platform_machine == 'x86_64' and extra == 'extra-7-nemo-rl-fsdp') or (platform_machine == 'x86_64' and extra == 'extra-7-nemo-rl-mcore') or (platform_machine == 'x86_64' and extra != 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm')" }, { name = "timm" }, { name = "torch" }, { name = "torchdata" }, { name = "torchvision" }, { name = "transferqueue" }, - { name = "transformers", version = "5.5.0", source = { registry = "https://pypi.org/simple" }, marker = "extra == 'extra-7-nemo-rl-automodel' or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-vllm' and extra == 'extra-8-nemo-gym-vllm')" }, - { name = "transformers", version = "5.5.4", source = { registry = "https://pypi.org/simple" }, marker = "extra == 'extra-7-nemo-rl-trtllm' or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-vllm' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-trtllm' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-mcore' and extra != 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-mcore' and extra != 'extra-7-nemo-rl-trtllm' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra != 'extra-7-nemo-rl-mcore' and extra != 'extra-7-nemo-rl-trtllm' and extra != 'extra-7-nemo-rl-vllm' and extra != 'extra-8-nemo-gym-vllm') or (extra != 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-mcore' and extra != 'extra-7-nemo-rl-sglang' and extra != 'extra-7-nemo-rl-vllm' and extra != 'extra-8-nemo-gym-vllm')" }, - { name = "transformers", version = "5.6.0", source = { registry = "https://pypi.org/simple" }, marker = "(extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-8-nemo-gym-vllm') or (extra != 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra != 'extra-7-nemo-rl-fsdp' and extra != 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-8-nemo-gym-vllm') or (extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-8-nemo-gym-vllm') or (extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm' and extra == 'extra-8-nemo-gym-vllm')" }, - { name = "transformers", version = "5.8.1", source = { registry = "https://pypi.org/simple" }, marker = "extra == 'extra-7-nemo-rl-mcore' or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-8-nemo-gym-vllm') or (extra != 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-vllm') or (extra != 'extra-7-nemo-rl-automodel' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra != 'extra-7-nemo-rl-fsdp' and extra != 'extra-7-nemo-rl-sglang' and extra != 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra != 'extra-7-nemo-rl-fsdp' and extra != 'extra-7-nemo-rl-sglang' and extra != 'extra-7-nemo-rl-trtllm' and extra == 'extra-8-nemo-gym-vllm')" }, + { name = "transformers", version = "5.5.0", source = { registry = "https://pypi.org/simple" }, marker = "extra == 'extra-7-nemo-rl-automodel' or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm')" }, + { name = "transformers", version = "5.5.4", source = { registry = "https://pypi.org/simple" }, marker = "extra == 'extra-7-nemo-rl-trtllm' or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-mcore' and extra != 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra != 'extra-7-nemo-rl-mcore' and extra != 'extra-7-nemo-rl-trtllm' and extra != 'extra-7-nemo-rl-vllm') or (extra != 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-mcore' and extra != 'extra-7-nemo-rl-sglang' and extra != 'extra-7-nemo-rl-vllm')" }, + { name = "transformers", version = "5.6.0", source = { registry = "https://pypi.org/simple" }, marker = "(extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra != 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra != 'extra-7-nemo-rl-fsdp' and extra != 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm')" }, + { name = "transformers", version = "5.8.1", source = { registry = "https://pypi.org/simple" }, marker = "extra == 'extra-7-nemo-rl-mcore' or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra != 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra != 'extra-7-nemo-rl-fsdp' and extra != 'extra-7-nemo-rl-sglang' and extra != 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm')" }, { name = "triton" }, { name = "wandb" }, { name = "zstandard" }, @@ -4317,35 +3606,35 @@ dependencies = [ [package.optional-dependencies] automodel = [ { name = "causal-conv1d" }, - { name = "deep-ep", version = "1.2.1+29d31c0", source = { git = "https://github.com/deepseek-ai/DeepEP.git?rev=29d31c095796f3c8ece47ee9cdcc167051bbeed9#29d31c095796f3c8ece47ee9cdcc167051bbeed9" }, marker = "(platform_machine == 'x86_64' and extra == 'extra-7-nemo-rl-automodel') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-vllm' and extra == 'extra-8-nemo-gym-vllm')" }, - { name = "deep-ep", version = "1.2.1+29d31c0", source = { git = "https://github.com/deepseek-ai/DeepEP.git?rev=a48493600c4886c1b297aaa78db0e1ebc2d8dd6c#a48493600c4886c1b297aaa78db0e1ebc2d8dd6c" }, marker = "(platform_machine != 'x86_64' and extra == 'extra-7-nemo-rl-automodel') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-vllm' and extra == 'extra-8-nemo-gym-vllm')" }, - { name = "flash-attn", version = "2.8.1+cu13torch2.10cxx11abitrue", source = { url = "https://github.com/Dao-AILab/flash-attention/releases/download/v2.8.1/flash_attn-2.8.1+cu13torch2.10cxx11abiTRUE-cp313-cp313-linux_aarch64.whl" }, marker = "(platform_machine != 'x86_64' and extra == 'extra-7-nemo-rl-automodel') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-vllm' and extra == 'extra-8-nemo-gym-vllm')" }, - { name = "flash-attn", version = "2.8.1+cu13torch2.10cxx11abitrue", source = { url = "https://github.com/Dao-AILab/flash-attention/releases/download/v2.8.1/flash_attn-2.8.1+cu13torch2.10cxx11abiTRUE-cp313-cp313-linux_x86_64.whl" }, marker = "(platform_machine == 'x86_64' and extra == 'extra-7-nemo-rl-automodel') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-vllm' and extra == 'extra-8-nemo-gym-vllm')" }, + { name = "deep-ep", version = "1.2.1+29d31c0", source = { git = "https://github.com/deepseek-ai/DeepEP.git?rev=29d31c095796f3c8ece47ee9cdcc167051bbeed9#29d31c095796f3c8ece47ee9cdcc167051bbeed9" }, marker = "(platform_machine == 'x86_64' and extra == 'extra-7-nemo-rl-automodel') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm')" }, + { name = "deep-ep", version = "1.2.1+29d31c0", source = { git = "https://github.com/deepseek-ai/DeepEP.git?rev=a48493600c4886c1b297aaa78db0e1ebc2d8dd6c#a48493600c4886c1b297aaa78db0e1ebc2d8dd6c" }, marker = "(platform_machine != 'x86_64' and extra == 'extra-7-nemo-rl-automodel') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm')" }, + { name = "flash-attn", version = "2.8.1+cu13torch2.10cxx11abitrue", source = { url = "https://github.com/Dao-AILab/flash-attention/releases/download/v2.8.1/flash_attn-2.8.1+cu13torch2.10cxx11abiTRUE-cp313-cp313-linux_aarch64.whl" }, marker = "(platform_machine != 'x86_64' and extra == 'extra-7-nemo-rl-automodel') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm')" }, + { name = "flash-attn", version = "2.8.1+cu13torch2.10cxx11abitrue", source = { url = "https://github.com/Dao-AILab/flash-attention/releases/download/v2.8.1/flash_attn-2.8.1+cu13torch2.10cxx11abiTRUE-cp313-cp313-linux_x86_64.whl" }, marker = "(platform_machine == 'x86_64' and extra == 'extra-7-nemo-rl-automodel') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm')" }, { name = "mamba-ssm" }, { name = "mistral-common" }, - { name = "nemo-automodel", extra = ["moe"], marker = "extra == 'extra-7-nemo-rl-automodel' or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-vllm' and extra == 'extra-8-nemo-gym-vllm')" }, + { name = "nemo-automodel", extra = ["moe"], marker = "extra == 'extra-7-nemo-rl-automodel' or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm')" }, { name = "nv-grouped-gemm" }, - { name = "transformer-engine", marker = "extra == 'extra-7-nemo-rl-automodel' or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-vllm' and extra == 'extra-8-nemo-gym-vllm')" }, + { name = "transformer-engine", marker = "extra == 'extra-7-nemo-rl-automodel' or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm')" }, { name = "transformers", version = "5.5.0", source = { registry = "https://pypi.org/simple" } }, ] fsdp = [ { name = "causal-conv1d" }, - { name = "flash-attn", version = "2.8.1+cu13torch2.10cxx11abitrue", source = { url = "https://github.com/Dao-AILab/flash-attention/releases/download/v2.8.1/flash_attn-2.8.1+cu13torch2.10cxx11abiTRUE-cp313-cp313-linux_aarch64.whl" }, marker = "(platform_machine != 'x86_64' and extra == 'extra-7-nemo-rl-fsdp') or (platform_machine != 'x86_64' and extra == 'extra-7-nemo-rl-mcore') or (platform_machine != 'x86_64' and extra != 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-vllm' and extra == 'extra-8-nemo-gym-vllm')" }, - { name = "flash-attn", version = "2.8.1+cu13torch2.10cxx11abitrue", source = { url = "https://github.com/Dao-AILab/flash-attention/releases/download/v2.8.1/flash_attn-2.8.1+cu13torch2.10cxx11abiTRUE-cp313-cp313-linux_x86_64.whl" }, marker = "(platform_machine == 'x86_64' and extra == 'extra-7-nemo-rl-fsdp') or (platform_machine == 'x86_64' and extra == 'extra-7-nemo-rl-mcore') or (platform_machine == 'x86_64' and extra != 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-vllm' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-8-nemo-gym-vllm')" }, + { name = "flash-attn", version = "2.8.1+cu13torch2.10cxx11abitrue", source = { url = "https://github.com/Dao-AILab/flash-attention/releases/download/v2.8.1/flash_attn-2.8.1+cu13torch2.10cxx11abiTRUE-cp313-cp313-linux_aarch64.whl" }, marker = "(platform_machine != 'x86_64' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm')" }, + { name = "flash-attn", version = "2.8.1+cu13torch2.10cxx11abitrue", source = { url = "https://github.com/Dao-AILab/flash-attention/releases/download/v2.8.1/flash_attn-2.8.1+cu13torch2.10cxx11abiTRUE-cp313-cp313-linux_x86_64.whl" }, marker = "(platform_machine == 'x86_64' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm')" }, { name = "mamba-ssm" }, ] mcore = [ { name = "causal-conv1d" }, { name = "cupy-cuda13x" }, - { name = "deep-ep", version = "1.2.1+29d31c0", source = { git = "https://github.com/deepseek-ai/DeepEP.git?rev=29d31c095796f3c8ece47ee9cdcc167051bbeed9#29d31c095796f3c8ece47ee9cdcc167051bbeed9" }, marker = "(platform_machine == 'x86_64' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-vllm' and extra == 'extra-8-nemo-gym-vllm')" }, - { name = "deep-ep", version = "1.2.1+29d31c0", source = { git = "https://github.com/deepseek-ai/DeepEP.git?rev=a48493600c4886c1b297aaa78db0e1ebc2d8dd6c#a48493600c4886c1b297aaa78db0e1ebc2d8dd6c" }, marker = "(platform_machine != 'x86_64' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-vllm' and extra == 'extra-8-nemo-gym-vllm')" }, - { name = "flash-attn", version = "2.8.1+cu13torch2.10cxx11abitrue", source = { url = "https://github.com/Dao-AILab/flash-attention/releases/download/v2.8.1/flash_attn-2.8.1+cu13torch2.10cxx11abiTRUE-cp313-cp313-linux_aarch64.whl" }, marker = "(platform_machine != 'x86_64' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-vllm' and extra == 'extra-8-nemo-gym-vllm')" }, - { name = "flash-attn", version = "2.8.1+cu13torch2.10cxx11abitrue", source = { url = "https://github.com/Dao-AILab/flash-attention/releases/download/v2.8.1/flash_attn-2.8.1+cu13torch2.10cxx11abiTRUE-cp313-cp313-linux_x86_64.whl" }, marker = "(platform_machine == 'x86_64' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-vllm' and extra == 'extra-8-nemo-gym-vllm')" }, + { name = "deep-ep", version = "1.2.1+29d31c0", source = { git = "https://github.com/deepseek-ai/DeepEP.git?rev=29d31c095796f3c8ece47ee9cdcc167051bbeed9#29d31c095796f3c8ece47ee9cdcc167051bbeed9" }, marker = "(platform_machine == 'x86_64' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm')" }, + { name = "deep-ep", version = "1.2.1+29d31c0", source = { git = "https://github.com/deepseek-ai/DeepEP.git?rev=a48493600c4886c1b297aaa78db0e1ebc2d8dd6c#a48493600c4886c1b297aaa78db0e1ebc2d8dd6c" }, marker = "(platform_machine != 'x86_64' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm')" }, + { name = "flash-attn", version = "2.8.1+cu13torch2.10cxx11abitrue", source = { url = "https://github.com/Dao-AILab/flash-attention/releases/download/v2.8.1/flash_attn-2.8.1+cu13torch2.10cxx11abiTRUE-cp313-cp313-linux_aarch64.whl" }, marker = "(platform_machine != 'x86_64' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm')" }, + { name = "flash-attn", version = "2.8.1+cu13torch2.10cxx11abitrue", source = { url = "https://github.com/Dao-AILab/flash-attention/releases/download/v2.8.1/flash_attn-2.8.1+cu13torch2.10cxx11abiTRUE-cp313-cp313-linux_x86_64.whl" }, marker = "(platform_machine == 'x86_64' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm')" }, { name = "flashinfer-cubin", version = "0.6.8.post1", source = { registry = "https://pypi.org/simple" } }, { name = "flashinfer-jit-cache", version = "0.6.8.post1+cu130", source = { registry = "https://flashinfer.ai/whl/cu130" } }, { name = "flashinfer-python", version = "0.6.8.post1", source = { registry = "https://pypi.org/simple" } }, { name = "mamba-ssm" }, - { name = "megatron-bridge", extra = ["ssm", "te"], marker = "extra == 'extra-7-nemo-rl-mcore' or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-vllm' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-mcore' and extra == 'extra-8-nemo-gym-vllm')" }, + { name = "megatron-bridge", extra = ["ssm", "te"], marker = "extra == 'extra-7-nemo-rl-mcore' or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm')" }, { name = "nvshmem4py-cu13" }, ] modelopt = [ @@ -4372,16 +3661,16 @@ trtllm = [ ] vllm = [ { name = "cuda-python" }, - { name = "deep-ep", version = "1.2.1+29d31c0", source = { git = "https://github.com/deepseek-ai/DeepEP.git?rev=29d31c095796f3c8ece47ee9cdcc167051bbeed9#29d31c095796f3c8ece47ee9cdcc167051bbeed9" }, marker = "(platform_machine == 'x86_64' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-vllm' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-8-nemo-gym-vllm')" }, - { name = "deep-ep", version = "1.2.1+29d31c0", source = { git = "https://github.com/deepseek-ai/DeepEP.git?rev=a48493600c4886c1b297aaa78db0e1ebc2d8dd6c#a48493600c4886c1b297aaa78db0e1ebc2d8dd6c" }, marker = "(platform_machine != 'x86_64' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-vllm' and extra == 'extra-8-nemo-gym-vllm')" }, + { name = "deep-ep", version = "1.2.1+29d31c0", source = { git = "https://github.com/deepseek-ai/DeepEP.git?rev=29d31c095796f3c8ece47ee9cdcc167051bbeed9#29d31c095796f3c8ece47ee9cdcc167051bbeed9" }, marker = "(platform_machine == 'x86_64' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm')" }, + { name = "deep-ep", version = "1.2.1+29d31c0", source = { git = "https://github.com/deepseek-ai/DeepEP.git?rev=a48493600c4886c1b297aaa78db0e1ebc2d8dd6c#a48493600c4886c1b297aaa78db0e1ebc2d8dd6c" }, marker = "(platform_machine != 'x86_64' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm')" }, { name = "deep-gemm" }, { name = "flashinfer-cubin", version = "0.6.13", source = { registry = "https://pypi.org/simple" } }, { name = "flashinfer-jit-cache", version = "0.6.13+cu130", source = { registry = "https://flashinfer.ai/whl/cu130" } }, { name = "flashinfer-python", version = "0.6.13", source = { registry = "https://pypi.org/simple" } }, { name = "num2words" }, - { name = "nvidia-cutlass-dsl", version = "4.5.2", source = { registry = "https://pypi.org/simple" }, extra = ["cu13"], marker = "extra == 'extra-7-nemo-rl-vllm' or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-vllm' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-mcore' and extra != 'extra-7-nemo-rl-vllm' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra != 'extra-7-nemo-rl-vllm' and extra == 'extra-8-nemo-gym-vllm')" }, - { name = "vllm", version = "0.25.1", source = { url = "https://github.com/vllm-project/vllm/releases/download/v0.25.1/vllm-0.25.1-cp38-abi3-manylinux_2_28_aarch64.whl" }, marker = "(platform_machine != 'x86_64' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-vllm' and extra == 'extra-8-nemo-gym-vllm')" }, - { name = "vllm", version = "0.25.1", source = { url = "https://github.com/vllm-project/vllm/releases/download/v0.25.1/vllm-0.25.1-cp38-abi3-manylinux_2_28_x86_64.whl" }, marker = "(platform_machine == 'x86_64' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-vllm' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-8-nemo-gym-vllm')" }, + { name = "nvidia-cutlass-dsl", version = "4.5.2", source = { registry = "https://pypi.org/simple" }, extra = ["cu13"], marker = "(extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra != 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-vllm') or (extra != 'extra-7-nemo-rl-fsdp' and extra != 'extra-7-nemo-rl-mcore' and extra != 'extra-7-nemo-rl-sglang' and extra != 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm')" }, + { name = "vllm", version = "0.25.1", source = { url = "https://github.com/vllm-project/vllm/releases/download/v0.25.1/vllm-0.25.1-cp38-abi3-manylinux_2_28_aarch64.whl" }, marker = "(platform_machine != 'x86_64' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm')" }, + { name = "vllm", version = "0.25.1", source = { url = "https://github.com/vllm-project/vllm/releases/download/v0.25.1/vllm-0.25.1-cp38-abi3-manylinux_2_28_x86_64.whl" }, marker = "(platform_machine == 'x86_64' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm')" }, ] [package.dev-dependencies] @@ -4423,156 +3712,6 @@ test = [ { name = "pytest-timeout" }, ] -[package.metadata] -requires-dist = [ - { name = "accelerate", specifier = ">=0.26" }, - { name = "awscrt", specifier = ">=0.35.0" }, - { name = "blobfile" }, - { name = "causal-conv1d", marker = "extra == 'automodel'", git = "https://github.com/Dao-AILab/causal-conv1d?rev=4f6ae4e26ae5fe8af9372f8d312ab25cc4595223" }, - { name = "causal-conv1d", marker = "extra == 'fsdp'", git = "https://github.com/Dao-AILab/causal-conv1d?rev=4f6ae4e26ae5fe8af9372f8d312ab25cc4595223" }, - { name = "causal-conv1d", marker = "extra == 'mcore'", git = "https://github.com/Dao-AILab/causal-conv1d?rev=4f6ae4e26ae5fe8af9372f8d312ab25cc4595223" }, - { name = "colored", specifier = "==2.2.3" }, - { name = "cuda-bindings", marker = "sys_platform != 'darwin'" }, - { name = "cuda-python", marker = "extra == 'vllm'" }, - { name = "cupy-cuda13x", marker = "extra == 'mcore'" }, - { name = "datasets", specifier = ">=4.0.0" }, - { name = "debugpy" }, - { name = "deep-ep", marker = "platform_machine == 'aarch64' and extra == 'automodel'", git = "https://github.com/deepseek-ai/DeepEP.git?rev=a48493600c4886c1b297aaa78db0e1ebc2d8dd6c" }, - { name = "deep-ep", marker = "platform_machine == 'aarch64' and extra == 'mcore'", git = "https://github.com/deepseek-ai/DeepEP.git?rev=a48493600c4886c1b297aaa78db0e1ebc2d8dd6c" }, - { name = "deep-ep", marker = "platform_machine == 'aarch64' and extra == 'vllm'", git = "https://github.com/deepseek-ai/DeepEP.git?rev=a48493600c4886c1b297aaa78db0e1ebc2d8dd6c" }, - { name = "deep-ep", marker = "platform_machine == 'x86_64' and extra == 'automodel'", git = "https://github.com/deepseek-ai/DeepEP.git?rev=29d31c095796f3c8ece47ee9cdcc167051bbeed9" }, - { name = "deep-ep", marker = "platform_machine == 'x86_64' and extra == 'mcore'", git = "https://github.com/deepseek-ai/DeepEP.git?rev=29d31c095796f3c8ece47ee9cdcc167051bbeed9" }, - { name = "deep-ep", marker = "platform_machine == 'x86_64' and extra == 'vllm'", git = "https://github.com/deepseek-ai/DeepEP.git?rev=29d31c095796f3c8ece47ee9cdcc167051bbeed9" }, - { name = "deep-gemm", marker = "extra == 'vllm'", git = "https://github.com/deepseek-ai/DeepGEMM.git?rev=67fc64863d43521080bf2005e6528d0fceee9510" }, - { name = "fastokens-b10", specifier = ">=0.1.1" }, - { name = "flash-attn", marker = "platform_machine == 'aarch64' and sys_platform == 'linux' and extra == 'automodel'", url = "https://github.com/Dao-AILab/flash-attention/releases/download/v2.8.1/flash_attn-2.8.1+cu13torch2.10cxx11abiTRUE-cp313-cp313-linux_aarch64.whl" }, - { name = "flash-attn", marker = "platform_machine == 'aarch64' and sys_platform == 'linux' and extra == 'fsdp'", url = "https://github.com/Dao-AILab/flash-attention/releases/download/v2.8.1/flash_attn-2.8.1+cu13torch2.10cxx11abiTRUE-cp313-cp313-linux_aarch64.whl" }, - { name = "flash-attn", marker = "platform_machine == 'aarch64' and sys_platform == 'linux' and extra == 'mcore'", url = "https://github.com/Dao-AILab/flash-attention/releases/download/v2.8.1/flash_attn-2.8.1+cu13torch2.10cxx11abiTRUE-cp313-cp313-linux_aarch64.whl" }, - { name = "flash-attn", marker = "platform_machine == 'x86_64' and sys_platform == 'linux' and extra == 'automodel'", url = "https://github.com/Dao-AILab/flash-attention/releases/download/v2.8.1/flash_attn-2.8.1+cu13torch2.10cxx11abiTRUE-cp313-cp313-linux_x86_64.whl" }, - { name = "flash-attn", marker = "platform_machine == 'x86_64' and sys_platform == 'linux' and extra == 'fsdp'", url = "https://github.com/Dao-AILab/flash-attention/releases/download/v2.8.1/flash_attn-2.8.1+cu13torch2.10cxx11abiTRUE-cp313-cp313-linux_x86_64.whl" }, - { name = "flash-attn", marker = "platform_machine == 'x86_64' and sys_platform == 'linux' and extra == 'mcore'", url = "https://github.com/Dao-AILab/flash-attention/releases/download/v2.8.1/flash_attn-2.8.1+cu13torch2.10cxx11abiTRUE-cp313-cp313-linux_x86_64.whl" }, - { name = "flash-attn", marker = "(platform_machine != 'aarch64' and platform_machine != 'x86_64' and extra == 'automodel') or (sys_platform != 'linux' and extra == 'automodel')", specifier = "==2.8.1" }, - { name = "flash-attn", marker = "(platform_machine != 'aarch64' and platform_machine != 'x86_64' and extra == 'fsdp') or (sys_platform != 'linux' and extra == 'fsdp')", specifier = "==2.8.1" }, - { name = "flash-attn", marker = "(platform_machine != 'aarch64' and platform_machine != 'x86_64' and extra == 'mcore') or (sys_platform != 'linux' and extra == 'mcore')", specifier = "==2.8.1" }, - { name = "flashinfer-cubin", marker = "extra == 'mcore'", specifier = "==0.6.8.post1" }, - { name = "flashinfer-cubin", marker = "extra == 'sglang'", specifier = "==0.6.11.post1" }, - { name = "flashinfer-cubin", marker = "extra == 'vllm'", specifier = "==0.6.13" }, - { name = "flashinfer-jit-cache", marker = "extra == 'mcore'", specifier = "==0.6.8.post1", index = "https://flashinfer.ai/whl/cu130" }, - { name = "flashinfer-jit-cache", marker = "extra == 'sglang'", specifier = "==0.6.11.post1", index = "https://flashinfer.ai/whl/cu130" }, - { name = "flashinfer-jit-cache", marker = "extra == 'vllm'", specifier = "==0.6.13", index = "https://flashinfer.ai/whl/cu130" }, - { name = "flashinfer-python", marker = "extra == 'mcore'", specifier = "==0.6.8.post1" }, - { name = "flashinfer-python", marker = "extra == 'sglang'", specifier = "==0.6.11.post1" }, - { name = "flashinfer-python", marker = "extra == 'vllm'", specifier = "==0.6.13" }, - { name = "hydra-core" }, - { name = "kernels", marker = "extra == 'sglang'", specifier = ">=0.12.0,<0.13" }, - { name = "mamba-ssm", marker = "extra == 'automodel'", git = "https://github.com/state-spaces/mamba.git?rev=a14b1dff0454a3bc27d9eb31355dc01e4b2490ec" }, - { name = "mamba-ssm", marker = "extra == 'fsdp'", git = "https://github.com/state-spaces/mamba.git?rev=a14b1dff0454a3bc27d9eb31355dc01e4b2490ec" }, - { name = "mamba-ssm", marker = "extra == 'mcore'", git = "https://github.com/state-spaces/mamba.git?rev=a14b1dff0454a3bc27d9eb31355dc01e4b2490ec" }, - { name = "math-verify" }, - { name = "matplotlib" }, - { name = "megatron-bridge", extras = ["ssm", "te"], marker = "extra == 'mcore'", editable = "3rdparty/Megatron-Bridge-workspace/Megatron-Bridge" }, - { name = "mistral-common", marker = "extra == 'automodel'", specifier = ">=1.11.0" }, - { name = "mlflow", specifier = ">=3.13.0" }, - { name = "mooncake-transfer-engine-cuda13", marker = "(platform_machine == 'aarch64' and sys_platform == 'linux') or (platform_machine == 'x86_64' and sys_platform == 'linux')", specifier = "==0.3.11.post1" }, - { name = "nccl4py", marker = "sys_platform != 'darwin'" }, - { name = "nemo-automodel", extras = ["moe"], marker = "extra == 'automodel'", editable = "3rdparty/Automodel-workspace/Automodel" }, - { name = "nemo-gym", marker = "extra == 'nemo-gym'", editable = "3rdparty/Gym-workspace/Gym" }, - { name = "ninja" }, - { name = "nixl", marker = "(platform_machine == 'aarch64' and sys_platform == 'linux') or (platform_machine == 'x86_64' and sys_platform == 'linux')", specifier = "==1.3.0" }, - { name = "num2words", specifier = ">=0.5.14" }, - { name = "num2words", marker = "extra == 'vllm'", specifier = ">=0.5.14" }, - { name = "numpy" }, - { name = "nv-grouped-gemm", marker = "extra == 'automodel'", git = "https://github.com/fanshiqing/grouped_gemm?tag=v1.1.4.post7" }, - { name = "nvidia-cudnn-cu13", marker = "sys_platform != 'darwin'", specifier = "==9.20.0.48" }, - { name = "nvidia-cutlass-dsl", extras = ["cu13"], marker = "extra == 'vllm'", specifier = "==4.5.2" }, - { name = "nvidia-ml-py" }, - { name = "nvidia-modelopt", marker = "extra == 'modelopt'", git = "https://github.com/NVIDIA/Model-Optimizer?rev=c3b913b9cc1d82d5a0af9fa77b4db87829e6f158" }, - { name = "nvidia-nvshmem-cu13", marker = "(platform_machine == 'aarch64' and sys_platform == 'linux') or (platform_machine == 'x86_64' and sys_platform == 'linux')", specifier = ">=3.6.5" }, - { name = "nvidia-resiliency-ext", marker = "extra == 'nvrx'", git = "https://github.com/NVIDIA/nvidia-resiliency-ext.git?tag=v0.6.0" }, - { name = "nvshmem4py-cu13", marker = "extra == 'mcore'", specifier = ">=0.2.1" }, - { name = "nvtx" }, - { name = "omegaconf" }, - { name = "open-clip-torch", specifier = ">=3.2.0" }, - { name = "pillow", specifier = ">=12.3.0" }, - { name = "pip" }, - { name = "plotly" }, - { name = "pybase64" }, - { name = "pyzmq" }, - { name = "ray", extras = ["default"], specifier = ">=2.55.1" }, - { name = "rich" }, - { name = "sentencepiece" }, - { name = "setuptools" }, - { name = "sglang", marker = "extra == 'sglang'", specifier = "==0.5.12.post1" }, - { name = "sglang-kernel", marker = "extra == 'sglang'", specifier = "==0.4.2.post2" }, - { name = "sglang-router", marker = "extra == 'sglang'" }, - { name = "soundfile", specifier = ">=0.13.1" }, - { name = "swanlab" }, - { name = "sympy", specifier = ">=1.14.0" }, - { name = "tensorboard" }, - { name = "tensordict" }, - { name = "tensorrt-llm", marker = "extra == 'trtllm'", directory = "3rdparty/TensorRT-LLM-workspace" }, - { name = "tiktoken" }, - { name = "tilelang", marker = "platform_machine == 'x86_64' and sys_platform == 'linux'" }, - { name = "timm" }, - { name = "torch", marker = "sys_platform != 'darwin'", specifier = "==2.11.0", index = "https://download.pytorch.org/whl/cu130" }, - { name = "torch", marker = "sys_platform == 'darwin'", specifier = "==2.11.0", index = "https://pypi.org/simple" }, - { name = "torchdata" }, - { name = "torchvision", marker = "sys_platform != 'darwin'", specifier = "==0.26.0", index = "https://download.pytorch.org/whl/cu130" }, - { name = "torchvision", marker = "sys_platform == 'darwin'", specifier = "==0.26.0", index = "https://pypi.org/simple" }, - { name = "transferqueue", git = "https://github.com/Ascend/TransferQueue.git?rev=c51614308b68c8d7a87c9b3ef62d59e14c69bde2" }, - { name = "transformer-engine", extras = ["core-cu13", "pytorch"], marker = "extra == 'automodel'", git = "https://github.com/NVIDIA/TransformerEngine.git?rev=v2.14.1" }, - { name = "transformers", specifier = ">=5.5.0,<5.9.0" }, - { name = "transformers", marker = "extra == 'automodel'", specifier = ">=5.5.0,<5.6.0" }, - { name = "transformers", marker = "extra == 'sglang'", specifier = "==5.6.0" }, - { name = "triton", marker = "(platform_machine == 'aarch64' and sys_platform == 'linux') or (platform_machine == 'x86_64' and sys_platform == 'linux')", index = "https://download.pytorch.org/whl/cu130" }, - { name = "vllm", marker = "platform_machine == 'aarch64' and sys_platform == 'linux' and extra == 'vllm'", url = "https://github.com/vllm-project/vllm/releases/download/v0.25.1/vllm-0.25.1-cp38-abi3-manylinux_2_28_aarch64.whl" }, - { name = "vllm", marker = "platform_machine == 'x86_64' and sys_platform == 'linux' and extra == 'vllm'", url = "https://github.com/vllm-project/vllm/releases/download/v0.25.1/vllm-0.25.1-cp38-abi3-manylinux_2_28_x86_64.whl" }, - { name = "vllm", marker = "(platform_machine != 'aarch64' and platform_machine != 'x86_64' and extra == 'vllm') or (sys_platform != 'linux' and extra == 'vllm')", specifier = "==0.25.1" }, - { name = "wandb", specifier = ">=0.28.0" }, - { name = "zstandard" }, -] -provides-extras = ["fsdp", "automodel", "vllm", "sglang", "mcore", "trtllm", "modelopt", "nvrx", "nemo-gym"] - -[package.metadata.requires-dev] -build = [ - { name = "einops" }, - { name = "hatchling" }, - { name = "packaging" }, - { name = "psutil" }, - { name = "pybind11" }, - { name = "setuptools" }, - { name = "torch", marker = "sys_platform != 'darwin'", specifier = "==2.11.0", index = "https://download.pytorch.org/whl/cu130" }, - { name = "torch", marker = "sys_platform == 'darwin'", specifier = "==2.11.0", index = "https://pypi.org/simple" }, -] -dev = [ - { name = "pre-commit", specifier = ">=4.2.0" }, - { name = "pyrefly", specifier = "==0.24.2" }, - { name = "ruff", specifier = "==0.9.9" }, - { name = "types-pyyaml" }, - { name = "types-requests" }, -] -docs = [ - { name = "gitpython", specifier = ">=3.1.45" }, - { name = "myst-parser" }, - { name = "nvidia-sphinx-theme" }, - { name = "python-dotenv" }, - { name = "sphinx" }, - { name = "sphinx-autobuild" }, - { name = "sphinx-autodoc2" }, - { name = "sphinx-copybutton" }, - { name = "sphinx-design" }, - { name = "sphinxcontrib-mermaid" }, - { name = "swagger-plugin-for-sphinx" }, -] -test = [ - { name = "pytest", specifier = ">=8.4.2" }, - { name = "pytest-asyncio" }, - { name = "pytest-cov" }, - { name = "pytest-shard" }, - { name = "pytest-testmon" }, - { name = "pytest-timeout" }, -] - [[package]] name = "networkx" version = "3.6.1" @@ -5114,18 +4253,6 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/38/56/c7e8645061cc2fc23f3a54f33e1e340df59216f07dcfb97d46b8ae7dd26c/nvtx-0.2.15-cp313-cp313t-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:3741edac4678b92f03d22a3f0a2dfd469f422f85e63db71b038e02525b2404ad", size = 788639, upload-time = "2026-03-18T10:12:01.69Z" }, ] -[[package]] -name = "obstore" -version = "0.11.0" -source = { registry = "https://pypi.org/simple" } -sdist = { url = "https://files.pythonhosted.org/packages/2e/2f/f83afaab7945509d72245b2b00af0b4834ce78fdd2d9ae9f0ad1a3036a91/obstore-0.11.0.tar.gz", hash = "sha256:a2f55163bcd348b4a60d12e6893eac50eddc742bad8032a1705d49140b992204", size = 130565, upload-time = "2026-06-25T18:29:49.405Z" } -wheels = [ - { url = "https://files.pythonhosted.org/packages/5d/59/3f47822683ee2b6db8685faa25829946d6343a561251ec2704548455d946/obstore-0.11.0-cp311-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:a2ff6d3ed553298828fb760b4aef6347fbcc7b5c5e3ce3f8381ce805c370021a", size = 5638724, upload-time = "2026-06-25T18:28:43.897Z" }, - { url = "https://files.pythonhosted.org/packages/23/50/1df335fdf9b527b3933f1e94ab6fc720ad314260fab8591cb0b6668ff192/obstore-0.11.0-cp311-abi3-manylinux_2_24_aarch64.whl", hash = "sha256:39d04b324fcf984e7050734ebda77b81764025b0c011750201a0d8954087f7aa", size = 5413508, upload-time = "2026-06-25T18:28:45.624Z" }, - { url = "https://files.pythonhosted.org/packages/de/dc/a259aba149b841ca7c91fea177df9972a60a636b54077beed1a35b254994/obstore-0.11.0-cp311-abi3-musllinux_1_2_aarch64.whl", hash = "sha256:37c0d15d775b1370ef5204ee3919a5ddf7e2592d11815213105f8db031f2ab8d", size = 5619995, upload-time = "2026-06-25T18:28:47.599Z" }, - { url = "https://files.pythonhosted.org/packages/57/b7/577a965f440e9ea64243518663f9d16be7df8eafc7123818e8e841fa21ce/obstore-0.11.0-cp311-abi3-musllinux_1_2_x86_64.whl", hash = "sha256:9c8fd2a544e2e0b926669c47fcfb8d2314e234abc240ea165dae04ee42e1d7ac", size = 5869187, upload-time = "2026-06-25T18:28:53.166Z" }, -] - [[package]] name = "omegaconf" version = "2.3.1" @@ -5284,36 +4411,6 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/10/68/162c97ea78c957d68ecf78a5c5041d2e25bd5562bdf5d89a6cbf7f8429bf/opencensus_context-0.1.3-py2.py3-none-any.whl", hash = "sha256:073bb0590007af276853009fac7e4bab1d523c3f03baf4cb4511ca38967c6039", size = 5060, upload-time = "2022-08-03T22:20:20.352Z" }, ] -[[package]] -name = "opensandbox" -version = "0.1.15" -source = { registry = "https://pypi.org/simple" } -dependencies = [ - { name = "attrs" }, - { name = "httpx" }, - { name = "pydantic" }, - { name = "python-dateutil" }, -] -sdist = { url = "https://files.pythonhosted.org/packages/43/21/654a3d69815b09690e926d553f3f4a178640d1206000a1b49f5e22c8eb68/opensandbox-0.1.15.tar.gz", hash = "sha256:017abc9b399b88da51bf077d6fb94ee89b1783e601495e85ba85fed478fed1b0", size = 228729, upload-time = "2026-07-24T09:36:29.988Z" } -wheels = [ - { url = "https://files.pythonhosted.org/packages/d5/5c/ab87ea696531210790feb8f575471036fccba29dedaff201736f15bbb3a7/opensandbox-0.1.15-py3-none-any.whl", hash = "sha256:992b01490551f4d8e3f99caa25e34cb9d1690f0c5027eeebab912738291957d1", size = 538522, upload-time = "2026-07-24T09:36:28.277Z" }, -] - -[[package]] -name = "openshell" -version = "0.0.102" -source = { registry = "https://pypi.org/simple" } -dependencies = [ - { name = "cloudpickle" }, - { name = "grpcio" }, - { name = "httpx" }, - { name = "protobuf" }, -] -wheels = [ - { url = "https://files.pythonhosted.org/packages/5d/4e/8fc4660a1cd57e92f4bc160234c29ef95ffef12b7b442f04a033b7cf7fe2/openshell-0.0.102-py3-none-manylinux_2_39_aarch64.whl", hash = "sha256:f9905a5e8fdab13300fac1a911619e3af3bab12e75059dd3b5bdba5067fbdc69", size = 8564345, upload-time = "2026-08-10T15:06:32.369Z" }, - { url = "https://files.pythonhosted.org/packages/f2/b7/6444a2320d75d244a872a00db479592e1c587cc44b8acd4461491eba80a7/openshell-0.0.102-py3-none-manylinux_2_39_x86_64.whl", hash = "sha256:0a4e9160e32ba8eabd28ac1eba1dff679a80b59e285b0a7b567fbcb41bde6a63", size = 9047752, upload-time = "2026-08-10T15:06:55.301Z" }, -] - [[package]] name = "opentelemetry-api" version = "1.44.0" @@ -5401,37 +4498,6 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/82/0d/d7cdc030edeed0fea03448446b88f1a1f8c4a565bad30dac0f5477ebe290/opentelemetry_exporter_prometheus-0.65b0-py3-none-any.whl", hash = "sha256:3b3d24b586d0ad9712c7b52b7d19c8a9dfbb318b9b284121b5f95e90ed019367", size = 13031, upload-time = "2026-07-16T15:25:20.906Z" }, ] -[[package]] -name = "opentelemetry-instrumentation" -version = "0.65b0" -source = { registry = "https://pypi.org/simple" } -dependencies = [ - { name = "opentelemetry-api" }, - { name = "opentelemetry-semantic-conventions" }, - { name = "packaging" }, - { name = "wrapt" }, -] -sdist = { url = "https://files.pythonhosted.org/packages/13/91/3c58961cb0360cd60509064734f0be4275383c8681d73c580a40ca83ddce/opentelemetry_instrumentation-0.65b0.tar.gz", hash = "sha256:071d9d9eced9bd6460444ec3b0c77229870ed05a881c22c84fdede58e4eed09b", size = 42689, upload-time = "2026-07-16T15:25:50.275Z" } -wheels = [ - { url = "https://files.pythonhosted.org/packages/40/7b/85eab1215f72adf0e68d3dc4a679b9bff993fa679ff34cd8dd378e2659fd/opentelemetry_instrumentation-0.65b0-py3-none-any.whl", hash = "sha256:ea967a72b9939b5fcfdad572753b4306c59dcb99e3f382d95dae04286805e137", size = 36717, upload-time = "2026-07-16T15:24:51.424Z" }, -] - -[[package]] -name = "opentelemetry-instrumentation-aiohttp-client" -version = "0.65b0" -source = { registry = "https://pypi.org/simple" } -dependencies = [ - { name = "opentelemetry-api" }, - { name = "opentelemetry-instrumentation" }, - { name = "opentelemetry-semantic-conventions" }, - { name = "opentelemetry-util-http" }, - { name = "wrapt" }, -] -sdist = { url = "https://files.pythonhosted.org/packages/a9/33/3ff7230b035e8b696db6be54f5c52dfa409829d634d91431c076ad789820/opentelemetry_instrumentation_aiohttp_client-0.65b0.tar.gz", hash = "sha256:85906a2806ee5641756b5c33274e9aa75c3cc2441e3b830aa5804cf0e1fa9dd1", size = 19042, upload-time = "2026-07-16T15:25:51.632Z" } -wheels = [ - { url = "https://files.pythonhosted.org/packages/71/f9/5c8459224f175829601cabbcffaeab0f9041903c086e4a0368f9971093a5/opentelemetry_instrumentation_aiohttp_client-0.65b0-py3-none-any.whl", hash = "sha256:3a060efa53fa44d02ba7372a7ed2b42cdfa6be6df81b089845067ad840e25729", size = 13677, upload-time = "2026-07-16T15:24:53.361Z" }, -] - [[package]] name = "opentelemetry-proto" version = "1.44.0" @@ -5484,15 +4550,6 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/55/22/41fb05f1dc5fda2c468e05a41814c20859016c85117b66c8a257cae814f6/opentelemetry_semantic_conventions_ai-0.5.1-py3-none-any.whl", hash = "sha256:25aeb22bd261543b4898a73824026d96770e5351209c7d07a0b1314762b1f6e4", size = 11250, upload-time = "2026-03-26T14:20:37.108Z" }, ] -[[package]] -name = "opentelemetry-util-http" -version = "0.65b0" -source = { registry = "https://pypi.org/simple" } -sdist = { url = "https://files.pythonhosted.org/packages/32/a9/d7525a59fdd240e69b5af4a6338e78fafa1b4203394122cbd6701fb5f84a/opentelemetry_util_http-0.65b0.tar.gz", hash = "sha256:84f82d826978bba416ab453460ff6a7391cdc3534c93a786595e4068680016b7", size = 11243, upload-time = "2026-07-16T15:26:27.898Z" } -wheels = [ - { url = "https://files.pythonhosted.org/packages/23/3f/ab8d29df207ce5f470a07fa96ebb48af4e95b7fab7e7635311b9a32f2fab/opentelemetry_util_http-0.65b0-py3-none-any.whl", hash = "sha256:7553b606f963097cb190536dc30556cce85090692e471a422fff30ca29b04348", size = 8245, upload-time = "2026-07-16T15:25:46.482Z" }, -] - [[package]] name = "optimum" version = "2.2.0" @@ -5578,8 +4635,8 @@ dependencies = [ { name = "numpy" }, { name = "python-dateutil" }, { name = "pytz" }, - { name = "tzdata", version = "2025.3", source = { registry = "https://pypi.org/simple" }, marker = "extra == 'extra-7-nemo-rl-mcore' or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-vllm' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-mcore' and extra == 'extra-8-nemo-gym-vllm')" }, - { name = "tzdata", version = "2026.3", source = { registry = "https://pypi.org/simple" }, marker = "extra == 'extra-7-nemo-rl-automodel' or extra != 'extra-7-nemo-rl-mcore' or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-8-nemo-gym-vllm')" }, + { name = "tzdata", version = "2025.3", source = { registry = "https://pypi.org/simple" }, marker = "extra == 'extra-7-nemo-rl-mcore' or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm')" }, + { name = "tzdata", version = "2026.3", source = { registry = "https://pypi.org/simple" }, marker = "extra == 'extra-7-nemo-rl-automodel' or extra != 'extra-7-nemo-rl-mcore' or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm')" }, ] sdist = { url = "https://files.pythonhosted.org/packages/33/01/d40b85317f86cf08d853a4f495195c73815fdf205eef3993821720274518/pandas-2.3.3.tar.gz", hash = "sha256:e05e1af93b977f7eafa636d043f9f94c7ee3ac81af99c13508215942e64c993b", size = 4495223, upload-time = "2025-09-29T23:34:51.853Z" } wheels = [ @@ -5634,8 +4691,8 @@ dependencies = [ { name = "safetensors" }, { name = "torch" }, { name = "tqdm" }, - { name = "transformers", version = "5.5.4", source = { registry = "https://pypi.org/simple" }, marker = "extra == 'extra-7-nemo-rl-trtllm' or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-vllm' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-trtllm' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-mcore' and extra != 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-mcore' and extra != 'extra-7-nemo-rl-trtllm' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra != 'extra-7-nemo-rl-mcore' and extra != 'extra-7-nemo-rl-trtllm' and extra != 'extra-7-nemo-rl-vllm' and extra != 'extra-8-nemo-gym-vllm') or (extra != 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-mcore' and extra != 'extra-7-nemo-rl-sglang' and extra != 'extra-7-nemo-rl-vllm' and extra != 'extra-8-nemo-gym-vllm')" }, - { name = "transformers", version = "5.8.1", source = { registry = "https://pypi.org/simple" }, marker = "extra == 'extra-7-nemo-rl-mcore' or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-vllm' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-mcore' and extra == 'extra-8-nemo-gym-vllm')" }, + { name = "transformers", version = "5.5.4", source = { registry = "https://pypi.org/simple" }, marker = "extra == 'extra-7-nemo-rl-trtllm' or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-mcore' and extra != 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra != 'extra-7-nemo-rl-mcore' and extra != 'extra-7-nemo-rl-trtllm' and extra != 'extra-7-nemo-rl-vllm') or (extra != 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-mcore' and extra != 'extra-7-nemo-rl-sglang' and extra != 'extra-7-nemo-rl-vllm')" }, + { name = "transformers", version = "5.8.1", source = { registry = "https://pypi.org/simple" }, marker = "extra == 'extra-7-nemo-rl-mcore' or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm')" }, ] sdist = { url = "https://files.pythonhosted.org/packages/86/cf/037f1e3d5186496c05513a6754639e2dab3038a05f384284d49a9bd06a2d/peft-0.19.1.tar.gz", hash = "sha256:0d97542fe96dcdaa20d3b81c06f26f988618f416a73544ab23c3618ccb674a40", size = 763738, upload-time = "2026-04-16T15:46:45.105Z" } wheels = [ @@ -6269,19 +5326,6 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/fa/b6/3127540ecdf1464a00e5a01ee60a1b09175f6913f0644ac748494d9c4b21/pytest_timeout-2.4.0-py3-none-any.whl", hash = "sha256:c42667e5cdadb151aeb5b26d114aff6bdf5a907f176a007a30b940d3d865b5c2", size = 14382, upload-time = "2025-05-05T19:44:33.502Z" }, ] -[[package]] -name = "pytest-xdist" -version = "3.8.0" -source = { registry = "https://pypi.org/simple" } -dependencies = [ - { name = "execnet" }, - { name = "pytest" }, -] -sdist = { url = "https://files.pythonhosted.org/packages/78/b4/439b179d1ff526791eb921115fca8e44e596a13efeda518b9d845a619450/pytest_xdist-3.8.0.tar.gz", hash = "sha256:7e578125ec9bc6050861aa93f2d59f1d8d085595d6551c2c90b6f4fad8d3a9f1", size = 88069, upload-time = "2025-07-01T13:30:59.346Z" } -wheels = [ - { url = "https://files.pythonhosted.org/packages/ca/31/d4e37e9e550c2b92a9cbc2e4d0b7420a27224968580b5a447f420847c975/pytest_xdist-3.8.0-py3-none-any.whl", hash = "sha256:202ca578cfeb7370784a8c33d6d05bc6e13b4f25b5053c30a152269fd10f0b88", size = 46396, upload-time = "2025-07-01T13:30:56.632Z" }, -] - [[package]] name = "python-box" version = "6.1.0" @@ -6325,18 +5369,6 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/0b/d7/1959b9648791274998a9c3526f6d0ec8fd2233e4d4acce81bbae76b44b2a/python_dotenv-1.2.2-py3-none-any.whl", hash = "sha256:1d8214789a24de455a8b8bd8ae6fe3c6b69a5e3d64aa8a8e5d68e694bbcb285a", size = 22101, upload-time = "2026-03-01T16:00:25.09Z" }, ] -[[package]] -name = "python-engineio" -version = "4.13.3" -source = { registry = "https://pypi.org/simple" } -dependencies = [ - { name = "simple-websocket" }, -] -sdist = { url = "https://files.pythonhosted.org/packages/fb/a0/f75491f942184d9960b15e763270f765fe9f239745ca5f9e16289011aed4/python_engineio-4.13.3.tar.gz", hash = "sha256:572b7783e341fed21edbc7cea297ccd378dad79265fdde96aa4664420a7c06c9", size = 79734, upload-time = "2026-06-20T22:53:52.197Z" } -wheels = [ - { url = "https://files.pythonhosted.org/packages/5b/96/82f6328e410515fab21d5602ba35b9377a47b5a141a0c1f9efa00ce21eb4/python_engineio-4.13.3-py3-none-any.whl", hash = "sha256:1f60ecaf1358190f0e26c48c578a60428dc02a8f1295bc3dbf53d1b31116821f", size = 59993, upload-time = "2026-06-20T22:53:50.775Z" }, -] - [[package]] name = "python-json-logger" version = "4.1.0" @@ -6355,28 +5387,6 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/e1/04/e8135ebd1ad02c56ec633277529b2602ff99ff634be76cdba5744cf554fd/python_multipart-0.0.32-py3-none-any.whl", hash = "sha256:ff6d3f776f16878c894e52e107296ffc890e913c611b1a4ec6c44e2821fe2e23", size = 30042, upload-time = "2026-06-04T16:18:57.319Z" }, ] -[[package]] -name = "python-socketio" -version = "5.16.3" -source = { registry = "https://pypi.org/simple" } -dependencies = [ - { name = "bidict" }, - { name = "python-engineio" }, -] -sdist = { url = "https://files.pythonhosted.org/packages/32/2d/ffce71017c106b75099fea569df6518c63fee5d6202ce0cfe7b01e6f22c3/python_socketio-5.16.3.tar.gz", hash = "sha256:89b136f677ae65607a84cecda9b4d6c5377b40a97582c504c25df89af16d520e", size = 128095, upload-time = "2026-06-15T22:07:04.003Z" } -wheels = [ - { url = "https://files.pythonhosted.org/packages/0a/38/8c5e72d53ff8eb27497c4f268a7f6d9121e727a50b65248288ad79a93053/python_socketio-5.16.3-py3-none-any.whl", hash = "sha256:e7ad14202a5e6448824c7c2f86161d04e13dec05992257df5c709e6a2798c041", size = 82087, upload-time = "2026-06-15T22:07:02.498Z" }, -] - -[package.optional-dependencies] -asyncio-client = [ - { name = "aiohttp" }, -] -client = [ - { name = "requests" }, - { name = "websocket-client" }, -] - [[package]] name = "pytz" version = "2026.2" @@ -6412,7 +5422,7 @@ name = "pyzmq" version = "27.1.0" source = { registry = "https://pypi.org/simple" } dependencies = [ - { name = "cffi", marker = "implementation_name == 'pypy' or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-vllm' and extra == 'extra-8-nemo-gym-vllm')" }, + { name = "cffi", marker = "implementation_name == 'pypy' or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm')" }, ] sdist = { url = "https://files.pythonhosted.org/packages/04/0b/3c9baedbdf613ecaa7aa07027780b8867f57b6293b6ee50de316c9f3222b/pyzmq-27.1.0.tar.gz", hash = "sha256:ac0765e3d44455adb6ddbf4417dcce460fc40a05978c08efdf2948072f6db540", size = 281750, upload-time = "2025-09-08T23:10:18.157Z" } wheels = [ @@ -6433,12 +5443,12 @@ source = { registry = "https://pypi.org/simple" } dependencies = [ { name = "apache-tvm-ffi" }, { name = "einops" }, - { name = "nvidia-cutlass-dsl", version = "4.5.0", source = { registry = "https://pypi.org/simple" }, marker = "extra == 'extra-7-nemo-rl-trtllm' or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-vllm' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-trtllm' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-mcore' and extra != 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-mcore' and extra != 'extra-7-nemo-rl-trtllm' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra != 'extra-7-nemo-rl-mcore' and extra != 'extra-7-nemo-rl-trtllm' and extra != 'extra-7-nemo-rl-vllm' and extra != 'extra-8-nemo-gym-vllm') or (extra != 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-mcore' and extra != 'extra-7-nemo-rl-sglang' and extra != 'extra-7-nemo-rl-vllm' and extra != 'extra-8-nemo-gym-vllm')" }, - { name = "nvidia-cutlass-dsl", version = "4.5.1", source = { registry = "https://pypi.org/simple" }, marker = "(extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-8-nemo-gym-vllm') or (extra != 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra != 'extra-7-nemo-rl-fsdp' and extra != 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-8-nemo-gym-vllm') or (extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-8-nemo-gym-vllm') or (extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm' and extra == 'extra-8-nemo-gym-vllm')" }, - { name = "nvidia-cutlass-dsl", version = "4.5.2", source = { registry = "https://pypi.org/simple" }, marker = "(extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-8-nemo-gym-vllm') or (extra != 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-vllm') or (extra != 'extra-7-nemo-rl-automodel' and extra == 'extra-8-nemo-gym-vllm') or (extra != 'extra-7-nemo-rl-fsdp' and extra != 'extra-7-nemo-rl-mcore' and extra != 'extra-7-nemo-rl-sglang' and extra != 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra != 'extra-7-nemo-rl-fsdp' and extra != 'extra-7-nemo-rl-mcore' and extra != 'extra-7-nemo-rl-sglang' and extra != 'extra-7-nemo-rl-trtllm' and extra == 'extra-8-nemo-gym-vllm')" }, + { name = "nvidia-cutlass-dsl", version = "4.5.0", source = { registry = "https://pypi.org/simple" }, marker = "extra == 'extra-7-nemo-rl-trtllm' or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-mcore' and extra != 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra != 'extra-7-nemo-rl-mcore' and extra != 'extra-7-nemo-rl-trtllm' and extra != 'extra-7-nemo-rl-vllm') or (extra != 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-mcore' and extra != 'extra-7-nemo-rl-sglang' and extra != 'extra-7-nemo-rl-vllm')" }, + { name = "nvidia-cutlass-dsl", version = "4.5.1", source = { registry = "https://pypi.org/simple" }, marker = "(extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra != 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra != 'extra-7-nemo-rl-fsdp' and extra != 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm')" }, + { name = "nvidia-cutlass-dsl", version = "4.5.2", source = { registry = "https://pypi.org/simple" }, marker = "(extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra != 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-vllm') or (extra != 'extra-7-nemo-rl-fsdp' and extra != 'extra-7-nemo-rl-mcore' and extra != 'extra-7-nemo-rl-sglang' and extra != 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm')" }, { name = "torch" }, - { name = "torch-c-dlpack-ext", version = "0.1.3", source = { registry = "https://pypi.org/simple" }, marker = "extra == 'extra-7-nemo-rl-trtllm' or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-vllm' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-trtllm' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-mcore' and extra != 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-mcore' and extra != 'extra-7-nemo-rl-trtllm' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra != 'extra-7-nemo-rl-mcore' and extra != 'extra-7-nemo-rl-trtllm' and extra != 'extra-7-nemo-rl-vllm' and extra != 'extra-8-nemo-gym-vllm') or (extra != 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-mcore' and extra != 'extra-7-nemo-rl-sglang' and extra != 'extra-7-nemo-rl-vllm' and extra != 'extra-8-nemo-gym-vllm')" }, - { name = "torch-c-dlpack-ext", version = "0.1.5", source = { registry = "https://pypi.org/simple" }, marker = "(extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-8-nemo-gym-vllm') or (extra != 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra != 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-vllm') or (extra != 'extra-7-nemo-rl-automodel' and extra == 'extra-8-nemo-gym-vllm') or (extra != 'extra-7-nemo-rl-fsdp' and extra != 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra != 'extra-7-nemo-rl-fsdp' and extra != 'extra-7-nemo-rl-mcore' and extra != 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra != 'extra-7-nemo-rl-fsdp' and extra != 'extra-7-nemo-rl-mcore' and extra != 'extra-7-nemo-rl-trtllm' and extra == 'extra-8-nemo-gym-vllm')" }, + { name = "torch-c-dlpack-ext", version = "0.1.3", source = { registry = "https://pypi.org/simple" }, marker = "extra == 'extra-7-nemo-rl-trtllm' or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-mcore' and extra != 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra != 'extra-7-nemo-rl-mcore' and extra != 'extra-7-nemo-rl-trtllm' and extra != 'extra-7-nemo-rl-vllm') or (extra != 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-mcore' and extra != 'extra-7-nemo-rl-sglang' and extra != 'extra-7-nemo-rl-vllm')" }, + { name = "torch-c-dlpack-ext", version = "0.1.5", source = { registry = "https://pypi.org/simple" }, marker = "(extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra != 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra != 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-vllm') or (extra != 'extra-7-nemo-rl-fsdp' and extra != 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra != 'extra-7-nemo-rl-fsdp' and extra != 'extra-7-nemo-rl-mcore' and extra != 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm')" }, ] sdist = { url = "https://files.pythonhosted.org/packages/2a/58/58b82e91b236539f424ff5681e7095b1f2860ddfb7778fe0be14d8fb58de/quack_kernels-0.4.1.tar.gz", hash = "sha256:9d7d6ba412bc0c8a9b1331c52a73db76280adb9dc2f2750df4851ddabef1466b", size = 274766, upload-time = "2026-04-30T14:37:55.65Z" } wheels = [ @@ -6573,18 +5583,6 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/a0/f4/c67b0b3f1b9245e8d266f0f112c500d50e5b4e83cb6f3b71b6528104182a/requests-2.34.2-py3-none-any.whl", hash = "sha256:2a0d60c172f83ac6ab31e4554906c0f3b3588d37b5cb939b1c061f4907e278e0", size = 73075, upload-time = "2026-05-14T19:25:26.443Z" }, ] -[[package]] -name = "requests-mock" -version = "1.12.1" -source = { registry = "https://pypi.org/simple" } -dependencies = [ - { name = "requests" }, -] -sdist = { url = "https://files.pythonhosted.org/packages/92/32/587625f91f9a0a3d84688bf9cfc4b2480a7e8ec327cefd0ff2ac891fd2cf/requests-mock-1.12.1.tar.gz", hash = "sha256:e9e12e333b525156e82a3c852f22016b9158220d2f47454de9cae8a77d371401", size = 60901, upload-time = "2024-03-29T03:54:29.446Z" } -wheels = [ - { url = "https://files.pythonhosted.org/packages/97/ec/889fbc557727da0c34a33850950310240f2040f3b1955175fdb2b36a8910/requests_mock-1.12.1-py2.py3-none-any.whl", hash = "sha256:b1e37054004cdd5e56c84454cc7df12b25f90f382159087f4b6915aaeef39563", size = 27695, upload-time = "2024-03-29T03:54:27.64Z" }, -] - [[package]] name = "requests-toolbelt" version = "1.0.0" @@ -6839,7 +5837,7 @@ dependencies = [ { name = "msgspec" }, { name = "ninja" }, { name = "numpy" }, - { name = "nvidia-cutlass-dsl", version = "4.5.1", source = { registry = "https://pypi.org/simple" }, extra = ["cu13"], marker = "(extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-8-nemo-gym-vllm') or (extra != 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra != 'extra-7-nemo-rl-fsdp' and extra != 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-8-nemo-gym-vllm') or (extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-8-nemo-gym-vllm') or (extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm' and extra == 'extra-8-nemo-gym-vllm')" }, + { name = "nvidia-cutlass-dsl", version = "4.5.1", source = { registry = "https://pypi.org/simple" }, extra = ["cu13"], marker = "(extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra != 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra != 'extra-7-nemo-rl-fsdp' and extra != 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm')" }, { name = "nvidia-ml-py" }, { name = "openai" }, { name = "openai-harmony" }, @@ -6923,18 +5921,6 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/e0/f9/0595336914c5619e5f28a1fb793285925a8cd4b432c9da0a987836c7f822/shellingham-1.5.4-py2.py3-none-any.whl", hash = "sha256:7ecfff8f2fd72616f7481040475a65b2bf8af90a56c89140852d1120324e8686", size = 9755, upload-time = "2023-10-24T04:13:38.866Z" }, ] -[[package]] -name = "simple-websocket" -version = "1.1.0" -source = { registry = "https://pypi.org/simple" } -dependencies = [ - { name = "wsproto" }, -] -sdist = { url = "https://files.pythonhosted.org/packages/b0/d4/bfa032f961103eba93de583b161f0e6a5b63cebb8f2c7d0c6e6efe1e3d2e/simple_websocket-1.1.0.tar.gz", hash = "sha256:7939234e7aa067c534abdab3a9ed933ec9ce4691b0713c78acb195560aa52ae4", size = 17300, upload-time = "2024-10-10T22:39:31.412Z" } -wheels = [ - { url = "https://files.pythonhosted.org/packages/52/59/0782e51887ac6b07ffd1570e0364cf901ebc36345fea669969d2084baebb/simple_websocket-1.1.0-py3-none-any.whl", hash = "sha256:4af6069630a38ed6c561010f0e11a5bc0d4ca569b36306eb257cd9a192497c8c", size = 13842, upload-time = "2024-10-10T22:39:29.645Z" }, -] - [[package]] name = "simplejson" version = "4.1.1" @@ -7431,25 +6417,6 @@ test = [ { name = "pytest-timeout" }, ] -[package.metadata] -requires-dist = [{ name = "nemo-rl", editable = "." }] - -[package.metadata.requires-dev] -test = [ - { name = "pytest", specifier = ">=7.0.0" }, - { name = "pytest-cov" }, - { name = "pytest-timeout" }, -] - -[[package]] -name = "tenacity" -version = "9.1.4" -source = { registry = "https://pypi.org/simple" } -sdist = { url = "https://files.pythonhosted.org/packages/47/c6/ee486fd809e357697ee8a44d3d69222b344920433d3b6666ccd9b374630c/tenacity-9.1.4.tar.gz", hash = "sha256:adb31d4c263f2bd041081ab33b498309a57c77f9acf2db65aadf0898179cf93a", size = 49413, upload-time = "2026-02-07T10:45:33.841Z" } -wheels = [ - { url = "https://files.pythonhosted.org/packages/d7/c1/eb8f9debc45d3b7918a32ab756658a0904732f75e555402972246b0b8e71/tenacity-9.1.4-py3-none-any.whl", hash = "sha256:6095a360c919085f28c6527de529e76a06ad89b23659fa881ae0649b867a9d55", size = 28926, upload-time = "2026-02-07T10:45:32.24Z" }, -] - [[package]] name = "tensorboard" version = "2.21.0" @@ -7571,9 +6538,9 @@ dependencies = [ { name = "numpy" }, { name = "nvidia-cuda-nvrtc" }, { name = "nvidia-cuda-tileiras" }, - { name = "nvidia-cutlass-dsl", version = "4.5.0", source = { registry = "https://pypi.org/simple" }, extra = ["cu13"], marker = "extra == 'extra-7-nemo-rl-trtllm' or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-vllm' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-trtllm' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-mcore' and extra != 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-mcore' and extra != 'extra-7-nemo-rl-trtllm' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra != 'extra-7-nemo-rl-mcore' and extra != 'extra-7-nemo-rl-trtllm' and extra != 'extra-7-nemo-rl-vllm' and extra != 'extra-8-nemo-gym-vllm') or (extra != 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-mcore' and extra != 'extra-7-nemo-rl-sglang' and extra != 'extra-7-nemo-rl-vllm' and extra != 'extra-8-nemo-gym-vllm')" }, + { name = "nvidia-cutlass-dsl", version = "4.5.0", source = { registry = "https://pypi.org/simple" }, extra = ["cu13"], marker = "extra == 'extra-7-nemo-rl-trtllm' or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-mcore' and extra != 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra != 'extra-7-nemo-rl-mcore' and extra != 'extra-7-nemo-rl-trtllm' and extra != 'extra-7-nemo-rl-vllm') or (extra != 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-mcore' and extra != 'extra-7-nemo-rl-sglang' and extra != 'extra-7-nemo-rl-vllm')" }, { name = "nvidia-ml-py" }, - { name = "nvidia-modelopt", marker = "extra == 'extra-7-nemo-rl-trtllm' or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-vllm' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-trtllm' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-mcore' and extra != 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-mcore' and extra != 'extra-7-nemo-rl-trtllm' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra != 'extra-7-nemo-rl-mcore' and extra != 'extra-7-nemo-rl-trtllm' and extra != 'extra-7-nemo-rl-vllm' and extra != 'extra-8-nemo-gym-vllm') or (extra != 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-mcore' and extra != 'extra-7-nemo-rl-sglang' and extra != 'extra-7-nemo-rl-vllm' and extra != 'extra-8-nemo-gym-vllm')" }, + { name = "nvidia-modelopt", marker = "extra == 'extra-7-nemo-rl-trtllm' or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-mcore' and extra != 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra != 'extra-7-nemo-rl-mcore' and extra != 'extra-7-nemo-rl-trtllm' and extra != 'extra-7-nemo-rl-vllm') or (extra != 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-mcore' and extra != 'extra-7-nemo-rl-sglang' and extra != 'extra-7-nemo-rl-vllm')" }, { name = "nvidia-nccl-cu13" }, { name = "nvtx" }, { name = "omegaconf" }, @@ -7593,7 +6560,7 @@ dependencies = [ { name = "psutil" }, { name = "pulp" }, { name = "pydantic" }, - { name = "pydantic-settings", extra = ["yaml"], marker = "extra == 'extra-7-nemo-rl-trtllm' or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-vllm' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-trtllm' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-mcore' and extra != 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-mcore' and extra != 'extra-7-nemo-rl-trtllm' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra != 'extra-7-nemo-rl-mcore' and extra != 'extra-7-nemo-rl-trtllm' and extra != 'extra-7-nemo-rl-vllm' and extra != 'extra-8-nemo-gym-vllm') or (extra != 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-mcore' and extra != 'extra-7-nemo-rl-sglang' and extra != 'extra-7-nemo-rl-vllm' and extra != 'extra-8-nemo-gym-vllm')" }, + { name = "pydantic-settings", extra = ["yaml"], marker = "extra == 'extra-7-nemo-rl-trtllm' or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-mcore' and extra != 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra != 'extra-7-nemo-rl-mcore' and extra != 'extra-7-nemo-rl-trtllm' and extra != 'extra-7-nemo-rl-vllm') or (extra != 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-mcore' and extra != 'extra-7-nemo-rl-sglang' and extra != 'extra-7-nemo-rl-vllm')" }, { name = "python-multipart" }, { name = "pyzmq" }, { name = "sentencepiece" }, @@ -7613,85 +6580,6 @@ dependencies = [ { name = "xgrammar" }, ] -[package.metadata] -requires-dist = [ - { name = "accelerate", specifier = ">=1.7.0" }, - { name = "aenum" }, - { name = "apache-tvm-ffi", specifier = "==0.1.6" }, - { name = "backoff" }, - { name = "blake3" }, - { name = "blobfile" }, - { name = "cache-dit", specifier = ">=1.3.5" }, - { name = "click", specifier = ">=8.3.1" }, - { name = "click-option-group" }, - { name = "colored" }, - { name = "cuda-python", specifier = ">=13" }, - { name = "cuda-tile", specifier = ">=1.0.1" }, - { name = "diffusers", specifier = ">=0.37.1" }, - { name = "einops" }, - { name = "etcd-sdk-python", specifier = "==0.0.7" }, - { name = "fastapi", specifier = ">=0.120.1" }, - { name = "flash-attn-4", specifier = "==4.0.0b11" }, - { name = "flashinfer-python", specifier = "==0.6.12" }, - { name = "ftfy" }, - { name = "h5py", specifier = "==3.12.1" }, - { name = "jsonschema" }, - { name = "lark" }, - { name = "librosa" }, - { name = "llguidance", specifier = ">=1.3.0,<1.4.0" }, - { name = "llist" }, - { name = "matplotlib" }, - { name = "mistral-common", specifier = ">=1.10.0" }, - { name = "mpi4py" }, - { name = "mpmath", specifier = ">=1.3.0" }, - { name = "msgpack" }, - { name = "numexpr" }, - { name = "numpy", specifier = ">=2.0.0,<2.4" }, - { name = "nvidia-cuda-nvrtc" }, - { name = "nvidia-cuda-tileiras", specifier = ">=13.1,<13.2" }, - { name = "nvidia-cutlass-dsl", extras = ["cu13"], specifier = "==4.5.0" }, - { name = "nvidia-ml-py", specifier = ">=13" }, - { name = "nvidia-modelopt", specifier = ">=0.37.0" }, - { name = "nvidia-nccl-cu13", specifier = ">=2.28.9,<=2.30.7" }, - { name = "nvtx" }, - { name = "omegaconf" }, - { name = "onnx", specifier = ">=1.21.0" }, - { name = "onnx-graphsurgeon", specifier = ">=0.5.2" }, - { name = "openai" }, - { name = "openai-harmony", specifier = "==0.0.4" }, - { name = "opencv-python-headless" }, - { name = "optimum" }, - { name = "ordered-set" }, - { name = "pandas" }, - { name = "partial-json-parser" }, - { name = "peft", specifier = ">=0.18.1" }, - { name = "pillow" }, - { name = "plotly" }, - { name = "prometheus-client" }, - { name = "prometheus-fastapi-instrumentator", specifier = ">=8.0.2" }, - { name = "psutil" }, - { name = "pulp" }, - { name = "pydantic", specifier = ">=2.9.1" }, - { name = "pydantic-settings", extras = ["yaml"] }, - { name = "python-multipart" }, - { name = "pyzmq" }, - { name = "sentencepiece", specifier = ">=0.1.99" }, - { name = "smg-grpc-proto", specifier = ">=0.4.2" }, - { name = "soundfile" }, - { name = "starlette", specifier = ">=0.49.1" }, - { name = "strenum" }, - { name = "tensorrt", specifier = "~=10.16.1" }, - { name = "tiktoken" }, - { name = "torch", specifier = ">=2.11.0" }, - { name = "torch-c-dlpack-ext", specifier = "==0.1.3" }, - { name = "torchao", specifier = ">=0.14.1,<0.16.0" }, - { name = "torchvision" }, - { name = "transformers", specifier = "==5.5.4" }, - { name = "uvicorn" }, - { name = "xdsl", specifier = ">=0.59.0" }, - { name = "xgrammar", specifier = ">=0.1.32" }, -] - [[package]] name = "tensorstore" version = "0.1.84" @@ -7744,17 +6632,17 @@ resolution-markers = [ "platform_machine == 'aarch64' and sys_platform == 'linux'", ] dependencies = [ - { name = "apache-tvm-ffi", marker = "(extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-8-nemo-gym-vllm') or (extra != 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra != 'extra-7-nemo-rl-fsdp' and extra != 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-8-nemo-gym-vllm') or (extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-8-nemo-gym-vllm') or (extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm' and extra == 'extra-8-nemo-gym-vllm')" }, - { name = "cloudpickle", marker = "(extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-8-nemo-gym-vllm') or (extra != 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra != 'extra-7-nemo-rl-fsdp' and extra != 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-8-nemo-gym-vllm') or (extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-8-nemo-gym-vllm') or (extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm' and extra == 'extra-8-nemo-gym-vllm')" }, - { name = "ml-dtypes", marker = "(extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-8-nemo-gym-vllm') or (extra != 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra != 'extra-7-nemo-rl-fsdp' and extra != 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-8-nemo-gym-vllm') or (extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-8-nemo-gym-vllm') or (extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm' and extra == 'extra-8-nemo-gym-vllm')" }, - { name = "numpy", marker = "(extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-8-nemo-gym-vllm') or (extra != 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra != 'extra-7-nemo-rl-fsdp' and extra != 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-8-nemo-gym-vllm') or (extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-8-nemo-gym-vllm') or (extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm' and extra == 'extra-8-nemo-gym-vllm')" }, - { name = "psutil", marker = "(extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-8-nemo-gym-vllm') or (extra != 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra != 'extra-7-nemo-rl-fsdp' and extra != 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-8-nemo-gym-vllm') or (extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-8-nemo-gym-vllm') or (extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm' and extra == 'extra-8-nemo-gym-vllm')" }, - { name = "setuptools", marker = "(extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-8-nemo-gym-vllm') or (extra != 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra != 'extra-7-nemo-rl-fsdp' and extra != 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-8-nemo-gym-vllm') or (extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-8-nemo-gym-vllm') or (extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm' and extra == 'extra-8-nemo-gym-vllm')" }, - { name = "torch", marker = "(extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-8-nemo-gym-vllm') or (extra != 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra != 'extra-7-nemo-rl-fsdp' and extra != 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-8-nemo-gym-vllm') or (extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-8-nemo-gym-vllm') or (extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm' and extra == 'extra-8-nemo-gym-vllm')" }, - { name = "torch-c-dlpack-ext", version = "0.1.5", source = { registry = "https://pypi.org/simple" }, marker = "(extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-8-nemo-gym-vllm') or (extra != 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra != 'extra-7-nemo-rl-fsdp' and extra != 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-8-nemo-gym-vllm') or (extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-8-nemo-gym-vllm') or (extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm' and extra == 'extra-8-nemo-gym-vllm')" }, - { name = "tqdm", marker = "(extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-8-nemo-gym-vllm') or (extra != 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra != 'extra-7-nemo-rl-fsdp' and extra != 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-8-nemo-gym-vllm') or (extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-8-nemo-gym-vllm') or (extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm' and extra == 'extra-8-nemo-gym-vllm')" }, - { name = "typing-extensions", marker = "(extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-8-nemo-gym-vllm') or (extra != 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra != 'extra-7-nemo-rl-fsdp' and extra != 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-8-nemo-gym-vllm') or (extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-8-nemo-gym-vllm') or (extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm' and extra == 'extra-8-nemo-gym-vllm')" }, - { name = "z3-solver", marker = "(extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-8-nemo-gym-vllm') or (extra != 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra != 'extra-7-nemo-rl-fsdp' and extra != 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-8-nemo-gym-vllm') or (extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-8-nemo-gym-vllm') or (extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm' and extra == 'extra-8-nemo-gym-vllm')" }, + { name = "apache-tvm-ffi", marker = "(extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra != 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra != 'extra-7-nemo-rl-fsdp' and extra != 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm')" }, + { name = "cloudpickle", marker = "(extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra != 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra != 'extra-7-nemo-rl-fsdp' and extra != 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm')" }, + { name = "ml-dtypes", marker = "(extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra != 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra != 'extra-7-nemo-rl-fsdp' and extra != 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm')" }, + { name = "numpy", marker = "(extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra != 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra != 'extra-7-nemo-rl-fsdp' and extra != 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm')" }, + { name = "psutil", marker = "(extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra != 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra != 'extra-7-nemo-rl-fsdp' and extra != 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm')" }, + { name = "setuptools", marker = "(extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra != 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra != 'extra-7-nemo-rl-fsdp' and extra != 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm')" }, + { name = "torch", marker = "(extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra != 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra != 'extra-7-nemo-rl-fsdp' and extra != 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm')" }, + { name = "torch-c-dlpack-ext", version = "0.1.5", source = { registry = "https://pypi.org/simple" }, marker = "(extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra != 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra != 'extra-7-nemo-rl-fsdp' and extra != 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm')" }, + { name = "tqdm", marker = "(extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra != 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra != 'extra-7-nemo-rl-fsdp' and extra != 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm')" }, + { name = "typing-extensions", marker = "(extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra != 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra != 'extra-7-nemo-rl-fsdp' and extra != 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm')" }, + { name = "z3-solver", marker = "(extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra != 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra != 'extra-7-nemo-rl-fsdp' and extra != 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm')" }, ] sdist = { url = "https://files.pythonhosted.org/packages/e6/27/6e363f48f878389078e2899756b8fecc326388b585122fd7f8a86590dfab/tilelang-0.1.8.tar.gz", hash = "sha256:da967821698eb7a79a76d27fbe25e314a3273f2b12ba4833e981658139d0e6d9", size = 93247335, upload-time = "2026-02-16T14:03:28.706Z" } wheels = [ @@ -7771,18 +6659,18 @@ resolution-markers = [ "platform_machine == 'aarch64' and sys_platform == 'linux'", ] dependencies = [ - { name = "apache-tvm-ffi", marker = "(platform_machine == 'x86_64' and extra == 'extra-7-nemo-rl-automodel') or (platform_machine == 'x86_64' and extra == 'extra-7-nemo-rl-fsdp') or (platform_machine == 'x86_64' and extra == 'extra-7-nemo-rl-mcore') or (platform_machine == 'x86_64' and extra != 'extra-7-nemo-rl-sglang') or extra == 'extra-7-nemo-rl-vllm' or extra == 'extra-8-nemo-gym-vllm' or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm')" }, - { name = "cloudpickle", marker = "(platform_machine == 'x86_64' and extra == 'extra-7-nemo-rl-automodel') or (platform_machine == 'x86_64' and extra == 'extra-7-nemo-rl-fsdp') or (platform_machine == 'x86_64' and extra == 'extra-7-nemo-rl-mcore') or (platform_machine == 'x86_64' and extra != 'extra-7-nemo-rl-sglang') or extra == 'extra-7-nemo-rl-vllm' or extra == 'extra-8-nemo-gym-vllm' or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm')" }, - { name = "ml-dtypes", marker = "(platform_machine == 'x86_64' and extra == 'extra-7-nemo-rl-automodel') or (platform_machine == 'x86_64' and extra == 'extra-7-nemo-rl-fsdp') or (platform_machine == 'x86_64' and extra == 'extra-7-nemo-rl-mcore') or (platform_machine == 'x86_64' and extra != 'extra-7-nemo-rl-sglang') or extra == 'extra-7-nemo-rl-vllm' or extra == 'extra-8-nemo-gym-vllm' or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm')" }, - { name = "numpy", marker = "(platform_machine == 'x86_64' and extra == 'extra-7-nemo-rl-automodel') or (platform_machine == 'x86_64' and extra == 'extra-7-nemo-rl-fsdp') or (platform_machine == 'x86_64' and extra == 'extra-7-nemo-rl-mcore') or (platform_machine == 'x86_64' and extra != 'extra-7-nemo-rl-sglang') or extra == 'extra-7-nemo-rl-vllm' or extra == 'extra-8-nemo-gym-vllm' or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm')" }, - { name = "psutil", marker = "(platform_machine == 'x86_64' and extra == 'extra-7-nemo-rl-automodel') or (platform_machine == 'x86_64' and extra == 'extra-7-nemo-rl-fsdp') or (platform_machine == 'x86_64' and extra == 'extra-7-nemo-rl-mcore') or (platform_machine == 'x86_64' and extra != 'extra-7-nemo-rl-sglang') or extra == 'extra-7-nemo-rl-vllm' or extra == 'extra-8-nemo-gym-vllm' or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm')" }, - { name = "setuptools", marker = "(platform_machine == 'x86_64' and extra == 'extra-7-nemo-rl-automodel') or (platform_machine == 'x86_64' and extra == 'extra-7-nemo-rl-fsdp') or (platform_machine == 'x86_64' and extra == 'extra-7-nemo-rl-mcore') or (platform_machine == 'x86_64' and extra != 'extra-7-nemo-rl-sglang') or extra == 'extra-7-nemo-rl-vllm' or extra == 'extra-8-nemo-gym-vllm' or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm')" }, - { name = "torch", marker = "(platform_machine == 'x86_64' and extra == 'extra-7-nemo-rl-automodel') or (platform_machine == 'x86_64' and extra == 'extra-7-nemo-rl-fsdp') or (platform_machine == 'x86_64' and extra == 'extra-7-nemo-rl-mcore') or (platform_machine == 'x86_64' and extra != 'extra-7-nemo-rl-sglang') or extra == 'extra-7-nemo-rl-vllm' or extra == 'extra-8-nemo-gym-vllm' or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm')" }, - { name = "torch-c-dlpack-ext", version = "0.1.3", source = { registry = "https://pypi.org/simple" }, marker = "(platform_machine == 'x86_64' and extra == 'extra-7-nemo-rl-trtllm') or (platform_machine == 'x86_64' and extra == 'extra-7-nemo-rl-fsdp' and extra != 'extra-7-nemo-rl-mcore' and extra != 'extra-7-nemo-rl-vllm' and extra != 'extra-8-nemo-gym-vllm') or (platform_machine == 'x86_64' and extra != 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-mcore' and extra != 'extra-7-nemo-rl-sglang' and extra != 'extra-7-nemo-rl-vllm' and extra != 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-vllm' and extra == 'extra-8-nemo-gym-vllm')" }, - { name = "torch-c-dlpack-ext", version = "0.1.5", source = { registry = "https://pypi.org/simple" }, marker = "(platform_machine == 'x86_64' and extra == 'extra-7-nemo-rl-automodel') or (platform_machine == 'x86_64' and extra == 'extra-7-nemo-rl-mcore') or extra == 'extra-7-nemo-rl-vllm' or extra == 'extra-8-nemo-gym-vllm' or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm')" }, - { name = "tqdm", marker = "(platform_machine == 'x86_64' and extra == 'extra-7-nemo-rl-automodel') or (platform_machine == 'x86_64' and extra == 'extra-7-nemo-rl-fsdp') or (platform_machine == 'x86_64' and extra == 'extra-7-nemo-rl-mcore') or (platform_machine == 'x86_64' and extra != 'extra-7-nemo-rl-sglang') or extra == 'extra-7-nemo-rl-vllm' or extra == 'extra-8-nemo-gym-vllm' or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm')" }, - { name = "typing-extensions", marker = "(platform_machine == 'x86_64' and extra == 'extra-7-nemo-rl-automodel') or (platform_machine == 'x86_64' and extra == 'extra-7-nemo-rl-fsdp') or (platform_machine == 'x86_64' and extra == 'extra-7-nemo-rl-mcore') or (platform_machine == 'x86_64' and extra != 'extra-7-nemo-rl-sglang') or extra == 'extra-7-nemo-rl-vllm' or extra == 'extra-8-nemo-gym-vllm' or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm')" }, - { name = "z3-solver", marker = "(platform_machine == 'x86_64' and extra == 'extra-7-nemo-rl-automodel') or (platform_machine == 'x86_64' and extra == 'extra-7-nemo-rl-fsdp') or (platform_machine == 'x86_64' and extra == 'extra-7-nemo-rl-mcore') or (platform_machine == 'x86_64' and extra != 'extra-7-nemo-rl-sglang') or extra == 'extra-7-nemo-rl-vllm' or extra == 'extra-8-nemo-gym-vllm' or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm')" }, + { name = "apache-tvm-ffi", marker = "(platform_machine == 'x86_64' and extra == 'extra-7-nemo-rl-automodel') or (platform_machine == 'x86_64' and extra == 'extra-7-nemo-rl-fsdp') or (platform_machine == 'x86_64' and extra == 'extra-7-nemo-rl-mcore') or (platform_machine == 'x86_64' and extra != 'extra-7-nemo-rl-sglang') or extra == 'extra-7-nemo-rl-vllm' or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm')" }, + { name = "cloudpickle", marker = "(platform_machine == 'x86_64' and extra == 'extra-7-nemo-rl-automodel') or (platform_machine == 'x86_64' and extra == 'extra-7-nemo-rl-fsdp') or (platform_machine == 'x86_64' and extra == 'extra-7-nemo-rl-mcore') or (platform_machine == 'x86_64' and extra != 'extra-7-nemo-rl-sglang') or extra == 'extra-7-nemo-rl-vllm' or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm')" }, + { name = "ml-dtypes", marker = "(platform_machine == 'x86_64' and extra == 'extra-7-nemo-rl-automodel') or (platform_machine == 'x86_64' and extra == 'extra-7-nemo-rl-fsdp') or (platform_machine == 'x86_64' and extra == 'extra-7-nemo-rl-mcore') or (platform_machine == 'x86_64' and extra != 'extra-7-nemo-rl-sglang') or extra == 'extra-7-nemo-rl-vllm' or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm')" }, + { name = "numpy", marker = "(platform_machine == 'x86_64' and extra == 'extra-7-nemo-rl-automodel') or (platform_machine == 'x86_64' and extra == 'extra-7-nemo-rl-fsdp') or (platform_machine == 'x86_64' and extra == 'extra-7-nemo-rl-mcore') or (platform_machine == 'x86_64' and extra != 'extra-7-nemo-rl-sglang') or extra == 'extra-7-nemo-rl-vllm' or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm')" }, + { name = "psutil", marker = "(platform_machine == 'x86_64' and extra == 'extra-7-nemo-rl-automodel') or (platform_machine == 'x86_64' and extra == 'extra-7-nemo-rl-fsdp') or (platform_machine == 'x86_64' and extra == 'extra-7-nemo-rl-mcore') or (platform_machine == 'x86_64' and extra != 'extra-7-nemo-rl-sglang') or extra == 'extra-7-nemo-rl-vllm' or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm')" }, + { name = "setuptools", marker = "(platform_machine == 'x86_64' and extra == 'extra-7-nemo-rl-automodel') or (platform_machine == 'x86_64' and extra == 'extra-7-nemo-rl-fsdp') or (platform_machine == 'x86_64' and extra == 'extra-7-nemo-rl-mcore') or (platform_machine == 'x86_64' and extra != 'extra-7-nemo-rl-sglang') or extra == 'extra-7-nemo-rl-vllm' or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm')" }, + { name = "torch", marker = "(platform_machine == 'x86_64' and extra == 'extra-7-nemo-rl-automodel') or (platform_machine == 'x86_64' and extra == 'extra-7-nemo-rl-fsdp') or (platform_machine == 'x86_64' and extra == 'extra-7-nemo-rl-mcore') or (platform_machine == 'x86_64' and extra != 'extra-7-nemo-rl-sglang') or extra == 'extra-7-nemo-rl-vllm' or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm')" }, + { name = "torch-c-dlpack-ext", version = "0.1.3", source = { registry = "https://pypi.org/simple" }, marker = "(platform_machine == 'x86_64' and extra == 'extra-7-nemo-rl-trtllm') or (platform_machine == 'x86_64' and extra == 'extra-7-nemo-rl-fsdp' and extra != 'extra-7-nemo-rl-mcore' and extra != 'extra-7-nemo-rl-vllm') or (platform_machine == 'x86_64' and extra != 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-mcore' and extra != 'extra-7-nemo-rl-sglang' and extra != 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm')" }, + { name = "torch-c-dlpack-ext", version = "0.1.5", source = { registry = "https://pypi.org/simple" }, marker = "(platform_machine == 'x86_64' and extra == 'extra-7-nemo-rl-automodel') or (platform_machine == 'x86_64' and extra == 'extra-7-nemo-rl-mcore') or extra == 'extra-7-nemo-rl-vllm' or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm')" }, + { name = "tqdm", marker = "(platform_machine == 'x86_64' and extra == 'extra-7-nemo-rl-automodel') or (platform_machine == 'x86_64' and extra == 'extra-7-nemo-rl-fsdp') or (platform_machine == 'x86_64' and extra == 'extra-7-nemo-rl-mcore') or (platform_machine == 'x86_64' and extra != 'extra-7-nemo-rl-sglang') or extra == 'extra-7-nemo-rl-vllm' or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm')" }, + { name = "typing-extensions", marker = "(platform_machine == 'x86_64' and extra == 'extra-7-nemo-rl-automodel') or (platform_machine == 'x86_64' and extra == 'extra-7-nemo-rl-fsdp') or (platform_machine == 'x86_64' and extra == 'extra-7-nemo-rl-mcore') or (platform_machine == 'x86_64' and extra != 'extra-7-nemo-rl-sglang') or extra == 'extra-7-nemo-rl-vllm' or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm')" }, + { name = "z3-solver", marker = "(platform_machine == 'x86_64' and extra == 'extra-7-nemo-rl-automodel') or (platform_machine == 'x86_64' and extra == 'extra-7-nemo-rl-fsdp') or (platform_machine == 'x86_64' and extra == 'extra-7-nemo-rl-mcore') or (platform_machine == 'x86_64' and extra != 'extra-7-nemo-rl-sglang') or extra == 'extra-7-nemo-rl-vllm' or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm')" }, ] sdist = { url = "https://files.pythonhosted.org/packages/56/70/5051f65821baa30a3d61fc48f8ba10c776490315e8c90f82559b92089756/tilelang-0.1.9.tar.gz", hash = "sha256:287f727c913bb648fcf6c1968809ba3390e55eeed257a5c6bb9a80bc05966af4", size = 93395292, upload-time = "2026-04-22T09:19:11.988Z" } wheels = [ @@ -7868,15 +6756,6 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/73/cf/6d0bb055e0fff9ef9284ec5455ea5f4d8d2e76f89cc4c853f7be69fdeef8/tokenspeed_triton-3.8.10.post20260709-cp312-abi3-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:15040e5514671ceb05a22dbead49030cf87ac3becb47361a67ed2f779433d04d", size = 87150923, upload-time = "2026-07-10T15:29:07.961Z" }, ] -[[package]] -name = "toml" -version = "0.10.2" -source = { registry = "https://pypi.org/simple" } -sdist = { url = "https://files.pythonhosted.org/packages/be/ba/1f744cdc819428fc6b5084ec34d9b30660f6f9daaf70eead706e3203ec3c/toml-0.10.2.tar.gz", hash = "sha256:b3bda1d108d5dd99f4a20d24d9c348e91c4db7ab1b749200bded2f839ccbe68f", size = 22253, upload-time = "2020-11-01T01:40:22.204Z" } -wheels = [ - { url = "https://files.pythonhosted.org/packages/44/6f/7120676b6d73228c96e17f1f794d8ab046fc910d781c8d151120c3f1569e/toml-0.10.2-py2.py3-none-any.whl", hash = "sha256:806143ae5bfb6a3c6e736a764057db0e6a0e05e338b5630894a5f779cabb4f9b", size = 16588, upload-time = "2020-11-01T01:40:20.672Z" }, -] - [[package]] name = "torch" version = "2.11.0+cu130" @@ -7913,7 +6792,7 @@ resolution-markers = [ "platform_machine == 'aarch64' and sys_platform == 'linux'", ] dependencies = [ - { name = "torch", marker = "extra == 'extra-7-nemo-rl-trtllm' or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-vllm' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-trtllm' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-mcore' and extra != 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-mcore' and extra != 'extra-7-nemo-rl-trtllm' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra != 'extra-7-nemo-rl-mcore' and extra != 'extra-7-nemo-rl-trtllm' and extra != 'extra-7-nemo-rl-vllm' and extra != 'extra-8-nemo-gym-vllm') or (extra != 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-mcore' and extra != 'extra-7-nemo-rl-sglang' and extra != 'extra-7-nemo-rl-vllm' and extra != 'extra-8-nemo-gym-vllm')" }, + { name = "torch", marker = "extra == 'extra-7-nemo-rl-trtllm' or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-mcore' and extra != 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra != 'extra-7-nemo-rl-mcore' and extra != 'extra-7-nemo-rl-trtllm' and extra != 'extra-7-nemo-rl-vllm') or (extra != 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-mcore' and extra != 'extra-7-nemo-rl-sglang' and extra != 'extra-7-nemo-rl-vllm')" }, ] sdist = { url = "https://files.pythonhosted.org/packages/34/1a/adafdc49546ffe0041f52471da51c084243094382b99332b5706e90ba26a/torch_c_dlpack_ext-0.1.3.tar.gz", hash = "sha256:4b5da66432af7224dcf02aad4f13cc416eeef5331cd153588b7e081a193f4972", size = 3334, upload-time = "2025-11-22T03:50:48.089Z" } wheels = [ @@ -7930,7 +6809,7 @@ resolution-markers = [ "platform_machine == 'aarch64' and sys_platform == 'linux'", ] dependencies = [ - { name = "torch", marker = "(platform_machine == 'x86_64' and extra == 'extra-7-nemo-rl-automodel') or (platform_machine == 'x86_64' and extra == 'extra-7-nemo-rl-mcore') or extra == 'extra-7-nemo-rl-sglang' or extra == 'extra-7-nemo-rl-vllm' or extra == 'extra-8-nemo-gym-vllm' or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm')" }, + { name = "torch", marker = "(platform_machine == 'x86_64' and extra == 'extra-7-nemo-rl-automodel') or (platform_machine == 'x86_64' and extra == 'extra-7-nemo-rl-mcore') or extra == 'extra-7-nemo-rl-sglang' or extra == 'extra-7-nemo-rl-vllm' or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm')" }, ] sdist = { url = "https://files.pythonhosted.org/packages/37/de/921b6491efce5c389a5ef9bbed3d2d6660005840dae488124173180859ab/torch_c_dlpack_ext-0.1.5.tar.gz", hash = "sha256:d06f0357d575d22a168cc77acb9020fc4bae30968ceb6718a055dcbe92bacabe", size = 12913, upload-time = "2026-01-12T11:25:08.484Z" } wheels = [ @@ -8062,8 +6941,8 @@ name = "transferqueue" version = "0.1.9" source = { git = "https://github.com/Ascend/TransferQueue.git?rev=c51614308b68c8d7a87c9b3ef62d59e14c69bde2#c51614308b68c8d7a87c9b3ef62d59e14c69bde2" } dependencies = [ - { name = "hydra-core", version = "1.3.2", source = { registry = "https://pypi.org/simple" }, marker = "extra == 'extra-7-nemo-rl-mcore' or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-vllm' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-mcore' and extra == 'extra-8-nemo-gym-vllm')" }, - { name = "hydra-core", version = "1.3.4", source = { registry = "https://pypi.org/simple" }, marker = "extra == 'extra-7-nemo-rl-automodel' or extra != 'extra-7-nemo-rl-mcore' or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-8-nemo-gym-vllm')" }, + { name = "hydra-core", version = "1.3.2", source = { registry = "https://pypi.org/simple" }, marker = "extra == 'extra-7-nemo-rl-mcore' or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm')" }, + { name = "hydra-core", version = "1.3.4", source = { registry = "https://pypi.org/simple" }, marker = "extra == 'extra-7-nemo-rl-automodel' or extra != 'extra-7-nemo-rl-mcore' or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm')" }, { name = "msgspec" }, { name = "numpy" }, { name = "omegaconf" }, @@ -8094,15 +6973,15 @@ resolution-markers = [ "platform_machine == 'aarch64' and sys_platform == 'linux'", ] dependencies = [ - { name = "huggingface-hub", marker = "extra == 'extra-7-nemo-rl-automodel' or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-vllm' and extra == 'extra-8-nemo-gym-vllm')" }, - { name = "numpy", marker = "extra == 'extra-7-nemo-rl-automodel' or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-vllm' and extra == 'extra-8-nemo-gym-vllm')" }, - { name = "packaging", marker = "extra == 'extra-7-nemo-rl-automodel' or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-vllm' and extra == 'extra-8-nemo-gym-vllm')" }, - { name = "pyyaml", marker = "extra == 'extra-7-nemo-rl-automodel' or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-vllm' and extra == 'extra-8-nemo-gym-vllm')" }, - { name = "regex", marker = "extra == 'extra-7-nemo-rl-automodel' or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-vllm' and extra == 'extra-8-nemo-gym-vllm')" }, - { name = "safetensors", marker = "extra == 'extra-7-nemo-rl-automodel' or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-vllm' and extra == 'extra-8-nemo-gym-vllm')" }, - { name = "tokenizers", marker = "extra == 'extra-7-nemo-rl-automodel' or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-vllm' and extra == 'extra-8-nemo-gym-vllm')" }, - { name = "tqdm", marker = "extra == 'extra-7-nemo-rl-automodel' or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-vllm' and extra == 'extra-8-nemo-gym-vllm')" }, - { name = "typer", marker = "extra == 'extra-7-nemo-rl-automodel' or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-vllm' and extra == 'extra-8-nemo-gym-vllm')" }, + { name = "huggingface-hub", marker = "extra == 'extra-7-nemo-rl-automodel' or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm')" }, + { name = "numpy", marker = "extra == 'extra-7-nemo-rl-automodel' or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm')" }, + { name = "packaging", marker = "extra == 'extra-7-nemo-rl-automodel' or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm')" }, + { name = "pyyaml", marker = "extra == 'extra-7-nemo-rl-automodel' or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm')" }, + { name = "regex", marker = "extra == 'extra-7-nemo-rl-automodel' or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm')" }, + { name = "safetensors", marker = "extra == 'extra-7-nemo-rl-automodel' or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm')" }, + { name = "tokenizers", marker = "extra == 'extra-7-nemo-rl-automodel' or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm')" }, + { name = "tqdm", marker = "extra == 'extra-7-nemo-rl-automodel' or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm')" }, + { name = "typer", marker = "extra == 'extra-7-nemo-rl-automodel' or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm')" }, ] sdist = { url = "https://files.pythonhosted.org/packages/ff/9d/fb46e729b461985f41a5740167688b924a4019141e5c164bea77548d3d9e/transformers-5.5.0.tar.gz", hash = "sha256:c8db656cf51c600cd8c75f06b20ef85c72e8b8ff9abc880c5d3e8bc70e0ddcbd", size = 8237745, upload-time = "2026-04-02T16:13:08.113Z" } wheels = [ @@ -8118,15 +6997,15 @@ resolution-markers = [ "platform_machine == 'aarch64' and sys_platform == 'linux'", ] dependencies = [ - { name = "huggingface-hub", marker = "extra == 'extra-7-nemo-rl-trtllm' or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-vllm' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-trtllm' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-mcore' and extra != 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-mcore' and extra != 'extra-7-nemo-rl-trtllm' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra != 'extra-7-nemo-rl-mcore' and extra != 'extra-7-nemo-rl-trtllm' and extra != 'extra-7-nemo-rl-vllm' and extra != 'extra-8-nemo-gym-vllm') or (extra != 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-mcore' and extra != 'extra-7-nemo-rl-sglang' and extra != 'extra-7-nemo-rl-vllm' and extra != 'extra-8-nemo-gym-vllm')" }, - { name = "numpy", marker = "extra == 'extra-7-nemo-rl-trtllm' or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-vllm' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-trtllm' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-mcore' and extra != 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-mcore' and extra != 'extra-7-nemo-rl-trtllm' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra != 'extra-7-nemo-rl-mcore' and extra != 'extra-7-nemo-rl-trtllm' and extra != 'extra-7-nemo-rl-vllm' and extra != 'extra-8-nemo-gym-vllm') or (extra != 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-mcore' and extra != 'extra-7-nemo-rl-sglang' and extra != 'extra-7-nemo-rl-vllm' and extra != 'extra-8-nemo-gym-vllm')" }, - { name = "packaging", marker = "extra == 'extra-7-nemo-rl-trtllm' or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-vllm' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-trtllm' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-mcore' and extra != 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-mcore' and extra != 'extra-7-nemo-rl-trtllm' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra != 'extra-7-nemo-rl-mcore' and extra != 'extra-7-nemo-rl-trtllm' and extra != 'extra-7-nemo-rl-vllm' and extra != 'extra-8-nemo-gym-vllm') or (extra != 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-mcore' and extra != 'extra-7-nemo-rl-sglang' and extra != 'extra-7-nemo-rl-vllm' and extra != 'extra-8-nemo-gym-vllm')" }, - { name = "pyyaml", marker = "extra == 'extra-7-nemo-rl-trtllm' or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-vllm' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-trtllm' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-mcore' and extra != 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-mcore' and extra != 'extra-7-nemo-rl-trtllm' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra != 'extra-7-nemo-rl-mcore' and extra != 'extra-7-nemo-rl-trtllm' and extra != 'extra-7-nemo-rl-vllm' and extra != 'extra-8-nemo-gym-vllm') or (extra != 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-mcore' and extra != 'extra-7-nemo-rl-sglang' and extra != 'extra-7-nemo-rl-vllm' and extra != 'extra-8-nemo-gym-vllm')" }, - { name = "regex", marker = "extra == 'extra-7-nemo-rl-trtllm' or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-vllm' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-trtllm' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-mcore' and extra != 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-mcore' and extra != 'extra-7-nemo-rl-trtllm' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra != 'extra-7-nemo-rl-mcore' and extra != 'extra-7-nemo-rl-trtllm' and extra != 'extra-7-nemo-rl-vllm' and extra != 'extra-8-nemo-gym-vllm') or (extra != 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-mcore' and extra != 'extra-7-nemo-rl-sglang' and extra != 'extra-7-nemo-rl-vllm' and extra != 'extra-8-nemo-gym-vllm')" }, - { name = "safetensors", marker = "extra == 'extra-7-nemo-rl-trtllm' or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-vllm' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-trtllm' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-mcore' and extra != 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-mcore' and extra != 'extra-7-nemo-rl-trtllm' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra != 'extra-7-nemo-rl-mcore' and extra != 'extra-7-nemo-rl-trtllm' and extra != 'extra-7-nemo-rl-vllm' and extra != 'extra-8-nemo-gym-vllm') or (extra != 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-mcore' and extra != 'extra-7-nemo-rl-sglang' and extra != 'extra-7-nemo-rl-vllm' and extra != 'extra-8-nemo-gym-vllm')" }, - { name = "tokenizers", marker = "extra == 'extra-7-nemo-rl-trtllm' or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-vllm' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-trtllm' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-mcore' and extra != 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-mcore' and extra != 'extra-7-nemo-rl-trtllm' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra != 'extra-7-nemo-rl-mcore' and extra != 'extra-7-nemo-rl-trtllm' and extra != 'extra-7-nemo-rl-vllm' and extra != 'extra-8-nemo-gym-vllm') or (extra != 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-mcore' and extra != 'extra-7-nemo-rl-sglang' and extra != 'extra-7-nemo-rl-vllm' and extra != 'extra-8-nemo-gym-vllm')" }, - { name = "tqdm", marker = "extra == 'extra-7-nemo-rl-trtllm' or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-vllm' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-trtllm' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-mcore' and extra != 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-mcore' and extra != 'extra-7-nemo-rl-trtllm' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra != 'extra-7-nemo-rl-mcore' and extra != 'extra-7-nemo-rl-trtllm' and extra != 'extra-7-nemo-rl-vllm' and extra != 'extra-8-nemo-gym-vllm') or (extra != 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-mcore' and extra != 'extra-7-nemo-rl-sglang' and extra != 'extra-7-nemo-rl-vllm' and extra != 'extra-8-nemo-gym-vllm')" }, - { name = "typer", marker = "extra == 'extra-7-nemo-rl-trtllm' or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-vllm' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-trtllm' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-mcore' and extra != 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-mcore' and extra != 'extra-7-nemo-rl-trtllm' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra != 'extra-7-nemo-rl-mcore' and extra != 'extra-7-nemo-rl-trtllm' and extra != 'extra-7-nemo-rl-vllm' and extra != 'extra-8-nemo-gym-vllm') or (extra != 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-mcore' and extra != 'extra-7-nemo-rl-sglang' and extra != 'extra-7-nemo-rl-vllm' and extra != 'extra-8-nemo-gym-vllm')" }, + { name = "huggingface-hub", marker = "extra == 'extra-7-nemo-rl-trtllm' or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-mcore' and extra != 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra != 'extra-7-nemo-rl-mcore' and extra != 'extra-7-nemo-rl-trtllm' and extra != 'extra-7-nemo-rl-vllm') or (extra != 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-mcore' and extra != 'extra-7-nemo-rl-sglang' and extra != 'extra-7-nemo-rl-vllm')" }, + { name = "numpy", marker = "extra == 'extra-7-nemo-rl-trtllm' or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-mcore' and extra != 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra != 'extra-7-nemo-rl-mcore' and extra != 'extra-7-nemo-rl-trtllm' and extra != 'extra-7-nemo-rl-vllm') or (extra != 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-mcore' and extra != 'extra-7-nemo-rl-sglang' and extra != 'extra-7-nemo-rl-vllm')" }, + { name = "packaging", marker = "extra == 'extra-7-nemo-rl-trtllm' or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-mcore' and extra != 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra != 'extra-7-nemo-rl-mcore' and extra != 'extra-7-nemo-rl-trtllm' and extra != 'extra-7-nemo-rl-vllm') or (extra != 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-mcore' and extra != 'extra-7-nemo-rl-sglang' and extra != 'extra-7-nemo-rl-vllm')" }, + { name = "pyyaml", marker = "extra == 'extra-7-nemo-rl-trtllm' or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-mcore' and extra != 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra != 'extra-7-nemo-rl-mcore' and extra != 'extra-7-nemo-rl-trtllm' and extra != 'extra-7-nemo-rl-vllm') or (extra != 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-mcore' and extra != 'extra-7-nemo-rl-sglang' and extra != 'extra-7-nemo-rl-vllm')" }, + { name = "regex", marker = "extra == 'extra-7-nemo-rl-trtllm' or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-mcore' and extra != 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra != 'extra-7-nemo-rl-mcore' and extra != 'extra-7-nemo-rl-trtllm' and extra != 'extra-7-nemo-rl-vllm') or (extra != 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-mcore' and extra != 'extra-7-nemo-rl-sglang' and extra != 'extra-7-nemo-rl-vllm')" }, + { name = "safetensors", marker = "extra == 'extra-7-nemo-rl-trtllm' or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-mcore' and extra != 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra != 'extra-7-nemo-rl-mcore' and extra != 'extra-7-nemo-rl-trtllm' and extra != 'extra-7-nemo-rl-vllm') or (extra != 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-mcore' and extra != 'extra-7-nemo-rl-sglang' and extra != 'extra-7-nemo-rl-vllm')" }, + { name = "tokenizers", marker = "extra == 'extra-7-nemo-rl-trtllm' or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-mcore' and extra != 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra != 'extra-7-nemo-rl-mcore' and extra != 'extra-7-nemo-rl-trtllm' and extra != 'extra-7-nemo-rl-vllm') or (extra != 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-mcore' and extra != 'extra-7-nemo-rl-sglang' and extra != 'extra-7-nemo-rl-vllm')" }, + { name = "tqdm", marker = "extra == 'extra-7-nemo-rl-trtllm' or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-mcore' and extra != 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra != 'extra-7-nemo-rl-mcore' and extra != 'extra-7-nemo-rl-trtllm' and extra != 'extra-7-nemo-rl-vllm') or (extra != 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-mcore' and extra != 'extra-7-nemo-rl-sglang' and extra != 'extra-7-nemo-rl-vllm')" }, + { name = "typer", marker = "extra == 'extra-7-nemo-rl-trtllm' or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-mcore' and extra != 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra != 'extra-7-nemo-rl-mcore' and extra != 'extra-7-nemo-rl-trtllm' and extra != 'extra-7-nemo-rl-vllm') or (extra != 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-mcore' and extra != 'extra-7-nemo-rl-sglang' and extra != 'extra-7-nemo-rl-vllm')" }, ] sdist = { url = "https://files.pythonhosted.org/packages/a5/1e/1e244ab2ab50a863e6b52cc55761910567fa532b69a6740f6e99c5fdbd98/transformers-5.5.4.tar.gz", hash = "sha256:2e67cadba81fc7608cc07c4dd54f524820bc3d95b1cabd0ef3db7733c4f8b82e", size = 8227649, upload-time = "2026-04-13T16:55:55.181Z" } wheels = [ @@ -8142,15 +7021,15 @@ resolution-markers = [ "platform_machine == 'aarch64' and sys_platform == 'linux'", ] dependencies = [ - { name = "huggingface-hub", marker = "(extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-8-nemo-gym-vllm') or (extra != 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra != 'extra-7-nemo-rl-fsdp' and extra != 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-8-nemo-gym-vllm') or (extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-8-nemo-gym-vllm') or (extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm' and extra == 'extra-8-nemo-gym-vllm')" }, - { name = "numpy", marker = "(extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-8-nemo-gym-vllm') or (extra != 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra != 'extra-7-nemo-rl-fsdp' and extra != 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-8-nemo-gym-vllm') or (extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-8-nemo-gym-vllm') or (extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm' and extra == 'extra-8-nemo-gym-vllm')" }, - { name = "packaging", marker = "(extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-8-nemo-gym-vllm') or (extra != 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra != 'extra-7-nemo-rl-fsdp' and extra != 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-8-nemo-gym-vllm') or (extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-8-nemo-gym-vllm') or (extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm' and extra == 'extra-8-nemo-gym-vllm')" }, - { name = "pyyaml", marker = "(extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-8-nemo-gym-vllm') or (extra != 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra != 'extra-7-nemo-rl-fsdp' and extra != 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-8-nemo-gym-vllm') or (extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-8-nemo-gym-vllm') or (extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm' and extra == 'extra-8-nemo-gym-vllm')" }, - { name = "regex", marker = "(extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-8-nemo-gym-vllm') or (extra != 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra != 'extra-7-nemo-rl-fsdp' and extra != 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-8-nemo-gym-vllm') or (extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-8-nemo-gym-vllm') or (extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm' and extra == 'extra-8-nemo-gym-vllm')" }, - { name = "safetensors", marker = "(extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-8-nemo-gym-vllm') or (extra != 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra != 'extra-7-nemo-rl-fsdp' and extra != 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-8-nemo-gym-vllm') or (extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-8-nemo-gym-vllm') or (extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm' and extra == 'extra-8-nemo-gym-vllm')" }, - { name = "tokenizers", marker = "(extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-8-nemo-gym-vllm') or (extra != 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra != 'extra-7-nemo-rl-fsdp' and extra != 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-8-nemo-gym-vllm') or (extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-8-nemo-gym-vllm') or (extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm' and extra == 'extra-8-nemo-gym-vllm')" }, - { name = "tqdm", marker = "(extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-8-nemo-gym-vllm') or (extra != 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra != 'extra-7-nemo-rl-fsdp' and extra != 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-8-nemo-gym-vllm') or (extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-8-nemo-gym-vllm') or (extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm' and extra == 'extra-8-nemo-gym-vllm')" }, - { name = "typer", marker = "(extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-8-nemo-gym-vllm') or (extra != 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra != 'extra-7-nemo-rl-fsdp' and extra != 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-8-nemo-gym-vllm') or (extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-8-nemo-gym-vllm') or (extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm' and extra == 'extra-8-nemo-gym-vllm')" }, + { name = "huggingface-hub", marker = "(extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra != 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra != 'extra-7-nemo-rl-fsdp' and extra != 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm')" }, + { name = "numpy", marker = "(extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra != 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra != 'extra-7-nemo-rl-fsdp' and extra != 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm')" }, + { name = "packaging", marker = "(extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra != 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra != 'extra-7-nemo-rl-fsdp' and extra != 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm')" }, + { name = "pyyaml", marker = "(extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra != 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra != 'extra-7-nemo-rl-fsdp' and extra != 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm')" }, + { name = "regex", marker = "(extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra != 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra != 'extra-7-nemo-rl-fsdp' and extra != 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm')" }, + { name = "safetensors", marker = "(extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra != 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra != 'extra-7-nemo-rl-fsdp' and extra != 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm')" }, + { name = "tokenizers", marker = "(extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra != 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra != 'extra-7-nemo-rl-fsdp' and extra != 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm')" }, + { name = "tqdm", marker = "(extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra != 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra != 'extra-7-nemo-rl-fsdp' and extra != 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm')" }, + { name = "typer", marker = "(extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra != 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra != 'extra-7-nemo-rl-fsdp' and extra != 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm')" }, ] sdist = { url = "https://files.pythonhosted.org/packages/88/14/e3eb58bd4c9df45af154f9de59a6e66a2ece30dd64434c710e40e5a4b1f1/transformers-5.6.0.tar.gz", hash = "sha256:291951976b79a6f93ec06d6ab14489a99aecdbc8f05aaabab538ea1d508c9a97", size = 8311711, upload-time = "2026-04-22T15:42:03.393Z" } wheels = [ @@ -8166,15 +7045,15 @@ resolution-markers = [ "platform_machine == 'aarch64' and sys_platform == 'linux'", ] dependencies = [ - { name = "huggingface-hub", marker = "extra == 'extra-7-nemo-rl-mcore' or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-8-nemo-gym-vllm') or (extra != 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-vllm') or (extra != 'extra-7-nemo-rl-automodel' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra != 'extra-7-nemo-rl-fsdp' and extra != 'extra-7-nemo-rl-sglang' and extra != 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra != 'extra-7-nemo-rl-fsdp' and extra != 'extra-7-nemo-rl-sglang' and extra != 'extra-7-nemo-rl-trtllm' and extra == 'extra-8-nemo-gym-vllm')" }, - { name = "numpy", marker = "extra == 'extra-7-nemo-rl-mcore' or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-8-nemo-gym-vllm') or (extra != 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-vllm') or (extra != 'extra-7-nemo-rl-automodel' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra != 'extra-7-nemo-rl-fsdp' and extra != 'extra-7-nemo-rl-sglang' and extra != 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra != 'extra-7-nemo-rl-fsdp' and extra != 'extra-7-nemo-rl-sglang' and extra != 'extra-7-nemo-rl-trtllm' and extra == 'extra-8-nemo-gym-vllm')" }, - { name = "packaging", marker = "extra == 'extra-7-nemo-rl-mcore' or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-8-nemo-gym-vllm') or (extra != 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-vllm') or (extra != 'extra-7-nemo-rl-automodel' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra != 'extra-7-nemo-rl-fsdp' and extra != 'extra-7-nemo-rl-sglang' and extra != 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra != 'extra-7-nemo-rl-fsdp' and extra != 'extra-7-nemo-rl-sglang' and extra != 'extra-7-nemo-rl-trtllm' and extra == 'extra-8-nemo-gym-vllm')" }, - { name = "pyyaml", marker = "extra == 'extra-7-nemo-rl-mcore' or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-8-nemo-gym-vllm') or (extra != 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-vllm') or (extra != 'extra-7-nemo-rl-automodel' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra != 'extra-7-nemo-rl-fsdp' and extra != 'extra-7-nemo-rl-sglang' and extra != 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra != 'extra-7-nemo-rl-fsdp' and extra != 'extra-7-nemo-rl-sglang' and extra != 'extra-7-nemo-rl-trtllm' and extra == 'extra-8-nemo-gym-vllm')" }, - { name = "regex", marker = "extra == 'extra-7-nemo-rl-mcore' or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-8-nemo-gym-vllm') or (extra != 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-vllm') or (extra != 'extra-7-nemo-rl-automodel' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra != 'extra-7-nemo-rl-fsdp' and extra != 'extra-7-nemo-rl-sglang' and extra != 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra != 'extra-7-nemo-rl-fsdp' and extra != 'extra-7-nemo-rl-sglang' and extra != 'extra-7-nemo-rl-trtllm' and extra == 'extra-8-nemo-gym-vllm')" }, - { name = "safetensors", marker = "extra == 'extra-7-nemo-rl-mcore' or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-8-nemo-gym-vllm') or (extra != 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-vllm') or (extra != 'extra-7-nemo-rl-automodel' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra != 'extra-7-nemo-rl-fsdp' and extra != 'extra-7-nemo-rl-sglang' and extra != 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra != 'extra-7-nemo-rl-fsdp' and extra != 'extra-7-nemo-rl-sglang' and extra != 'extra-7-nemo-rl-trtllm' and extra == 'extra-8-nemo-gym-vllm')" }, - { name = "tokenizers", marker = "extra == 'extra-7-nemo-rl-mcore' or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-8-nemo-gym-vllm') or (extra != 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-vllm') or (extra != 'extra-7-nemo-rl-automodel' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra != 'extra-7-nemo-rl-fsdp' and extra != 'extra-7-nemo-rl-sglang' and extra != 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra != 'extra-7-nemo-rl-fsdp' and extra != 'extra-7-nemo-rl-sglang' and extra != 'extra-7-nemo-rl-trtllm' and extra == 'extra-8-nemo-gym-vllm')" }, - { name = "tqdm", marker = "extra == 'extra-7-nemo-rl-mcore' or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-8-nemo-gym-vllm') or (extra != 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-vllm') or (extra != 'extra-7-nemo-rl-automodel' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra != 'extra-7-nemo-rl-fsdp' and extra != 'extra-7-nemo-rl-sglang' and extra != 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra != 'extra-7-nemo-rl-fsdp' and extra != 'extra-7-nemo-rl-sglang' and extra != 'extra-7-nemo-rl-trtllm' and extra == 'extra-8-nemo-gym-vllm')" }, - { name = "typer", marker = "extra == 'extra-7-nemo-rl-mcore' or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-8-nemo-gym-vllm') or (extra != 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-vllm') or (extra != 'extra-7-nemo-rl-automodel' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra != 'extra-7-nemo-rl-fsdp' and extra != 'extra-7-nemo-rl-sglang' and extra != 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra != 'extra-7-nemo-rl-fsdp' and extra != 'extra-7-nemo-rl-sglang' and extra != 'extra-7-nemo-rl-trtllm' and extra == 'extra-8-nemo-gym-vllm')" }, + { name = "huggingface-hub", marker = "extra == 'extra-7-nemo-rl-mcore' or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra != 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra != 'extra-7-nemo-rl-fsdp' and extra != 'extra-7-nemo-rl-sglang' and extra != 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm')" }, + { name = "numpy", marker = "extra == 'extra-7-nemo-rl-mcore' or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra != 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra != 'extra-7-nemo-rl-fsdp' and extra != 'extra-7-nemo-rl-sglang' and extra != 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm')" }, + { name = "packaging", marker = "extra == 'extra-7-nemo-rl-mcore' or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra != 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra != 'extra-7-nemo-rl-fsdp' and extra != 'extra-7-nemo-rl-sglang' and extra != 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm')" }, + { name = "pyyaml", marker = "extra == 'extra-7-nemo-rl-mcore' or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra != 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra != 'extra-7-nemo-rl-fsdp' and extra != 'extra-7-nemo-rl-sglang' and extra != 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm')" }, + { name = "regex", marker = "extra == 'extra-7-nemo-rl-mcore' or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra != 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra != 'extra-7-nemo-rl-fsdp' and extra != 'extra-7-nemo-rl-sglang' and extra != 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm')" }, + { name = "safetensors", marker = "extra == 'extra-7-nemo-rl-mcore' or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra != 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra != 'extra-7-nemo-rl-fsdp' and extra != 'extra-7-nemo-rl-sglang' and extra != 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm')" }, + { name = "tokenizers", marker = "extra == 'extra-7-nemo-rl-mcore' or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra != 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra != 'extra-7-nemo-rl-fsdp' and extra != 'extra-7-nemo-rl-sglang' and extra != 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm')" }, + { name = "tqdm", marker = "extra == 'extra-7-nemo-rl-mcore' or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra != 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra != 'extra-7-nemo-rl-fsdp' and extra != 'extra-7-nemo-rl-sglang' and extra != 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm')" }, + { name = "typer", marker = "extra == 'extra-7-nemo-rl-mcore' or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra != 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra != 'extra-7-nemo-rl-fsdp' and extra != 'extra-7-nemo-rl-sglang' and extra != 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm')" }, ] sdist = { url = "https://files.pythonhosted.org/packages/e7/e6/4134ea2fbea322cddc7ffc94a0d8ee47fe32ce8e876b320cd37d88edfc4d/transformers-5.8.1.tar.gz", hash = "sha256:4dd5b6de4105725104d84fd6abd74b305f4debfc251b38c648ee5dd087cf543b", size = 8532019, upload-time = "2026-05-13T03:21:57.234Z" } wheels = [ @@ -8342,93 +7221,6 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/c1/7c/4e7225d46d634a0d8d534dd8a6ce0c319d09b4d0cf0337eb314ca4789d8c/virtualenv-21.6.1-py3-none-any.whl", hash = "sha256:afe991df855715a2b2f60edfcc0107ef95a79fdfd8cb4cdaa71603d1c12e463b", size = 5506392, upload-time = "2026-07-10T19:33:51.629Z" }, ] -[[package]] -name = "vllm" -version = "0.24.0" -source = { registry = "https://pypi.org/simple" } -resolution-markers = [ - "platform_machine == 'x86_64' and sys_platform == 'linux'", - "platform_machine == 'aarch64' and sys_platform == 'linux'", -] -dependencies = [ - { name = "aiohttp" }, - { name = "anthropic" }, - { name = "apache-tvm-ffi" }, - { name = "blake3" }, - { name = "cachetools" }, - { name = "cbor2" }, - { name = "cloudpickle" }, - { name = "compressed-tensors", version = "0.17.0", source = { registry = "https://pypi.org/simple" } }, - { name = "depyf" }, - { name = "diskcache" }, - { name = "einops" }, - { name = "fastapi", version = "0.136.3", source = { registry = "https://pypi.org/simple" }, extra = ["standard"], marker = "(extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-8-nemo-gym-vllm') or (extra != 'extra-7-nemo-rl-automodel' and extra == 'extra-8-nemo-gym-vllm') or (extra != 'extra-7-nemo-rl-fsdp' and extra != 'extra-7-nemo-rl-mcore' and extra != 'extra-7-nemo-rl-sglang' and extra != 'extra-7-nemo-rl-trtllm' and extra != 'extra-7-nemo-rl-vllm' and extra == 'extra-8-nemo-gym-vllm')" }, - { name = "fastsafetensors" }, - { name = "filelock" }, - { name = "flashinfer-cubin", version = "0.6.12", source = { registry = "https://pypi.org/simple" } }, - { name = "flashinfer-python", version = "0.6.12", source = { registry = "https://pypi.org/simple" } }, - { name = "humming-kernels", version = "0.1.6", source = { registry = "https://pypi.org/simple" }, extra = ["cu13"], marker = "(extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-8-nemo-gym-vllm') or (extra != 'extra-7-nemo-rl-automodel' and extra == 'extra-8-nemo-gym-vllm') or (extra != 'extra-7-nemo-rl-fsdp' and extra != 'extra-7-nemo-rl-mcore' and extra != 'extra-7-nemo-rl-sglang' and extra != 'extra-7-nemo-rl-trtllm' and extra != 'extra-7-nemo-rl-vllm' and extra == 'extra-8-nemo-gym-vllm')" }, - { name = "ijson" }, - { name = "jsonschema" }, - { name = "lark" }, - { name = "llguidance" }, - { name = "lm-format-enforcer" }, - { name = "mcp" }, - { name = "mistral-common", marker = "(extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-8-nemo-gym-vllm') or (extra != 'extra-7-nemo-rl-automodel' and extra == 'extra-8-nemo-gym-vllm') or (extra != 'extra-7-nemo-rl-fsdp' and extra != 'extra-7-nemo-rl-mcore' and extra != 'extra-7-nemo-rl-sglang' and extra != 'extra-7-nemo-rl-trtllm' and extra != 'extra-7-nemo-rl-vllm' and extra == 'extra-8-nemo-gym-vllm')" }, - { name = "model-hosting-container-standards" }, - { name = "msgspec" }, - { name = "ninja" }, - { name = "numba" }, - { name = "numpy" }, - { name = "nvidia-cudnn-frontend" }, - { name = "nvidia-cutlass-dsl", version = "4.5.2", source = { registry = "https://pypi.org/simple" }, extra = ["cu13"], marker = "(extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-8-nemo-gym-vllm') or (extra != 'extra-7-nemo-rl-automodel' and extra == 'extra-8-nemo-gym-vllm') or (extra != 'extra-7-nemo-rl-fsdp' and extra != 'extra-7-nemo-rl-mcore' and extra != 'extra-7-nemo-rl-sglang' and extra != 'extra-7-nemo-rl-trtllm' and extra != 'extra-7-nemo-rl-vllm' and extra == 'extra-8-nemo-gym-vllm')" }, - { name = "openai" }, - { name = "openai-harmony" }, - { name = "opentelemetry-api" }, - { name = "opentelemetry-exporter-otlp" }, - { name = "opentelemetry-sdk" }, - { name = "opentelemetry-semantic-conventions-ai" }, - { name = "outlines-core" }, - { name = "partial-json-parser" }, - { name = "pillow" }, - { name = "prometheus-client" }, - { name = "prometheus-fastapi-instrumentator" }, - { name = "protobuf" }, - { name = "psutil" }, - { name = "py-cpuinfo" }, - { name = "pybase64" }, - { name = "pydantic" }, - { name = "python-json-logger" }, - { name = "pyyaml" }, - { name = "pyzmq" }, - { name = "quack-kernels" }, - { name = "regex" }, - { name = "requests" }, - { name = "safetensors" }, - { name = "sentencepiece" }, - { name = "setproctitle" }, - { name = "setuptools" }, - { name = "six" }, - { name = "starlette" }, - { name = "tiktoken" }, - { name = "tilelang", version = "0.1.9", source = { registry = "https://pypi.org/simple" } }, - { name = "tokenizers" }, - { name = "tokenspeed-mla", version = "0.1.2", source = { registry = "https://pypi.org/simple" } }, - { name = "torch" }, - { name = "torchaudio" }, - { name = "torchvision" }, - { name = "tqdm" }, - { name = "transformers", version = "5.8.1", source = { registry = "https://pypi.org/simple" } }, - { name = "typing-extensions" }, - { name = "watchfiles" }, - { name = "xgrammar" }, -] -sdist = { url = "https://files.pythonhosted.org/packages/d4/17/ea541f9ffb31d438ed6a5a8e1f7b9805410664abf97e326f28416147a5da/vllm-0.24.0.tar.gz", hash = "sha256:0862453adc1f3339f1a0c9dca1179c34d6ed6e118f87b6e5bddd120af614ac66", size = 37236989, upload-time = "2026-06-30T01:18:35.996Z" } -wheels = [ - { url = "https://files.pythonhosted.org/packages/9e/80/51a071305b4eed0f6f512dc1c1c6957cbb14ccce38db1be90ffcff2a2844/vllm-0.24.0-cp38-abi3-manylinux_2_28_aarch64.whl", hash = "sha256:700db71c3cf14697d42583521f38b12fac38db1e7a8ad062e8e4d63a5dadebd5", size = 271361241, upload-time = "2026-06-30T01:17:52.566Z" }, - { url = "https://files.pythonhosted.org/packages/00/33/3f0abda52acff437a471cf3a2bf204213eb4102975b2677512cd76f2b45e/vllm-0.24.0-cp38-abi3-manylinux_2_28_x86_64.whl", hash = "sha256:2d2831aeba311292250df0132dbc4d8e9f42c654536eaec48e6fe58acb1822cf", size = 279209310, upload-time = "2026-06-30T01:18:18.504Z" }, -] - [[package]] name = "vllm" version = "0.25.1" @@ -8448,26 +7240,26 @@ dependencies = [ { name = "depyf", marker = "platform_machine != 'x86_64'" }, { name = "diskcache", marker = "platform_machine != 'x86_64'" }, { name = "einops", marker = "platform_machine != 'x86_64'" }, - { name = "fastapi", version = "0.136.3", source = { registry = "https://pypi.org/simple" }, extra = ["standard"], marker = "(platform_machine != 'x86_64' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-vllm' and extra == 'extra-8-nemo-gym-vllm')" }, + { name = "fastapi", version = "0.136.3", source = { registry = "https://pypi.org/simple" }, extra = ["standard"], marker = "(platform_machine != 'x86_64' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm')" }, { name = "fastsafetensors", marker = "platform_machine != 'x86_64'" }, { name = "filelock", marker = "platform_machine != 'x86_64'" }, { name = "flashinfer-cubin", version = "0.6.13", source = { registry = "https://pypi.org/simple" }, marker = "platform_machine != 'x86_64'" }, { name = "flashinfer-python", version = "0.6.13", source = { registry = "https://pypi.org/simple" }, marker = "platform_machine != 'x86_64'" }, - { name = "humming-kernels", version = "0.1.10", source = { registry = "https://pypi.org/simple" }, extra = ["cu13"], marker = "(platform_machine != 'x86_64' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-vllm' and extra == 'extra-8-nemo-gym-vllm')" }, + { name = "humming-kernels", extra = ["cu13"], marker = "(platform_machine != 'x86_64' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm')" }, { name = "ijson", marker = "platform_machine != 'x86_64'" }, { name = "jsonschema", marker = "platform_machine != 'x86_64'" }, { name = "lark", marker = "platform_machine != 'x86_64'" }, { name = "llguidance", marker = "platform_machine != 'x86_64'" }, { name = "lm-format-enforcer", marker = "platform_machine != 'x86_64'" }, { name = "mcp", marker = "platform_machine != 'x86_64'" }, - { name = "mistral-common", marker = "(platform_machine != 'x86_64' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-vllm' and extra == 'extra-8-nemo-gym-vllm')" }, + { name = "mistral-common", marker = "(platform_machine != 'x86_64' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm')" }, { name = "model-hosting-container-standards", marker = "platform_machine != 'x86_64'" }, { name = "msgspec", marker = "platform_machine != 'x86_64'" }, { name = "ninja", marker = "platform_machine != 'x86_64'" }, { name = "numba", marker = "platform_machine != 'x86_64'" }, { name = "numpy", marker = "platform_machine != 'x86_64'" }, { name = "nvidia-cudnn-frontend", marker = "platform_machine != 'x86_64'" }, - { name = "nvidia-cutlass-dsl", version = "4.5.2", source = { registry = "https://pypi.org/simple" }, extra = ["cu13"], marker = "(platform_machine != 'x86_64' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-vllm' and extra == 'extra-8-nemo-gym-vllm')" }, + { name = "nvidia-cutlass-dsl", version = "4.5.2", source = { registry = "https://pypi.org/simple" }, extra = ["cu13"], marker = "(platform_machine != 'x86_64' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm')" }, { name = "nvtx", marker = "platform_machine != 'x86_64'" }, { name = "openai", marker = "platform_machine != 'x86_64'" }, { name = "openai-harmony", marker = "platform_machine != 'x86_64'" }, @@ -8516,109 +7308,6 @@ wheels = [ { url = "https://github.com/vllm-project/vllm/releases/download/v0.25.1/vllm-0.25.1-cp38-abi3-manylinux_2_28_aarch64.whl", hash = "sha256:902be760af4c5ebfad8af5b8ea07a53ae14e5a6c839c8ab56da30581abb75ad2" }, ] -[package.metadata] -requires-dist = [ - { name = "aiohttp", specifier = ">=3.13.3" }, - { name = "anthropic", specifier = ">=0.71.0" }, - { name = "apache-tvm-ffi", specifier = "==0.1.9" }, - { name = "av", marker = "extra == 'audio'" }, - { name = "blake3" }, - { name = "cachetools" }, - { name = "cbor2" }, - { name = "cloudpickle" }, - { name = "compressed-tensors", specifier = "==0.17.0" }, - { name = "datasets", marker = "extra == 'bench'" }, - { name = "depyf", specifier = "==0.20.0" }, - { name = "diskcache", specifier = "==5.6.3" }, - { name = "einops" }, - { name = "fastapi", extras = ["standard"], specifier = ">=0.133.0,<0.137.0" }, - { name = "fastsafetensors", specifier = ">=0.3.2" }, - { name = "fastsafetensors", marker = "extra == 'fastsafetensors'", specifier = ">=0.3.2" }, - { name = "filelock", specifier = ">=3.16.1" }, - { name = "flashinfer-cubin", specifier = "==0.6.13" }, - { name = "flashinfer-python", specifier = "==0.6.13" }, - { name = "helion", marker = "extra == 'helion'", specifier = "==1.1.0" }, - { name = "humming-kernels", extras = ["cu13"], specifier = "==0.1.10" }, - { name = "ijson" }, - { name = "instanttensor", marker = "extra == 'instanttensor'", specifier = ">=0.1.5" }, - { name = "jsonschema", specifier = ">=4.23.0" }, - { name = "lark", specifier = "==1.2.2" }, - { name = "llguidance", marker = "platform_machine == 'aarch64' or platform_machine == 'arm64' or platform_machine == 'ppc64le' or platform_machine == 'x86_64'", specifier = ">=1.7.0,<1.8.0" }, - { name = "lm-format-enforcer", specifier = "==0.11.3" }, - { name = "matplotlib", marker = "extra == 'bench'" }, - { name = "mcp" }, - { name = "mistral-common", extras = ["audio"], marker = "extra == 'audio'" }, - { name = "mistral-common", extras = ["image"], specifier = ">=1.11.5" }, - { name = "model-hosting-container-standards", specifier = ">=0.1.14,<1.0.0" }, - { name = "msgspec" }, - { name = "ninja" }, - { name = "numba", specifier = "==0.65.0" }, - { name = "numpy" }, - { name = "nvidia-cudnn-frontend", specifier = ">=1.19.1" }, - { name = "nvidia-cutlass-dsl", extras = ["cu13"], specifier = "==4.5.2" }, - { name = "nvtx", specifier = "==0.2.15" }, - { name = "openai", specifier = ">=2.0.0" }, - { name = "openai-harmony", specifier = ">=0.0.3" }, - { name = "opencv-python-headless", specifier = ">=4.13.0" }, - { name = "opentelemetry-api", specifier = ">=1.27.0" }, - { name = "opentelemetry-api", marker = "extra == 'otel'", specifier = ">=1.26.0" }, - { name = "opentelemetry-exporter-otlp", specifier = ">=1.27.0" }, - { name = "opentelemetry-exporter-otlp", marker = "extra == 'otel'", specifier = ">=1.26.0" }, - { name = "opentelemetry-sdk", specifier = ">=1.27.0" }, - { name = "opentelemetry-sdk", marker = "extra == 'otel'", specifier = ">=1.26.0" }, - { name = "opentelemetry-semantic-conventions-ai", specifier = ">=0.4.1" }, - { name = "opentelemetry-semantic-conventions-ai", marker = "extra == 'otel'", specifier = ">=0.4.1" }, - { name = "outlines-core", specifier = "==0.2.14" }, - { name = "pandas", marker = "extra == 'bench'" }, - { name = "partial-json-parser" }, - { name = "pillow" }, - { name = "plotly", marker = "extra == 'bench'" }, - { name = "prometheus-client", specifier = ">=0.18.0" }, - { name = "prometheus-fastapi-instrumentator", specifier = ">=8.0.0" }, - { name = "protobuf", specifier = ">=5.29.6,!=6.30.*,!=6.31.*,!=6.32.*,!=6.33.0.*,!=6.33.1.*,!=6.33.2.*,!=6.33.3.*,!=6.33.4.*" }, - { name = "psutil" }, - { name = "py-cpuinfo" }, - { name = "pybase64" }, - { name = "pydantic", specifier = ">=2.12.0" }, - { name = "pynvvideocodec", specifier = "==2.0.4" }, - { name = "python-json-logger" }, - { name = "pyyaml" }, - { name = "pyzmq", specifier = ">=25.0.0" }, - { name = "quack-kernels", specifier = ">=0.3.3" }, - { name = "regex" }, - { name = "requests", specifier = ">=2.26.0" }, - { name = "runai-model-streamer", extras = ["azure", "gcs", "s3"], marker = "extra == 'runai'", specifier = ">=0.15.7" }, - { name = "safetensors", specifier = ">=0.6.2" }, - { name = "scipy", marker = "extra == 'audio'" }, - { name = "scipy", marker = "extra == 'bench'" }, - { name = "seaborn", marker = "extra == 'bench'" }, - { name = "sentencepiece" }, - { name = "setproctitle" }, - { name = "setuptools", marker = "python_full_version >= '3.12'", specifier = ">=77.0.3,<81.0.0" }, - { name = "six", marker = "python_full_version >= '3.12'", specifier = ">=1.16.0" }, - { name = "smg-grpc-servicer", extras = ["vllm"], marker = "extra == 'grpc'", specifier = ">=0.5.2" }, - { name = "soundfile", marker = "extra == 'audio'" }, - { name = "soxr", marker = "extra == 'audio'" }, - { name = "starlette", specifier = ">=1.0.1" }, - { name = "tensorizer", marker = "extra == 'tensorizer'", specifier = "==2.10.1" }, - { name = "tiktoken", specifier = ">=0.6.0" }, - { name = "tilelang", specifier = "==0.1.9" }, - { name = "tokenizers", specifier = ">=0.21.1" }, - { name = "tokenspeed-mla", marker = "sys_platform == 'linux'", specifier = "==0.1.2" }, - { name = "torch", specifier = "==2.11.0" }, - { name = "torchaudio", specifier = "==2.11.0" }, - { name = "torchcodec", specifier = ">=0.14" }, - { name = "torchvision", specifier = "==0.26.0" }, - { name = "tqdm" }, - { name = "transformers", specifier = ">=5.5.3" }, - { name = "typing-extensions", specifier = ">=4.10" }, - { name = "vllm-gguf-plugin", marker = "extra == 'extra-quant'", specifier = ">=0.0.2" }, - { name = "watchfiles" }, - { name = "xgrammar", marker = "platform_machine == 'aarch64' or platform_machine == 'arm64' or platform_machine == 'ppc64le' or platform_machine == 's390x' or platform_machine == 'x86_64'", specifier = ">=0.2.1,<1.0.0" }, - { name = "zentorch", marker = "extra == 'zen'", specifier = "==2.11.0.0" }, -] -provides-extras = ["zen", "bench", "tensorizer", "fastsafetensors", "instanttensor", "runai", "audio", "video", "flashinfer", "helion", "grpc", "otel", "extra-quant"] - [[package]] name = "vllm" version = "0.25.1" @@ -8638,26 +7327,26 @@ dependencies = [ { name = "depyf", marker = "platform_machine == 'x86_64'" }, { name = "diskcache", marker = "platform_machine == 'x86_64'" }, { name = "einops", marker = "platform_machine == 'x86_64'" }, - { name = "fastapi", version = "0.136.3", source = { registry = "https://pypi.org/simple" }, extra = ["standard"], marker = "(platform_machine == 'x86_64' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-vllm' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-8-nemo-gym-vllm')" }, + { name = "fastapi", version = "0.136.3", source = { registry = "https://pypi.org/simple" }, extra = ["standard"], marker = "(platform_machine == 'x86_64' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm')" }, { name = "fastsafetensors", marker = "platform_machine == 'x86_64'" }, { name = "filelock", marker = "platform_machine == 'x86_64'" }, { name = "flashinfer-cubin", version = "0.6.13", source = { registry = "https://pypi.org/simple" }, marker = "platform_machine == 'x86_64'" }, { name = "flashinfer-python", version = "0.6.13", source = { registry = "https://pypi.org/simple" }, marker = "platform_machine == 'x86_64'" }, - { name = "humming-kernels", version = "0.1.10", source = { registry = "https://pypi.org/simple" }, extra = ["cu13"], marker = "(platform_machine == 'x86_64' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-vllm' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-8-nemo-gym-vllm')" }, + { name = "humming-kernels", extra = ["cu13"], marker = "(platform_machine == 'x86_64' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm')" }, { name = "ijson", marker = "platform_machine == 'x86_64'" }, { name = "jsonschema", marker = "platform_machine == 'x86_64'" }, { name = "lark", marker = "platform_machine == 'x86_64'" }, { name = "llguidance", marker = "platform_machine == 'x86_64'" }, { name = "lm-format-enforcer", marker = "platform_machine == 'x86_64'" }, { name = "mcp", marker = "platform_machine == 'x86_64'" }, - { name = "mistral-common", marker = "(platform_machine == 'x86_64' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-vllm' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-8-nemo-gym-vllm')" }, + { name = "mistral-common", marker = "(platform_machine == 'x86_64' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm')" }, { name = "model-hosting-container-standards", marker = "platform_machine == 'x86_64'" }, { name = "msgspec", marker = "platform_machine == 'x86_64'" }, { name = "ninja", marker = "platform_machine == 'x86_64'" }, { name = "numba", marker = "platform_machine == 'x86_64'" }, { name = "numpy", marker = "platform_machine == 'x86_64'" }, { name = "nvidia-cudnn-frontend", marker = "platform_machine == 'x86_64'" }, - { name = "nvidia-cutlass-dsl", version = "4.5.2", source = { registry = "https://pypi.org/simple" }, extra = ["cu13"], marker = "(platform_machine == 'x86_64' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-vllm' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-8-nemo-gym-vllm')" }, + { name = "nvidia-cutlass-dsl", version = "4.5.2", source = { registry = "https://pypi.org/simple" }, extra = ["cu13"], marker = "(platform_machine == 'x86_64' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm')" }, { name = "nvtx", marker = "platform_machine == 'x86_64'" }, { name = "openai", marker = "platform_machine == 'x86_64'" }, { name = "openai-harmony", marker = "platform_machine == 'x86_64'" }, @@ -8706,109 +7395,6 @@ wheels = [ { url = "https://github.com/vllm-project/vllm/releases/download/v0.25.1/vllm-0.25.1-cp38-abi3-manylinux_2_28_x86_64.whl", hash = "sha256:16fc7a28df1576eb6f7ca0455026551b8f9adb674c19c66059359ef3e964bd1e" }, ] -[package.metadata] -requires-dist = [ - { name = "aiohttp", specifier = ">=3.13.3" }, - { name = "anthropic", specifier = ">=0.71.0" }, - { name = "apache-tvm-ffi", specifier = "==0.1.9" }, - { name = "av", marker = "extra == 'audio'" }, - { name = "blake3" }, - { name = "cachetools" }, - { name = "cbor2" }, - { name = "cloudpickle" }, - { name = "compressed-tensors", specifier = "==0.17.0" }, - { name = "datasets", marker = "extra == 'bench'" }, - { name = "depyf", specifier = "==0.20.0" }, - { name = "diskcache", specifier = "==5.6.3" }, - { name = "einops" }, - { name = "fastapi", extras = ["standard"], specifier = ">=0.133.0,<0.137.0" }, - { name = "fastsafetensors", specifier = ">=0.3.2" }, - { name = "fastsafetensors", marker = "extra == 'fastsafetensors'", specifier = ">=0.3.2" }, - { name = "filelock", specifier = ">=3.16.1" }, - { name = "flashinfer-cubin", specifier = "==0.6.13" }, - { name = "flashinfer-python", specifier = "==0.6.13" }, - { name = "helion", marker = "extra == 'helion'", specifier = "==1.1.0" }, - { name = "humming-kernels", extras = ["cu13"], specifier = "==0.1.10" }, - { name = "ijson" }, - { name = "instanttensor", marker = "extra == 'instanttensor'", specifier = ">=0.1.5" }, - { name = "jsonschema", specifier = ">=4.23.0" }, - { name = "lark", specifier = "==1.2.2" }, - { name = "llguidance", marker = "platform_machine == 'aarch64' or platform_machine == 'arm64' or platform_machine == 'ppc64le' or platform_machine == 'x86_64'", specifier = ">=1.7.0,<1.8.0" }, - { name = "lm-format-enforcer", specifier = "==0.11.3" }, - { name = "matplotlib", marker = "extra == 'bench'" }, - { name = "mcp" }, - { name = "mistral-common", extras = ["audio"], marker = "extra == 'audio'" }, - { name = "mistral-common", extras = ["image"], specifier = ">=1.11.5" }, - { name = "model-hosting-container-standards", specifier = ">=0.1.14,<1.0.0" }, - { name = "msgspec" }, - { name = "ninja" }, - { name = "numba", specifier = "==0.65.0" }, - { name = "numpy" }, - { name = "nvidia-cudnn-frontend", specifier = ">=1.19.1" }, - { name = "nvidia-cutlass-dsl", extras = ["cu13"], specifier = "==4.5.2" }, - { name = "nvtx", specifier = "==0.2.15" }, - { name = "openai", specifier = ">=2.0.0" }, - { name = "openai-harmony", specifier = ">=0.0.3" }, - { name = "opencv-python-headless", specifier = ">=4.13.0" }, - { name = "opentelemetry-api", specifier = ">=1.27.0" }, - { name = "opentelemetry-api", marker = "extra == 'otel'", specifier = ">=1.26.0" }, - { name = "opentelemetry-exporter-otlp", specifier = ">=1.27.0" }, - { name = "opentelemetry-exporter-otlp", marker = "extra == 'otel'", specifier = ">=1.26.0" }, - { name = "opentelemetry-sdk", specifier = ">=1.27.0" }, - { name = "opentelemetry-sdk", marker = "extra == 'otel'", specifier = ">=1.26.0" }, - { name = "opentelemetry-semantic-conventions-ai", specifier = ">=0.4.1" }, - { name = "opentelemetry-semantic-conventions-ai", marker = "extra == 'otel'", specifier = ">=0.4.1" }, - { name = "outlines-core", specifier = "==0.2.14" }, - { name = "pandas", marker = "extra == 'bench'" }, - { name = "partial-json-parser" }, - { name = "pillow" }, - { name = "plotly", marker = "extra == 'bench'" }, - { name = "prometheus-client", specifier = ">=0.18.0" }, - { name = "prometheus-fastapi-instrumentator", specifier = ">=8.0.0" }, - { name = "protobuf", specifier = ">=5.29.6,!=6.30.*,!=6.31.*,!=6.32.*,!=6.33.0.*,!=6.33.1.*,!=6.33.2.*,!=6.33.3.*,!=6.33.4.*" }, - { name = "psutil" }, - { name = "py-cpuinfo" }, - { name = "pybase64" }, - { name = "pydantic", specifier = ">=2.12.0" }, - { name = "pynvvideocodec", specifier = "==2.0.4" }, - { name = "python-json-logger" }, - { name = "pyyaml" }, - { name = "pyzmq", specifier = ">=25.0.0" }, - { name = "quack-kernels", specifier = ">=0.3.3" }, - { name = "regex" }, - { name = "requests", specifier = ">=2.26.0" }, - { name = "runai-model-streamer", extras = ["azure", "gcs", "s3"], marker = "extra == 'runai'", specifier = ">=0.15.7" }, - { name = "safetensors", specifier = ">=0.6.2" }, - { name = "scipy", marker = "extra == 'audio'" }, - { name = "scipy", marker = "extra == 'bench'" }, - { name = "seaborn", marker = "extra == 'bench'" }, - { name = "sentencepiece" }, - { name = "setproctitle" }, - { name = "setuptools", marker = "python_full_version >= '3.12'", specifier = ">=77.0.3,<81.0.0" }, - { name = "six", marker = "python_full_version >= '3.12'", specifier = ">=1.16.0" }, - { name = "smg-grpc-servicer", extras = ["vllm"], marker = "extra == 'grpc'", specifier = ">=0.5.2" }, - { name = "soundfile", marker = "extra == 'audio'" }, - { name = "soxr", marker = "extra == 'audio'" }, - { name = "starlette", specifier = ">=1.0.1" }, - { name = "tensorizer", marker = "extra == 'tensorizer'", specifier = "==2.10.1" }, - { name = "tiktoken", specifier = ">=0.6.0" }, - { name = "tilelang", specifier = "==0.1.9" }, - { name = "tokenizers", specifier = ">=0.21.1" }, - { name = "tokenspeed-mla", marker = "sys_platform == 'linux'", specifier = "==0.1.2" }, - { name = "torch", specifier = "==2.11.0" }, - { name = "torchaudio", specifier = "==2.11.0" }, - { name = "torchcodec", specifier = ">=0.14" }, - { name = "torchvision", specifier = "==0.26.0" }, - { name = "tqdm" }, - { name = "transformers", specifier = ">=5.5.3" }, - { name = "typing-extensions", specifier = ">=4.10" }, - { name = "vllm-gguf-plugin", marker = "extra == 'extra-quant'", specifier = ">=0.0.2" }, - { name = "watchfiles" }, - { name = "xgrammar", marker = "platform_machine == 'aarch64' or platform_machine == 'arm64' or platform_machine == 'ppc64le' or platform_machine == 's390x' or platform_machine == 'x86_64'", specifier = ">=0.2.1,<1.0.0" }, - { name = "zentorch", marker = "extra == 'zen'", specifier = "==2.11.0.0" }, -] -provides-extras = ["zen", "bench", "tensorizer", "fastsafetensors", "instanttensor", "runai", "audio", "video", "flashinfer", "helion", "grpc", "otel", "extra-quant"] - [[package]] name = "wandb" version = "0.28.1" @@ -8886,15 +7472,6 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/d9/00/aca6beb3658dab4ed3dbb41a78e6e7f31342e0b41d28088f205525751601/webdataset-1.0.2-py3-none-any.whl", hash = "sha256:3dbfced32b25c0d199c6b9787937b6f85742bc3c84f652c846893075c1c082d9", size = 74956, upload-time = "2025-06-19T23:26:20.354Z" }, ] -[[package]] -name = "websocket-client" -version = "1.9.0" -source = { registry = "https://pypi.org/simple" } -sdist = { url = "https://files.pythonhosted.org/packages/2c/41/aa4bf9664e4cda14c3b39865b12251e8e7d239f4cd0e3cc1b6c2ccde25c1/websocket_client-1.9.0.tar.gz", hash = "sha256:9e813624b6eb619999a97dc7958469217c3176312b3a16a4bd1bc7e08a46ec98", size = 70576, upload-time = "2025-10-07T21:16:36.495Z" } -wheels = [ - { url = "https://files.pythonhosted.org/packages/34/db/b10e48aa8fff7407e67470363eac595018441cf32d5e1001567a7aeba5d2/websocket_client-1.9.0-py3-none-any.whl", hash = "sha256:af248a825037ef591efbf6ed20cc5faa03d3b47b9e5a2230a529eeee1c1fc3ef", size = 82616, upload-time = "2025-10-07T21:16:34.951Z" }, -] - [[package]] name = "websockets" version = "16.1.1" @@ -9014,9 +7591,9 @@ dependencies = [ { name = "numpy" }, { name = "pydantic" }, { name = "torch" }, - { name = "transformers", version = "5.5.4", source = { registry = "https://pypi.org/simple" }, marker = "extra == 'extra-7-nemo-rl-trtllm' or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-vllm' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-trtllm' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-mcore' and extra != 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-mcore' and extra != 'extra-7-nemo-rl-trtllm' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra != 'extra-7-nemo-rl-mcore' and extra != 'extra-7-nemo-rl-trtllm' and extra != 'extra-7-nemo-rl-vllm' and extra != 'extra-8-nemo-gym-vllm') or (extra != 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-mcore' and extra != 'extra-7-nemo-rl-sglang' and extra != 'extra-7-nemo-rl-vllm' and extra != 'extra-8-nemo-gym-vllm')" }, - { name = "transformers", version = "5.6.0", source = { registry = "https://pypi.org/simple" }, marker = "(extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-8-nemo-gym-vllm') or (extra != 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra != 'extra-7-nemo-rl-fsdp' and extra != 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-8-nemo-gym-vllm') or (extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-8-nemo-gym-vllm') or (extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm' and extra == 'extra-8-nemo-gym-vllm')" }, - { name = "transformers", version = "5.8.1", source = { registry = "https://pypi.org/simple" }, marker = "(extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-8-nemo-gym-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-8-nemo-gym-vllm') or (extra != 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-vllm') or (extra != 'extra-7-nemo-rl-automodel' and extra == 'extra-8-nemo-gym-vllm') or (extra != 'extra-7-nemo-rl-fsdp' and extra != 'extra-7-nemo-rl-mcore' and extra != 'extra-7-nemo-rl-sglang' and extra != 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra != 'extra-7-nemo-rl-fsdp' and extra != 'extra-7-nemo-rl-mcore' and extra != 'extra-7-nemo-rl-sglang' and extra != 'extra-7-nemo-rl-trtllm' and extra == 'extra-8-nemo-gym-vllm')" }, + { name = "transformers", version = "5.5.4", source = { registry = "https://pypi.org/simple" }, marker = "extra == 'extra-7-nemo-rl-trtllm' or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-mcore' and extra != 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra != 'extra-7-nemo-rl-mcore' and extra != 'extra-7-nemo-rl-trtllm' and extra != 'extra-7-nemo-rl-vllm') or (extra != 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-mcore' and extra != 'extra-7-nemo-rl-sglang' and extra != 'extra-7-nemo-rl-vllm')" }, + { name = "transformers", version = "5.6.0", source = { registry = "https://pypi.org/simple" }, marker = "(extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra != 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra != 'extra-7-nemo-rl-fsdp' and extra != 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-automodel' and extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra != 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm')" }, + { name = "transformers", version = "5.8.1", source = { registry = "https://pypi.org/simple" }, marker = "(extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-fsdp') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-mcore') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-fsdp' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-sglang') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-mcore' and extra == 'extra-7-nemo-rl-vllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-trtllm') or (extra == 'extra-7-nemo-rl-sglang' and extra == 'extra-7-nemo-rl-vllm') or (extra != 'extra-7-nemo-rl-automodel' and extra == 'extra-7-nemo-rl-vllm') or (extra != 'extra-7-nemo-rl-fsdp' and extra != 'extra-7-nemo-rl-mcore' and extra != 'extra-7-nemo-rl-sglang' and extra != 'extra-7-nemo-rl-trtllm' and extra == 'extra-7-nemo-rl-vllm')" }, { name = "triton", marker = "platform_machine == 'x86_64'" }, { name = "typing-extensions" }, ]