feat(vllm): forward session IDs to upstream routers

[NolenLiang]

Summary

• add optional session-affinity header forwarding to the vLLM model configuration
• optionally forward the stable NeMo Gym session ID in session_params.session_id for cache-aware routers that consume request-body identity
• preserve existing default headers, other session_params fields, and session-to-client reuse
• request and consume inline prompt/generation token IDs while retaining token-string and /tokenize fallbacks
• expose attach_trajectory_record as the public integration point for callers that use run_examples directly

Motivation

When NeMo Gym sends requests through one upstream Router endpoint, its in-process session-to-client mapping does not by itself expose a stable session identity to that Router. Header forwarding supports consistent-hash routing, while the optional request-body field supports vLLM Router 0.1.15 cache-aware routing.

NeMo RL also uses Gym's low-level run_examples path. Exporting the existing trajectory attachment helper lets that caller fold captured model-call timing, token usage, response metadata, and observation gaps into the standard ng_trajectory result without duplicating Gym logic or relying on out-of-band state.

Compatibility

Session-affinity forwarding is opt-in, so existing configurations retain their previous request shape. Existing session_params fields are preserved. Invalid non-mapping values are rejected instead of being silently overwritten.

External audit provenance

The two incremental commits originally published as 9ad35b3 and f075ce8 were consolidated only to correct the Git author identity. The current commit 4d6b55a has:

• author and committer: Nolen Liang <nliang@nvidia.com>
• DCO sign-off: Nolen Liang <nliang@nvidia.com>
• source tree: e0807bfc5c40cf282c07b33aa8178c124dda4e9f, exactly identical to the former f075ce8 tree

The parent NeMo RL Phase 2 matrix observed complete model-call timing for all six accepted formal runs using this source tree.

Testing

• focused tests for header forwarding, request-body session identity, field preservation, invalid types, inline token IDs, and capture behavior
• parent NeMo RL integration tests for cache-aware body forwarding and consistent-hash compatibility
• Ruff check and format check for the modified Gym files
• DCO check

[copy-pr-bot]

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.