[fix] Standardize rollout trajectory records and document producer coverage

[Glorf]

Summary

• add a versioned ng_trajectory projection for normalized model calls, token statistics, semantic turns, tool observations, and invocation-scoped model-visible history
• preserve provider-reported cached, reasoning, and total token usage across capture, conversion, and multi-turn agent loops
• preserve Responses lifecycle status as response_status without reinterpreting dialect-specific finish_reason
• add an opt-in reference trajectory producer for Simple Agent and document current V/O/X coverage for C1-C7 across all 34 agents

This PR establishes the shared schema and one reference producer. The capability matrix reports current support; it does not claim that every producer satisfies C1-C7.

Correctness and compatibility

• collector-derived task and rollout identities are canonical; producer mismatches are recorded and turn identities are normalized without dropping the producer trajectory
• producer invocations remain authoritative on duplicate IDs, observation-only invocations are appended, and model and tool records are merged only by exact identifiers
• projection failures retain an explicit gap and captured request and response payloads; successful projections remove the duplicate capture payloads
• Simple Agent records the actual per-turn model input, cumulative tool-step count, and an explicit gap when resolution is unavailable
• no existing fields are removed; response lifecycle status, provider token details and totals, and cached-token aggregation are intentional correctness changes; ng_trajectory is additive
• Simple Agent retains self-dispatch when observability is disabled or responses() is overridden
• trajectory collection remains gated by observability and independent of the token-ID capture correlation introduced by feat(token-id-capture): capture training tokens from external harnesses #2124
• captured request and response payloads are retained in persisted ng_trajectory; LabBench's multimodal_history_redacted gap omits those copies
• W&B rollout tables omit ng_trajectory and model-call request and response payloads during collection and reverification
• payload projection has no separate size cap or trajectory-specific opt-out; model-call capture remains opt-in

NVBug alignment

• NVBug 6535274 is addressed with its preferred lossless option: completed Responses calls preserve response_status="completed"; finish_reason remains unset when the dialect does not provide one. The inference_provider path now uses the shared Chat-to-Responses converter, so the fix applies to the backend named in the report.
• This is the schema and reference-producer phase of NVBug 6555643 / [epic] standardised trajectories format #1867. The capability matrix marks unsupported and path-dependent producers as X or O; it is not a claim of full producer coverage.

Follow-up pull requests

PRs #2115 and #2117-#2120 are stacked; later PRs include earlier stack changes. Open PRs describe planned evidence and do not affect the current capability matrix. #2153 provides Claude Code tool timing and status; this PR joins tool results to those observations. No available PR from #2115 through #2122 adds canonical C3 turns.

PR	Producer or path	Evidence added	Remaining gap
#2115	OpenClaw, PinchBench	Correlated model calls, conversations, parallel tool timing, and sandbox observations	No C3 turns
#2116	Claude Code	Superseded by #2153	See #2153 and this PR
#2117	Hermes	Parallel tool timing and agent observations	No C3 turns
#2118	Pi	Correlated model calls, conversations, and parallel tool timing	No C3 turns
#2119	OpenCode	Rollout-level model calls, retained conversations, and parallel tool timing	No exact per-invocation model-call ownership; no C3 turns
#2120	SWE OpenCode, OpenHands	OpenCode model-call correlation plus retained conversations and sandbox observations for both paths	OpenHands calls, tool timing, and C3 turns remain unavailable
#2121	—	No PR exists	—
#2122	Stirrup, GDPVal	GDPVal judge-call correlation; Stirrup and Tau2 model-call correlation is already present	No agent turns or tool observations

Validation

• 354 capture, conversion, trajectory, collector, reverification, Fern-link, and inference-provider tests passed, plus 5 subtests
• 10 Simple Agent producer and dispatch tests passed
• in-process endpoint-to-record test passed: prefixed Responses request → inference_provider → capture middleware → CaptureStore → ModelCallRecord
• producer-to-collector-to-JSON trajectory round trip passed with both available and unavailable resolution status
• 110 vLLM model and Responses conversion tests passed
• 67 Claude Code observation tests and 4 LabBench redaction tests passed
• the trajectory patch merges cleanly with feat(token-id-capture): capture training tokens from external harnesses #2124; combined-stack token-capture and trajectory tests passed
• Ruff check, Ruff format check, and git diff --check passed

Part of #1867.

[copy-pr-bot]

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

[github-actions]

🌿 Preview your docs: https://nvidia-preview-agent-standardize-rollout-trajectories.docs.buildwithfern.com/nemo/gym

Here are the markdown pages you've updated:

• nemo/gym/about/concepts/key-terminology
• nemo/gym/model-server/model-call-capture
• nemo/gym/reference
• nemo/gym/reference/trajectory-capabilities

[mlazuka]

Lgtm :)