(1/2) refactor(rollout): drop --generate-multi-samples and its per-turn sample semantics - #1916
Merged
Merged
Conversation
…ple semantics The flag made every turn a separate full-context sample by skipping the TITO merge. That is incorrect: sample boundaries belong to the trajectory, not a CLI switch, and per-turn siblings share one group_index and pollute group baselines. Linear multi-turn generation now accumulates one scalar sample. Agentic generation always merges the recorded linear session and returns list[Sample], including aborted paths, so a batch does not mix scalar and list shapes. Consequently, custom reward models on the agentic path receive list[Sample], dynamic filters flatten nested groups, and agentic use with group RM, partial rollout, or recompute-logprobs-via-prefill remains unsupported.
…ample]] A dynamic-filter group mixes scalar and list elements (agentic generate returns list[Sample] per call), which is exactly why _flatten_samples exists; the old list[Sample] annotation misdocumented that contract.
guapisolo
requested review from
Shi-Dong,
fzyzcjy and
yueming-yuan
as code owners
July 28, 2026 19:08
Contributor
|
Caution The consumer version of Gemini Code Assist on GitHub has been sunset. All code review activity has officially ceased. |
guapisolo
commented
Jul 28, 2026
| - `compute_prompt_ids_from_sample` and `compute_request_payload` from | ||
| `miles/rollout/generate_utils/generate_endpoint_utils.py` build `/generate` requests. | ||
| - For multi-sample outputs, set `--generate-multi-samples` and return a list. | ||
| - Returning a `list[Sample]` from a generate function is supported natively; no flag is needed. |
Collaborator
Author
There was a problem hiding this comment.
Returning a Sample or list[Sample].
No need to mention "flag is needed or supported natively"
Collaborator
Author
There was a problem hiding this comment.
Updated in eae3bdf: the docs now state that GenerateFnOutput.samples accepts a Sample or list[Sample], without tying that contract to a flag or “native support.”
guapisolo
commented
Jul 28, 2026
|
|
||
| <Warning> | ||
|
|
||
| **Agentic output is a `list[Sample]`.** `agentic_tool_call.generate` always returns a list |
Collaborator
Author
There was a problem hiding this comment.
Not always actually.
Collaborator
Author
There was a problem hiding this comment.
Updated in eae3bdf: I removed the list-only agentic warning. agentic_tool_call.generate now returns the merged scalar Sample on success and aborted paths, while GenerateFnOutput.samples continues to support both shapes for custom generators.
The swe-agent-v2 example is planned for deprecation, so this rollout refactor should not update its guidance. Restore the README to the main-branch version and keep the PR focused on supported rollout paths.
Removing --generate-multi-samples eliminates flag-controlled per-turn samples; it does not narrow GenerateFnOutput.samples. Return the merged linear agentic trajectory as a scalar Sample while preserving Sample | list[Sample] for custom generators.
guapisolo
added a commit
that referenced
this pull request
Jul 29, 2026
…nal overlay
Select the samples-wire allowlist when each tracer is created. V1 direct constructions retain the base fields and the exact {max_seq_len: ...} request body; v2 adds agent metadata to the collection body and decodes the reward field returned by tree assembly.
agentic_tool_call keeps the #1916 scalar contract for v1 and returns list[Sample] only for v2 multi-leaf output. V2 applies agent metadata on the server, while v1 retains the driver overlays. Collection failures continue to propagate, empty replies return an ABORTED value with the version-appropriate shape, and unsupported v2 combinations with --group-rm, --partial-rollout, or --recompute-logprobs-via-prefill fail before creating a session.
Pins cover the v1 payload, version-based field selection, the v2 metadata and reward channel, v2 success and empty shapes, failure propagation, and unsupported consumers. The rollout endpoint guide documents the conditional output contract.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
guapisolo
added a commit
that referenced
this pull request
Jul 30, 2026
…nal overlay
Select the samples-wire allowlist when each tracer is created. V1 direct constructions retain the base fields and the exact {max_seq_len: ...} request body; v2 adds agent metadata to the collection body and decodes the reward field returned by tree assembly.
agentic_tool_call keeps the #1916 scalar contract for v1 and returns list[Sample] only for v2 multi-leaf output. V2 applies agent metadata on the server, while v1 retains the driver overlays. Collection failures continue to propagate, empty replies return an ABORTED value with the version-appropriate shape, and unsupported v2 combinations with --group-rm, --partial-rollout, or --recompute-logprobs-via-prefill fail before creating a session.
Pins cover the v1 payload, version-based field selection, the v2 metadata and reward channel, v2 success and empty shapes, failure propagation, and unsupported consumers. The rollout endpoint guide documents the conditional output contract.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
guapisolo
added a commit
that referenced
this pull request
Jul 31, 2026
…nal overlay
Select the samples-wire allowlist when each tracer is created. V1 direct constructions retain the base fields and the exact {max_seq_len: ...} request body; v2 adds agent metadata to the collection body and decodes the reward field returned by tree assembly.
agentic_tool_call keeps the #1916 scalar contract for v1 and returns list[Sample] only for v2 multi-leaf output. V2 applies agent metadata on the server, while v1 retains the driver overlays. Collection failures continue to propagate, empty replies return an ABORTED value with the version-appropriate shape, and unsupported v2 combinations with --group-rm, --partial-rollout, or --recompute-logprobs-via-prefill fail before creating a session.
Pins cover the v1 payload, version-based field selection, the v2 metadata and reward channel, v2 success and empty shapes, failure propagation, and unsupported consumers. The rollout endpoint guide documents the conditional output contract.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
guapisolo
added a commit
that referenced
this pull request
Jul 31, 2026
…nal overlay
Select the samples-wire allowlist when each tracer is created. V1 direct constructions retain the base fields and the exact {max_seq_len: ...} request body; v2 adds agent metadata to the collection body and decodes the reward field returned by tree assembly.
agentic_tool_call keeps the #1916 scalar contract for v1 and returns list[Sample] only for v2 multi-leaf output. V2 applies agent metadata on the server, while v1 retains the driver overlays. Collection failures continue to propagate, empty replies return an ABORTED value with the version-appropriate shape, and unsupported v2 combinations with --group-rm, --partial-rollout, or --recompute-logprobs-via-prefill fail before creating a session.
Pins cover the v1 payload, version-based field selection, the v2 metadata and reward channel, v2 success and empty shapes, failure propagation, and unsupported consumers. The rollout endpoint guide documents the conditional output contract.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
guapisolo
added a commit
that referenced
this pull request
Aug 1, 2026
…nal overlay
Select the samples-wire allowlist when each tracer is created. V1 direct constructions retain the base fields and the exact {max_seq_len: ...} request body; v2 adds agent metadata to the collection body and decodes the reward field returned by tree assembly.
agentic_tool_call keeps the #1916 scalar contract for v1 and returns list[Sample] only for v2 multi-leaf output. V2 applies agent metadata on the server, while v1 retains the driver overlays. Collection failures continue to propagate, empty replies return an ABORTED value with the version-appropriate shape, and unsupported v2 combinations with --group-rm, --partial-rollout, or --recompute-logprobs-via-prefill fail before creating a session.
Pins cover the v1 payload, version-based field selection, the v2 metadata and reward channel, v2 success and empty shapes, failure propagation, and unsupported consumers. The rollout endpoint guide documents the conditional output contract.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
guapisolo
added a commit
that referenced
this pull request
Aug 1, 2026
…nal overlay
Select the samples-wire allowlist when each tracer is created. V1 direct constructions retain the base fields and the exact {max_seq_len: ...} request body; v2 adds agent metadata to the collection body and decodes the reward field returned by tree assembly.
agentic_tool_call keeps the #1916 scalar contract for v1 and returns list[Sample] only for v2 multi-leaf output. V2 applies agent metadata on the server, while v1 retains the driver overlays. Collection failures continue to propagate, empty replies return an ABORTED value with the version-appropriate shape, and unsupported v2 combinations with --group-rm, --partial-rollout, or --recompute-logprobs-via-prefill fail before creating a session.
Pins cover the v1 payload, version-based field selection, the v2 metadata and reward channel, v2 success and empty shapes, failure propagation, and unsupported consumers. The rollout endpoint guide documents the conditional output contract.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
guapisolo
added a commit
that referenced
this pull request
Aug 1, 2026
…nal overlay
Select the samples-wire allowlist when each tracer is created. V1 direct constructions retain the base fields and the exact {max_seq_len: ...} request body; v2 adds agent metadata to the collection body and decodes the reward field returned by tree assembly.
agentic_tool_call keeps the #1916 scalar contract for v1 and returns list[Sample] only for v2 multi-leaf output. V2 applies agent metadata on the server, while v1 retains the driver overlays. Collection failures continue to propagate, empty replies return an ABORTED value with the version-appropriate shape, and unsupported v2 combinations with --group-rm, --partial-rollout, or --recompute-logprobs-via-prefill fail before creating a session.
Pins cover the v1 payload, version-based field selection, the v2 metadata and reward channel, v2 success and empty shapes, failure propagation, and unsupported consumers. The rollout endpoint guide documents the conditional output contract.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
guapisolo
added a commit
that referenced
this pull request
Aug 1, 2026
…nal overlay
Select the samples-wire allowlist when each tracer is created. V1 direct constructions retain the base fields and the exact {max_seq_len: ...} request body; v2 adds agent metadata to the collection body and decodes the reward field returned by tree assembly.
agentic_tool_call keeps the #1916 scalar contract for v1 and returns list[Sample] only for v2 multi-leaf output. V2 applies agent metadata on the server, while v1 retains the driver overlays. Collection failures continue to propagate, empty replies return an ABORTED value with the version-appropriate shape, and unsupported v2 combinations with --group-rm, --partial-rollout, or --recompute-logprobs-via-prefill fail before creating a session.
Pins cover the v1 payload, version-based field selection, the v2 metadata and reward channel, v2 success and empty shapes, failure propagation, and unsupported consumers. The rollout endpoint guide documents the conditional output contract.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Remove
--generate-multi-sampleswithout narrowing the generate output contract.Motivation
The flag made every turn a separate full-context sample by skipping the TITO merge. Sample boundaries belong to trajectory topology, not a CLI switch; treating turns as siblings gives them one
group_indexand pollutes group baselines.Removing the flag does not make
GenerateFnOutput.sampleslist-only. The contract remainsSample | list[Sample]:list[Sample].Before / After
examples/experimental/swe-agent-v2/README.mdremains unchanged.Behavior Preservation
list[Sample]output.Verification
test_multi_turn.pypassed 24 tests with 7 expected skips.Review Focus
GenerateFnOutput.samplesremainsSample | list[Sample]even though the built-in linear generators return scalar samples./samplesendpoint, codec, or HTTP transport change.