docs: parent resolution, prefix supply and the read-route token - #2340
Closed
ananthsub wants to merge 1 commit into
Closed
docs: parent resolution, prefix supply and the read-route token#2340ananthsub wants to merge 1 commit into
ananthsub wants to merge 1 commit into
Conversation
|
Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually. Contributors can view more details about this message here. |
Contributor
|
🌿 Preview your docs: https://nvidia-preview-ananthsub-tokidcap-docs-extras.docs.buildwithfern.com/nemo/gym Here are the markdown pages you've updated: |
ananthsub
force-pushed
the
ananthsub/tokidcap/read-route
branch
from
August 5, 2026 01:13
3dea978 to
cbf548b
Compare
ananthsub
force-pushed
the
ananthsub/tokidcap/docs-extras
branch
from
August 5, 2026 01:13
307b288 to
4bf5fa3
Compare
ananthsub
force-pushed
the
ananthsub/tokidcap/read-route
branch
from
August 5, 2026 01:21
cbf548b to
cb2f6c6
Compare
ananthsub
force-pushed
the
ananthsub/tokidcap/docs-extras
branch
from
August 5, 2026 01:21
4bf5fa3 to
7a7a0ca
Compare
ananthsub
marked this pull request as ready for review
August 5, 2026 01:30
ananthsub
force-pushed
the
ananthsub/tokidcap/read-route
branch
from
August 5, 2026 02:13
cb2f6c6 to
6bad3e2
Compare
ananthsub
force-pushed
the
ananthsub/tokidcap/docs-extras
branch
from
August 5, 2026 02:13
7a7a0ca to
bdebca8
Compare
ananthsub
force-pushed
the
ananthsub/tokidcap/docs-extras
branch
from
August 5, 2026 15:21
bdebca8 to
b308f9d
Compare
This was referenced Aug 5, 2026
Adds the parts of the external-harness page that describe the optimizations on top of capture: how a rollout's calls are linked and what a miss costs, what prefix supply needs from a backend, and the bearer token on the read route. Signed-off-by: Ananth Subramaniam <ansubramania@nvidia.com>
ananthsub
force-pushed
the
ananthsub/tokidcap/docs-extras
branch
from
August 5, 2026 15:57
b308f9d to
3d0fdd0
Compare
ananthsub
force-pushed
the
ananthsub/tokidcap/read-route
branch
from
August 5, 2026 16:00
6bad3e2 to
79a1883
Compare
Contributor
Author
|
Closing and reopening against the correct base. This PR was based on |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Documents the two optimizations that sit on top of the base feature.
Extends the page added in #2341 with:
How a rollout's calls are linked (#2180). By default calls chain by token prefix, which is correct but cannot tell two attempts of a retried call apart. The model server therefore identifies the parent at request time, by hashing the model-authored turns of an incoming request, and the page explains the two checks that must pass before a match is used and why a miss is harmless.
Prefix supply (#2181). What
required_prefix_token_idsdoes, why re-rendering a conversation can split an assistant turn differently than the model sampled it, and that the backend has to honour the field for any of it to matter.Both sections lead with the symptom a reader would actually see,
delivered_fractionbelow 1.0 withchainsabove 1, since that is what sends someone to this page.Depends on #2181.