fix(vllm-model): consume native Dynamo token metadata - #1963
Closed
jthomson04 wants to merge 4 commits into
Closed
Conversation
…nize When per-message prompt_token_ids/generation_token_ids are attached to assistant messages (training mode), populate the top-level required_prefix_token_ids field on both the chat-completion request and the separate tokenize request. Mirrors NeMoRLOpenAIChatRequestMixin auto-derive in nemo-rl's custom vLLM serving (vllm_worker_async.py). Without this, Dynamo - which has the splice machinery server-side but no auto-derive - re-tokenizes the chat history each turn, breaking the byte-level token-contiguity invariant on multi-turn rollouts. The fix must apply to BOTH endpoints because the contiguity assert in nemo_rl/environments/nemo_gym.py reads prompt_token_ids from the tokenize response, not the chat response. Patching only chat fails at the tokenize step. Signed-off-by: jthomson04 <jwillthomson19@gmail.com>
Signed-off-by: jthomson04 <jwillthomson19@gmail.com>
Signed-off-by: jthomson04 <jwillthomson19@gmail.com>
Signed-off-by: jthomson04 <jwillthomson19@gmail.com>
Contributor
Author
|
Closing as a duplicate of #1784. The native Dynamo logprob fixes are now on the existing PR branch, rebased onto current main. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changed
nvext.engine_datacompletion_logprobsfrom the same native engine payload instead of pairing native IDs with post-processed choice logprobs/tokenizefallback when native engine metadata is absentWhy
Dynamo can post-process an OpenAI-compatible choice differently from the native engine completion. In the V5 NeMo-RL workload, the native response contained 566 completion token IDs while the post-processed choice contained 565 logprobs. Combining those two sources caused rollout construction to fail. Dynamo also does not expose the vLLM
/tokenizeendpoint used by the fallback path.Keeping token IDs and logprobs from the same native engine payload preserves one-to-one alignment and avoids unnecessary tokenization.
Validation
pytest responses_api_models/vllm_model/tests/test_app.py -k TokenIDInformation— 4 passedpytest responses_api_models/vllm_model/tests/test_app.py— 75 passed