Conversation
The repetition detector (check_sequence_repetition, FinishReason REPETITION) exists in the scheduler and is exposed per-request, but there is no way for an operator to enable it server-side: the get_diff_sampling_param allowlist drops it from generation configs (--generation-config / --override-generation-config), and the OpenAI protocols never fell back to default_sampling_params for it. Add repetition_detection to the allowlist, convert the raw JSON dict into typed RepetitionDetectionParams in ModelConfig, and fall back to the server default in Completion/Chat to_sampling_params when the request does not set its own - the same pattern used for stop_token_ids in vllm-project#22519. Co-authored-by: Kimi (AI coding assistant) Signed-off-by: Islam <islam.almersawi@openinnovation.ai>
|
Warning Review limit reachedNext included review available in 30 minutes. View limit detailsLimit details: You’ve used the included review currently available. You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository. Review configuration: ⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Team Run ID: 📒 Files selected for processing (5)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Operator-configurable repetition-detection defaults
Status: historical upstream backport, not enabled in the active Kimi service. Published for review at the operator's request; no new serving behavior is introduced by publication.
Prepared commit
95b96696e702preserves the server-default plumbing from upstream vllm-project/vllm#51036. Original authorship and attribution are retained.The change admits
repetition_detectionfrom model generation configuration, converts its JSON object toRepetitionDetectionParams, and lets chat/completion requests inherit that default when they do not supply their own setting. Request values still take precedence. Without an operator-configured default or request value the detector remains disabled.Important distinction
This configuration feature is not the fix for corrupted four-token prefill tails. Lab #687 corrects the underlying graph-selection defect. This branch only exposes an optional output-stopping policy and is not being used to conceal that defect or presented as a precision-preserving model optimization.
Validation and compatibility
The corresponding upstream PR records six protocol-default tests, generation-config conversion coverage and eight neighboring stop-token default tests. The prepared branch includes the protocol and model-config test additions. The publication audit inspected those changes and ran
git diff --check; it did not rerun the historical full test matrix or model evaluation.The branch targets the local-inference-lab integration line. It is a backport record of existing upstream work, not a duplicate upstream proposal. Open lab searches found no separate server-default repetition-detection PR.
Current-base review and tests are required before merge. No active generation configuration, repetition penalty, stopping policy, server or cache was changed. AI assistance was used for publication; human merge approval is not asserted.
🤖 Generated with Claude Code
https://claude.ai/code/session_01KxvNwugeU8RJFd7WRYwNLG