Repository navigation
fix: MM-aware KV routing for Phi-3, Qwen2-VL, Qwen2.5-VL #9441
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from 11 commits
Commits
Show all changes
27 commits
Select commit
Hold shift + click to select a range
fd0d572
fix(mm-routing): MM-aware KV routing for Phi-3, Qwen2-VL, Qwen2.5-VL
krishung5 d52d504
perf(mm-routing): pass formatted_prompt by reference to avoid per-req…
krishung5 b216723
test(mm-routing): strong-gate agg_router profiles on lightseek init +…
krishung5 253f73e
test(mm-routing): settle kv-router index between repeats to fix Phi-3…
krishung5 688b675
Revert "test(mm-routing): settle kv-router index between repeats to f…
krishung5 5032de3
fix(mm-routing): mirror vLLM decode-roundtrip for Phi-3 routing tokens
krishung5 0af2150
fix(mm-routing): disable MM-aware routing when DYN_TOKENIZER=fastokens
krishung5 324fb0f
chore(mm-routing): cargo fmt fixup
krishung5 261a871
test(mm-routing): metric-based router_kv_hit_rate gate for MM agg_rou…
krishung5 e50b044
Merge remote-tracking branch 'origin/main' into krish/phi3-mm-routing…
krishung5 ab42746
Merge remote-tracking branch 'origin/main' into krish/phi3-mm-routing…
krishung5 646fe91
Merge remote-tracking branch 'origin/main' into krish/phi3-mm-routing…
krishung5 19d263d
test(mm-routing): rename require_lightseek_init -> require_rust_proce…
krishung5 211d975
test(mm-routing): drop lightseek wording from PR-added log strings
krishung5 523c02b
refactor(mm-routing): centralize HF config parsing in lightseek_mm.rs
krishung5 79d2057
refactor(mm-routing): collapse redundant image-token fields
krishung5 4024fdc
refactor(mm-routing): unify BOS handling + Phi-3 family guard in spli…
krishung5 8c02d52
test(mm-routing): compose router_kv_hit_rate metric name from canonic…
krishung5 d6a21c4
test(mm-routing): simplify router-histogram scraper to return (sum, c…
krishung5 d2bef12
test(tokenizers): add merge_special_tokens_from_config gate test + wa…
krishung5 21f02de
test(mm-routing): narrow _scrape_router_kv_hit_rate except to Request…
krishung5 b759d58
docs(mm-routing): fix stale Qwen2-VL placeholder-token comment
krishung5 098de96
fix(mm-routing): apply tokenizer_config.json merge before extracting …
krishung5 8f6d7a6
Merge remote-tracking branch 'origin/main' into krish/phi3-mm-routing…
krishung5 ced13a9
style(mm-routing): cargo fmt + clippy fixes for pre-merge gates
krishung5 2d7a88a
docs(mm-routing): add TODO marker for RouterMetricsAssertion mixin fo…
krishung5 769f6ea
fix(test): align rust mm-router init-log assertion with renamed log line
krishung5 File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.