patches: serve-model-path-match — accept the model path /v1/models advertises - #167
Conversation
0ea5994 to
dbba103
Compare
a815160 to
b51f8e6
Compare
…vertises is_base_model matched the served name only, so a client echoing the root that /v1/models itself publishes got a 404. The patch accepts the served name, the full checkpoint path, or the path's basename — exact matches only. Independent of every other patch, appended at the series' block boundary. Cut from the extended cpuchip/vllm qwen38/0.28 branch, topic commit [qwen38] serve-model-path-match; kind: fix, retires when upstream takes it.
b51f8e6 to
c9414df
Compare
|
Merged (rebased for you). I did take your offer to consider dropping this one seriously, since #165 and #166 make the failure self-explaining, and I am keeping it for a reason worth writing down: the mismatch is this repo's doing, not the client's. Every launcher here passes a checkpoint path as Checked against the pinned source: |
The cell said 'none yet'; the upstream PR is vllm-project/vllm#58026. The link was lost in a rebase of #167.
What
Adds
patches/serve-model-path-match.patch, itspatches/seriesline and itsPATCHES.mdrow.is_base_modelnow accepts the served name, the full model path, or the basename of that path, all as exact matches.Upstream: vllm-project/vllm#58026.
Why
/v1/modelspublishes the checkpoint path inroot, but the name check matched the served name only, so echoingrootback returned 404. Measured on a live server from the current image:This repo is a steady source of that mismatch:
--modelis a path everywhere and the served name isqwen3.8-27beverywhere, so any client that discovers the model from/v1/modelspicks the path.There is no prefix match and no fuzzy match, so an unknown name still returns 404.
Of the five patches, this is the one I would drop first if you would rather keep the name space strict: #165 and #166 already make the failure self-explaining.
Verification
patch integrity(the workflow'sgit-applyjob) applies the whole series to a pristinevllm-project/vllmcheckout at the pin: passes on this PR.patch -p1 --fuzz 0 --dry-runof this file against the installed tree inghcr.io/syv-ai/hyperqwen:latest(06150174): applies, no fuzz.verify.shneeds no new entry: its loop readspatches/series, andpatches/_check_applied.pyparses the patch file itself.Not done: I have not rebuilt the image and restarted a server on this patch, so the runtime evidence above comes from the code path, not from a rebuilt server.