Repository navigation
[Frontend] Validate JSON body and request_ids in /inference/v1/abort_requests - #63302
Merged
vllm-bot merged 13 commits intoNov 2, 2026
Merged
Conversation
…requests Signed-off-by: zydo <zydo.git@gmail.com>
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
6 tasks done
Replace the hand-written isinstance checks with an AbortRequestsRequest model so invalid bodies get the standard 400 ErrorResponse. The body is still parsed manually so clients without a Content-Type header keep working on the legacy /abort_requests path. Signed-off-by: zydo <zydo.git@gmail.com>
The AbortRequestsRequest annotation already rejects non-object bodies and non-str/list[str] request_ids, so asserting those 400s only re-tests pydantic. Keep a focused test for the manual body parse, which accepts clients that send no Content-Type header. Signed-off-by: zydo <zydo.git@gmail.com>
The AbortRequestsRequest annotation enforces the request_ids contract by definition, so the endpoint test added nothing beyond re-testing pydantic. Signed-off-by: zydo <zydo.git@gmail.com>
DarkLight1337
added
the
ready
Member
|
/ci run |
DarkLight1337
enabled auto-merge (squash)
November 2, 2026 06:36
|
✅ @zydo, CI is now available for this PR.
|
|
✅ Triggered Buildkite CI #99091 for commit |
Contributor
|
Hi @zydo, the pre-commit checks have failed. Please run: uv pip install pre-commit>=4.5.1
pre-commit install
pre-commit run --all-filesThen, commit the changes and push to your branch. For future commits, |
Member
|
Please rebase from main to fix unrelated pre-commit issues |
vllm-project#64357 and vllm-project#63440 each added a process_weights_after_loading to MuseSpark1xForCausalLM, so the second definition silently replaced the first and broke pre-commit on main (ruff F811, mypy no-redef). Run both steps in one method: materialize the residual-path constants first, then attach the monokernel, which snapshots the final weights. Signed-off-by: zydo <zydo.git@gmail.com>
auto-merge was automatically disabled
November 2, 2026 06:50
Head branch was pushed to by a user without write access
zydo
requested review from
dllehr-amd,
hongxiayang and
tjtanaa
as code owners
November 2, 2026 06:50
zydo
changed the title
DarkLight1337
changed the title
DarkLight1337
enabled auto-merge (squash)
November 2, 2026 06:52
Member
|
/ci run |
|
✅ Triggered Buildkite CI #99097 for commit |
get_rope caches RotaryEmbedding instances by shape and dtype, not device. test_muse_spark_rocm_shared_rope.py runs earlier in the same pytest process and builds a CPU cache with the exact key the mono tiny model uses (head_dim 64, max_position 4096, theta 5e5, fp32), so the tiny model's target layers reused it and the fused QK-norm/RoPE kernel rejected the CPU tensor in all 26 stock-path tests. Clear the cache before building the tiny model and after tearing it down. Signed-off-by: zydo <zydo.git@gmail.com>
auto-merge was automatically disabled
November 2, 2026 08:04
Head branch was pushed to by a user without write access
This reverts commit c8128f4 per review: keep this PR scoped to the abort_requests change. Signed-off-by: zydo <zydo.git@gmail.com>
Member
|
/ci run |
DarkLight1337
enabled auto-merge (squash)
November 2, 2026 09:11
|
✅ Triggered Buildkite CI #99132 for commit |
hickeyma
approved these changes
Nov 2, 2026
Contributor
|
/ci run |
|
❌ @hickeyma, Only reviewers with write access can use CI commands before CI is delegated to the PR author. |
Member
|
Force merging |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Overview
Validate the body of
/inference/v1/abort_requests(and/abort_requests) invllm/entrypoints/scale_out/token_in_token_out/api_router.pywith a pydantic model, so malformed payloads return400 Bad Requestinstead of raising an unhandled500 Internal Server Error.Claims
AbortRequestsRequest(request_ids: str | list[str]) toprotocol.py; the annotation defines the contract, so non-object JSON bodies and non-str/list[str]request_idsare rejected by the model rather than by hand-written checks.["req-1"]) now return the standard400ErrorResponseinstead of crashing withAttributeError: 'list' object has no attribute 'get'.Content-Typeheader keep working on the legacy/abort_requestspath.request_idscontract holds by definition from the model annotation, so an endpoint test would only re-test pydantic (per review feedback).Validation
POST /inference/v1/abort_requestswith JSON body["req-1"]raisesAttributeError: 'list' object has no attribute 'get'(500 Internal Server Error).400 Bad Request, while{"request_ids": "req-1"}and{"request_ids": ["req-1", "req-2"]}return200 OK.TestClienttest asserting400for["req-1"],{},{"request_ids": 123}and{"request_ids": [1]},200for valid bodies, and200for a body sent with noContent-Type. That test was removed before merge per review, since the model enforces the contract by definition.35 passed;ruff-check,ruff-format,mypy, and the remaining pre-commit hooks all passed.Details
abort_requestscalledbody = await raw_request.json()and immediately accessedbody.get("request_ids"). Becauseraw_request.json()accepts any top-level JSON value (arrays, numbers, booleans, strings,null), non-dict bodies raised an unhandledAttributeError, and non-stringrequest_idsvalues were forwarded straight toengine_client(raw_request).abort(request_ids).AbortRequestsRequest.model_validate(body)rather than declaring the model as a body parameter. vLLM requires FastAPI >= 0.133, wherestrict_content_typedefaults toTrue, so a body parameter would reject clients that send noContent-Type— the legacy/abort_requestsbreakage this change avoids. Pydantic errors are converted toRequestValidationError, so the response format matches other endpoints. The limitation is that the model does not appear in the OpenAPI schema for this route.gh pr list --repo vllm-project/vllm --state open --search "abort_requests token_in_token_out". Open PR [Bugfix][Frontend] Keep a reference to the background abort task in disagg api_router #45249 retains a background task reference forasyncio.create_taskinabort_requestsand does not validate the request payload orrequest_idstypes.Pull Request Checklist
I used vLLM's
/pr-checklistskill. (Mandatory for agents, optional for humans).AI assistance was used during the creation of this PR.
Design Fit: Minimizes impact on core components, reuses existing functionality, and justifies added complexity.
Testing and Validation: Validates the change and ensures any added tests are meaningful and reliable, with CI coverage or documented CI resource constraints and validation performed outside CI.
Code Quality and Style: Keeps code and comments clear and concise, and updates relevant documentation and examples.
Pull Request Contents: Includes a brief summary and relevant links, supports claims with evidence, explains root causes and implementation trade-offs, and follows the contributing guide.