[TRTLLM-17263][refactor] Converge the three Anthropic orchestrations into one serving class by JunyiXu-nv · Pull Request #21551 · NVIDIA/TensorRT-LLM

[TRTLLM-17261][test] Add route, middleware-order and shim-identity snapshots and a CPU test seam for trtllm-serve

JunyiXu-nv

committed

b291cca

[TRTLLM-17261][test] Record the trtllm-serve route reference and test baselines at the merge base

JunyiXu-nv

committed

c9311ab

[TRTLLM-17261][test] Cover the attached-frontend route registration in the snapshot matrix

JunyiXu-nv

committed

81b0350

[None][test] Address review of the serve snapshot and layering tests

JunyiXu-nv

committed

2eb184a

[TRTLLM-17261][test] Refresh the serve route reference and baselines on the current main

JunyiXu-nv

committed

5138e15

[None][test] Make the route snapshot and the object.new ratchet fail on silent regressions

JunyiXu-nv

committed

045a69e

[TRTLLM-17262][test] Pin the disaggregated entry-point wrapper and its outside-subclass contract

JunyiXu-nv

committed

75fc8e7

[TRTLLM-17262][refactor] Split the disaggregated entry-point wrapper into prepare, run, finish and serialize steps

JunyiXu-nv

committed

533cb0b

[TRTLLM-17261][refactor] Move the msgpack route classes to tensorrt_llm/serve/_base/msgspec_route.py

JunyiXu-nv

committed

68344ed

[TRTLLM-17261][refactor] Add the trtllm-serve router skeleton and serve the system endpoints through it

JunyiXu-nv

committed

9f4a48e

[TRTLLM-17261][infra] Multi-GPU triggers and type checking for the new serve packages

JunyiXu-nv

committed

caf3a29

[TRTLLM-17261][test] Register the system router in the multi-frontend tests that build routes by hand

JunyiXu-nv

committed

4a6325d

[TRTLLM-17261][test] Match a request instead of reading route paths in the OpenEngine metrics test

JunyiXu-nv

committed

7e29509

[TRTLLM-17261][test] Classify endpoint-package imports by the imported name

JunyiXu-nv

committed

d3aacdd

[TRTLLM-17263][test] Pin the chat completion handler's HTTP answers before splitting it

JunyiXu-nv

committed

543322f

[TRTLLM-17263][refactor] Move the chat SSE generator to tensorrt_llm/serve/_base/sse.py

JunyiXu-nv

committed

6174ee4

[TRTLLM-17263][refactor] Split openai_chat into a typed create_chat_completion and an HTTP wrapper

JunyiXu-nv

committed

bd6b3d7

[TRTLLM-17263][infra] Type-check tensorrt_llm/serve/_base/errors.py

JunyiXu-nv

committed

936cc54

[TRTLLM-17263][test] Stub the chat response as a ChatCompletionResponse in the multimodal prompt test

JunyiXu-nv

committed

974e4bd

[TRTLLM-17263][test] Pin that an unencodable chat response is answered 400 without a traceback

JunyiXu-nv

committed

d8c56b5

[TRTLLM-17263][refactor] Move the Anthropic Messages modules into serve/api/anthropic

JunyiXu-nv

committed

752da8b

[TRTLLM-17263][test] Pin both servers' Anthropic Messages answers and mark the new package

JunyiXu-nv

committed

147c114

Merge branch 'dev-junyix-feat-serve-typed-chat-core' into dev-junyix-feat-serve-anthropic-serving-class (integration base, not part of the reviewable change)

JunyiXu-nv

committed

aa73407

Merge branch 'dev-junyix-feat-serve-disagg-entry-split' into dev-junyix-feat-serve-anthropic-serving-class (integration base, not part of the reviewable change)

JunyiXu-nv

committed

db50772

[TRTLLM-17263][refactor] Add the Anthropic count-tokens and batches serving classes

JunyiXu-nv

committed

31fa80b

[TRTLLM-17263][refactor] Move the Anthropic count-tokens and batch handlers out of OpenAIServer

JunyiXu-nv

committed

95bb1e1

[TRTLLM-17263][refactor] One Anthropic Messages serving class for the aggregated and disaggregated servers

JunyiXu-nv

committed

d5e971e

[TRTLLM-17263][test] Test the Anthropic serving class, the request context and the router on the real apps

JunyiXu-nv

committed

07016ff

[TRTLLM-17263][test] Mount the Messages usage test on the Anthropic router and pass the kv-hints handlers a request

JunyiXu-nv

committed

6998c2b

Navigation Menu

Repository navigation

[TRTLLM-17263][refactor] Converge the three Anthropic orchestrations into one serving class - #21551

[TRTLLM-17263][refactor] Converge the three Anthropic orchestrations into one serving class#21551
JunyiXu-nv wants to merge 29 commits into
NVIDIA:mainfrom
JunyiXu-nv:dev-junyix-feat-serve-anthropic-serving-class

Commits on Nov 4, 2026

[TRTLLM-17261][test] Add route, middleware-order and shim-identity snapshots and a CPU test seam for trtllm-serve

[TRTLLM-17261][test] Record the trtllm-serve route reference and test baselines at the merge base

[TRTLLM-17261][test] Cover the attached-frontend route registration in the snapshot matrix

[None][test] Address review of the serve snapshot and layering tests

[TRTLLM-17261][test] Refresh the serve route reference and baselines on the current main

[None][test] Make the route snapshot and the object.new ratchet fail on silent regressions

[TRTLLM-17262][test] Pin the disaggregated entry-point wrapper and its outside-subclass contract

[TRTLLM-17262][refactor] Split the disaggregated entry-point wrapper into prepare, run, finish and serialize steps

[TRTLLM-17261][refactor] Move the msgpack route classes to tensorrt_llm/serve/_base/msgspec_route.py

[TRTLLM-17261][refactor] Add the trtllm-serve router skeleton and serve the system endpoints through it

[TRTLLM-17261][infra] Multi-GPU triggers and type checking for the new serve packages

[TRTLLM-17261][test] Register the system router in the multi-frontend tests that build routes by hand

[TRTLLM-17261][test] Match a request instead of reading route paths in the OpenEngine metrics test

[TRTLLM-17261][test] Classify endpoint-package imports by the imported name

[TRTLLM-17263][test] Pin the chat completion handler's HTTP answers before splitting it

[TRTLLM-17263][refactor] Move the chat SSE generator to tensorrt_llm/serve/_base/sse.py

[TRTLLM-17263][refactor] Split openai_chat into a typed create_chat_completion and an HTTP wrapper

[TRTLLM-17263][infra] Type-check tensorrt_llm/serve/_base/errors.py

[TRTLLM-17263][test] Stub the chat response as a ChatCompletionResponse in the multimodal prompt test

[TRTLLM-17263][test] Pin that an unencodable chat response is answered 400 without a traceback

[TRTLLM-17263][refactor] Move the Anthropic Messages modules into serve/api/anthropic

[TRTLLM-17263][test] Pin both servers' Anthropic Messages answers and mark the new package

Merge branch 'dev-junyix-feat-serve-typed-chat-core' into dev-junyix-feat-serve-anthropic-serving-class (integration base, not part of the reviewable change)

Merge branch 'dev-junyix-feat-serve-disagg-entry-split' into dev-junyix-feat-serve-anthropic-serving-class (integration base, not part of the reviewable change)

[TRTLLM-17263][refactor] Add the Anthropic count-tokens and batches serving classes

[TRTLLM-17263][refactor] Move the Anthropic count-tokens and batch handlers out of OpenAIServer

[TRTLLM-17263][refactor] One Anthropic Messages serving class for the aggregated and disaggregated servers

[TRTLLM-17263][test] Test the Anthropic serving class, the request context and the router on the real apps

[TRTLLM-17263][test] Mount the Messages usage test on the Anthropic router and pass the kv-hints handlers a request