[TRTLLM-14510][feat] serve: multi-process HTTP frontends on the classic IPC executor path - #16523

Merged

longlee0622 merged 25 commits into

NVIDIA:mainfrom

lancelly:feat/serve-multi-frontend-main

Jul 22, 2026

Commits on Jul 17, 2026

[None][feat] serve: multi-process HTTP frontends on the classic IPC executor path
lancelly
committed
68abdb3
[None][fix] serve multi-frontend: clean up ipc artifacts and scope the env gate
lancelly
committed
06dbbd3
[None][chore] serve: fold multi-frontend mode resolution into a helper
lancelly
committed
21663cf
[None][chore] serve multi-frontend: direct client_id access on the response routing hot path
lancelly
committed
8840733
[None][feat] serve multi-frontend: replace the env gate with the num_serve_frontends knob
lancelly
committed
257d737
[None][chore] serve multi-frontend: cap num_serve_frontends at 64, drop the env fallback
lancelly
committed
c55c1ae
[None][chore] serve multi-frontend: drop the leftover env warning, tighten comments
lancelly
committed
f1e72ae
[None][chore] serve multi-frontend: explicit None-check instead of getattr for the frontend count
lancelly
committed
e089a40
[None][chore] serve multi-frontend: merge FrontendProxy id checks
lancelly
committed
a4f54aa
[None][chore] apply pre-commit yapf formatting
lancelly
committed
757660a
[None][chore] serve multi-frontend: pass attach info via env value, not a temp file
lancelly
committed
9a086ff
[None][chore] serve multi-frontend: slim MultiFrontendMode
lancelly
committed
a8d7b23
[None][chore] postproc: take the result-lane address list unconditionally
lancelly
committed
bd1fab2
[None][chore] serve multi-frontend: tighten _init_multi_frontend_mode
lancelly
committed
cbbf9ab
[None][fix] serve multi-frontend: harden the client_id namespace
lancelly
committed
6f0d39d
[None][chore] apply pre-commit isort/yapf formatting
lancelly
committed
d6a2eb6
[None][chore] fix D205 docstring layout in _init_multi_frontend_mode
lancelly
committed
eb3e425
Merge branch 'main' into feat/serve-multi-frontend-main
lancelly
authored
cdbc7d6

Navigation Menu

Uh oh!

[TRTLLM-14510][feat] serve: multi-process HTTP frontends on the classic IPC executor path - #16523

[TRTLLM-14510][feat] serve: multi-process HTTP frontends on the classic IPC executor path#16523
longlee0622 merged 25 commits into
NVIDIA:mainfrom
lancelly:feat/serve-multi-frontend-main

Commits on Jul 17, 2026

[None][feat] serve: multi-process HTTP frontends on the classic IPC executor path

[None][fix] serve multi-frontend: clean up ipc artifacts and scope the env gate

[None][chore] serve: fold multi-frontend mode resolution into a helper

[None][chore] serve multi-frontend: direct client_id access on the response routing hot path

[None][feat] serve multi-frontend: replace the env gate with the num_serve_frontends knob

[None][chore] serve multi-frontend: cap num_serve_frontends at 64, drop the env fallback

[None][chore] serve multi-frontend: drop the leftover env warning, tighten comments

[None][chore] serve multi-frontend: explicit None-check instead of getattr for the frontend count

[None][chore] serve multi-frontend: merge FrontendProxy id checks

[None][chore] apply pre-commit yapf formatting

[None][chore] serve multi-frontend: pass attach info via env value, not a temp file

[None][chore] serve multi-frontend: slim MultiFrontendMode

[None][chore] postproc: take the result-lane address list unconditionally

[None][chore] serve multi-frontend: tighten _init_multi_frontend_mode

[None][fix] serve multi-frontend: harden the client_id namespace

[None][chore] apply pre-commit isort/yapf formatting

[None][chore] fix D205 docstring layout in _init_multi_frontend_mode

Merge branch 'main' into feat/serve-multi-frontend-main

Commits on Jul 18, 2026

[None][chore] regenerate the LLM args telemetry golden manifest

[None][chore] add --num_serve_frontends to the serve CLI stability reference

Commits on Jul 20, 2026

[None][fix] serve: initialize attached-frontend proxy state and cover it in CI

[None][fix] serve: READY handshake and exception-safe cleanup for attached frontends

[None][fix] serve: disable stateful Responses API storage in multi-frontend mode

Commits on Jul 21, 2026

[None][test] mark multi-frontend routing tests cpu_only

[None][test] stop the dispatch thread in the frontend-proxy lifecycle tests