Skip to content

fix(mlx-grpc): per-step admission + own-thread BatchGenerator (eliminates chat-c4 TTFT regression and Stream(gpu,1) crash) - #1427

Merged
key4ng merged 6 commits into
mainfrom
keyang/mlx-grpc-per-step
May 1, 2026
Merged

fix(mlx-grpc): per-step admission + own-thread BatchGenerator (eliminates chat-c4 TTFT regression and Stream(gpu,1) crash)#1427
key4ng merged 6 commits into
mainfrom
keyang/mlx-grpc-per-step

refactor(mlx-grpc): mlx-lm.server-style abort signal; drop _gen_lock

bbdb868
Select commit
Loading
Failed to load commit list.

Select a check to view from the sidebar