feat(coordinator): put the decode leg on the tokens-in generate format - #3690

feat(coordinator): put the decode leg on the tokens-in generate format#3690

roytman wants to merge 3 commits into

llm-d:mainllm-d/llm-d-router:mainfrom

roytman:generate_allroytman/llm-d-inference-scheduler:generate_allCopy head branch name to clipboard