[guapisolo]

Summary

Forward SGLang's return_outputs_via_store on /generate and /v1/chat/completions.

Motivation

SGLang's Mooncake output store (sgl-project/sglang#42647) is opted into per request with return_outputs_via_store. The router re-serializes typed requests, so the flag was dropped: a Miles R3 training run with --sglang-output-store-backend mooncake silently got inline replay outputs through this router, while MilesRouter forwarded the flag.

Usage

curl -s localhost:30000/generate -d '{"input_ids": [1, 2, 3], "return_routed_experts": true,
  "return_outputs_via_store": true}' | jq '.meta_info.output_store_ref.fields'
# {"routed_experts": {"dtype": "int32", "shape": [<tokens>, <layers>, <topk>]}}

Design Notes

• Mental model: the HTTP router deserializes each request into GenerateRequest or ChatCompletionRequest, then serializes that struct to the worker, so only declared fields reach SGLang; non-streaming responses come back as raw bytes. Declaring the field therefore carries the flag to SGLang, while meta_info.output_store_ref already reaches the client.
• Option<bool> serialized only when set, like sampling_logprobs_mode in fix(protocol): allow 128 chat top_logprobs and forward sampling_logprobs_mode #21: requests without the flag are forwarded unchanged, so SGLang builds without the output store never see it.
• Chat still has no return_indexer_topk (generate has it); not changed here.

Verification

• miles-wheels run 37363412212 test, at the pre-merge head 36c4d14: spec_test 98 passed (1 ignored); routing_tests 91 passed with the same single failure as at 374ff28, test_manual_routing_without_header.
• Same run, x86_64 + aarch64 builds, check_router_forwarding.py with the wheel plus the standalone binary: on /generate and chat the flag reaches the worker only when set; output_store_ref comes back unchanged.
• The merge of main (fix(protocol): forward flat output top logprobs for chat completions #24) keeps both new chat fields side by side; the tests have not been rerun at the merged head yet.

Review Focus

• GenerateRequest / ChatCompletionRequest: the new field must stay out of the serialized request when unset.