[ZenAlexa]

Summary

On the MLX backend, return_logprob=True without --mlx-enable-sampling produced empty logprob arrays. /v1/score then returned an opaque 500 after the score result processor found no output logprobs.

The scheduler now rejects this unsupported request before admission with Output logprobs on the MLX backend require --mlx-enable-sampling. Non-MLX requests and MLX requests with sampling enabled keep their existing paths. The Apple Metal guide lists the flag beside the other MLX serving parameters and describes its logprob requirement.

Validation

• At 34c4378, python -m pytest test/registered/unit/managers/test_scheduler_uno_request_validation.py test/registered/unit/managers/test_scheduler_sampling_mask_validation.py -q completed with 3 passed and 2 subtests passed.
• Integrated main 74308cf1 at 27daff53. Source-isolated execution of the complete production request handler rechecked all eight combinations of MLX, sampling, and output-logprob flags on both main and candidate. Only MLX output-logprob requests with sampling disabled take the new abort path; the request retains the expected error and logprob state.
• At prior integrated head 03dd0754, repository pre-commit checks passed for AST validation, import ordering, Ruff, formatting, codespell, CI registry validation, and related hooks.
• git diff --check passed.
• Verified on Apple M2 by @luke-edward at 34c4378 on Apple M2 16 GB, macOS 15.7, Python 3.12.13, PyTorch 2.13.0, MLX 0.32.2, and Qwen/Qwen2.5-0.5B-Instruct bf16. With sampling disabled, logprob requests to /generate and /v1/score return 400 with the flag requirement; ordinary generation returns 200. With sampling enabled, both endpoints return populated logprob and score results.

Fixes #41211.
The earlier integration used the source-loaded handler and admission fixtures; full scheduler imports and model-backed endpoints were covered by the historical runs above.

---

CI States

Latest PR Test (Base): ❌ Run #41783506198
Latest PR Test (Extra): ❌ Run #41783506126
Latest PR Test (AMD ROCm 10): ❌ Run #41783506039

Tip

Base CI only covers the core cases, and its workload may not exercise your change. Use /rerun-test to run the tests your change touches, or copy this prompt for an agent:

Find the registered tests this PR affects and run them with /rerun-test, following https://github.com/sgl-project/sglang/blob/main/.agents/skills/rerun-test/SKILL.md

current main integration

merged main 426281d1bc2 at ca1566654e. #41916 added a speculative-decoding min_p rejection at the same point in handle_generate_request, so both checks stay as separate blocks before request admission, MLX first, and both use main's abort-and-enqueue path. the resolved scheduler.py differs from main by this PR's ten lines. e0721ceb82 fixes the lint failure on 3bc5edb, where the check-indirect-imports hook from #47992 rejected the test's DisaggregationMode import through disaggregation.utils, so the test now imports it from sglang.srt.disaggregation.mode. that hook passes on the changed files, along with isort, ruff, ruff-format and codespell.

the CPU admission regression and native MLX checks weren't rerun on this revision. please add run-ci for this head.

previous main integration

merged main c8704818d4f0 into 3bc5edb67a38 through a normal merge. main's scheduler request-holder and step-launch changes remain intact. unsupported MLX logprob requests continue through the existing abort-and-enqueue path before request admission.

on macOS arm64 with Python 3.11.18, source-loaded execution of the complete production request handler rechecked all eight combinations of MLX, sampling and output-logprob flags on main and candidate. the candidate rejected the unsupported combination and preserved the other seven paths. the registered admission regression passed with dependency seams, including the error message, logprob state and existing abort queue route.

complete scheduler imports, HTTP/model execution and native MLX checks remain unrun on this revision. please enable the applicable project CI lane for this head. the Apple M2 results above remain tied to 34c4378.

[Copilot]

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

[ZenAlexa]

The implementation and CPU regression are ready. The repository gate requires the run-ci label before the MLX workflow can start; the fork account cannot add repository labels. Please add run-ci when convenient so the staged MLX checks can run.

[luke-edward]

Ran the "Apple Metal server run" the description lists as pending, on this PR's head (34c4378) with the M2 that produced the report in #41211. Apple M2 16 GB, macOS 15.7, Python 3.12.13, PyTorch 2.13.0, MLX 0.32.2, Qwen/Qwen2.5-0.5B-Instruct bf16.

Without --mlx-enable-sampling (the case from the issue):

request	before (main @ 6103a8c)	this PR
/generate with return_logprob + token_ids_logprob, max_new_tokens=1	200, output_token_ids_logprobs: []	400 {"error": {"message": "Output logprobs on the MLX backend require --mlx-enable-sampling."}}
/v1/score with label_token_ids, apply_softmax	500 output_logprobs is empty for request …	400 BadRequestError, same message
/generate without logprobs (control)	200	200, 'Yes.'

With --mlx-enable-sampling, unchanged: /generate returns output_token_ids_logprobs [[[-0.0219, 9454, None], [-4.2719, 2753, None]]], /v1/score returns [[0.9859, 0.0141]].

test_scheduler_uno_request_validation passes here as well (2 tests). This does what I asked for in #41211, and the doc line is where I would have put it. Happy for this to close the issue.

[ZenAlexa]

Thanks for running this on the original M2 setup. I've added your results to the validation section, including both sampling modes and the ordinary-generation control.

[mintlify]

Preview approval required

This pull request comes from a fork, so Mintlify won't build a preview until someone with write access approves commit e0721ce.

Approve preview