Fix Overlap R3

[zyzshishui]

Motivation

Merged into #22911

Modifications

Accuracy Tests

Speed Tests and Profiling

Checklist

• Format your code according to the Format code with pre-commit.
• Add unit tests according to the Run and add unit tests.
• Update documentation according to Write documentations.
• Provide accuracy and speed benchmark results according to Test the accuracy and Benchmark the speed.
• Follow the SGLang code style guidance.

Review and Merge Process

1. Ping Merge Oncalls to start the process. See the PR Merge Process.
2. Get approvals from CODEOWNERS and other reviewers.
3. Trigger CI tests with comments or contact authorized users to do so.
   • Common commands include /tag-and-rerun-ci, /tag-run-ci-label, /rerun-failed-ci
4. After green CI and required approvals, ask Merge Oncalls or people with Write permission to merge the PR.

[gemini-code-assist]

Code Review

This pull request modifies the copy_to_cpu method in python/sglang/srt/managers/utils.py by removing the conditional check for return_routed_experts when copying routed expert outputs to the CPU. Feedback indicates that this change makes the return_routed_experts parameter unused and leads to inefficient memory management and unnecessary data transfers. It is recommended to restore the original conditional logic to maintain efficiency.

[gemini-code-assist]

The parameter return_routed_experts is now unused in this function. Removing the check and return_routed_experts and the else block that clears self.routed_experts_output leads to unnecessary D2H copies and keeps GPU tensors alive even when they are not requested by the user. If the intention was to always copy these experts, the parameter should be removed from the function signature. Otherwise, the previous conditional logic should be restored to maintain efficiency and proper memory management.

Suggested change

	if self.routed_experts_output is not None:
	self.routed_experts_output.copy_to_cpu()
	else:
	self.routed_experts_output = None
	if self.routed_experts_output is not None and return_routed_experts:
	self.routed_experts_output.copy_to_cpu()
	else:
	self.routed_experts_output = None

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 1a1c4236d6

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[chatgpt-codex-connector]

Restore routed-expert copy gating by request flag

copy_to_cpu() now always calls self.routed_experts_output.copy_to_cpu() whenever the capturer is enabled, even when no request asked for routed experts. In overlap mode, batch.return_routed_experts is computed as any(req.return_routed_experts for req in reqs) (see schedule_batch.py), so this removed guard turns an optional D2H path into a per-batch cost. For MoE models this can add large host transfers and finalize work on every step, materially reducing throughput/latency for workloads that enable routed-expert support but only occasionally request it.

Useful? React with 👍 / 👎.