Use 2x less memory for MoE (including the DisposableTensor small tool)

[fzyzcjy]

UPDATE: latest PR is in #6147

Brainstorm: maybe we can use the_tensor.set_(torch.empty((0,), device=...)) (e.g. if users do not need to access metadata of the tensor after this operation)

2025.04.21 Update

I observe something similar for forward_deepgemm_masked. Will update this PR containing that as well. For users who need it before this PR is extracted from my dev branch, please directly visit #5524

Original text

Do NOT look at the code now - it is based on two-batch-overlap branch (b/c that branch merges the tools branches). I will re-do the things on master branch later.
Updated

Before: peak 1.8+1.8+1.8GB = 5.4GB

After: peak is 1.8+0.9GB = 2.7GB

04.12: Updated the code, and re-checked the memory profile which still looks reasonable

Motivation

Modifications

Checklist

• Format your code according to the Code Formatting with Pre-Commit.
• Add unit tests as outlined in the Running Unit Tests.
• Update documentation / docstrings / example tutorials as needed, according to Writing Documentation.
• Provide throughput / latency benchmark results and accuracy evaluation results as needed, according to Benchmark and Profiling and Accuracy Results.
• For reviewers: If you haven't made any contributions to this PR and are only assisting with merging the main branch, please remove yourself as a co-author when merging the PR.
• Please feel free to join our Slack channel at https://slack.sglang.ai to discuss your PR.

[fzyzcjy]

Title seems to be inaccurate - 2x memory save is due to multiple changes