fix: graceful degradation when EAGLE draft pool fails to register with Mooncake standalone storage

[HyeonjeCho]

Motivation

Fixes #28873

When using EAGLE speculative decoding with Mooncake L3 storage in standalone (dummy client) mode, the server crashes during startup. The draft host pool is allocated with the default allocator (mmap) instead of MooncakeHostTensorAllocator, causing register_buffer() to fail with error code -1 because the buffer is outside Mooncake's shared memory segment.

Modifications

Wrap the register_mem_host_pool_v2() call in _maybe_register_draft_with_storage() with a try/except. On failure, log a warning and return early so that draft_page_get_func and draft_page_set_func remain None.

This disables draft L3 (SSD offload) while keeping:

• Draft L2 (host DRAM): fully functional (uses mem_pool_host_draft directly, independent of storage backend)
• Target model L2/L3: fully functional (target pool uses MooncakeHostTensorAllocator correctly)

Accuracy Tests

Not applicable — no changes to model forward code.

Speed Tests and Profiling

Not applicable — no changes to inference kernels. The fix only affects startup behavior.

Checklist

• Format your code according to the Format code with pre-commit.
• Add unit tests according to the Run and add unit tests.
• Update documentation according to Write documentations.
• Provide accuracy and speed benchmark results according to Test the accuracy and Benchmark the speed.
• Follow the SGLang code style guidance.

Review and Merge Process

1. Ping Merge Oncalls to start the process. See the PR Merge Process.
2. Get approvals from CODEOWNERS and other reviewers.
3. Trigger CI tests with comments or contact authorized users to do so.
   • Common commands include /tag-and-rerun-ci, /tag-run-ci-label, /rerun-failed-ci
4. After green CI and required approvals, ask Merge Oncalls or people with Write permission to merge the PR.

---

CI States

Latest PR Test (Base): ❌ Run #28023230096
Latest PR Test (Extra): ❌ Run #28023229767

[gemini-code-assist]

Code Review

This pull request wraps the registration of the draft memory host pool in a try-except block to gracefully handle registration failures with Mooncake Store, falling back to L2 cache. The reviewer suggests catching Exception instead of only RuntimeError to ensure all potential exceptions from C++ bindings are caught, preventing startup failures.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

[stmatengss]

An extra unit test is unnecessary.

[stmatengss]

Motivation

Fixes #28873

When using EAGLE speculative decoding with Mooncake L3 storage in standalone (dummy client) mode, the server crashes during startup. The draft host pool is allocated with the default allocator (mmap) instead of MooncakeHostTensorAllocator, causing register_buffer() to fail with error code -1 because the buffer is outside Mooncake's shared memory segment.

Modifications

Wrap the register_mem_host_pool_v2() call in _maybe_register_draft_with_storage() with a try/except. On failure, log a warning and return early so that draft_page_get_func and draft_page_set_func remain None.

This disables draft L3 (SSD offload) while keeping:

• Draft L2 (host DRAM): fully functional (uses mem_pool_host_draft directly, independent of storage backend)
• Target model L2/L3: fully functional (target pool uses MooncakeHostTensorAllocator correctly)

Accuracy Tests

Not applicable — no changes to model forward code.

Speed Tests and Profiling

Not applicable — no changes to inference kernels. The fix only affects startup behavior.

Checklist

• Format your code according to the Format code with pre-commit.
• Add unit tests according to the Run and add unit tests.
• Update documentation according to Write documentations.
• Provide accuracy and speed benchmark results according to Test the accuracy and Benchmark the speed.
• Follow the SGLang code style guidance.

Review and Merge Process

1. Ping Merge Oncalls to start the process. See the PR Merge Process.

2. Get approvals from CODEOWNERS and other reviewers.

3. Trigger CI tests with comments or contact authorized users to do so.

   • Common commands include /tag-and-rerun-ci, /tag-run-ci-label, /rerun-failed-ci

4. After green CI and required approvals, ask Merge Oncalls or people with Write permission to merge the PR.

CI States

Latest PR Test (Base): ❌ Run #28023230096 Latest PR Test (Extra): ❌ Run #28023229767

This issue likely stems from the dummy client's lack of draft model offloading support in standalone mode. MooncakeHostTensorAllocator could be a potential solution.

[huangtingwei9988]

Thanks for pointing out the issue. Because the draft didn't specify an allocator type, an inconsistent allocator was used, causing an error when registering the buffer. I have now fixed this problem. #29823