fix(gms): support SGLang 0.5.16 memory pool API

[galletas1712]

Summary

• Patch GMS memory accounting at SGLang 0.5.16's alloc_memory_pool API.
• Add preloaded weight memory to the persistent baseline once per runner, preserving post-capture KV sizing.
• Publish write-mode weights only after SGLang exits all managed GMS allocation regions, so writable buffer clones use the normal CUDA allocator after GMS reconnects read-only.
• Add focused CPU coverage for baseline accounting, publication ordering, RW-to-RO transition, nested-region failures, and finalization cleanup.

Validation

• Focused CPU tests — 14 passed:
  • components/src/dynamo/sglang/tests/test_sglang_gms_memory_saver.py
  • lib/gpu_memory_service/tests/test_sglang_patches.py
• Pytest marker validation — 14 checked, 0 missing marker sets.
• Ruff lint/format, Python syntax, and diff whitespace checks — passed.
• Narrow H100 validation: run 30593269879 — 2 passed:
  • test_gms_basic_pause_resume_sglang
  • test_gms_shadow_engine_failover_sglang
• H100 logs confirm deferred weight publication completed, alloc_memory_pool applied the preloaded-weight baseline adjustment, and pause/resume plus post-failover inference returned HTTP 200. All Early Flake Detection retries also passed.

Related Issues

• No related public GitHub issue
• Internal tracking: DYN-3690

[devin-ai-integration]

Devin Review found 3 potential issues.

[coderabbitai]

Walkthrough

patch_model_runner() now supports SGLang runners without init_memory_pool by patching alloc_memory_pool. New CPU-only tests cover legacy and fallback behavior, idempotency, retries, zero preload values, and unsupported APIs.

Changes

SGLang memory patching

Layer / File(s)	Summary
alloc_memory_pool fallback patch lib/gpu_memory_service/integrations/sglang/patches.py	Detects the alternate ModelRunner API, adjusts pre_model_load_memory once using preloaded weight size, delegates allocation, and updates patch flags.
SGLang patcher test coverage lib/gpu_memory_service/tests/test_sglang_patches.py	Adds isolated fake-runner tests for legacy and fallback APIs, retries, idempotency, zero preload values, and unsupported runners.

Estimated code review effort: 3 (Moderate) | ~20 minutes

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Title check	✅ Passed	The title clearly and concisely identifies the fix adding support for SGLang 0.5.16’s memory pool API.
Description check	✅ Passed	The description clearly explains the changes, testing, validation results, and related-issue status, despite using different section headings than the template.

---

_{Comment @coderabbitai help to get the list of available commands.}

[coderabbitai]

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@lib/gpu_memory_service/integrations/sglang/patches.py`:
- Around line 182-190: Update the baseline-adjustment logic around
_gms_memory_baseline_adjusted to access self.pre_model_load_memory directly and
remove the getattr fallback and numeric-type guard, so missing or invalid
baselines raise while preserving the existing one-time adjustment behavior.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: fc9f0a29-eb00-4028-8399-e4505230adf0

📥 Commits

Reviewing files that changed from the base of the PR and between 6cc7c4e and c53907e.

📒 Files selected for processing (2)

• lib/gpu_memory_service/integrations/sglang/patches.py
• lib/gpu_memory_service/tests/test_sglang_patches.py

[datadog-official]

🎯 Code Coverage (details)
• Patch Coverage: 100.00%
• Overall Coverage: 43.53% (-7.58%)

_{This comment will be updated automatically if new data arrives.
🔗 Commit SHA: 7ce711b | Docs | Datadog PR Page | Give us feedback!}