Skip to content

[mem] Flatten memory checkers into composable per-pool invariant checks#22562

Merged
hnyls2002 merged 16 commits intomainfrom
lsyin/checker-refactor
Apr 11, 2026
Merged

[mem] Flatten memory checkers into composable per-pool invariant checks#22562
hnyls2002 merged 16 commits intomainfrom
lsyin/checker-refactor

Conversation

@hnyls2002
Copy link
Copy Markdown
Collaborator

@hnyls2002 hnyls2002 commented Apr 10, 2026

Summary

  • Replace 3 monolithic checkers (_check_hybrid_memory, _check_mamba_memory, _check_radix_cache_memory) with composable per-pool checks: _check_full_pool, _check_swa_pool, _check_mamba_pool
  • All share a single _check_pool_invariant static method: available + evictable + protected + session_held + uncached == total
  • check_memory (idle) uses flat if/if instead of if/elif/else — naturally supports SWA + mamba coexistence
  • self_check_during_busy delegates to _check_full_pool(ps, uncached=N) instead of reimplementing the invariant
  • Extract _get_total_uncached_size helper from busy check boilerplate
  • Watchdog dump_info also uses the flat structure

Follows up on #22554 and #22559.

Test plan

  • test_scheduler_pause_generation.py — CPU, mock PoolStats path
  • test_metrics.py — exercises idle metrics path
  • Mamba / SWA model tests (existing CI coverage)

@gemini-code-assist
Copy link
Copy Markdown
Contributor

Warning

You have reached your daily quota limit. Please wait up to 24 hours and I will start processing your requests again!

@hnyls2002
Copy link
Copy Markdown
Collaborator Author

/rerun-test test_scheduler_pause_generation.py test_mamba_unittest.py test_metrics.py

@github-actions
Copy link
Copy Markdown
Contributor

ubuntu-latest (1 test): View workflow run

cd test/ && python3 registered/unit/managers/test_scheduler_pause_generation.py

1-gpu-5090 (2 tests): View workflow run

cd test/ && python3 registered/unit/mem_cache/test_mamba_unittest.py
cd test/ && python3 registered/observability/test_metrics.py

@hnyls2002
Copy link
Copy Markdown
Collaborator Author

/rerun-test test_disaggregation_basic.py test_pp_llama.py

@github-actions
Copy link
Copy Markdown
Contributor

test_disaggregation_basic.py: Ambiguous filename test_disaggregation_basic.py — matched 2 files:

  • test/registered/amd/disaggregation/test_disaggregation_basic.py
  • test/registered/disaggregation/test_disaggregation_basic.py

Please provide the full path, e.g. /rerun-test test/registered/disaggregation/test_disaggregation_basic.py

test_pp_llama.py: No test file found matching test_pp_llama.py under test/registered/.

@hnyls2002
Copy link
Copy Markdown
Collaborator Author

/rerun-test test/registered/disaggregation/test_disaggregation_basic.py test/registered/distributed/test_pp_single_node.py

@github-actions
Copy link
Copy Markdown
Contributor

2-gpu-h100 (1 test): View workflow run

cd test/ && python3 registered/disaggregation/test_disaggregation_basic.py

4-gpu-h100 (1 test): View workflow run

cd test/ && python3 registered/distributed/test_pp_single_node.py

@hnyls2002
Copy link
Copy Markdown
Collaborator Author

/rerun-test test/registered/distributed/test_disaggregation_pp.py test/registered/distributed/test_epd_disaggregation.py

@github-actions
Copy link
Copy Markdown
Contributor

8-gpu-h20 (1 test): View workflow run

cd test/ && python3 registered/distributed/test_disaggregation_pp.py

4-gpu-h100 (1 test): View workflow run

cd test/ && python3 registered/distributed/test_epd_disaggregation.py

@hnyls2002
Copy link
Copy Markdown
Collaborator Author

/rerun-test test_openai_function_calling.py

@github-actions
Copy link
Copy Markdown
Contributor

1-gpu-h100 (1 test): View workflow run

cd test/ && python3 registered/openai_server/function_call/test_openai_function_calling.py

@hnyls2002 hnyls2002 merged commit 61a62c6 into main Apr 11, 2026
39 of 93 checks passed
@hnyls2002 hnyls2002 deleted the lsyin/checker-refactor branch April 11, 2026 09:56
pyc96 pushed a commit to pyc96/sglang that referenced this pull request Apr 14, 2026
yhyang201 pushed a commit to yhyang201/sglang that referenced this pull request Apr 22, 2026
@hnyls2002 hnyls2002 mentioned this pull request Apr 29, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant