Skip to content

[ROCm] Raise HiCache JIT block quota for mapped-host throughput - #39036

Merged
HaiShaw merged 1 commit into
sgl-project:mainfrom
AMD-yanfeiwang:perf/rocm-hicache-jit-block-quota-32
Sep 11, 2026
Merged

HaiShaw merged 1 commit into
sgl-project:mainfrom
AMD-yanfeiwang:perf/rocm-hicache-jit-block-quota-32

Conversation

@AMD-yanfeiwang

@AMD-yanfeiwang AMD-yanfeiwang commented Sep 11, 2026 •

Copy link
Copy Markdown
Contributor

Summary

This PR is split from #35233 to keep each change focused and independently reviewable.

It contains only the ROCm-specific HiCache JIT block-quota setting.

Changes

  • Use a default JIT block quota of 32 on HIP runtimes.
  • Keep the CUDA default unchanged at 2.

Scope

This PR does not change kernel implementations, memory layouts, backend selection, or registered-host pointer handling.

The registered-host kernel/page_first capability is split into #39041. The complete implementation remains in #35233.

Validation

  • Repository pre-commit hooks passed.
  • ROCm JIT compilation and correctness checks passed.

CI States

Latest PR Test (Base): ❌ Run #34570728333
Latest PR Test (Extra): ❌ Run #34570727548
Latest PR Test (AMD ROCm 10): ❌ Run #34570727941

@HaiShaw

HaiShaw commented Sep 11, 2026

Copy link
Copy Markdown
Collaborator

HIP specific

@HaiShaw
HaiShaw merged commit 69aa46b into sgl-project:main Sep 11, 2026
209 of 239 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

hicache Hierarchical Caching for SGLang jit-kernel

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants