Skip to content

Memory pool fix for upstream change about eagle#4170

Merged
xiezhq-hermann merged 3 commits intomainfrom
xiezhq-mem-fix
Mar 7, 2025
Merged

Memory pool fix for upstream change about eagle#4170
xiezhq-hermann merged 3 commits intomainfrom
xiezhq-mem-fix

Conversation

@xiezhq-hermann
Copy link
Copy Markdown
Collaborator

Motivation

#3986 made substaintial change on memory pool which breaks hierarchical cache, this PR fix the interfaces problem around it.
There are separated PRs fixing other known issues with hierarchical caching, which will be merged later:
Large TP sizes: #4082
MLA support: #4009

Modifications

Primarily interface changes.

Checklist

@xiezhq-hermann xiezhq-hermann merged commit 9376ac3 into main Mar 7, 2025
34 of 38 checks passed
@xiezhq-hermann xiezhq-hermann deleted the xiezhq-mem-fix branch March 7, 2025 08:58
aoshen524 pushed a commit to aoshen524/sglang that referenced this pull request Mar 10, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants