Repository navigation
Conversation
Replace hardcoded torch.cuda.Stream/stream calls with device-agnostic helpers (_create_device_stream, _device_stream_context) that dispatch to torch.xpu on Intel XPU devices. Add XPU integration tests verifying store/retrieve round-trip, cold cache, negative slot indices, multi-cycle correctness, and bf16 dtype preservation. Signed-Off: Libin Tang <libin.tang@intel.com>
Signed-Off: Libin Tang <libin.tang@intel.com>
LMCacheLayerwiseConnector.__init__ requires config_file as a mandatory positional argument, but TestLMCacheXPUConnector.setUpClass constructed the connector without it, failing with: TypeError: __init__() missing 1 required positional argument: 'config_file' The test already resolves the bundled test_xpu_config.yaml into the LMCACHE_CONFIG_FILE env var; thread that path into the constructor. Signed-Off: Libin Tang <libin.tang@intel.com>
…Cache - Replace non-existent torch.current_stream(device=...) with torch.get_device_module(self.device).current_stream() in _mp_load_back - Simplify _create_device_stream/_device_stream_context to use torch.get_device_module() instead of xpu/cuda if-else branching - Fix _load_back to dispatch via load_fn(slot_mapping, prefix_pad) instead of hardcoding start_load_kv, so MP mode runs retrieve_kv Co-Authored-By: Libin Tang <libin.tang@intel.com>
The MP (multi-process) connector shares the KV cache with the LMCache daemon via CUDA IPC (Tensor._share_cuda_), which is unavailable on XPU and raises 'RuntimeError: CUDA is not available' during KV registration. Default to IP (in-process layerwise) mode on XPU while keeping MP as the default on CUDA, so enabling LMCache on XPU works out of the box. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…test_lmcache_connector_config.yaml. Co-Authored-By: Libin Tang <libin.tang@intel.com>
Co-authored-by: Libin Tang <libin.tang@intel.com>
…uantization on AMD GPUs" (sgl-project#28291) Co-authored-by: Bowen Bao <bowenbao@amd.com> Co-authored-by: HAI <hixiao@gmail.com> Co-authored-by: Claude <noreply@anthropic.com>
…gl-project#31308) Co-authored-by: Zhangheng <hzh0425@apache.org>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> Co-authored-by: Shangming Cai <csmthu@gmail.com>
…fix clamping (sgl-project#31443) Co-authored-by: AlejandroParedesLT <alejandroparedeslatorre@gmail.com> Co-authored-by: Kevin Flansburg <kevin.flansburg@gmail.com> Co-authored-by: Chao Shi <chao.shi@alibaba-inc.com>
…gl-project#30247) Co-authored-by: Claude Fable 5 <noreply@anthropic.com> Co-authored-by: Halcyon <56064364+VAthree@users.noreply.github.com>
…ct#31576) Co-authored-by: siyu <liusy58@linux.alibaba.com>
Co-authored-by: Zijie Xia <zijie.xia@radixark.ai> Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com> Co-authored-by: zijiexia <37504505+zijiexia@users.noreply.github.com>
PR sgl-project#23534 registered test_lmcache_connector / test_lmcache_radix_cache on stage-b-test-1-gpu-xpu but only added `pip install lmcache` to the stage-a Install Dependency step. Each XPU stage starts a fresh container, so stage-b never had lmcache and failed with ModuleNotFoundError. Move the lmcache install to stage-b (where the tests actually run) and leave stage-a as a light smoke gate. Co-authored-by: Ma Mingfei <mingfei.ma@intel.com>
github-actions
Bot
added
documentation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
20 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes the XPU CI failure on #23534:
Root cause:
test_lmcache_connector.py/test_lmcache_radix_cache.pyare registered onstage-b-test-1-gpu-xpu, but#23534only addedpip install "lmcache>=0.3.9"to the stage-a Install Dependency step. Each XPU stage starts a fresh container, so stage-b never hadlmcache.Fix: install
lmcachein stage-b (where the tests run), and keep stage-a as a light smoke gate without it.This branch is based on
#23534head (libinta/xpu_lmcache) plus the one workflow fix. Prefer cherry-picking commitff3c3e842onto#23534if that PR should remain the landing vehicle.Minimal patch for sgl-project#23534
In
.github/workflows/pr-test-xpu.ymlstage-b Install Dependency, change:docker exec ci_sglang_xpu /opt/venv/bin/python3 -m pip install pytest expecttest ray huggingface_hub tabulateto:
docker exec ci_sglang_xpu /opt/venv/bin/python3 -m pip install pytest expecttest ray huggingface_hub tabulate "lmcache>=0.3.9"And remove
"lmcache>=0.3.9"from the stage-a install line (tests are not registered there).Test plan
PR Test (XPU)on#23534after applying the patchlmcachetest_lmcache_radix_cache.pyandtest_lmcache_connector.pyno longer fail at importCI States
Latest PR Test (Base): ❌ Run #29889756586
Latest PR Test (Extra): ❌ Run #29889756410