Repository navigation
[CI] [AMD] install: Cache the AOT kernel wheel and stop wiping venv bytecode - #46703
Merged
bingxche merged 1 commit intoOct 28, 2026
Merged
Conversation
…ytecode Every AMD CI job rebuilt python/sglang/kernels/aot from source (~30 s) and deleted every .pyc under /opt/venv (~6 s) before running a test. - Build sglang-kernel as a wheel and keep it on the persistent /sgl-data volume, keyed by the aot source tree hash, the image ID and the image's GPU arch. Later jobs on the same host install the cached wheel instead of compiling. A dirty tree builds without caching, publishing is atomic, and entries unused for 7 days are pruned. - Only clear bytecode in the checkout. pip already removed the old sglang files from /opt/venv, so walking site-packages only made every job recompile torch's bytecode.
bingxche
approved these changes
Oct 28, 2026
Open
1 task done
This was referenced Oct 29, 2026
TwelveNights
added a commit
to TwelveNights/sglang
that referenced
this pull request
Nov 5, 2026
…y source tree and image Replace the stage-a artifact handoff with a cache on the persistent /sgl-data volume, next to the AOT wheel cache from sgl-project#46703 and with the same conventions. The artifact only reached pull_request runs of pr-test-amd.yml; the install script runs in every AMD job, so MI35x, extra, nightly and later runs now hit too. amd_ci_install_dependency.sh computes one entry per Cargo package: sglang-radix-tree (mem_cache, mem_cache_inspection) and sglang-mm (_multimodal). The key hashes the git tree of the package's sources, the loader, the toolchain file and .cargo, plus the installed torch version and SOABI, followed by the image ID and GPU arch. A dirty tree, a missing path or a container without a /sgl-data mount gets no key, with the reason logged, and nothing is built eagerly; the tests build at runtime as before. amd_ci_rust_ext_cache.py sync copies a cached entry into the loader's own cache, then loads every module of the package through the loader, which finds it in its cache or builds it. The loader stays in auto mode, so its fingerprint still gates every load. An entry counts as a hit only if it supplied every module that loaded; otherwise (a toolchain or RUSTFLAGS the key does not cover, files lost to a killed prune) it is replaced, not refreshed. A miss publishes the modules that built atomically (mkdtemp + rename, the loser drops its copy), then prunes entries unused for 7 days, renaming each aside before removing it. sglang-radix-tree is skipped up front when torch is outside the Rust TreeCore range. Errors are warnings and never fail the job. The workflow changes and amd_ci_rust_ext_cache.sh are dropped.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Two cuts to the setup that every AMD CI job runs in
amd_ci_install_dependency.shbefore its first test:sglang-kernel) wheel on the runner's persistent/sgl-datavolume.pycunder/opt/venv"Per week" assumes the ~41,900 AMD GPU jobs measured in the week of Oct 19–26 and a high hit rate. The real hit rate depends on how often
kernels/aotand the CI image change. It will be measured after merge.Why
The install step takes ~2.3 of the ~2.5 min of per-job setup:
python/sglang/kernels/aotfrom source withsetup_rocm.py install, in every job, although that tree rarely changes.find /opt/venv -name "*.pyc" -deleteand the__pycache__cleanup. Thepip uninstallcalls just before it already remove the old sglang files, so the walk only throws away torch's bytecode, which every job then recompiles on first import.Notes for the reviewer
python/sglang/kernels/aot, the container's image ID, and the image's GPU arch. Those are the only inputs to the build:setup_rocm.pyreads nothing outside that directory, and the image fixes torch, HIP and Python./sgl-datamount gets a container-local cache, so it just misses every time, same as today.setup_rocm.py install(an egg pluseasy-install.pth) tosetup_rocm.py bdist_wheelandpip install --no-deps --force-reinstall, so a cached build and a fresh one install the same way. Nothing else in the AMD scripts reads the egg orkernels/aot/dist.mv -T; the loser drops its copy. Entries not used for 7 days are pruned, and a hit refreshes the entry's mtime.__pycache__in the checkout is kept.Test plan
pr-test-amd.ymldispatch on rocm10 (run 40430315918):sgl-kernel-unit-test-amdpasses against the wheel install.stage-b-test-1-gpu-small-amd-mi35x (0)passes.test_openai_server.py(cannot import name 'ResponseFunctionShellToolCall'). That's the OpenAI SDK bump from [Chore] deps: Bump the OpenAI SDK to 2.8.0 for Responses shell tools (2/8) #45291, fixed on main by [CI] [AMD] deps: Install openai 2.8.0 in AMD CI for the Responses API #46774, not this change.CI States
Latest PR Test (Base): ✅ Run #40430396805
Latest PR Test (Extra): ❌ Run #40430396795
Latest PR Test (AMD ROCm 10): ❌ Run #40430396869