feat(sglang): cut over to a fork-built image, retire our own build - #4256
Merged
Conversation
…utover mattbucci/2x-R9700-RDNA4-GFX1201-sglang-inference now publishes its own ghcr.io/mattbucci/sglang-rdna4 image with the same patch series we vendored, so our Dockerfile/build workflow/renovate tracking are dead weight. The deployed InferenceService keeps running the last self-built digest unchanged. Cutover to the upstream image is blocked on mattbucci#6 (HiCache OpenSSL headers fix) merging; SGLANG_RDNA4_DISABLE_STORE_CACHE=1 is prepped now so the swap is a one-line image bump once that lands.
📝 WalkthroughWalkthroughThe repository retires its custom RDNA4 SGLang image assets, documents the fork-published replacement, updates Renovate metadata, and changes the Qwen SGLang deployment to a new image digest with revised cache settings. ChangesRDNA4 SGLang cutover
Estimated code review effort: 3 (Moderate) | ~20 minutes Possibly related PRs
Suggested labels: 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Comment |
Tanguille
marked this pull request as draft
July 29, 2026 20:06
@@ spec.env @@
# inference.llmkube.dev/v1alpha1/InferenceService/ai/qwen36-27b
! + one list entry added:
+ - name: SGLANG_RDNA4_DISABLE_STORE_CACHE
+ value: "1"
@@ spec.env.SGLANG_HICACHE_FILE_BACKEND_STORAGE_DIR.value @@
# inference.llmkube.dev/v1alpha1/InferenceService/ai/qwen36-27b
! ± value change
- /hicache/sglang-v0.5.16_awq-f541031d
+ /hicache/sglang-v0.5.16_awq-f541031d_fork-cb7b760
@@ spec.image @@
# inference.llmkube.dev/v1alpha1/InferenceService/ai/qwen36-27b
! ± value change
- ghcr.io/tanguille/sglang-rdna4:v0.5.16-gfx1201@sha256:615cb4265400f5468da4c32221f0086df075b126ce8dbfa78d998e844a5e424d
+ ghcr.io/tanguille/sglang-rdna4:sha-cb7b76050cbf8dda9e5a78c07b24df1a783caeb3@sha256:f961fc274eaad4af065097e0921b88ae5a97a478baa0e384e6c5ee440c355b29
|
AI Automated ReviewAnalysis engine: qwen-3.6-fast@http://litellm.ai.svc.cluster.local/v1 (openai) Recommendation: Approve. This PR cleanly retires the self-built SGLang image pipeline, removes dead build artifacts and Renovate managers, and preps the Change-by-Change Findings
Standards Compliance
Must Check Items
Unknowns or Needs Verification
|
…upstream ghcr.io/tanguille/sglang-rdna4 now points at an image built by our fork (Tanguille/2x-R9700-RDNA4-GFX1201-sglang-inference) rather than the last self-built digest. Includes mattbucci's full patch series plus our HiCache OpenSSL-headers fix (upstream PR mattbucci#6, still open) and the CI disk-space/GHCR-lowercase fixes needed to publish it. SGLANG_RDNA4_DISABLE_STORE_CACHE=1 is now load-bearing, not prep. Bumped the HiCache storage-dir suffix per the existing convention (any image change invalidates cached pages). Re-added the renovate digest-review rule since the image is live and rebuildable again.
…' into refactor/drop-sglang-image-build
Tanguille
changed the title
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@docs/llm-hosting/sglang-blockers.md`:
- Line 115: Update the image reference in the Impact statement to match the
deployed `ghcr.io/tanguille/sglang-rdna4` image used elsewhere in the document,
or explicitly identify `ghcr.io/mattbucci/sglang-rdna4` as the upstream/source
image rather than the deployed image.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro Plus
Run ID: b5150629-d1b4-4900-b72d-c760e5adb3cd
📒 Files selected for processing (8)
.github/workflows/build-sglang-rdna4.yaml.renovaterc.json5docker/sglang-rdna4/Dockerfiledocker/sglang-rdna4/README.mddocker/sglang-rdna4/entrypoint.shdocs/llm-hosting/engine-benchmarks-gfx1201.mddocs/llm-hosting/sglang-blockers.mdkubernetes/apps/ai/llmkube/models/qwen36-27b-sglang.yaml
🔗 Linked repositories identified
CodeRabbit considers these linked repositories for cross-repo context during reviews:
Tanguille/LLMKube(auto-detected)Tanguille/2x-R9700-RDNA4-GFX1201-sglang-inference(auto-detected)
💤 Files with no reviewable changes (4)
- docker/sglang-rdna4/Dockerfile
- docker/sglang-rdna4/entrypoint.sh
- docker/sglang-rdna4/README.md
- .github/workflows/build-sglang-rdna4.yaml
| ## Blocker 5 — No official SGLang Docker image for gfx1201 | ||
|
|
||
| **Impact:** Operational. Requires maintaining a custom Dockerfile and image (defined in `docker/sglang-rdna4/`). | ||
| **Impact:** Operational. Previously required maintaining our own Dockerfile and build pipeline; the fork now publishes `ghcr.io/mattbucci/sglang-rdna4` itself, retiring ours (see "Current approach" above). |
There was a problem hiding this comment.
📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win
Align the documented image with the deployed image.
Line 115 names ghcr.io/mattbucci/sglang-rdna4, but the current deployment and Line 5 use ghcr.io/tanguille/sglang-rdna4. Update this line or explicitly label the Mattbucci image as the upstream/source image.
Proposed fix
-**Impact:** Operational. Previously required maintaining our own Dockerfile and build pipeline; the fork now publishes `ghcr.io/mattbucci/sglang-rdna4` itself, retiring ours (see "Current approach" above).
+**Impact:** Operational. Previously required maintaining our own Dockerfile and build pipeline; our fork now publishes `ghcr.io/tanguille/sglang-rdna4`, retiring ours (see "Current approach" above).📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
Suggested change
| **Impact:** Operational. Previously required maintaining our own Dockerfile and build pipeline; the fork now publishes `ghcr.io/mattbucci/sglang-rdna4` itself, retiring ours (see "Current approach" above). | |
| **Impact:** Operational. Previously required maintaining our own Dockerfile and build pipeline; our fork now publishes `ghcr.io/tanguille/sglang-rdna4`, retiring ours (see "Current approach" above). |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@docs/llm-hosting/sglang-blockers.md` at line 115, Update the image reference
in the Impact statement to match the deployed `ghcr.io/tanguille/sglang-rdna4`
image used elsewhere in the document, or explicitly identify
`ghcr.io/mattbucci/sglang-rdna4` as the upstream/source image rather than the
deployed image.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
mattbucci/2x-R9700-RDNA4-GFX1201-sglang-inferencenow publishes its own image with the same patch series we used to vendor — our build pipeline is dead weight. Deletesdocker/sglang-rdna4/,.github/workflows/build-sglang-rdna4.yaml, and the renovate rules tied to them.InferenceService.imagenow points atghcr.io/tanguille/sglang-rdna4:sha-cb7b7605..., built by our own fork (Tanguille/2x-R9700-RDNA4-GFX1201-sglang-inference,main) via itsbuild-image.yamlpublishjob — not a pipeline in this repo, and not waiting on upstream review.mattbucci#6, still open/unmerged there) and two CI fixes we had to make to get the fork's own workflow to actually build and publish (disk-space exhaustion onload: true, GHCR repo name not lowercased).SGLANG_RDNA4_DISABLE_STORE_CACHE=1is now load-bearing: LLMKube invokeslaunch_serverdirectly, bypassing the image'sentrypoint.sh/gpu-selection.sh, the only place this env var is normally set for TP=1. Verified directly against the published image'skvcache.py— without it, the JIT store_cache kernel crashes on first request.ghcr.io/tanguille/sglang-rdna4— removed earlier when the image was frozen, restored now that it's live and rebuildable again.Production impact
This rolls the live sglang serving pod on the next reconcile (
InferenceServiceusesRecreateon the single GPU).Verified before merge
SGLANG_RDNA4_DISABLE_STORE_CACHErequirement confirmed by pulling the actual published image and reading the realkvcache.py, not by reading build scripts--trust-remote-code/SGLANG_TRUST_REMOTE_CODEconflict checked — the env var is only read by the bypassed entrypoint wrapper, not bysglang.launch_serveritself.renovaterc.json5validated withjson5parserqwen36-27b-sglang.yamlvalidated withyaml.safe_load_allSummary by CodeRabbit
Updates
Documentation