Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
26 commits
Select commit Hold shift + click to select a range
dccd493
fix(qwen): alias minimal effort to low
ktsaou Sep 14, 2026
3c96585
wip(hicache): preserve Qwen auxiliary cache state
ktsaou Sep 14, 2026
de9abbe
Merge pull request #6 from ktsaou/fix/qwen-minimal-effort
kanadaj Sep 14, 2026
aa990bc
fix(responses): preserve Qwen phase and item order
ktsaou Sep 14, 2026
ed43202
fix(responses): preserve nonstream phase order
Sep 14, 2026
df22281
test(responses): capture PR5 merge-gate regressions
Sep 14, 2026
6968c13
test(responses): capture independent follow-up regressions
Sep 14, 2026
76f04a2
test(responses): capture second review edge cases
Sep 14, 2026
1cdf383
test(responses): capture native payload marker regression
Sep 14, 2026
7fe50de
test(responses): capture implicit-close and scope regressions
Sep 14, 2026
8b33799
test(responses): capture renewed reasoning after implicit close
Sep 14, 2026
a42cf78
test(responses): capture final parser attribution blockers
Sep 14, 2026
9c91e42
fix(responses): finalize ordered parser artifacts
Sep 14, 2026
460545b
Merge pull request #5 from ktsaou/fix/qwen-responses-phase-order
kanadaj Sep 14, 2026
179cbb4
fix(qwen): enable release multimodal processor paths
ktsaou Sep 14, 2026
84ec70a
fix(packaging): compose multimodal alias after responses
Sep 14, 2026
e5d9338
Merge pull request #7 from ktsaou/fix/qwen-multimodal-alias
kanadaj Sep 14, 2026
3fba7f0
fix(runtime): surface invalid generated token failures
ktsaou Sep 14, 2026
eebf6b5
fix(runtime): complete invalid token failure propagation
Sep 14, 2026
cb8a491
[verified] fix(packaging): finalize cumulative invalid-token profile
Sep 14, 2026
fae68fb
fix(responses): emit Harmony failure terminal event
Sep 14, 2026
facd7be
Merge pull request #8 from ktsaou/fix/invalid-token-failure
kanadaj Sep 14, 2026
dc9b10b
docs: record cumulative compatibility image publication
Sep 14, 2026
b524794
Merge pull request #10 from kanadaj/docs/production-cumulative-compat…
kanadaj Sep 14, 2026
b6c8bcc
fix(hicache): restore Qwen companion state and backport ordering fixes
ktsaou Sep 14, 2026
3c13bfa
Merge preserved HiCache draft history after updating onto current main
ktsaou Sep 14, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 14 additions & 0 deletions Dockerfile.hicache-wip
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
# EXPERIMENTAL: opt-in HiCache profile, isolated from the default images.
# Configuration-specific qualification is recorded in docs/hicache-wip.md.
FROM docker.io/kanadaj/sglang-qwen38fn-sm120-turbo@sha256:f2859d1ccf824a5295088cf578eba89b0f3eeefff6ae7679c3f5d64af0689458
ARG SOURCE_REVISION
LABEL org.opencontainers.image.source="https://github.com/kanadaj/sglang" \
org.opencontainers.image.revision="${SOURCE_REVISION}" \
org.opencontainers.image.description="Opt-in Qwen HiCache state-transfer profile; qualify the deployment configuration"
COPY patches/0020-hicache-ple-state.patch patches/0021-hicache-file-integrity.patch patches/0022-hicache-qsa-sidecar.patch patches/0023-qsa-sparse-gather-memory-safety.patch patches/0024-router-pdl-bias-order.patch patches/0025-hicache-load-order.patch patches/0026-qsa-short-extend-bounds.patch patches/series.hicache-wip /opt/qwen-hicache-wip/patches/
COPY provenance/invalid-token-failure-runtime-files.json provenance/hicache-wip.json /opt/qwen-hicache-wip/provenance/
COPY scripts/verify_hicache_wip.py /opt/qwen-hicache-wip/scripts/verify_hicache_wip.py
RUN python3 -B /opt/qwen-hicache-wip/scripts/verify_hicache_wip.py \
--tree /sgl-workspace/sglang --apply
ENTRYPOINT ["python3", "-m", "sglang.launch_server"]
CMD ["--help"]
19 changes: 19 additions & 0 deletions Dockerfile.invalid-token-failure
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
# Cumulative CPU-only overlay: patches 0015 through 0019.
# All model paths, serving arguments, and runtime settings remain external.
FROM docker.io/kanadaj/sglang-qwen38fn-sm120-turbo@sha256:872a2bda228e39aa9c1af729b47cc28f7862e7859e448f1a8868b85a4051f404
ARG SOURCE_REVISION
LABEL org.opencontainers.image.source="https://github.com/kanadaj/sglang" \
org.opencontainers.image.revision="${SOURCE_REVISION}"
COPY runtime.invalid-token-failure/python/sglang/srt/entrypoints/openai/serving_chat.py /sgl-workspace/sglang/python/sglang/srt/entrypoints/openai/serving_chat.py
COPY runtime.invalid-token-failure/python/sglang/srt/entrypoints/openai/serving_completions.py /sgl-workspace/sglang/python/sglang/srt/entrypoints/openai/serving_completions.py
COPY runtime/python/sglang/srt/entrypoints/openai/protocol.py /sgl-workspace/sglang/python/sglang/srt/entrypoints/openai/protocol.py
COPY runtime.invalid-token-failure/python/sglang/srt/entrypoints/openai/serving_responses.py /sgl-workspace/sglang/python/sglang/srt/entrypoints/openai/serving_responses.py
COPY runtime/python/sglang/srt/entrypoints/openai/responses_compat.py /sgl-workspace/sglang/python/sglang/srt/entrypoints/openai/responses_compat.py
COPY runtime/python/sglang/srt/function_call/qwen3_coder_detector.py /sgl-workspace/sglang/python/sglang/srt/function_call/qwen3_coder_detector.py
COPY runtime/python/sglang/srt/multimodal/processors/qwen_vl.py /sgl-workspace/sglang/python/sglang/srt/multimodal/processors/qwen_vl.py
COPY runtime.invalid-token-failure/python/sglang/srt/managers/schedule_batch.py /sgl-workspace/sglang/python/sglang/srt/managers/schedule_batch.py
COPY provenance/invalid-token-failure-runtime-files.json /tmp/invalid-token-failure-runtime-files.json
RUN python3 -B -c 'import hashlib,json,pathlib; root=pathlib.Path("/sgl-workspace/sglang"); expected=json.loads(pathlib.Path("/tmp/invalid-token-failure-runtime-files.json").read_text()); actual={str(p.relative_to(root)) for p in (root/"python/sglang").rglob("*") if p.is_file() and "__pycache__" not in p.parts and p.suffix != ".pyc"}; assert actual == set(expected), (len(actual), len(expected)); bad=[n for n,h in expected.items() if hashlib.sha256((root/n).read_bytes()).hexdigest()!=h]; assert not bad, bad; files=[root/"python/sglang/srt/entrypoints/openai"/n for n in ("serving_chat.py", "serving_completions.py", "protocol.py", "serving_responses.py", "responses_compat.py")]+[root/"python/sglang/srt/function_call/qwen3_coder_detector.py", root/"python/sglang/srt/multimodal/processors/qwen_vl.py", root/"python/sglang/srt/managers/schedule_batch.py"]; [compile(f.read_bytes(), str(f), "exec") for f in files]' \
&& rm /tmp/invalid-token-failure-runtime-files.json
ENTRYPOINT ["python3", "-m", "sglang.launch_server"]
CMD ["--help"]
17 changes: 17 additions & 0 deletions Dockerfile.qwen-multimodal-alias
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
# Cumulative CPU-only overlay: patches 0015, 0016, 0017, then 0018.
# All model paths, serving arguments, and runtime settings remain external.
FROM docker.io/kanadaj/sglang-qwen38fn-sm120-turbo@sha256:872a2bda228e39aa9c1af729b47cc28f7862e7859e448f1a8868b85a4051f404
ARG SOURCE_REVISION
LABEL org.opencontainers.image.source="https://github.com/kanadaj/sglang" \
org.opencontainers.image.revision="${SOURCE_REVISION}"
COPY runtime/python/sglang/srt/entrypoints/openai/serving_chat.py /sgl-workspace/sglang/python/sglang/srt/entrypoints/openai/serving_chat.py
COPY runtime/python/sglang/srt/entrypoints/openai/protocol.py /sgl-workspace/sglang/python/sglang/srt/entrypoints/openai/protocol.py
COPY runtime/python/sglang/srt/entrypoints/openai/serving_responses.py /sgl-workspace/sglang/python/sglang/srt/entrypoints/openai/serving_responses.py
COPY runtime/python/sglang/srt/entrypoints/openai/responses_compat.py /sgl-workspace/sglang/python/sglang/srt/entrypoints/openai/responses_compat.py
COPY runtime/python/sglang/srt/function_call/qwen3_coder_detector.py /sgl-workspace/sglang/python/sglang/srt/function_call/qwen3_coder_detector.py
COPY runtime/python/sglang/srt/multimodal/processors/qwen_vl.py /sgl-workspace/sglang/python/sglang/srt/multimodal/processors/qwen_vl.py
COPY provenance/qwen-multimodal-alias-runtime-files.json /tmp/qwen-multimodal-alias-runtime-files.json
RUN python3 -B -c 'import hashlib,json,pathlib; root=pathlib.Path("/sgl-workspace/sglang"); expected=json.loads(pathlib.Path("/tmp/qwen-multimodal-alias-runtime-files.json").read_text()); actual={str(p.relative_to(root)) for p in (root/"python/sglang").rglob("*") if p.is_file() and "__pycache__" not in p.parts and p.suffix != ".pyc"}; assert actual == set(expected), (len(actual), len(expected)); bad=[n for n,h in expected.items() if hashlib.sha256((root/n).read_bytes()).hexdigest()!=h]; assert not bad, bad; files=[root/"python/sglang/srt/entrypoints/openai"/n for n in ("serving_chat.py", "protocol.py", "serving_responses.py", "responses_compat.py")]+[root/"python/sglang/srt/function_call/qwen3_coder_detector.py", root/"python/sglang/srt/multimodal/processors/qwen_vl.py"]; [compile(f.read_bytes(), str(f), "exec") for f in files]' \
&& rm /tmp/qwen-multimodal-alias-runtime-files.json
ENTRYPOINT ["python3", "-m", "sglang.launch_server"]
CMD ["--help"]
2 changes: 1 addition & 1 deletion Dockerfile.responses-compat
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# API-only compatibility overlay. All serving arguments remain external.
# API-only compatibility and phase/order overlay. All serving arguments remain external.
# Immutable rollback/base; no private snapshot, checkpoint or docker commit.
FROM docker.io/kanadaj/sglang-qwen38fn-sm120-turbo@sha256:872a2bda228e39aa9c1af729b47cc28f7862e7859e448f1a8868b85a4051f404
ARG SOURCE_REVISION
Expand Down
21 changes: 18 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,8 +1,23 @@
# Qwen TP2 packed-PLE vision on SM120

**Unpublished CPU candidate:** [Responses namespace/custom compatibility](docs/responses-compat.md)
adds a separately attested boundary backport after the effort-alias profile.
Historical production profiles below are unchanged; no deployment is implied.
**Published cumulative compatibility runtime:**
[`production-cumulative-compat-20260914-v3`](docs/production-cumulative-compat-20260914.md)
contains the ordered `0015` → `0016` → `0017` → `0018` → `0019` stack at
`docker.io/kanadaj/sglang-qwen38fn-sm120-turbo@sha256:f2859d1ccf824a5295088cf578eba89b0f3eeefff6ae7679c3f5d64af0689458`.
It includes the Qwen effort aliases, [Responses namespace/custom and phase/order
compatibility](docs/responses-compat.md), [Flash-Next multimodal processor
aliases](docs/qwen-multimodal-alias.md), and [invalid generated-token failure
propagation](docs/invalid-token-failure.md). The image was rebuilt from clean
`main`, verified against all 4,392 source hashes, and anonymously pulled by tag
and digest. Publication does not imply production deployment.

**Opt-in HiCache profile:** [Qwen RAM/file state transfer](docs/hicache-wip.md)
adds companion-state handling and upstream QSA, router and restore-order fixes
after the cumulative runtime. See the configuration-specific live results and
remaining limits before enabling it; default images and patch series are unchanged.

Historical production profiles below remain available as rollback and audit
records.

Publishable source and deployment package for the locally accepted Qwen3.8
Flash-Next LIL NVFP4 stack. **No model weights, container archives, credentials,
Expand Down
Loading