Repository navigation
Add mimo-v2-6-flash-rl-sglang-mia-dual (MiaAI-Lab SGLang, two Sparks) - #195
Merged
Merged
Conversation
CarstVaartjes
enabled auto-merge (squash)
September 29, 2026 20:31
…rks via MiaAI-Lab's SGLang kit Pins MiaAI-Lab/MiMo-V2.6-Flash-2x-DGX-Sparks 201be3e7 (AGPL-3.0; boot.py and patches/sitecustomize.py vendored byte-identical) on lmsysorg/sglang nightly-dev-cu13-20260921-0f6761b5 (arm64 manifest 9e1fb4c3). Reuses the existing MiMo-V2.6-Flash-RL model at 5711b268. Option drafter: EAGLE MTP (upstream default), DFlash, off. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NxX8uY9ruX1ZqmuABxwaSa
CarstVaartjes
force-pushed
the
recipe/mimo-v2-6-flash-rl-sglang-mia-dual
branch
from
September 29, 2026 20:37
606e9ae to
3d7e10a
Compare
56 tasks done
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds
mimo-v2-6-flash-rl-sglang-mia-dual: XiaomiMiMo/MiMo-V2.6-Flash-RL on two Sparks with SGLang (TP2/EP2), from MiaAI-Lab's kit.Upstream: https://github.com/MiaAI-Lab/MiMo-V2.6-Flash-2x-DGX-Sparks @
201be3e743d215291feee09fe470822ec453c63d(2026-09-22, no releases/tags, so release 1.0.0 / 2026-09-22). Licence: AGPL-3.0 (LICENSE file, kept asLICENSE.upstream); model MIT.Image:
docker.io/lmsysorg/sglang@sha256:9e1fb4c395b9c406136e10aa445b8784d06bca3839623b52cbe4a3b231a157a8(linux/arm64 manifest ofnightly-dev-cu13-20260921-0f6761b5, the first nightly with sgl-project/sglang#40448 that keeps the MXFP4 experts packed).Model: reuses
mimo-v2-6-flash-rl-5711b268(HF5711b268, MIT). Upstream pins HF3b38d063. The two revisions differ only in README.md, the technical-report PDF anddflash/config.json(the newer one fixes the trailing comma that upstream's boot.py repairs at run time under/state/dflash); all weights are identical. The DFlash drafter isdflash/inside the same repo, so no extra Model.Adapter
adapters/mimo/mia-sglang-dual:boot.py,patches/sitecustomize.pyand the licence are byte-identical to upstream (check-vendored-upstream passes for this adapter). Ours:Dockerfile,sglang-serve(maps Vonk's--nnodes/--node-rank/--dist-init-addr,--context-length,--max-running-requestsand the rendezvous env to the env boot.py reads),verify-runtime.py,requirements-audio.txt,NOTICE.upstream.Divergence from upstream's Dockerfile: upstream apt-installs ffmpeg and pip-installs an unpinned torchcodec. The base already ships ffmpeg; torchcodec is pinned to 0.16.0 by wheel hash (newest release on the upstream commit date),
--no-deps. Build hosts: pypi.org, files.pythonhosted.org. Upstream's import/ffprobe assertion is kept.Option
drafter:eagle(default, upstream default: EAGLE MTP, 4 draft tokens, decode CUDA graphs off, pin 2743808),dflash(SPEC_ALGO=DFLASH, 8 tokens, graphs on for 8, pin 2299968),off. Runtime env holds upstream's.env.exampledefaults (mem fraction 0.93, page 64, chunk 2048, fp8_e4m3 KV, swa ratio 0.03, flashinfer_mxfp4 MoE, triton attention, NCCL settings). Settings: context 1048576, concurrency 8. Endpoint port 8888, served nameMiMo-v2.6-Flash.Not expressible / not done: upstream's start.sh orchestration, NFS head export and worker mount, host NCCL overlay,
OFFLOAD_MODE=ssd(needs a mount), MEM_FRACTION_STATIC/MoE-marlin knobs (upstream says the pin must be re-derived; unmeasured). Untested: arm64 image build and the torchcodec import on torch 2.13 (no build here), any physical Spark run. Upstream figures (EAGLE prose 35.0 / code 46.0 tok/s, DFlash prose 25.8 / code 69.8, single stream) are upstream claims. Qualification definition added; nothing marked qualified. The first start loads about 82 GiB per rank (upstream allows 40 minutes).Checks: full
pytest -m "not lane"passes, check-recipe-model-references, check-recipe-topology, check-build-context-sources,git diff --check.🤖 Generated with Claude Code
https://claude.ai/code/session_01NxX8uY9ruX1ZqmuABxwaSa