feat(flydsl): sglang-kimi-k26-flydsl-mi355x eval instance - #16
Conversation
8x MI355X Qwen3-30B-A3B MoE pretraining MFU optimization. Built on ghcr.io/amdpilot-org/primus-mi355x-ready:v1. Two task.yaml variants for A/B testing the Phase 1 baseline agent: - task.yaml: phase1_baseline: true - task_nophase1.yaml: phase1_baseline: false Both pin the executor to Kimi-K2.6 at 10.235.24.154:30000.
The previous base (ghcr.io/amdpilot-org/primus-mi355x-ready:v1) was a slimmed copy that did NOT include /workspace/primus_train/Primus. Phase 1 spent its full max_turns budget trying to bootstrap from scratch. Switch to primus-mi355x-flat:v1 (locally available, has Primus + Primus-Turbo pre-installed and patched for triton 3.4.0). Also install uv at /root/.local/bin in our Dockerfile so the kimi-cli runtime's source $HOME/.local/bin/env succeeds — without this, executor trials exit 137 immediately after image switch. Tag the new image primus-qwen3-30b-mfu-base:v1 so amdpilot triggers a build the first time it runs.
Two cumulative fixes for n08-09 (8x MI355X): 1. Without PYTORCH_ROCM_ARCH=gfx950 (and AITER_ROCM_ARCH, HSA_NO_SCRATCH_RECLAIM, HIP_FORCE_DEV_KERNARG, etc) the torch HIP runtime can't dispatch kernels and benchmarks die immediately with hipErrorInvalidDeviceFunction. These were set in xiao/baizhou's working containers but missing from amdpilot's docker run line. Add to both Dockerfile ENV and task.yaml container.env so they apply via either path. 2. Replace /workspace/detect_interface.sh with a /proc-based detector (the original needed `ip` from iproute2, unavailable in the slim base). bench_mfu.sh now auto-detects GLOO/NCCL socket IFNAME at runtime if bench_config.env doesn't pin one — without this, Megatron's distributed init fails fast (3-5s) before training starts.
Enables tag + push of the Phase 1 baseline image to
docker.io/jhinpan/primus-qwen3-30b-mfu-phase1 after a successful
phase1 commit. Template: {date}-{metric} (e.g. 20260422-278p80) +
:latest. Other nodes can then `docker pull` that tag and skip Phase 1
entirely.
Switch repository from docker.io/jhinpan to ghcr.io/amdpilot-org so all nodes in the org can pull the verified phase1-baseline image directly. Bump push timeout_s to 9000 (2.5h) to absorb the one-time 42 GB base-layer seed; subsequent pushes only upload the phase1-commit delta (~500 MB - 1 GB) via GHCR cross-repo mount.
![devin-ai-integration[bot]](https://avatars.githubusercontent.com/in/811515?s=60&v=4){kind=small}
| # even while the gating env vars are off. Safe no-op until | ||
| # AITER_USE_FLYDSL_MOE is flipped to 1 during a trial. | ||
| DSL2_ROOT: "/opt/FlyDSL" | ||
| MLIR_PATH: "/opt/mlir_install" |
There was a problem hiding this comment.
🔴 MLIR_PATH in task.yaml overrides Dockerfile ENV with wrong path, breaking FlyDSL at runtime
The Dockerfile builds and installs MLIR at /opt/llvm-project/mlir_install (instances/sglang-kimi-k26-flydsl-mi355x/Dockerfile:36 and Dockerfile:77-79), and correctly sets ENV MLIR_PATH=/opt/llvm-project/mlir_install. However, task.yaml:62 sets MLIR_PATH: "/opt/mlir_install" in container.env, which gets passed as a -e flag to docker run and overrides the Dockerfile's ENV at runtime. Since /opt/mlir_install does not exist in the image, FlyDSL will fail to find the MLIR compiler when trials enable it (Phases B–G). The same wrong path is propagated in task_description.md:28,201,254 and ISSUE.md:56, so the agent would not self-correct.
| MLIR_PATH: "/opt/mlir_install" | |
| MLIR_PATH: "/opt/llvm-project/mlir_install" |
Was this helpful? React with 👍 or 👎 to provide feedback.
Adds a hand-authored eval instance that targets the FlyDSL fused-MoE port from AMD's Kimi-K2.5 MI300X optimization blog to Kimi-K2.6 on 4x MI355X, on top of the baseline established by sgl-project/sglang#23381. Contents: - task.yaml : optimize-type task, phase1_baseline=true, frontier_model=true, TP=4, 4xMI355X, FlyDSL env OFF at start so Phase 1 reproduces PR #23381 numbers (DSL2_ROOT + MLIR_PATH + CK_TILE_FLOAT_TO_BFLOAT16 pre-set so import works as soon as agent flips gates during a trial) - Dockerfile : layers FlyDSL + AITER dev/kimi-K2.5 on top of jhinpan/sglang-k26-mi355x:v0.5.10rc0-rocm720-20260420 - bench_flydsl_k26.sh : single bench script emitting one canonical line: output_throughput_tok_s: <v> | concurrency=40 in=10240 out=512 decode_bs1_in8k=<guard> - task_description.md : full phased plan A-G, env-var reference table, BS=1 decode guard (>= 0.98x PR #23381 ~38.05 tok/s), front-loaded supervisor-visible gating block - test_harness.py : GSM8K accuracy gate (lm_eval limit=50 >= 0.90) - metadata.json : links to GH issue amdpilot-org/sglang#2 + PR #23381 - ISSUE.md : condensed body as filed on amdpilot-org/sglang#2 Tracks GitHub issue amdpilot-org/sglang#2.
56a0dda to
ba57612
Compare
| env: | ||
| PYTORCH_ROCM_ARCH: "gfx950" | ||
| AITER_ROCM_ARCH: "gfx942;gfx950" | ||
| HSA_NO_SCRATCH_RECLAIM: "1" | ||
| HIP_FORCE_DEV_KERNARG: "1" | ||
| HSA_FORCE_FINE_GRAIN_PCIE: "1" | ||
| TOKENIZERS_PARALLELISM: "false" | ||
| LD_LIBRARY_PATH: "/opt/rocm/lib:/usr/local/lib:" |
There was a problem hiding this comment.
🔴 task_nophase1.yaml missing TMPDIR override, will trigger HIP kernel-loading failures
The main task.yaml explicitly sets TMPDIR: "/tmp" (line 61) with a comment explaining it's needed to override amdpilot's default TMPDIR=/scratch/tmp which "triggers torchrun / primus-cli HIP kernel-loading failures on this box" (instances/primus-qwen3-30b-mfu/task.yaml:58-61). task_nophase1.yaml claims to be "same as task.yaml but Phase 1 is disabled" (line 7) yet its container.env block (lines 31-38) is entirely different — it omits TMPDIR, GLOO_SOCKET_IFNAME, NCCL_SOCKET_IFNAME, GPU_COREDUMP_ENABLE, and NVTE_CK_USES_BWD_V3, while adding GPU arch vars (PYTORCH_ROCM_ARCH, HSA_FORCE_FINE_GRAIN_PCIE) that the main task.yaml says are "baked into the base image and MUST NOT be overridden". The missing TMPDIR will cause training to fail on the target node.
Prompt for agents
The task_nophase1.yaml container.env block should mirror the main task.yaml's env block (with only phase1-related fields differing), but it currently has a completely different set of environment variables. The main task.yaml (lines 47-61) sets GLOO_SOCKET_IFNAME, NCCL_SOCKET_IFNAME, IP_INTERFACE, TOKENIZERS_PARALLELISM, GPU_COREDUMP_ENABLE, NVTE_CK_USES_BWD_V3, and critically TMPDIR=/tmp. The nophase1 variant instead sets PYTORCH_ROCM_ARCH, AITER_ROCM_ARCH, HSA_NO_SCRATCH_RECLAIM, HIP_FORCE_DEV_KERNARG, HSA_FORCE_FINE_GRAIN_PCIE, TOKENIZERS_PARALLELISM, and LD_LIBRARY_PATH -- which the main task.yaml explicitly says should NOT be overridden (they are baked into the base image). Copy the container.env block from the main task.yaml to task_nophase1.yaml so both variants run with the same runtime environment.
Was this helpful? React with 👍 or 👎 to provide feedback.
Keep FlyDSL's runtime MLIR path consistent with the Dockerfile and make new eval metadata consumable by the registry tooling. Mirror the Primus no-phase1 runtime env so the control variant does not hit known HIP temp path failures.
1 participant
Summary
Hand-authored amdpilot eval instance that targets the FlyDSL fused-MoE port from AMD's Kimi-K2.5 MI300X optimization blog to Kimi-K2.6 on 4x MI355X, on top of the baseline established by source project sgl-project/sglang; issue or pull request number 23381 (source project sgl-project/sglang; pull request number 23381). Tracks GitHub issue amdpilot-org/sglang#2.
What lands
evals/instances/sglang-kimi-k26-flydsl-mi355x/:task.yaml— optimize-type task withphase1_baseline: true,frontier_model: true, TP=4, 4xMI355X, FlyDSL gates OFF at container start (AITER_USE_FLYDSL_MOE=0, STAGE1=0, STAGE2=0, AITER_ENFORCE_DSL=0) so Phase 1 reproduces PR #23381 numbers. DSL2_ROOT / MLIR_PATH / CK_TILE_FLOAT_TO_BFLOAT16_DEFAULT pre-set so FlyDSL import succeeds as soon as the agent flips gates during a trial.Dockerfile— layers FlyDSL main + AITERdev/kimi-K2.5branch on top ofjhinpan/sglang-k26-mi355x:v0.5.10rc0-rocm720-20260420. Builds LLVM+MLIR from upstream source via FlyDSL's ownscripts/build_llvm.sh(first-time ~30-45 min; cached for all later trials). Patches-DMLIR_ENABLE_ROCM_RUNNER=OFFsince the MLIR ROCm runner's amdgpu-arch probe requires GPU access that Dockerbuildcan't provide — FlyDSL still gets AMDGPU codegen viaLLVM_TARGETS_TO_BUILD=AMDGPU.bench_flydsl_k26.sh— single bench script emitting one canonical metric line:output_throughput_tok_s: <v> | concurrency=40 in=10240 out=512 decode_bs1_in8k=<guard>. Primary metric (concurrency=40) + BS=1 decode guard in one pass.task_description.md— phased plan A-G (load / stage1 / stage2 / hybrid sweep / torch.compile / disable-radix-cache / GSM8K), env-var reference table, BS=1 decode guard (>= 0.98x PR #23381 ~38.05 tok/s). Front-loaded "Key gating env vars and guards" block within first 1600 chars so supervisor + nudge see the critical constraints in their prompt window.test_harness.py— GSM8K accuracy gate (lm_eval limit=50, exact_match_flexible >= 0.90).metadata.json— issue_url link, PR #23381 baseline, delivery branchamdpilot/sglang-kimi-k26-flydsl-mi355x.ISSUE.md— body as filed on amdpilot-org/sglang#2.Integration with amdpilot
:phase1-baselinebefore trials run.amdpilot/src/amdpilot/orchestrator/optimize_guidance.pygainshas_flydsl_context()+_render_flydsl_optimize_section_body()(same PR as the submodule bump) so supervisor + planner + nudge all receive the FlyDSL playbook when this eval is loaded.Success criteria (deferred to full amdpilot run)
decode_bs1_in8kwithin 2% of 38.05 tok/s at input=8192.output_throughput_tok_sat concurrency=40 captured intobaseline_contract.expected_metric.exact_match_flexible@ limit=50 >= 0.90.Test plan
docker build -t amdpilot-eval-sglang-kimi-k26-flydsl-mi355x evals/instances/sglang-kimi-k26-flydsl-mi355x/completes (one-time ~40 min on any MI355X node).uv run amdpilot run evals/instances/sglang-kimi-k26-flydsl-mi355x/task.yaml --hours 6on a 4xMI355X node.results/<job>/phase1/baseline_contract.jsonis populated anddocker images | grep phase1-baselineshows the committed snapshot.docker exec <container> ls /workspace/skills | grep flydslreturns 7 folders (from skills submodule).Tracking: amdpilot-org/sglang#2, amdpilot-org/amdpilot#119, amdpilot-org/AMD-Skills#12.