Skip to content

[opus] fix(gfx942): retune stale GLM5 BF16 GEMM config - #4301

Merged
junhaha666 merged 1 commit into
mainfrom
bf16_csv_debug
Jul 20, 2026
Merged

[opus] fix(gfx942): retune stale GLM5 BF16 GEMM config#4301
junhaha666 merged 1 commit into
mainfrom
bf16_csv_debug

Conversation

@yifehuan

Copy link
Copy Markdown
Contributor

Motivation

The GLM5 BF16 tuned GEMM configuration referenced gfx942 Opus kernel 10211, which was removed in #4204. The stale row was later introduced in #4236.

Because 10211 is no longer present in the current kernel registry, codegen silently filters this tuning entry and it is not included in the runtime lookup. This PR replaces it with the best valid kernel from a fresh tuning run.

Technical Details

Updated the gfx942 BF16 configuration for:

  • M=768
  • N=256
  • K=6144
  • BF16 input/output
  • No bias

The configuration changes from:

  • Kernel: 10211
  • splitK: 3
  • Latency: 36.6162 us
  • Error ratio: 0.0083

To:

  • Kernel: 10201
  • splitK: 3
  • Latency: 34.5862 us
  • Error ratio: 0
  • Throughput: 69.85 TFLOPS
  • Bandwidth: 375.18 GB/s

No kernel implementation or dispatch code is changed.

Test Plan

  • Restore 10211 temporarily and tune all valid gfx942 Opus candidates.

  • Remove 10211, rebuild in an isolated JIT directory, and repeat the same full candidate sweep.

  • Run a paired benchmark of 10211/splitK=3 and 10201/splitK=3 using:

    • The same input tensors and module
    • 30 warmup iterations
    • 9 timing samples
    • 200 launches per sample
  • Validate every gfx942 Opus entry in the BF16 model-config CSVs against the current kernel registry.

Test Result

Full candidate sweep:

Configuration Winner Latency Error ratio
━━━━━━━━━━━━━━━━━━━━━ ━━━━━━━━━━━━━━━━ ━━━━━━━━━━━━ ━━━━━━━━━━━━━
With 10211 restored 10211/splitK=3 34.4598 us 0.0084
───────────────────── ──────────────── ──────────── ─────────────
Current registry 10201/splitK=3 34.5862 us 0

Paired benchmark median:

Kernel Workspace Latency Mean absolute error Bad ratio
━━━━━━━━━━━━━━━━ ━━━━━━━━━━━ ━━━━━━━━━━━━ ━━━━━━━━━━━━━━━━━━━━━ ━━━━━━━━━━━
10211/splitK=3 BF16 34.3804 us 0.198291 0.009013
──────────────── ─────────── ──────────── ───────────────────── ───────────
10201/splitK=3 FP32 34.5818 us 0.0000139 0

10211 was only 0.58% faster in the paired benchmark while producing less accurate results. Restoring an additional kernel is therefore not justified.

After updating the row, all gfx942 Opus BF16 model-config entries reference valid kernel IDs.

Submission Checklist

@yifehuan
yifehuan requested a review from a team July 20, 2026 09:07
@github-actions

Copy link
Copy Markdown
Contributor

🏷️ CI Guide

Runs automatically on every PR:

  • ✅ Pre-checks (submodule verification, code formatting)
  • ✅ Aiter op tests (gfx942 + gfx950)
  • ✅ Triton tests on MI35X (only when aiter/ops/triton/** or related paths are changed)

Extended tests (opt-in via labels):

Label Tests
ci:triton-300x Run an additional Triton test job on MI300X in PRs; main branch always runs both MI35X and MI300X
ci:sglang SGLang integration tests: DeepSeek-R1-MXFP4 accuracy, Qwen 3.5 accuracy
ci:atom ATOM benchmark: DeepSeek-R1-0528, GPT-OSS-120B
ci:atom_full ATOM accuracy suite for PR and main models from ATOM models_accuracy.json
ci:vllm vLLM benchmark: GPT-OSS-120B, DeepSeek-R1-0528, Kimi-K2.5
ci:all All standard extended tests (excludes ci:atom_full)

Only add ci:atom_full for FlyDSL or Triton upgrades.
Add labels via the sidebar or gh pr edit 4301 --add-label <label>

@junhaha666
junhaha666 merged commit 52eabe3 into main Jul 20, 2026
40 checks passed
@junhaha666
junhaha666 deleted the bf16_csv_debug branch July 20, 2026 11:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants