Repository navigation
Conversation
…put memset gemm_a8w8_blockscale_bpreshuffle_asm zero-fills `out` with hipMemsetAsync before every split-K launch, because the splits accumulate into it atomically. A caller whose producer already writes those zeros (a fused norm + quant kernel, say) pays for a second fill and one more stream / graph node per GEMM. out_zeroed=True tells the launcher the output already holds zeros, so it launches the kernel alone. - C launcher: an `int out_zeroed` argument; the memset is skipped when set. - gemm_a8w8_blockscale_bpreshuffle_asm and the tuned dispatcher gemm_a8w8_blockscale_bpreshuffle take out_zeroed=False; the dispatcher forwards it to asm rows only (every other row overwrites `out`) and requires the caller's `out` with it. - op test: the zeroed launch matches the default one at split-K 2 and 3, and a non-zero `out` shows through, so the memset really is skipped. Defaults keep today's behaviour for every existing caller. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Kevin Mi <mikevin920@yahoo.com>
🏷️ CI GuideRuns automatically on every PR:
Extended tests (opt-in via labels):
One backend per PR: PR title tags & labels: |
|
Updated against main while preserving the new fp16 fallback, output-row stride handling and split-K alignment checks. On gfx950 / ROCm 10, the output-layout suite passed all 22 cases. The added cases cover the direct asm entry and tuned dispatcher, contiguous and row-padded pre-zeroed outputs, sentinel preservation, deliberate nonzero input proving the clear was skipped, and rejection of misaligned outputs even when the flag is set. Black, Ruff and whitespace checks pass. The flag is still absent upstream and remains useful for a producer that clears the output. Keeping the existing draft status; the original end-to-end performance claim was not re-benchmarked in this pass. |
…tput The block-fp8 GEMM after a gfx950 Gemma norm launched aiter's asm split-K kernel into an output the norm zero-filled, saving aiter's memset launch. End to end it measured within run-to-run noise, and ROCm/aiter#6786 adds out_zeroed to aiter's own launcher. This removes the local launcher (aiter_asm_blockscale, hipcc's 2-D grid), the norm's zero fill, the GEMM shape each norm was bound to, and their tests. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…tput The block-fp8 GEMM after a gfx950 Gemma norm launched aiter's asm split-K kernel into an output the norm zero-filled, saving aiter's memset launch. End to end it measured within run-to-run noise, and ROCm/aiter#6786 adds out_zeroed to aiter's own launcher. This removes the local launcher (aiter_asm_blockscale, hipcc's 2-D grid), the norm's zero fill, the GEMM shape each norm was bound to, and their tests. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Summary
Add
out_zeroed=Falseto the blockscale bpreshuffle asm entrypoint and tuned dispatcher. Without_zeroed=True, the caller promises the output already contains zeros, allowing the split-K launcher to skip its redundant clear. This lets a fused producer clear the output without requiring a copied custom GEMM launcher.The default preserves the existing clear. The opt-in flag preserves upstream’s output alignment checks, row-stride support, and fp16 fallback. A row-padded output keeps its padding untouched. The dispatcher requires an explicit output when the flag is enabled.
Validation
Against main
1dd826c42, on gfx950 / ROCm 10 / PyTorch 2.11:op_tests/test_gemm_a8w8_blockscale_bpreshuffle_out.pypassed.The flag is still absent upstream. This remains a draft; the original producer-integrated performance result has not been re-benchmarked on this updated branch.