Skip to content

fix(rdna): follow-ups from the 2026-10-16 cleanup PRs (#101-#106) #118

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
BlivionIaG wants to merge 1 commit into
base: rdna_extras
Choose a base branch
from
Open

fix(rdna): follow-ups from the 2026-10-16 cleanup PRs (#101-#106) #118

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 4 additions & 3 deletions docs/rdna2/v0.31.0-port-notes.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -74,8 +74,9 @@ Static checks stay under "What was verified (no GPU)". Current knobs:
does not reach that assert (T-MOE).
8. Debug knobs still read by this tree (env-gated, off by default):
`VLLM_MOE_NAN_DEBUG`, `VLLM_LAYER_NAN_DEBUG`,
`VLLM_TRACE_PREFIX_CACHE`, `VLLM_DEBUG_PREFIX_HIT`, `VLLM_NO_PARTIAL_HASH`,
`VLLM_BT_DEBUG`, `VLLM_MAMBA_COPY_DEBUG`, `DBG_VLLM_STEP_TIMING`.
`VLLM_TRACE_PREFIX_CACHE`, `VLLM_DEBUG_PREFIX_HIT`, `VLLM_NO_PARTIAL_HASH`.
(`VLLM_BT_DEBUG`, `VLLM_MAMBA_COPY_DEBUG` and `DBG_VLLM_STEP_TIMING` were
dropped in #103; nothing reads them any more.)

## RDNA gating

Expand Down Expand Up @@ -250,7 +251,7 @@ model-gated features; candidates to upstream):
captured graph" rule extended to upstream's new UVA `write_contents` buffer.
- `v1/worker/gpu/block_table.py`: upstream slot-mapping kernel (has
`slot_mapping_enabled` + dummy-request PAD); fork row-zero fix on overwrite
and `VLLM_BT_DEBUG` kept.
kept (its `VLLM_BT_DEBUG` debug knob was later dropped in #103).
- `v1/worker/gpu_model_runner.py` (V1): mrope contiguous workspace now sized by
upstream's `mrope_num_dims` (upstream removed XD-RoPE buffers); upstream's
cached `_get_mamba_state_copy_funcs()` taken.
Expand Down
18 changes: 12 additions & 6 deletions tests/kernels/quantization/test_gdn_prefill_kkt_rdna2.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24,19 +24,25 @@
Run `pytest tests/kernels/quantization/test_gdn_prefill_kkt_rdna2.py`.
"""

import os

import pytest
import torch

from vllm.platforms import current_platform

# Pre-existing on 0.28 and v0.31: this module fails, then the process aborts
# in hsa_signal_wait. See docs/rdna2/v0.31.0-port-notes.md.
pytest.skip(
"Recorded GPU hang: test_gdn_prefill_kkt_rdna2 fails, then the process "
"aborts in hsa_signal_wait (docs/rdna2/v0.31.0-port-notes.md). "
"Run in isolation only (ONLY=gdn_prefill_kkt).",
allow_module_level=True,
)
if os.environ.get("VLLM_RUN_GDN_KKT_HANG_TEST") != "1":
pytest.skip(
"Skipped by default: recorded GPU hang (test_gdn_prefill_kkt_rdna2 "
"fails, then the process aborts in hsa_signal_wait, see "
"docs/rdna2/v0.31.0-port-notes.md). To run it, set "
"VLLM_RUN_GDN_KKT_HANG_TEST=1 and run this file in its own process "
"(e.g. VLLM_RUN_GDN_KKT_HANG_TEST=1 ONLY=gdn_prefill_kkt "
"bash tools/rdna/port_v031/unit.sh).",
allow_module_level=True,
)

if not current_platform.is_rocm():
pytest.skip("RDNA2 GDN prefill KKT kernel is ROCm-only", allow_module_level=True)
Expand Down
14 changes: 9 additions & 5 deletions tests/kernels/test_gdn_prefill_rdna2.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,8 @@
64-token FLA chunk).
"""

import os

import pytest
import torch

Expand Down Expand Up @@ -36,12 +38,14 @@ def _inputs(T, FLA_CHUNK_SIZE, seed=0):
return q, k, v, g_raw, beta, initial_state, cu_seqlens


@pytest.mark.skip(
@pytest.mark.skipif(
os.environ.get("VLLM_RUN_GDN_KKT_HANG_TEST") != "1",
reason=(
"Recorded GPU hang: the HIP GDN prefill chain calls "
"gdn_prefill_kkt_rdna2, which fails then aborts in hsa_signal_wait "
"(docs/rdna2/v0.31.0-port-notes.md)"
)
"Skipped by default: recorded GPU hang, the HIP GDN prefill chain "
"calls gdn_prefill_kkt_rdna2, which fails then aborts in "
"hsa_signal_wait (docs/rdna2/v0.31.0-port-notes.md). To run it, set "
"VLLM_RUN_GDN_KKT_HANG_TEST=1 and run this file in its own process."
),
)
@pytest.mark.parametrize(
"T",
Expand Down
2 changes: 2 additions & 0 deletions tools/rdna/port_v031/unit.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,8 @@
# TAG=baseline-028 GPU=8 bash tools/rdna/port_v031/unit.sh
# SKIP='gdn_prefill_kkt' bash tools/rdna/port_v031/unit.sh # skip a file
# ONLY='gdn_prefill_kkt' TAG=kkt bash tools/rdna/port_v031/unit.sh
# (gdn_prefill_kkt and test_gdn_prefill_rdna2 skip themselves by default
# because of a recorded GPU hang; opt in with VLLM_RUN_GDN_KKT_HANG_TEST=1)
#
# Summary: ~/w4a8_runs/port-v031/unit-$TAG/summary.txt
set -uo pipefail
Expand Down
67 changes: 0 additions & 67 deletions tools/rdna2_028/probe_force_split.py
View file
Open in desktop

This file was deleted.

4 changes: 2 additions & 2 deletions tunableop/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@ deltas and regeneration recipe).

`profiles.json` is the registry: `name -> {rocblas, rocm, torch, lib_sha256,
dir, rows_per_rank, status}`. The helper
(`tools/rdna2_028/tunableop_env.sh`, mirror `tools/rdna2/tunableop_env.sh`)
(`tools/rdna2_028/tunableop_env.sh`)
resolves a profile from it:

```bash
Expand Down Expand Up @@ -74,7 +74,7 @@ the selected rows path; harvest into the profile directory when validated).

All serve launchers under `tools/rdna/` (through
`tools/rdna/rdna_launcher_common.sh`) and the in-tree bench/capture drivers
go through this helper (or the `tools/rdna2/` mirror); a launcher with a
go through this helper; a launcher with a
`TUNABLEOP=0` opt-out sets `PYTORCH_TUNABLEOP_ENABLED=0` and writes nothing.

## Cache-aligned prefill
Expand Down