feat(hipdnn): complete TensorAttributes metadata Python bindings by NikharAsthana · Pull Request #14332 · ROCm/rocm-libraries

NikharAsthana · 2026-10-23T21:23:06Z

Summary

Completes the hipDNN Python bindings for TensorAttributes metadata, as requested in #10586: alignment, ragged-offset and ragged-offset-multiplier accessors, scalar-state accessors, plain-Python getters for stored scalar values, and Graph.tensor_scalar with the ScalarType enum. Behavior follows the maintainer's guidance in the issue discussion.

ISSUE ID : #10586

Risk Assessment

Level 2. Additive Python bindings over existing C++ methods. No change to existing behavior, C++ headers, schemas or build files, and every new method has unit tests. It adds new public Python API surface (new methods, a new enum and the Graph.tensor_scalar factory), which puts it at the boundary with level 3.

ASIC Coverage

Architecture-independent: host-side tensor metadata only, no device code. Passing standard PR CI is sufficient; no specific-ASIC or multi-arch run is needed.

Testing Summary

Python binding unit tests for every new method: getter/setter round trips, method chaining, None handling for the ragged offset, validate() rules for alignment and the ragged-offset multiplier, value conversion for every supported scalar type and scalar state, and the data type and scalar type chosen by Graph.tensor_scalar.

Testing Checklist

Python binding tests - python -m pytest -q projects/hipdnn/python/frontend_wheel_package/tests -m "not gpu" - Status: Passed (238 passed, 13 skipped; GPU tests deselected; run in the hipDNN Docker dev image without a GPU)
Pre-commit hooks - pre-commit run --files <changed files> - Status: Passed
PR CI - GitHub PR checks - Status: Pending

Technical Changes

tensor_bindings.cpp: bind get_alignment/set_alignment; get_ragged_offset/set_ragged_offset (accepts None)/has_ragged_offset; the ragged-offset multiplier getter, setter and has_ check; clear_value, set_as_runtime_parameter, get_has_compile_time_constant.
Add get_value_variant, get_pass_by_value and get_compile_time_constant, sharing one helper that returns a plain Python value (half/bfloat16/float/double as float, integer types as int, bool as bool, no value as None); get_data_type() gives the exact type.
Add set_compile_time_constant as an alias of set_value.
Bind the ScalarType enum and add Graph.tensor_scalar(value, scalar_type) in graph_bindings.cpp as four overloads (float, double, int32_t, int64_t, tried in that order), each calling Graph::tensor, as specified in the issue discussion. A Python 0.5 becomes FLOAT, 0.1 DOUBLE, 3 INT32 and 2**40 INT64.
test_tensor.py: tests for all of the above.

therock-pr-bot · 2026-10-23T21:28:14Z

✅ All Checks Passed — Ready for Review

Check	Status	Details
📝 PR Description	✅ Pass	—
⛔ Forbidden Files	✅ Pass	—
🧪 Unit Test	✅ Pass	—
🔎 pre-commit	✅ Pass	—
🚫 Draft PR	🔜 To Be Enabled	—
🚩 Feature Flag	🔜 To Be Enabled	—
📊 Code Coverage	🔜 To Be Enabled	—
🤖 therock-pr-bot	✅ Pass	—

🎉 All checks passed! This PR is ready for review.

📖 Need help? See the Policy FAQ for details on every check and how to fix failures.

🙋 Wish to Override Policy?

therock-pr-bot · 2026-10-23T21:43:29Z

🎉 All checks passed! This PR is ready for review.

NikharAsthana · 2026-10-27T10:34:53Z

Hi @SamuelReeder , this draft PR is ready for a first look. It covers the methods in #10586 except for Graph.tensor_scalar and the ScalarType enum, which I'm working on now and will push to this branch.

As an outside contributor I can't start the CI workflows: 7 are waiting for maintainer approval, and the pre-commit check and the PR bot are waiting on them. Could you or another hipDNN team member approve the workflows and run CI on this branch when you have a moment? The Python tests and the pre-commit hooks pass locally.

Thanks!

Bind the following TensorAttributes methods: - get_alignment / set_alignment - get_ragged_offset / set_ragged_offset (accepts None) / has_ragged_offset - get_ragged_offset_multiplier / set_ragged_offset_multiplier / has_ragged_offset_multiplier - clear_value / set_as_runtime_parameter / get_has_compile_time_constant Add tests for getter/setter round trips, method chaining, None handling and the validate() rules for alignment and the ragged offset multiplier. Part of ROCm#10586.

Add get_value_variant, get_pass_by_value and get_compile_time_constant to the Tensor bindings. All three share one helper that converts the stored scalar to a plain Python value: half, bfloat16, float and double become float, integer types become int, bool stays bool, and no stored value becomes None. Use get_data_type() for the exact type. Add tests for every supported scalar type and for the value, runtime pass-by-value and cleared states. Part of ROCm#10586.

Bind set_compile_time_constant as an alias of set_value, with the same arguments (value, optional data_type). In C++ it is a thin wrapper around set_value, so both names share one implementation. Add tests for baking a constant, inferring the data type and rejecting a multi-element tensor. Part of ROCm#10586.

Bind the ScalarType enum and add Graph.tensor_scalar(value, scalar_type) as four overloads (float, double, int32_t, int64_t, tried in that order) that call Graph::tensor. A Python 0.5 becomes FLOAT, 0.1 becomes DOUBLE, 3 becomes INT32 and 2**40 becomes INT64. scalar_type selects a compile-time constant or a runtime parameter with a default value. Add tests for the data type chosen for each Python value and for both scalar types. Part of ROCm#10586.

NikharAsthana · 2026-10-29T11:02:44Z

Hi @SamuelReeder ,
Update: I've pushed the last item from #10586 (the ScalarType enum and Graph.tensor_scalar with the four overloads in the order you described) and rebased on the latest develop. The PR now covers everything in the issue. The Python tests and the pre-commit hooks pass locally on the rebased branch.

The CI workflows are still waiting for maintainer approval. Could you or another hipDNN team member approve and run them when you have a moment? I'll mark the PR ready for review once CI is green, and I'm happy to change anything you'd prefer done differently.

SamuelReeder · 2026-10-29T18:53:19Z

HI @NikharAsthana, thanks for the PR! Workflows are running and I'll review shortly.

…e-parameter test Address review feedback on ROCm#14332: - Add a test that places a Graph.tensor_scalar tensor in a graph and checks the serialized data type, value and minimum engine API version for both scalar types. - Assert that set_as_runtime_parameter() clears the stored value, which distinguishes it from set_is_pass_by_value(True). Part of ROCm#10586.

Bind get_ragged_offset and set_ragged_offset (None clears; returns self) with the signatures of open PR #14332, so packed varlen SDPA graphs can be described from Python. Adds wheel tests for the round trip and the serialized ragged_offset_tensor_uid.

NikharAsthana · 2026-11-02T19:30:45Z

Hi @SamuelReeder, a gentle follow-up when you have a moment: both review comments are addressed in 84d098b, and the CI workflows are waiting for approval again. Thanks!

SamuelReeder

LGTM assuming CI passes. Thanks for the updates and the contribution!

Please ping me if there's any issues merging.

Bind get_ragged_offset and set_ragged_offset (None clears; returns self) with the signatures of open PR #14332, so packed varlen SDPA graphs can be described from Python. Adds wheel tests for the round trip and the serialized ragged_offset_tensor_uid.

BrianHarrisonAMD · 2026-11-03T17:32:24Z

Swapping to open, and looking at breaks.
Ill approve the workflow shortly, but might need to update from develop if this branched at a window that had issues.

BrianHarrisonAMD · 2026-11-03T17:34:13Z

Okay heads up @NikharAsthana, I need to update from develop for CI to pass.
Ill update & kick-off CI.

…accessors

BrianHarrisonAMD · 2026-11-04T16:45:53Z

Alright CI passing.
Thanks for the contribution @NikharAsthana!

I am going to merge since these bindings are needed for other ongoing work.

NikharAsthana · 2026-11-04T20:50:54Z

Thank you @BrianHarrisonAMD for updating the branch and merging, and @SamuelReeder for the review and the guidance on the issue. This was my first contribution to ROCm, and I learned a lot from it. I'm glad the bindings are useful for the other work, and I'm happy to help with follow-ups around the hipDNN Python bindings.

…plus vLLM forms Extends the FlyDSL SDPA forward engines of #14111 to match all of AOTriton's forward features, verified on AOTriton's forward test suite (forward only; backward is separate work), and serves the forms vLLM needs. Families: RDNA3 (gfx11-generic), RDNA4 (gfx12-generic) and CDNA (gfx90a, gfx942, gfx950). Sits directly on develop, which carries #14111 (FlyDSL core), #14358 (SDPA references) and #14332 (TensorAttributes Python bindings); this commit adds only the parity work on top of them. Features (all families unless noted): - Variable length: padding by per-batch lengths (padded rows O = 0, LSE = -inf); packed THD via ragged offsets; packed + both lengths (PyTorch seqused_k); ragged Q with seq_len_kv alone (SEM-028), which develop's frontend and references already accept (cuDNN 9.25 spells it cu_seq_len_q + seq_len_kv; hipDNN maps that once #14560 lands; no API change here). - Dropout: Philox4x32-10 bit-compatible with AOTriton (0.14.2b/0.15 word order), the rng_dump mask writer op, dropout with bias and under padding, at every AOTriton head-dim width on RDNA and CDNA. - Packed + Philox dropout (PyTorch's varlen flash path) on CDNA, gfx11 and flydsl_sdpa_aiw, compact or with per-batch lengths where the family serves them: each sequence draws AOTriton's varlen counters (its index, positions within it, the graph's declared extents), so dense, padded and packed storage of one problem draw one mask, and the rng_dump stays the dense FLOAT [B, Hq, Sq, Skv]. Matchers only, no object changes; packed with a bias still declines. SdpaSemanticsRegister SEM-015 records the mask layout. - fp32 I/O: exact fp32 on CDNA (with bias, dropout, any scale, head dims 136-248) and exact IEEE fp32 on the RDNA prefill engine (VALU path). - Bias at every head-dim tier, f32 or query dtype, vectorized and strided (any layout incl. key-broadcast), padded bias on the vectorized objects. - Attention sinks in FLOAT, cuDNN's dtype (SEM-022): every FlyDSL family now reads the FLOAT sink the frontend accepts (it read only a query-dtype sink before); any other sink dtype is declined. - Zero, negative and subnormal softmax scales. - Head dims 264-512, Q/K != V head-dim pairs, MLA 192/128, Hk != Hv on CDNA. - Head dims that are not a multiple of 8 (any row pitch) through padded copies (new sdpa_head_pad helper op), which like the engines declines K off Q's head dim and O off V's; multiple-of-8 graphs are unchanged. - Windows/bands: left-only windows on CDNA; FA/vLLM window mapping (SEM-027). No FlyDSL kernel reads V at keys no row of its block sees (window, band, past seq_len_kv), so NaN/Inf in nulled or stale KV no longer reaches O. - Exact softmax exponent at the max key (fixes two #14111 findings: NaN at large bf16 scores, and a 2^-8 output bias at e.g. head_dim 512, scale 1.2): CDNA generic/decode round the max down and clamp P in v_exp_f32 with DX10_CLAMP off (NaN rows stay NaN); CDNA dual-wave and RDNA take s - m first; flydsl_sdpa_aiw no longer contracts the scale into the exponent. - bf16 P rounded to nearest before the second product in every bf16 FlyDSL SDPA forward (truncation biased O about 2^-8.5 low wherever V does not have zero mean, AOTriton's lesson 16): CDNA generic and decode, bias and dropout builds included (gfx950 v_cvt_pk_bf16_f32; gfx942/gfx90a AOTriton's bf16_round_nonan_pack_v8), and flydsl_sdpa_aiw; the dual-wave and RDNA packs already rounded. The gfx942 bf16 generic objects cost +0.9% geomean (COVERAGE 7.5, deferred item 27). - gfx950 16-bit decode takes the rounded-down max like every other plain build: rounded to nearest, c * m could round up where |c * m| passes 2^31 (Q = K = V = 132,096), every exponent of the row fell below -126, and O was 0. - tools/check_trans_hazards.py (REGEN.md step 4) flags a transcendental result read by the next VALU instruction in the gfx942/gfx950 objects (the hazard an inline-asm convert hid from the backend's hazard recognizer). - gfx950 decode: the 16-bit decode builds pack bf16 P rounded to nearest, so the exact exponent no longer serializes their K loads (Llama-style long-cache decode was 14-22% slower; now within 0.1% geomean of before, at most +1.7%), and the gfx950 DMA V path reads a causal block's V window through a per-workgroup buffer descriptor instead of clamping every row (decode head_dim 128 causal +2.3-3.3% -> +0.2-1.4%). - Performance: CDNA head_dim 256 tiles with K chunking on gfx942/gfx90a, and gfx950 head_dim 256 on the dual-wave kernel (4 waves, ported from AOTriton's gfx950 FlyDSL forward). - Determinism: the FlyDSL SDPA forward kernels are deterministic by construction (no atomics; fixed-order split-KV combine; per-wave LDS partials); the split count follows the device's CU count. Documented in COVERAGE 5.7; no determinism tests yet. Frontend and references: - p = 0 dropout without an RNG dump (and with fp8 scaling) is no dropout in the references. - The CPU and GPU references never read V at keys a window or block mask removes (NaN/Inf in nulled or stale KV blocks no longer leaks into O). - GPU reference: a padded query row exits before the softmax max, and the fp8 dequantize pass skips Q/K/V padding, so padding is never read (SEM-008). Tests now exercise the native ragged plans (CPU BF16 densify comparison, GPU executor routing) and a paged fp8 K/V case with scalar descales (follow-ups from Adam Dickin's #14358 review). - Fix: frontend descriptor unpacking keeps ragged offsets, the ragged multiplier and byte alignment (graphs loaded from JSON and re-lowered were made dense). It asks for the multiplier only on a ragged tensor, and reads an extension a backend of the same major version does not serve (NOT_SUPPORTED, or BAD_PARAM from a therock-10.0 backend) as unset, so dense graphs still deserialize there. - Python: test_sdpa_ragged.py builds a packed THD SDPA graph through #14332's ragged-offset accessors, checks what validation and JSON/binary serialization keep, and compares it on the CPU reference with its padded equivalent and a float64 oracle (that case skips without HIPDNN_REF_CAPI_LIB). - Folds in #14696 (Daming Feng, ALMIOPEN-3094): dense attention-sink quick-tier bundle (quick/SdpaFwd/SinkToken: FLOAT [1, H_q, 1, 1] sink, GQA 8:2, causal, stats) and tests (an untyped sink rejected through Graph::validate, sink lowering, dense CPU plan execution with a sink and a dangling sink uid, CPU-reference sink rescaling per query head, -inf sink, wrong-shape sink); its validation and CPU-reference sink support were already on develop via #14358. Provider: - A per-engine graph-schema guard under engine plugin API 1.4.0: each engine sees only graphs up to the schema it declares. The built-in engines (HIP_MLOPS, HIP_FLASH2, ASM_SDPA) declare 1.4.0, which #14202 reviewed them against, so ASM's ragged (group-mode) path still reaches it; the FlyDSL engines declare 1.4.0 through their UEDs; every other engine stays at the provider's 1.2.0 baseline. The FlyDSL engines decline override-shape graphs. The guard holds engines back one by one inside the threaded applicability path, so the engines it admits keep their pool offload. - ASM_SDPA forward: its group-mode kernels are now checked against the CPU reference (THD, head dims 128 and 192/128, no mask and bottom-right causal per sequence); test only, the engine is unchanged. - FlyDSL SDPA integration lanes (test configuration only): the generated lane TOMLs grade the LSE/stats output with allclose_matching_infinities, since a padded row's LSE is -inf in the engine and the reference alike, and the lanes compare on the host (HIPDNN_TEST_VALIDATOR=cpu), which that validator needs; O keeps the default validator. Support claims for the FlyDSL engines on the SinkToken and padneq quick bundles (gfx942, gfx1201). Validation: AOTriton's forward test grid run through hipDNN with dnn-benchmarking, checked against PyTorch/AOTriton and hipDNN's GPU and CPU references, on gfx90a, gfx942, gfx950, gfx1201 and gfx1100 (details and results in the PR). Every existing #14111 object either changed only through a listed kernel modification or stayed byte-identical; see flydsl/COVERAGE.md and the vendored-kernel modification lists in each kernel README. AOT objects (DVC remote ingestor): SDPA objects 688 -> 1,912 across the five targets (incl. the dropout-mask and head-pad helpers); FlyDSL object dirs 25.6 -> 64.8 MiB. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

) ## Summary Completes the hipDNN Python bindings for `TensorAttributes` metadata, as requested in #10586: alignment, ragged-offset and ragged-offset-multiplier accessors, scalar-state accessors, plain-Python getters for stored scalar values, and `Graph.tensor_scalar` with the `ScalarType` enum. Behavior follows the maintainer's guidance in the issue discussion. ISSUE ID : #10586 ## Risk Assessment Level 2. Additive Python bindings over existing C++ methods. No change to existing behavior, C++ headers, schemas or build files, and every new method has unit tests. It adds new public Python API surface (new methods, a new enum and the `Graph.tensor_scalar` factory), which puts it at the boundary with level 3. ## ASIC Coverage Architecture-independent: host-side tensor metadata only, no device code. Passing standard PR CI is sufficient; no specific-ASIC or multi-arch run is needed. ## Testing Summary - Python binding unit tests for every new method: getter/setter round trips, method chaining, `None` handling for the ragged offset, `validate()` rules for alignment and the ragged-offset multiplier, value conversion for every supported scalar type and scalar state, and the data type and scalar type chosen by `Graph.tensor_scalar`. ## Testing Checklist - [x] Python binding tests - `python -m pytest -q projects/hipdnn/python/frontend_wheel_package/tests -m "not gpu"` - Status: Passed (238 passed, 13 skipped; GPU tests deselected; run in the hipDNN Docker dev image without a GPU) - [x] Pre-commit hooks - `pre-commit run --files <changed files>` - Status: Passed - [ ] PR CI - GitHub PR checks - Status: Pending ## Technical Changes - `tensor_bindings.cpp`: bind `get_alignment`/`set_alignment`; `get_ragged_offset`/`set_ragged_offset` (accepts `None`)/`has_ragged_offset`; the ragged-offset multiplier getter, setter and `has_` check; `clear_value`, `set_as_runtime_parameter`, `get_has_compile_time_constant`. - Add `get_value_variant`, `get_pass_by_value` and `get_compile_time_constant`, sharing one helper that returns a plain Python value (`half`/`bfloat16`/`float`/`double` as `float`, integer types as `int`, `bool` as `bool`, no value as `None`); `get_data_type()` gives the exact type. - Add `set_compile_time_constant` as an alias of `set_value`. - Bind the `ScalarType` enum and add `Graph.tensor_scalar(value, scalar_type)` in `graph_bindings.cpp` as four overloads (`float`, `double`, `int32_t`, `int64_t`, tried in that order), each calling `Graph::tensor`, as specified in the issue discussion. A Python `0.5` becomes FLOAT, `0.1` DOUBLE, `3` INT32 and `2**40` INT64. - `test_tensor.py`: tests for all of the above. --------- Co-authored-by: BrianHarrisonAMD <169072757+BrianHarrisonAMD@users.noreply.github.com>

assistant-librarian · 2026-11-06T12:54:30Z

TheRock Submodule Bump Activity

Newest first

2026-11-06 — Included in TheRock via ROCm/TheRock#9759.

github-actions Bot added the project: hipdnn label Oct 23, 2026

assistant-librarian Bot added the external contribution Code contribution from users community.. label Oct 23, 2026

NikharAsthana added 4 commits October 29, 2026 03:07

NikharAsthana force-pushed the users/NikharAsthana/hipdnn-python-tensor-accessors branch from ce48dc3 to 8ddf63d Compare October 28, 2026 21:55

SamuelReeder reviewed Oct 30, 2026

View reviewed changes

Comment thread projects/hipdnn/python/frontend_wheel_package/tests/test_tensor.py Resolved

Comment thread projects/hipdnn/python/frontend_wheel_package/tests/test_tensor.py Resolved

SamuelReeder approved these changes Nov 2, 2026

View reviewed changes

BrianHarrisonAMD marked this pull request as ready for review November 3, 2026 17:30

BrianHarrisonAMD requested a review from a team as a code owner November 3, 2026 17:31

Merge branch 'develop' into users/NikharAsthana/hipdnn-python-tensor-…

1bf13ff

…accessors

BradPepersAMD mentioned this pull request Nov 4, 2026

feat(hip-kernel-provider): FlyDSL SDPA forward parity with AOTriton, plus vLLM forms #15066

Open

BrianHarrisonAMD merged commit 654f2a2 into ROCm:develop Nov 4, 2026
235 of 239 checks passed

BrianHarrisonAMD mentioned this pull request Nov 4, 2026

[Python bindings] Complete TensorAttributes metadata bindings #10586

Closed

GarbyGilbertAMD mentioned this pull request Nov 6, 2026

feat(hipdnn): add SDPA conformance tool running AOTriton's test suite on hipDNN #15232

Open

4 of 5 tasks

Conversation

NikharAsthana commented Oct 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Risk Assessment

ASIC Coverage

Testing Summary

Testing Checklist

Technical Changes

Uh oh!

therock-pr-bot Bot commented Oct 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

✅ All Checks Passed — Ready for Review

Uh oh!

therock-pr-bot Bot commented Oct 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

NikharAsthana commented Oct 27, 2026

Uh oh!

NikharAsthana commented Oct 29, 2026

Uh oh!

SamuelReeder commented Oct 29, 2026

Uh oh!

Uh oh!

Uh oh!

NikharAsthana commented Nov 2, 2026

Uh oh!

SamuelReeder left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

BrianHarrisonAMD commented Nov 3, 2026

Uh oh!

BrianHarrisonAMD commented Nov 3, 2026

Uh oh!

BrianHarrisonAMD commented Nov 4, 2026

Uh oh!

Uh oh!

NikharAsthana commented Nov 4, 2026

Uh oh!

assistant-librarian Bot commented Nov 6, 2026

TheRock Submodule Bump Activity

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

NikharAsthana commented Oct 23, 2026 •

edited

Loading

therock-pr-bot Bot commented Oct 23, 2026 •

edited

Loading

therock-pr-bot Bot commented Oct 23, 2026 •

edited

Loading

SamuelReeder left a comment •

edited

Loading