Repository navigation
Conversation
…port - dense_attention_reference gains optional seqlen_q / seqlen_kv. None keeps the exact unpadded arithmetic. When set, keys past seqlen_kv are masked, a row with no valid key yields O = 0, and padded query rows are zeroed. - _sdpa_seqlen.py: host adapter validating hipDNN's per-sequence lengths at the boundary (integer dtype, per-batch count, 0 <= len <= S_max) and returning the flat per-batch int32 arrays the shell reads. - _sdpa_padding_mask.py: host specification of the kernel-side mask -- keep_key (the shell's extra_mask_predicate hook) and row_is_valid (the epilogue's padded-row zeroing), with the emitter form documented. The live IR emitter is deferred to the shell, where it carries its byte-identity C++ mirror. This module is the host reference it must match, cross-checked against dense_attention_reference. Kernel wiring and GPU validation wait on the shell (ST4) and the arbitrary-length buffer clipping (ticket 8).
Ticket 12 (varlen/THD forward) needs hipDNN's ragged per-sequence lengths converted into the cumulative cu_seqlens the varlen kernels (fmha_varlen) address with. Extend the shared SDPA length adapter with sdpa_cu_seqlens, which validates the per-sequence counts and returns (B+1,) int32 exclusive prefix sums (leading 0, last element = packed token total). The validator is refactored to make the upper bound optional, since ragged tokens are packed with no padding and therefore no per-sequence S_max. Host numpy only, no kernel emission. Covers the host/adapter half of ticket 12, the non-causal unified no-mask path and arbitrary-length tails are separate work.
github-actions
Bot
added
project: hip-kernel-provider
rocke-lib: attention
⏳ Waiting on Required Checks⏳ Required checks have not all concluded yet. This bot run is exiting without failing so those checks can report independently.
📖 Need help? See the Policy FAQ for details on every check and how to fix failures. |
|
⏳ Waiting for required checks to start or finish:
|
Contributor
Author
|
Moved to this PR: #13953 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
Ticket 12 (varlen / THD forward) needs hipDNN's ragged per-sequence lengths turned into the cumulative cu_seqlens the varlen kernels address with. This is the host-side, non-blocked slice of that ticket, and it extends the same length adapter introduced for padding (ticket 10) rather than adding a parallel one.
Technical Details
batchcounts, non-negative) and returns (cu_seqlens_q, cu_seqlens_kv) as contiguous (B+1,) int32 exclusive prefix sums with a leading 0, whose last element is the packed token total.kvto match hipDNN's SEQ_LEN_KV and the padded path, the kernel param iscu_seqlens_k.)Test Plan
Test Result
Related
JIRA ID : AICK-2521
Submission Checklist