Skip to content
Merged
Show file tree
Hide file tree
Changes from 2 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 5 additions & 5 deletions docs/fe-oss-apis/dsa.md
Original file line number Diff line number Diff line change
Expand Up @@ -331,11 +331,11 @@ d_index_q, d_weights, d_index_k = (

When compressed forward returns its fused `softmax`, backward can skip both
indexer Q@K score recompute and the separate logits softmax. Pass `softmax`
directly as `index_score`; backward consumes and overwrites this buffer, so
pass `softmax.clone()` if it must be preserved. `attn_score` must use the same
valid-slot mask. Because compressed forward returns global indices by default,
also pass `topk_indices_global=True` unless forward used
`topk_indices_global=False`. The public sparse `indexer_backward_wrapper` has a
directly as `index_score`; backward treats this buffer as read-only.
`attn_score` must use the same valid-slot mask. Because compressed forward
returns global indices by default, also pass `topk_indices_global=True` unless
forward used `topk_indices_global=False`. The public sparse
`indexer_backward_wrapper` has a
BSHD-shaped interface; BF16 THD tensors can use zero-copy `B=1` views (squeeze
the singleton K head and add a batch dimension) together with global Top-K
indices. FP8 and MXFP8 indexer backward are not currently supported because
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -785,9 +785,9 @@ def indexer_backward_wrapper(

``index_score`` contains the predict softmax aligned slot-for-slot with
``topk_indices``; the fused softmax returned by compressed indexer forward
can be passed directly. ``attn_score`` and ``index_score`` are consumed
in-place: the kernel overwrites ``attn_score`` with ``grad_signal`` and
``index_score`` with ``sum_grad`` during the score-grad precompute stage.
can be passed directly. The kernel overwrites ``attn_score`` with
``grad_signal`` during the score-grad precompute stage and treats
``index_score`` as read-only.
Comment thread
coderabbitai[bot] marked this conversation as resolved.
Resolved

Args:
topk_indices_global: whether ``topk_indices`` already contains global
Expand All @@ -801,7 +801,7 @@ def indexer_backward_wrapper(
index_score: FP32 predict probabilities over the selected indexer
logits. The caller must use the same valid-slot mask when
constructing ``attn_score`` so target and predict describe
identical slots. This buffer is overwritten in-place.
identical slots. This buffer is preserved.
grad_loss: single-element float32 tensor on the same CUDA device as
``index_q``. The kernel reads its value at runtime, including on
CUDA Graph replay.
Expand Down
Loading