GQA support sliding_window_cache

docs/ContribOperators.md

@@ -2637,6 +2637,8 @@ This version of the operator has been available since version 1 of the 'com.micr
 <dd>Rotate using interleaved pattern. Default value is 0 (False).</dd>
 <dt><tt>scale</tt> : float</dt>
 <dd>Custom scale will be used if specified. Default value is 1/sqrt(head_size)</dd>
+<dt><tt>sliding_window_cache</tt> : int</dt>
+<dd>Set to 1 when the past/present KV buffers are window-sized instead of holding the whole sequence. The op then keeps only the min(total_sequence_length, cache_capacity) most recent tokens, contiguously, using cache-relative indexing and evicting from the front as needed. Requires local_window_size > 0 and a cache capacity of at least local_window_size. Multi-token steps may use a temporary staging buffer, so the capacity need not cover the entire step. Default value is 0 (full-length cache).</dd>
 <dt><tt>smooth_softmax</tt> : int</dt>
 <dd>Use a smooth factor in softmax.</dd>
 <dt><tt>softcap</tt> : float</dt>
@@ -2973,6 +2975,7 @@ This version of the operator has been available since version 1 of the 'com.micr
 <dd>Constrain scalar scales to FP32.</dd>
 </dl>
 
+
 ### <a name="com.microsoft.MatMulBlockQuantizedFp8Weight"></a><a name="com.microsoft.matmulblockquantizedfp8weight">**com.microsoft.MatMulBlockQuantizedFp8Weight**</a>
 
   Weight-only block-scaled FP8 (E4M3) matrix multiplication.

docs/contrib_ops/cuda/gqa.md

@@ -56,6 +56,7 @@ Selected attributes:
 | `scale` | Softmax scale. Defaults to `1/sqrt(head_size)`. |
 | `softcap` | Optional logit soft-capping value. `0` disables it. |
 | `local_window_size` | Left window size for local attention. `-1` means global attention. |
+| `sliding_window_cache` | Set to `1` when using a windowed (sliding-window) KV cache instead of full-length. When enabled, the operator keeps only the most recent tokens, using cache-relative indexing and evicting from the front as needed. Requires `local_window_size > 0`. Defaults to `0` (full-length cache). |
 | `do_rotary` / `rotary_interleaved` | Enable RoPE and select interleaved vs. half-rotary layout. |
 | `smooth_softmax` | Add a smooth factor to the softmax denominator. |
 | `qk_norm_epsilon` | Epsilon for the fused per-head Q/K RMSNorm (QK-Norm) prologue. Defaults to `1e-6`. |
@@ -155,6 +156,40 @@ holds (the past and present tensors alias the same memory), the cache length is
 sequence length and new keys/values are appended in place. This shared-buffer mode is required by
 the XQA decode path and by the Flash-Decoding fast path.
 
+### Sliding Window (Windowed) KV Cache
+
+When the `sliding_window_cache` attribute is set to `1`, the KV cache operates in a windowed
+(sliding-window) mode, keeping only the most recent `local_window_size` tokens instead of growing
+to the full sequence length. This significantly reduces memory usage for long-context models that
+employ local attention (e.g., GPT-OSS with layer-wise sliding windows).
+
+**Key behaviors:**
+
+- **Cache capacity:** The cache buffer size is fixed to `kv_cache_capacity` (the initial allocation
+  size), which must be at least `local_window_size`. Allocating a little more than the window is
+  worthwhile on CPU: the surplus is what lets the append point drift, so the cache is compacted once
+  every `kv_cache_capacity - local_window_size + 1` steps instead of on every step. A modest surplus
+  (order tens of entries) is the sweet spot; a much larger buffer starts costing more in attention
+  over out-of-window entries than it saves in compaction.
+- **Cache-relative indexing:** New keys/values are appended at cache-relative positions, not global
+  positions. Once the window is full, old tokens are evicted from the front.
+- **RoPE position bounds:** The `rotary_max_position` parameter controls the upper bound (exclusive)
+  for RoPE position indices, decoupling absolute sequence positions from cache buffer indices.
+- **Multi-token staging:** For multi-token (prompt) steps that exceed window size, a temporary
+  staging buffer is used internally to handle compaction and eviction.
+- **Present shape:** When using windowed cache with `past_present_share_buffer`, the `present_key`
+  and `present_value` shapes remain bounded by `kv_cache_capacity` in the sequence dimension,
+  rather than growing with `total_sequence_length`.
+- **Constraints:** `sliding_window_cache=1` requires `local_window_size > 0`. Windowed caches are
+  incompatible with the Flash-Attention fast-decode path and instead use the XQA or standard
+  attention backends.
+- **CPU support:** The CPU kernel implements the same contract. Entries live at `[0, end)` and are
+  appended at `end`, so a step only moves memory when the append would run past the capacity; at
+  that point the surviving entries are compacted to the front in one go. Multi-token steps that
+  would drop entries the step itself still reads are staged instead, so results match a full-length
+  cache exactly. On CPU the `attention_bias` input, the `qk_output` attribute and a shared KV layout
+  (`key`/`value` folded into `query`) are not supported together with `sliding_window_cache=1`.
+
 ### Quantized KV cache
 
 To reduce the KV-cache memory footprint, the cache may be stored quantized while `query` stays

onnxruntime/contrib_ops/cpu/bert/attention_parameters.h

@@ -104,6 +104,21 @@ struct GroupQueryAttentionParameters : AttentionParameters {
   KVQuantizationType k_quant_type = KVQuantizationType::NONE;
   KVQuantizationType v_quant_type = KVQuantizationType::NONE;
   int kv_cache_bit_width = 0;
+
+  // Windowed (sliding-window) KV cache. Set from the sliding_window_cache attribute.
+  // When true, past/present KV buffers are allocated with capacity C = kv_cache_capacity, which may
+  // be much smaller than total_sequence_length. The cache then holds only the min(T, C) most recent
+  // tokens at cache indices [0, L) and the kernel uses cache-relative indexing plus a shift
+  // compaction step. Requires local_window_size > 0. See docs: windowed KV cache design.
+  bool is_windowed_kv_cache = false;
+  int kv_cache_capacity = 0;       // Capacity of the KV buffer used by this step.
+  int kv_cache_real_capacity = 0;  // C: allocated sequence dim of the past/present KV buffers.
+  // A multi-token step runs against a longer staging buffer (see GroupQueryAttention::ComputeInternal),
+  // in which case kv_cache_capacity is the staging capacity and differs from kv_cache_real_capacity.
+
+  // Upper bound (exclusive) for absolute RoPE positions: dim 0 of the cos/sin caches.
+  // 0 when rotary is not configured.
+  int rotary_max_position = 0;
 };
 
 // Parameters deduced from node attributes and inputs/outputs.