Skip to content

Fix bug in Squeeze for getting the value of total_seq_len #1886

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
kunal-vaishnavi merged 5 commits into from
Nov 26, 2025
+17 −16
Merged

Fix bug in Squeeze for getting the value of total_seq_len #1886

Changes from 1 commit
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
d89a1e8
Fix bug in Squeeze for getting the value of total_seq_len
Honry Nov 21, 2025
60caa6e
Address comments
Honry Nov 24, 2025
2d38382
update the ReduceSum in make_attention_mask_standard_reformatting_for…
Honry Nov 24, 2025
435172c
update ReduceSum for the attention mask subgraph as well
Honry Nov 24, 2025
245a0e2
Simplify make_reduce_sum by using keepdims=False by default
Honry Nov 24, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion src/python/py/models/builders/base.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4196,7 +4196,7 @@ def make_attention_mask_graph_capture_reformatting_for_gqa(self, attn_mask_basen

# Right branch: Squeeze to get int value for total_seq_len
squeeze_name = f"{attn_mask_basename}/Squeeze"
squeeze_inputs = [f"{reduce_sum_name}/output_0", "/model/constants/INT64/[0]"]
squeeze_inputs = [f"{reduce_sum_name}/output_0"]
self.make_squeeze(squeeze_name, squeeze_inputs, dtype=ir.DataType.INT32, shape=[])

self.mask_attrs["seqlens_k"] = sub_name
Expand Down
Loading