Skip to content
Merged
Changes from 5 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -188,11 +188,14 @@ def legacy_routing(
if sm_first:
logits = torch.softmax(logits, dim=-1)
sparse_logits = topk(logits, n_expts_act, apply_softmax=not sm_first)
return legacy_routing_from_sparsematrix(

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

after you remove this legacy_routing_from_sparsematrix is not used any more. Let's remove it.

Moreover, please disclose the accuracy of the models.

I am not familiar with this part of the code.

CC @robertgshaw2-redhat @bnellnm

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I removed that legacy routing logic. I enabled also GPQA Eval tests too for GPT-OSS accuracy.

sparse_logits,
logits.shape[-1],
n_expts_act,
)
if isinstance(sparse_logits, tuple):
topk_weights, topk_ids_raw, bitmatrix = sparse_logits
return legacy_routing_from_bitmatrix(
bitmatrix, topk_weights, topk_ids_raw, logits.shape[-1], n_expts_act
)
topk_ids = sparse_logits.indx.to(torch.long)
topk_weights = sparse_logits.vals
return make_routing_data(topk_ids, topk_weights, logits.shape[-1])


def triton_kernel_moe_forward(
Expand Down
Loading