Skip to content

Conversation

@keshavvinayak01
Copy link
Contributor

@keshavvinayak01 keshavvinayak01 commented Nov 27, 2025

Following the discussion from #22441

Depending on the backend, certain computations may benefit from directly using exp instead of exp2, since there might be accuracy losses due to FP-reassociation. It's helpful to add flag incase the user tracks losses to this particular computation and might favour directly using exp.

The use_exp2 flag is mostly unused in dialect conversions and passes, I presume it's used as a KernelOption. The changes here will not modify the default behavior.

@keshavvinayak01 keshavvinayak01 changed the title [LinalgExt] Added toggle for using useexp2 for onlineAttention Decomposition [LinalgExt] Added toggle for using useExp2 for onlineAttention Decomposition Nov 27, 2025
Signed-off-by: Keshav Vinayak Jha <[email protected]>
Copy link
Collaborator

@MaheshRavishankar MaheshRavishankar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it would be simpler to just add an optional attribute to the op itself to use exp2 for decomposition. This doesnt need to be a separate attribute that is not part of the op definition.

@keshavvinayak01
Copy link
Contributor Author

I think it would be simpler to just add an optional attribute to the op itself to use exp2 for decomposition. This doesnt need to be a separate attribute that is not part of the op definition.

I was going to do that, but then I saw

%result = iree_linalg_ext.attention {decomposition_config = {pv_attrs = {x}, qk_attrs = {y}, use_exp2}, indexing_maps = [#map, #map1, #map2, #map3, #map4], compilation_info = #compilation} ins(%arg0, %arg1, %arg2, %arg3 : tensor<2x10x6x4xf16>, tensor<2x10x4x4xf16>, tensor<2x10x4x4xf16>, f16) outs(%init : tensor<2x10x6x4xf16>) {

Where it's part of the decomposition config itself. So I thought I'd better refine the attribute and continue using it. Making it an optional attribute of the op itself might introduce redundancy since this already exists @MaheshRavishankar ?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants