[Pytorch] Support for Swiglu Activation used in GPT OSS - #2161

Merged

vthumbe1503 merged 49 commits into

NVIDIA:mainfrom

vthumbe1503:users/vthumbe/gpt_oss_swiglu_integration

Sep 30, 2025

Commits on Sep 19, 2025

Test working as I think it should work
vthumbe1503
committed
2a27823
initial draft of changes to get GPT oss based swiglu integrated, gated kernels needs to be fixed
vthumbe1503
committed
d4c06c5
redundant implementation for the pytorch to te hook up, refactoring to be done later
vthumbe1503
committed
1f596af
all gated kernels modified, pytest working for oss swiglu
vthumbe1503
committed
42f85c3
[pre-commit.ci] auto fixes from pre-commit.com hooks

pre-commit-ci[bot]
authored and
vthumbe1503
committed
c9d3311
fix the merge conflict
vthumbe1503
committed
5d06c2a
[pre-commit.ci] auto fixes from pre-commit.com hooks

pre-commit-ci[bot]
authored and
vthumbe1503
committed
025ce6b
accidentally had removed some activations, minor bug in the templated function
vthumbe1503
committed
d964b24
[pre-commit.ci] auto fixes from pre-commit.com hooks

pre-commit-ci[bot]
authored and
vthumbe1503
committed
de9ef2f
parent de9ef2fe450daae0d4ea1b647a37219f72814f66

committed
8e17473
accidentally removed the copyright
vthumbe1503
committed
1f2c65b
fix linting issue
vthumbe1503
committed
75c4b13
minor issue in comments
vthumbe1503
committed
288e926
Commit is for another PR
vthumbe1503
committed
448eceb
revert changes since this belongs to another PR
vthumbe1503
committed
23b5822
[pre-commit.ci] auto fixes from pre-commit.com hooks

pre-commit-ci[bot]
authored and
vthumbe1503
committed
a1a5794
Revert change back since belongs to another PR
vthumbe1503
committed
0d6a3ea
[pre-commit.ci] auto fixes from pre-commit.com hooks

pre-commit-ci[bot]
authored and
vthumbe1503
committed
33c3364
Changes belong to another PR
vthumbe1503
committed
a724c2d
[pre-commit.ci] auto fixes from pre-commit.com hooks

pre-commit-ci[bot]
authored and
vthumbe1503
committed
34d9815
Revert changes here

committed
3475264
address review comments
vthumbe1503
committed
5e687d1
cleanup
vthumbe1503
committed
8535dfb
[pre-commit.ci] auto fixes from pre-commit.com hooks

pre-commit-ci[bot]
authored and
vthumbe1503
committed
fa0e9a9
fix linting error
vthumbe1503
committed
aee3fb9
[pre-commit.ci] auto fixes from pre-commit.com hooks

pre-commit-ci[bot]
authored and
vthumbe1503
committed
87ae3d1
Address review comments, fix mxfp8 kernel bug: was not passing clamped swiglu parameter correctly
vthumbe1503
committed
3858eab
[pre-commit.ci] auto fixes from pre-commit.com hooks

pre-commit-ci[bot]
authored and
vthumbe1503
committed
de3080e
[pre-commit.ci] auto fixes from pre-commit.com hooks

pre-commit-ci[bot]
authored and
vthumbe1503
committed
7bf0bc4
Use limit=0.75 in clamped SwiGLU test

timmoon10
authored and
vthumbe1503
committed
fe93c01
Address review comments
vthumbe1503
committed
5d3b169
Merge branch 'main' into users/vthumbe/gpt_oss_swiglu_integration
vthumbe1503
authored
439e5a1
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot]
committed
3a1cd45
accidentally removed a line while resolving merge conflict
vthumbe1503
committed
3417290
match pytorch implementation: dclamp should be 1 for borders of clamping limits as well
vthumbe1503
committed
724b37a

Commits on Sep 20, 2025

Commits on Sep 23, 2025

float should rather be bool: fix by copilot
vthumbe1503
committed
1132edf

Commits on Sep 29, 2025

Commits on Sep 30, 2025

{ missed in activation.cpp
vthumbe1503
committed
04510b7
merge conflict
vthumbe1503
committed
442eac0
minor formatting change
vthumbe1503
committed
f1e3599
nvfp4 change
vthumbe1503
committed
b2dc251
Merge branch 'main' into users/vthumbe/gpt_oss_swiglu_integration
vthumbe1503
authored
0e8d139