Fix missing norm call in Mlp forward (not used by default, but can be…

… enabled for normformer MLP scale). Fix #1851 fix #1852
huggingface · Aug 3, 2023 · 76d1669 · 76d1669
1 parent 8e4480e
commit 76d1669
Showing 1 changed file with 1 addition and 0 deletions.
diff --git a/timm/layers/mlp.py b/timm/layers/mlp.py
@@ -42,6 +42,7 @@ def forward(self, x):
         x = self.fc1(x)
         x = self.act(x)
         x = self.drop1(x)
+        x = self.norm(x)
         x = self.fc2(x)
         x = self.drop2(x)
         return x