Skip to content

Commit 8ea419b

Browse files
authored
Fix value_dim in TransformerDecoder's cross-attn layer (#667)
* Fix in TransformerDecoder's cross-attn layer * Revert UT
1 parent 7dadc61 commit 8ea419b

File tree

1 file changed

+1
-1
lines changed

1 file changed

+1
-1
lines changed

keras_nlp/layers/transformer_decoder.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -165,7 +165,7 @@ def _build(self, input_shape, has_cross_attention):
165165
self._cross_attention_layer = keras.layers.MultiHeadAttention(
166166
num_heads=self.num_heads,
167167
key_dim=head_dim,
168-
value_dim=hidden_dim,
168+
value_dim=head_dim,
169169
dropout=self.dropout,
170170
kernel_initializer=clone_initializer(self.kernel_initializer),
171171
bias_initializer=clone_initializer(self.bias_initializer),

0 commit comments

Comments
 (0)