Skip to content

Fix accuracy issues with Gemma models #1448

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
kunal-vaishnavi merged 29 commits into from
May 9, 2025
Merged

Fix accuracy issues with Gemma models #1448

Changes from 28 commits
Commits
Show all changes
29 commits
Select commit Hold shift + click to select a range
cd5fc2c
Initial implementation of bfloat16 support
kunal-vaishnavi Apr 15, 2025
5fa15f2
Refactor multi-cache calculation
kunal-vaishnavi Apr 21, 2025
084936a
Fix how caches are saved in if node
kunal-vaishnavi Apr 21, 2025
eb6f0fb
Fix node names for logit softcapping
kunal-vaishnavi Apr 24, 2025
d3b5ee0
Remove extra multiply
kunal-vaishnavi Apr 24, 2025
4880922
Add missing final norm for Gemma-3 multimodal
kunal-vaishnavi Apr 24, 2025
2e00e80
Fix naming of model type
kunal-vaishnavi Apr 24, 2025
196c1c2
Add Gemma-3 system prompt to chat example
kunal-vaishnavi Apr 24, 2025
e82b240
Merge branch 'kvaishnavi/bf16' into kvaishnavi/gemma-mixed-precision
kunal-vaishnavi Apr 25, 2025
7d2fc55
Generate Gemma-3 4B text-only model
kunal-vaishnavi Apr 25, 2025
89040f6
Cast MatMul from FP32 to BF16
kunal-vaishnavi Apr 25, 2025
795d20b
Merge branch 'main' into kvaishnavi/gemma3-mm
kunal-vaishnavi Apr 28, 2025
c24d40d
Add missing name for constant node
kunal-vaishnavi Apr 28, 2025
b1ec3bd
GQA bf16 support (#1429)
nenad1002 Apr 29, 2025
1e541b6
Fast Gelu
nenad1002 Apr 29, 2025
be4e533
Add Gemma3 bf16 related changes (#1441)
nenad1002 May 2, 2025
1ca152d
Add layernorms
nenad1002 May 7, 2025
0fa139a
Merge branch 'main' into kvaishnavi/gemma3-mm
kunal-vaishnavi May 7, 2025
b1e4378
Fix how final norm is accessed
kunal-vaishnavi May 7, 2025
19654b2
Add guard methods
nenad1002 May 7, 2025
ed9eda0
Refactor layernorm
nenad1002 May 8, 2025
30077d0
Make sure bias is not used before being defined
nenad1002 May 8, 2025
577e4b3
Redesign and refactor how casting is done
kunal-vaishnavi May 8, 2025
a6bb465
Fix embedding check to be more generic
kunal-vaishnavi May 8, 2025
01b2a49
Add missing cast name
kunal-vaishnavi May 8, 2025
b6d8b94
Revert quantization import consolidation until CIs are updated
kunal-vaishnavi May 8, 2025
3eb173d
Remove unneeded parentheses
kunal-vaishnavi May 8, 2025
f8f6e02
Add more granular control over LayerNorm casts
kunal-vaishnavi May 8, 2025
cdd273c
Use same chat template for Gemma models
kunal-vaishnavi May 9, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Loading