Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
29 commits
Select commit Hold shift + click to select a range
cd5fc2c
Initial implementation of bfloat16 support
kunal-vaishnavi Apr 15, 2025
5fa15f2
Refactor multi-cache calculation
kunal-vaishnavi Apr 21, 2025
084936a
Fix how caches are saved in if node
kunal-vaishnavi Apr 21, 2025
eb6f0fb
Fix node names for logit softcapping
kunal-vaishnavi Apr 24, 2025
d3b5ee0
Remove extra multiply
kunal-vaishnavi Apr 24, 2025
4880922
Add missing final norm for Gemma-3 multimodal
kunal-vaishnavi Apr 24, 2025
2e00e80
Fix naming of model type
kunal-vaishnavi Apr 24, 2025
196c1c2
Add Gemma-3 system prompt to chat example
kunal-vaishnavi Apr 24, 2025
e82b240
Merge branch 'kvaishnavi/bf16' into kvaishnavi/gemma-mixed-precision
kunal-vaishnavi Apr 25, 2025
7d2fc55
Generate Gemma-3 4B text-only model
kunal-vaishnavi Apr 25, 2025
89040f6
Cast MatMul from FP32 to BF16
kunal-vaishnavi Apr 25, 2025
795d20b
Merge branch 'main' into kvaishnavi/gemma3-mm
kunal-vaishnavi Apr 28, 2025
c24d40d
Add missing name for constant node
kunal-vaishnavi Apr 28, 2025
b1ec3bd
GQA bf16 support (#1429)
nenad1002 Apr 29, 2025
1e541b6
Fast Gelu
nenad1002 Apr 29, 2025
be4e533
Add Gemma3 bf16 related changes (#1441)
nenad1002 May 2, 2025
1ca152d
Add layernorms
nenad1002 May 7, 2025
0fa139a
Merge branch 'main' into kvaishnavi/gemma3-mm
kunal-vaishnavi May 7, 2025
b1e4378
Fix how final norm is accessed
kunal-vaishnavi May 7, 2025
19654b2
Add guard methods
nenad1002 May 7, 2025
ed9eda0
Refactor layernorm
nenad1002 May 8, 2025
30077d0
Make sure bias is not used before being defined
nenad1002 May 8, 2025
577e4b3
Redesign and refactor how casting is done
kunal-vaishnavi May 8, 2025
a6bb465
Fix embedding check to be more generic
kunal-vaishnavi May 8, 2025
01b2a49
Add missing cast name
kunal-vaishnavi May 8, 2025
b6d8b94
Revert quantization import consolidation until CIs are updated
kunal-vaishnavi May 8, 2025
3eb173d
Remove unneeded parentheses
kunal-vaishnavi May 8, 2025
f8f6e02
Add more granular control over LayerNorm casts
kunal-vaishnavi May 8, 2025
cdd273c
Use same chat template for Gemma models
kunal-vaishnavi May 9, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion examples/python/model-chat.py
Original file line number Diff line number Diff line change
Expand Up @@ -56,7 +56,7 @@ def main(args):
print("Using Chat Template for LLAMA 3, if you are using LLAMA 2 please pass the argument --chat_template '{input} [/INST]')")
elif model_type.startswith("qwen2"):
args.chat_template = '<|im_start|>user\n{input}<|im_end|>\n<|im_start|>assistant\n'
elif model_type == "gemma3_text":
elif model_type.startswith("gemma"):
args.chat_template = '<start_of_turn>user\n{system_prompt}{input}<end_of_turn>\n<start_of_turn>model\n'
else:
raise ValueError(f"Chat Template for model type {model_type} is not known. Please provide chat template using --chat_template")
Expand Down
4 changes: 2 additions & 2 deletions examples/python/model-qa.py
Original file line number Diff line number Diff line change
Expand Up @@ -56,7 +56,7 @@ def main(args):
print("Using Chat Template for LLAMA 3, if you are using LLAMA 2 please pass the argument --chat_template '{input} [/INST]')")
elif model_type.startswith("qwen2"):
args.chat_template = '{system_prompt}<|im_start|>user\n{input}<|im_end|>\n<|im_start|>assistant\n'
elif model_type == "gemma3_text":
elif model_type.startswith("gemma"):
args.chat_template = '<start_of_turn>user\n{system_prompt}{input}<end_of_turn>\n<start_of_turn>model\n'
else:
raise ValueError(f"Chat Template for model type {model_type} is not known. Please provide chat template using --chat_template")
Expand All @@ -75,7 +75,7 @@ def main(args):
print("Using System Prompt for LLAMA 3, if you are using LLAMA 2 please pass the argument --system_prompt '<s>[INST] <<SYS>>\\n{args.system_prompt}\\n<</SYS>>')")
elif model_type.startswith("qwen2"):
system_prompt = f"<|im_start|>system\n{args.system_prompt}<|im_end|>\n"
elif model_type == "gemma3_text":
elif model_type.startswith("gemma"):
system_prompt = f"{args.system_prompt}"
else:
system_prompt = args.system_prompt
Expand Down
Loading