Skip to content

[model] Add Gemma 4 support#1

Closed
marksverdhei wants to merge 1 commit into
mainfrom
nb/gemma4
Closed

[model] Add Gemma 4 support#1
marksverdhei wants to merge 1 commit into
mainfrom
nb/gemma4

Conversation

@marksverdhei
Copy link
Copy Markdown
Member

Summary

  • Ports upstream PR [model] gemma4 hiyouga/LlamaFactory#10346 to add Gemma 4 model family support
  • Adds Gemma4 and Gemma4n multimodal plugins (image processing with pan-and-scan, audio support for Gemma4n)
  • Adds chat templates, tool calling utilities, model constants, and visual module support
  • Includes HyperParallel workflow compatibility fix

Upstream status

The upstream PR is still open/WIP with known limitations:

  • gemma4-e*B variants are not yet ready for training
  • Gemma4n audio plugin and forwarding tests are pending

Files changed

File Change
data/mm_plugin.py Gemma4/Gemma4n multimodal plugins
data/template.py Chat templates for gemma4, gemma4n
data/tool_utils.py Tool calling support
extras/constants.py Model registry entries
model/model_utils/visual.py Visual module support
train/hyper_parallel/workflow.py HyperParallel compatibility
tests/data/test_mm_plugin.py Plugin tests

Test plan

  • Verify Gemma 4 text-only SFT training works
  • Test Gemma 4 multimodal (image) training
  • Test Gemma 4n with audio inputs
  • Verify no regressions on existing models

Port upstream PR hiyouga#10346 to add Gemma 4 model support:
- Gemma4 multimodal plugin (image processing, pan-and-scan)
- Gemma4n multimodal plugin with audio support
- Chat templates for gemma4 and gemma4n
- Tool calling utilities for Gemma 4
- Model constants and visual module support
- HyperParallel workflow compatibility
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant