Skip to content

Conversation

@douglas-reid
Copy link
Contributor

What does this PR do?

Fixes the rope_parameters in the weights conversion script for Gemma 3.

These should be:

local => default @ 10_000.0
global => linear(8.0) @ 1_000_000.0

@github-actions
Copy link
Contributor

[For maintainers] Suggested jobs to run (before merge)

run-slow: gemma3

@RyanMullins
Copy link
Contributor

cc @zucchini-nlp

@Rocketknight1
Copy link
Member

Yes, good fix - rope_parameters is assumed to be keyed by layer type in the modeling file, so this conversion script would generate a broken config in at least some cases.

@Rocketknight1 Rocketknight1 enabled auto-merge (squash) October 30, 2025 11:48
@Rocketknight1 Rocketknight1 merged commit b47b356 into huggingface:main Oct 30, 2025
14 checks passed
@HuggingFaceDocBuilderDev

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

i3hz pushed a commit to i3hz/transformers that referenced this pull request Oct 30, 2025
…#41922)

Fix rope_parameters for gemma3 weights conversion script.

Co-authored-by: Douglas Reid <[email protected]>
SangbumChoi pushed a commit to SangbumChoi/transformers that referenced this pull request Jan 23, 2026
…#41922)

Fix rope_parameters for gemma3 weights conversion script.

Co-authored-by: Douglas Reid <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants