Skip to content

Patch should_convert_module for transformers 5.x substring matching #491

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
danielhanchen merged 1 commit into from
Feb 10, 2026
Merged

Patch should_convert_module for transformers 5.x substring matching #491

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
32 changes: 32 additions & 0 deletions unsloth_zoo/patching_utils.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -751,6 +751,38 @@ def add_score_code(match):
transformers.integrations.bitsandbytes._replace_with_bnb_linear = _unsloth_replace_with_bnb_linear
pass

# Patch for transformers 5.x: should_convert_module uses re.match (prefix-anchored)
# and endswith, but does not do substring component matching. This means entries like
# "vision_tower" in llm_int8_skip_modules fail to match module names like
# "model.vision_tower.vision_model.encoder.layers.0.self_attn.q_proj".
# On 4.x, Unsloth patches _replace_with_bnb_linear with check_conversion_mappings
# (substring matching). On 5.x, _replace_with_bnb_linear doesn't exist, so we
# patch should_convert_module instead.
import transformers.quantizers.quantizers_utils as _quantizers_utils
if not hasattr(transformers.integrations.bitsandbytes, "_replace_with_bnb_linear") and \
hasattr(_quantizers_utils, "should_convert_module") and \
getattr(_quantizers_utils.should_convert_module, "__name__", "") != "_unsloth_should_convert_module":

_original_should_convert_module = _quantizers_utils.should_convert_module

def _unsloth_should_convert_module(full_name, patterns=None):
if patterns is None:
return True
should_not_convert = any(
re.match(f"{key}\\.", full_name) or
re.match(f"{key}", full_name) or
Comment on lines +772 to +773
Copy link
Copy Markdown
Contributor

@gemini-code-assist gemini-code-assist Bot Feb 10, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The check re.match(f\"{key}\\\\.\", full_name) is redundant. If this condition is true, full_name starts with key + '.', which implies it also starts with key. Therefore, the subsequent condition re.match(f\"{key}\", full_name) will also be true. You can remove this redundant check to simplify the code.

            re.match(f\"{key}\", full_name) or

full_name.endswith(key) or
f".{key}." in f".{full_name}."
for key in patterns
)
return not should_not_convert

_quantizers_utils.should_convert_module = _unsloth_should_convert_module
# Also patch the imported reference in bitsandbytes module
if hasattr(transformers.integrations.bitsandbytes, "should_convert_module"):
transformers.integrations.bitsandbytes.should_convert_module = _unsloth_should_convert_module
pass

# Unsloth Zoo - Utilities for Unsloth
# Copyright 2023-present Daniel Han-Chen, Michael Han-Chen & the Unsloth team. All rights reserved.
#
Expand Down