import_fixes: stub transformers.conversion_mapping so peft 0.19.x imports on transformers 4.x

[danielhanchen]

What's broken

On the (peft 0.19.x, transformers 4.57.x) combination,

from peft.utils import transformers_weight_conversion

raises

ModuleNotFoundError: No module named 'transformers.conversion_mapping'

because peft's transformers_weight_conversion module unconditionally imports two transformers-v5 submodules at module top (transformers.conversion_mapping and transformers.core_model_loading), neither of which exists on transformers < 5. peft itself only USES those submodules inside an is_transformers_ge_v5 branch, but the top-of-file import still explodes the moment anything tries to load the module.

Our existing patch_peft_weight_converter_compatibility (unsloth/import_fixes.py lines 1375-1454) opens with

try:
    from peft.utils import transformers_weight_conversion as twc
except (ImportError, AttributeError):
    return

so the bare except silently no-ops, the kwargs compat wrap never gets installed, and any downstream code that later does from peft.utils import transformers_weight_conversion blows up with the same ModuleNotFoundError.

What this PR does

Adds fix_peft_transformers_weight_conversion_import to unsloth/import_fixes.py. When (and only when) the import is currently broken AND the underlying transformers really is missing those two submodules, it injects minimal stub modules into sys.modules exposing the symbols peft pulls in at module top. Then it forces a fresh import of peft.utils.transformers_weight_conversion so the existing patch_peft_weight_converter_compatibility (the kwargs compat wrap) can run on top.

Concatenate and ConversionOps are real subclassable classes since peft subclasses them as PeftConcatenate / FlattenDims / PermuteDims at module top, so peft's own class creation succeeds. None of the stubbed callables actually fire on the 4.x branch because peft's runtime is_transformers_ge_v5 gate keeps them unreachable.

Wired into unsloth/_gpu_init.py to run BEFORE patch_peft_weight_converter_compatibility (otherwise that function's bare except would still silently no-op).

This mirrors the equivalent fix shipping in unsloth-zoo (zoo applies its own copy at apply_import_fixes() time), but a user can run unsloth against an older unsloth_zoo that doesn't have the workaround, so the unsloth side needs to own a copy too.

Gating contract

Strict no-op outside the (peft 0.19.x, transformers 4.x) combination:

• No-op if peft is not installed.
• No-op if peft.utils.transformers_weight_conversion already imports clean (transformers v5+, or any peft fork off the v5 path).
• Strictly additive: only stubs submodules that are currently missing from sys.modules / find_spec. We never overwrite the real transformers.conversion_mapping / transformers.core_model_loading on transformers v5+.
• Idempotent: a sentinel attribute (__unsloth_stub__) on the stub modules makes a second call return False, a third call return False, and so on.
• Surfaces drift unchanged: if peft fails for some reason OTHER than these two specific missing submodules, the original ImportError is left for the caller's own try / except to take over.

Compatibility matrix

• transformers 4.57.6 (no submodule) -> install stubs, peft imports clean.
• transformers 5.x (real submodule) -> first-import probe succeeds, return False, never touch sys.modules.
• TRL 0.22 / 0.27 / 1.x -- none import either submodule directly; they reach the peft conversion module (if at all) through peft.tuners.tuners_utils, behind peft's own is_transformers_ge_v5 gate. Stubs are therefore unreachable from TRL on a 4.x install, and on a 5.x install the real submodules win the import race.
• peft 0.18 / 0.19 / 0.20 -- the symbols stubbed cover the union of what peft pulls at module top across the 0.19.x line; older peft that doesn't import the v5 submodules at all hits the cheap first-import-probe exit and we never touch sys.modules.
• torch 2.4 - 2.11, vllm latest -- none of these paths participate in the stub injection.

Tests

tests/conftest.py is updated to pre-apply this specific fix via the standalone import-fixes module so the GPU-free drift detector test (tests/test_import_fixes_drift.py::test_peft_transformers_weight_conversion_importable_and_signature, shipping in a sibling PR) sees the same patched state a real import unsloth would. The pattern mirrors unsloth-zoo's tests/conftest.py _apply_zoo_import_fixes_for_tests helper, scoped to just the peft fix.

Local verification on (peft 0.19.1, transformers 4.57.6, torch 2.9.1+cu128):

$ python3 -m pytest tests/test_import_fixes_drift.py::test_peft_transformers_weight_conversion_importable_and_signature -v
tests/test_import_fixes_drift.py::test_peft_transformers_weight_conversion_importable_and_signature PASSED
============================== 1 passed in 0.01s ===============================

Full suite: 16/18 pass. The two remaining failures (test_triton_compiled_kernel_has_num_ctas_and_cluster_dims, test_vllm_guided_decoding_params_or_structured_outputs_present) are independent of this PR -- they were failing before our changes for unrelated drift items (triton 3.6+ CompiledKernel shape, vLLM PR #22772 GuidedDecodingParams rename) that this PR does not address. They are also handled by the equivalent zoo fixes already shipping, and can be wired into the same conftest helper in a follow-up.

End-to-end smoke (import unsloth -> existing patch_peft_weight_converter_compatibility actually installs):

>>> import unsloth
>>> from peft.utils import transformers_weight_conversion as twc
>>> twc._unsloth_weight_converter_compat_patch
True

Idempotence:

>>> r1 = fix_peft_transformers_weight_conversion_import()  # True (applied)
>>> r2 = fix_peft_transformers_weight_conversion_import()  # False (no-op)
>>> r3 = fix_peft_transformers_weight_conversion_import()  # False (no-op)

Transformers v5+ simulation (real submodules pre-installed, no sentinel):

>>> result = fix_peft_transformers_weight_conversion_import()  # False (cheap exit)
>>> getattr(sys.modules['transformers.conversion_mapping'], '__unsloth_stub__', False)
False  # real module untouched

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: c06ab2c3ba

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[chatgpt-codex-connector]

Preserve the guarded AttributeError fallback

In environments where peft.utils.transformers_weight_conversion raises an AttributeError during import because of another upstream drift, import unsloth now fails here before reaching patch_peft_weight_converter_compatibility(), even though that existing patch deliberately catches AttributeError and no-ops. Please treat non-target AttributeErrors the same as before, e.g. catch it in this probe and return False, so this new workaround only changes the specific missing-transformers-submodule case.

Useful? React with 👍 / 👎.

[gemini-code-assist]

Code Review

This pull request addresses a compatibility issue between peft 0.19.x and transformers 4.x by stubbing missing transformers submodules (conversion_mapping and core_model_loading) required for peft imports. The fix is implemented in unsloth/import_fixes.py and integrated into both the GPU initialization process and the test suite configuration. Review feedback highlights a discrepancy in the fix_peft_transformers_weight_conversion_import docstring regarding its return value when the patch is already applied. Additionally, it is recommended to dynamically generate the log message to accurately report which specific modules were stubbed.

[gemini-code-assist]

The docstring states that the function returns True if the patch "had been applied previously". However, the implementation at line 1666 returns False if the module already imports cleanly (which is the case if the patch was applied in a previous call). This contradiction should be resolved by updating the docstring to reflect that it returns False if no action was taken in the current call.

[gemini-code-assist]

The log message unconditionally states that both transformers.conversion_mapping and transformers.core_model_loading were stubbed. However, the logic in step 4 (lines 1700-1706) allows for stubbing only one of them if the other is already present. It would be more accurate to dynamically generate the message based on which modules were actually patched, or use a more general phrasing.

    if patched_any:
        stubbed_names = []
        if sys.modules.get("transformers.conversion_mapping").__dict__.get(_UNSLOTH_STUB_SENTINEL):
            stubbed_names.append("transformers.conversion_mapping")
        if sys.modules.get("transformers.core_model_loading").__dict__.get(_UNSLOTH_STUB_SENTINEL):
            stubbed_names.append("transformers.core_model_loading")
        
        logger.info(
            f"Unsloth: stubbed {' / '.join(stubbed_names)} so peft.utils."
            "transformers_weight_conversion imports cleanly on "
            "transformers <5."
        )

References

1. User-facing warning messages should be dynamically generated to include the specific configuration values they refer to, rather than using hardcoded examples, to ensure accuracy and avoid confusion.

[danielhanchen]

FYI - I just opened unslothai/unsloth-zoo#639 to remove unsloth_zoo/import_fixes.py from zoo. Six of the seven fix_* / patch_* functions in that file mirror identically-named functions on unsloth/import_fixes.py; the seventh - fix_peft_transformers_weight_conversion_import - is the novel one this PR ports to unsloth. Once this merges zoo will lean on unsloth's copy alone, which removes the drift surface between the two mirrors.

The zoo PR's tests/conftest.py now triggers import unsloth so the GPU-free test harness still sees the patched state, and the drift detectors continue to fire on real upstream regressions. No behaviour change at runtime: unsloth_zoo/__init__.py already raises ImportError unless find_spec("unsloth") succeeds, so unsloth's import_fixes.py has always run before zoo gets a chance to apply its mirror copy.