Skip to content

Fix _patch_transformers_hybrid_cache for peft #4844

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
albertvillanova merged 1 commit into from
Jan 16, 2026
+28 −2
Merged

Fix _patch_transformers_hybrid_cache for peft #4844

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
30 changes: 28 additions & 2 deletions trl/_compat.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -180,9 +180,35 @@ def _patch_transformers_hybrid_cache() -> None:
_is_package_version_below("liger_kernel", "0.6.5") or _is_package_version_below("peft", "0.18.0")
):
try:
import transformers.cache_utils as cache_utils
import transformers.cache_utils
from transformers.utils.import_utils import _LazyModule

Cache = transformers.cache_utils.Cache

# Patch for liger_kernel: Add HybridCache as an alias for Cache in the cache_utils module
transformers.cache_utils.HybridCache = Cache

# Patch for peft: Patch _LazyModule.__init__ to add HybridCache to transformers' lazy loading structures
_original_lazy_module_init = _LazyModule.__init__

def _patched_lazy_module_init(self, name, *args, **kwargs):
_original_lazy_module_init(self, name, *args, **kwargs)
if name == "transformers":
# Update _LazyModule's internal structures
if hasattr(self, "_import_structure") and "cache_utils" in self._import_structure:
if "HybridCache" not in self._import_structure["cache_utils"]:
self._import_structure["cache_utils"].append("HybridCache")

if hasattr(self, "_class_to_module"):
self._class_to_module["HybridCache"] = "cache_utils"

if hasattr(self, "__all__") and "HybridCache" not in self.__all__:
self.__all__.append("HybridCache")

self.HybridCache = Cache

_LazyModule.__init__ = _patched_lazy_module_init
Comment on lines +192 to +210

@qgallouedec qgallouedec Jan 16, 2026 •
edited
Loading

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm fine with this solution, but any reason not to simply use?

transformers.HybridCache = Cache

is it to leverage lazy import?

@albertvillanova albertvillanova Jan 16, 2026

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@qgallouedec, that was my first approach, but it did not work: after the patch, when peft imports transformers, the transformers module was being replaced with a fresh _LazyModule instance that didn't have the HybridCache patch applied. This happened because:

  1. Transformers' __init__.py replaces itself with a _LazyModule at initialization
  2. When peft imported transformers, this caused the module to be re-initialized
  3. The new _LazyModule instance didn't have our HybridCache patch


cache_utils.HybridCache = cache_utils.Cache
except Exception as e:
warnings.warn(f"Failed to patch transformers HybridCache compatibility: {e}", stacklevel=2)

Expand Down
Loading