Guard optional vLLM imports when extension is broken

[danielhanchen]

Summary

• add a dynamic vLLM blocker similar to disable_broken_causal_conv1d
• when vLLM is detected as ABI-broken, mark it as unavailable by:
  • clearing vllm* entries from sys.modules
  • patching importlib.util.find_spec to hide vllm
  • installing a sys.meta_path blocker that raises ModuleNotFoundError for vllm*
• wire fix_vllm_guided_decoding_params to trigger this blocker on broken-extension ImportError/OSError
• call disable_broken_vllm() early in unsloth/__init__.py so broken optional vLLM does not crash import unsloth

Why

In environments where vllm/_C.abi3.so is incompatible with the installed torch/CUDA stack, import-time failures should degrade gracefully for non-vLLM workflows, similar to the causal-conv1d fallback behavior.

Validation

• Runtime repro check in the same broken-vLLM environment:
  • PYTHONPATH=<patched-branch> python -u -c "import importlib.util, unsloth; print(importlib.util.find_spec('vllm')); import vllm"
  • observed behavior:
    • import unsloth succeeds
    • find_spec('vllm') returns None
    • import vllm raises ModuleNotFoundError

[gemini-code-assist]

Summary of Changes

Hello @danielhanchen, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request significantly improves the robustness of Unsloth's import mechanism, particularly when users have an installed vLLM library with a broken or incompatible native extension. By gracefully handling import errors related to vLLM's binary components and switching to lazy loading for vLLM utilities, Unsloth can now import successfully without being blocked by an optional, misconfigured dependency. This ensures a smoother user experience, allowing core Unsloth functionalities to remain accessible even if vLLM is not perfectly set up, while still providing clear warnings when vLLM-specific features might be unavailable.

Highlights

• Robust vLLM Imports: Prevented optional vLLM import failures from crashing import unsloth when vLLM is installed but its native extension is ABI-incompatible.
• Error Handling in fix_vllm_guided_decoding_params: Modified fix_vllm_guided_decoding_params to catch ImportError and OSError related to broken vLLM extensions, logging a warning and skipping the compatibility patch instead of hard-failing import.
• Robustness in fix_vllm_pdl_blackwell: Enhanced fix_vllm_pdl_blackwell to make module spec checks robust against ImportError and OSError that can occur during vLLM submodule resolution.
• Lazy Imports for vllm_utils: Converted eager unsloth_zoo.vllm_utils imports to lazy imports in unsloth/models/loader_utils.py, unsloth/dataprep/synthetic.py, and unsloth/models/vision.py to avoid import-time crashes.

🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console.

Changelog

• tests/test_import_fixes_vllm_guided_decoding.py
  • Added new test cases to verify that fix_vllm_guided_decoding_params skips when a broken vLLM binary import occurs.
  • Included a test to confirm that fix_vllm_guided_decoding_params still raises an error for transformers-vLLM mismatch.
  • Added a test to ensure fix_vllm_pdl_blackwell handles broken vLLM spec checks without raising exceptions.
• unsloth/dataprep/synthetic.py
  • Removed eager top-level imports of load_vllm, patch_vllm, and delete_vllm from unsloth_zoo.vllm_utils.
  • Introduced a new helper function _load_vllm_utils to lazily import vLLM utilities.
  • Updated the __init__ method to call _load_vllm_utils for lazy loading.
  • Modified the cleanup method to conditionally call _delete_vllm only if it has been loaded.
• unsloth/import_fixes.py
  • Implemented _is_broken_vllm_extension to detect specific error patterns indicating a broken vLLM native extension.
  • Added _warn_and_skip to log a warning and return early when a broken vLLM extension is detected, preventing hard failures.
  • Modified fix_vllm_guided_decoding_params to catch OSError in addition to ImportError and apply the new warning/skip logic.
  • Expanded the except clause in _spec_exists within fix_vllm_pdl_blackwell to also catch ImportError and OSError for increased robustness.
• unsloth/models/loader_utils.py
  • Removed the eager top-level import of _get_torchao_fp8_config from unsloth_zoo.vllm_utils.
  • Created a new wrapper function _get_torchao_fp8_config that lazily imports the actual implementation from unsloth_zoo.vllm_utils.
• unsloth/models/vision.py
  • Removed unused eager top-level imports of convert_lora_modules and return_lora_modules from unsloth_zoo.vllm_utils.

Activity

• New test cases were added and passed, specifically tests/test_import_fixes_vllm_guided_decoding.py reported '3 passed'.
• A runtime sanity check was performed, confirming that import unsloth now completes successfully even with a broken vLLM installation, logging a warning as expected.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[gemini-code-assist]

Code Review

This pull request provides a robust solution to prevent import unsloth from crashing when an optional dependency, vLLM, is installed but has a broken native extension. The changes are well-implemented, using lazy loading for vLLM utilities to defer imports until they are actually needed, and adding careful error handling to catch and gracefully manage import failures. The new logic correctly distinguishes between critical configuration mismatches (like with transformers) that should still fail, and ABI-incompatibility issues that can be safely bypassed with a warning. The addition of a new test suite with clever use of mocking and AST parsing ensures the new import guarding mechanisms are thoroughly validated. Overall, this is an excellent improvement that significantly enhances the robustness and user experience of the library.

[Datta0]

Why are these scoped within fix_vllm_guided_decoding_params and not outside?