Fix: prevent load_in_fp8 kwarg from reaching Qwen3MoeForCausalLM constructor (Fix #3649)

[bhuvanprakash]

PR Description

This PR fixes the issue reported in #3649 where calling:

FastModel.from_pretrained(
    "unsloth/Qwen3-30B-A3B-Instruct-2507-FP8",
    load_in_fp8=True,
)

TypeError: Qwen3MoeForCausalLM.init() got an unexpected keyword argument 'load_in_fp8'
The problem was that load_in_fp8 was an internal flag for Unsloth, but it was still being passed to the Hugging Face model constructor and models like Qwen3MoeForCausalLM do not accept that argument -->which caused the crash.

---

What I changed

The parameter is now removed from the location where the model is passed to the HF loader:

load_in_fp8 = kwargs.pop("load_in_fp8", None)

I kept the value around if it's needed later, but it no longer gets to the HF constructor.

---

Notes

I don’t have FP8 hardware to perform a full training run, so the fix is based on the reading the execution path and matching it with the similar logic that’s already implemented in the codebase fixes (like the change in PR #3621).
if additional adjustments are needed, I’m happy to update the PR.

Fixes​‍​‌‍​‍‌​‍​‌‍​‍‌ #3649

[gemini-code-assist]

Summary of Changes

Hello @bhuvanprakash, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request addresses a critical bug where an internal Unsloth flag, load_in_fp8, was being incorrectly passed to the underlying Hugging Face model constructor during model loading. This resulted in a TypeError for models that do not recognize this argument. The fix ensures that this specific argument is intercepted and removed before reaching the Hugging Face model, thereby resolving the loading issue and allowing models to be loaded correctly.

Highlights

• Bug Fix: Resolves a TypeError that occurred when load_in_fp8 was passed to the Qwen3MoeForCausalLM constructor, which does not accept this argument.
• Argument Handling: The load_in_fp8 keyword argument is now explicitly removed from the kwargs dictionary before being passed to the Hugging Face model's from_pretrained method, preventing it from reaching constructors that do not expect it.
• Internal Flag Management: Ensures that load_in_fp8, an internal Unsloth flag, is properly handled and not inadvertently forwarded to external library constructors.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[gemini-code-assist]

Code Review

This pull request correctly addresses a TypeError that occurs when using the load_in_fp8 argument. The issue was that this internal Unsloth flag was being passed down to the Hugging Face model constructor, which does not recognize it. The fix involves popping the load_in_fp8 argument from the keyword arguments before calling the Hugging Face from_pretrained method. This is a clean and effective solution that prevents the unexpected keyword argument from causing a crash. The change is consistent with similar fixes in the codebase for other internal flags. I've reviewed the change and it looks good to me.

[danielhanchen]

Oh thank you!!