Fix is_contiguous() method call and remove duplicate imports

[yurekami]

Summary

• Fix bug in rope_embedding.py where is_contiguous was used without parentheses, causing the method object (always truthy) to be evaluated instead of calling the method
• Remove duplicate imports in rl.py and vision.py

Bug Fix (Fixes #3781)

The is_contiguous method was used without parentheses on 4 lines in Fast_RoPE_Embedding_QK:

# Before (BUG - always evaluates to True):
Q_out = Q.clone() if not Q.is_contiguous else Q

# After (FIXED):
Q_out = Q.clone() if not Q.is_contiguous() else Q

Since is_contiguous is a method (not a property), the condition always evaluated to True (method objects are truthy), meaning clones were never made for non-contiguous tensors. This caused fast rope backpropagation to fail with zero-strided tensors.

Files Changed

• unsloth/kernels/rope_embedding.py - Fixed 4 instances (lines 315, 316, 398, 399)
• unsloth/models/rl.py - Removed duplicate import torch
• unsloth/models/vision.py - Removed duplicate import functools and import types

Test plan

• Verified Python syntax is valid
• Confirmed no other instances of the bug exist in the codebase
• Test with non-contiguous tensors in RoPE forward/backward pass

🤖 Generated with Claude Code

[gemini-code-assist]

Summary of Changes

Hello @yurekami, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request primarily addresses a critical bug in the rope_embedding.py kernel where the is_contiguous method was mistakenly treated as a property. This oversight prevented proper cloning of non-contiguous tensors, which could cause issues during backpropagation. Additionally, the PR includes minor code clean-up by eliminating duplicate import statements in rl.py and vision.py, enhancing overall code quality and maintainability.

Highlights

• Bug Fix: is_contiguous() Method Call: Corrected the usage of is_contiguous from a property to a method call (is_contiguous()) in unsloth/kernels/rope_embedding.py. This resolves a bug where non-contiguous tensors were not being cloned, leading to potential backpropagation failures with zero-strided tensors.
• Code Clean-up: Duplicate Imports: Removed redundant import torch from unsloth/models/rl.py and duplicate import functools and import types from unsloth/models/vision.py, improving code cleanliness.

🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[gemini-code-assist]

Code Review

This pull request correctly fixes a bug where is_contiguous was used as a property instead of a method, and also removes some duplicate imports. The changes are good, but I have a suggestion to make the bug fix more robust and idiomatic.

In rope_embedding.py, the fix uses Q.clone() if not Q.is_contiguous() else Q. However, Q.clone() does not guarantee a contiguous tensor. A better approach is to use Q.contiguous(), which is the standard way to ensure a tensor is contiguous, creating a copy only when necessary. I've added comments with code suggestions to apply this improvement.

[gemini-code-assist]

While adding the parentheses fixes the immediate bug of not calling is_contiguous, the logic can be improved for correctness and clarity.

Q.clone() does not guarantee that the resulting tensor is contiguous. If Q is not contiguous, Q.clone() will also be non-contiguous by default (preserving memory format).

A more idiomatic and correct way to ensure a tensor is contiguous is to use torch.Tensor.contiguous(). This method returns the tensor itself if it's already contiguous, and creates a contiguous copy otherwise. This achieves the goal of avoiding an unnecessary copy while ensuring the output tensor is always contiguous for the Triton kernel.

I suggest replacing these lines with:

Suggested change

	Q_out = Q.clone() if not Q.is_contiguous() else Q
	K_out = K.clone() if not K.is_contiguous() else K
	Q_out = Q.contiguous()
	K_out = K.contiguous()

[gemini-code-assist]

Similar to my previous comment, using .contiguous() is the more correct and idiomatic way to ensure the tensors are contiguous for the kernel. clone() does not guarantee contiguity.

Suggested change

	dQ_out = dQ.clone() if not dQ.is_contiguous() else dQ
	dK_out = dK.clone() if not dK.is_contiguous() else dK
	dQ_out = dQ.contiguous()
	dK_out = dK.contiguous()

[danielhanchen]

Thanks @yurekami ! I guess this is superseded by #3790 ?