Skip to content

[Bugfix] Enforce installed FlashInfer version to match optional requirement#24671

Closed
mgoin wants to merge 1 commit intovllm-project:mainfrom
neuralmagic:enforce-flashinfer-version
Closed

[Bugfix] Enforce installed FlashInfer version to match optional requirement#24671
mgoin wants to merge 1 commit intovllm-project:mainfrom
neuralmagic:enforce-flashinfer-version

Conversation

@mgoin
Copy link
Copy Markdown
Member

@mgoin mgoin commented Sep 11, 2025

Purpose

Previously there was no enforcement on the installed version of FlashInfer, just that the imported function exists. This enforcement should help guide users to install FI using the official pathway and reduce user error when FI function behavior changes.

Test Plan

Test Result


Essential Elements of an Effective PR Description Checklist
  • The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
  • The test plan, such as providing test command.
  • The test results, such as pasting the results comparison before and after, or e2e results
  • (Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model.
  • (Optional) Release notes update. If your change is user facing, please update the release notes draft in the Google Doc.

@mergify mergify bot added the ci/build label Sep 11, 2025
Copy link
Copy Markdown
Contributor

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request effectively addresses the need to enforce a specific version of FlashInfer. By centralizing the version number in vllm/version.py and using it in setup.py, you've improved maintainability. The addition of a runtime version check in vllm/utils/flashinfer.py is a great step towards preventing user errors and ensuring compatibility. I have one suggestion to make the version validation even more robust.

Comment on lines +56 to +57
except (ImportError, AttributeError):
return False
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

The try...except block does not handle a potential ValueError (or packaging.version.InvalidVersion) that can be raised by packaging.version.parse() if flashinfer.__version__ is not a valid PEP 440 version string. An unhandled exception here would crash the application during initialization. It's safer to catch this exception to prevent such crashes.

Suggested change
except (ImportError, AttributeError):
return False
except (ImportError, AttributeError, ValueError):
return False

# Parse both versions to handle version comparison properly
expected = parse(FLASHINFER_VERSION)
actual = parse(flashinfer.__version__)
return actual == expected
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not >=?

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agree >= would work

Copy link
Copy Markdown
Collaborator

@pavanimajety pavanimajety left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, I agree enforcing FI version is good considering we test a given commit / release of vLLM against a specific FI version.

@mergify
Copy link
Copy Markdown
Contributor

mergify bot commented Sep 16, 2025

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @mgoin.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

@mergify mergify bot added the needs-rebase label Sep 16, 2025
@mgoin
Copy link
Copy Markdown
Member Author

mgoin commented Oct 9, 2025

Not needed with #26443

@mgoin mgoin closed this Oct 9, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants