fix: prevent HuggingFace access when SGLANG_USE_MODELSCOPE is enabled#8286
Closed
yankay wants to merge 1 commit intosgl-project:mainfrom
Closed
fix: prevent HuggingFace access when SGLANG_USE_MODELSCOPE is enabled#8286yankay wants to merge 1 commit intosgl-project:mainfrom
yankay wants to merge 1 commit intosgl-project:mainfrom
Conversation
Contributor
|
Warning You have reached your daily quota limit. Please wait up to 24 hours and I will start processing your requests again! |
Contributor
Author
|
HI @merrymercy |
|
Ping for a review. |
c324027 to
72c73f1
Compare
4 tasks
Signed-off-by: Kay Yan <kay.yan@daocloud.io>
72c73f1 to
974151e
Compare
Contributor
Author
|
HI @Fridge003 and @Kangyan-Zhou Could you please help review it again? |
Contributor
Author
|
close because of #12039 merged |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
When SGLANG_USE_MODELSCOPE=true is set, the system was still making calls to HuggingFace endpoints during model configuration loading, which could lead to:
Fix: #8275
Modifications
Referenced Vllm: vllm-project/vllm#5233
This PR fixes the issue by adding conditional import logic for AutoConfig and HfApi.
The problem can be reliably reproduced by setting
HF_ENDPOINT: https://127.0.0.1, so a test case has been modified to ensure no HuggingFace access occurs whenSGLANG_USE_MODELSCOPEis enabled.Checklist