Skip to content

Fix Mistral-native format detection when the Hub is unreachable #43446

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
alisonshao merged 2 commits into from
Oct 12, 2026
Merged

Fix Mistral-native format detection when the Hub is unreachable #43446

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
3b67801
Fix Mistral-native format detection when the Hub is unreachable
alisonshao Oct 10, 2026
d13a032
Merge branch 'main' into fix-mistral-native-detect-offline
alisonshao Oct 11, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
29 changes: 16 additions & 13 deletions python/sglang/srt/arg_groups/model_path_hook.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@
resolving_view,
run_post_process_pass,
)
from sglang.srt.utils.common import is_remote_url
from sglang.srt.utils.common import find_local_repo_dir, is_remote_url
from sglang.srt.utils.hf_transformers_utils import check_gguf_file
from sglang.srt.utils.runai_utils import ObjectStorageModel, is_runai_obj_uri

Expand Down Expand Up @@ -351,16 +351,19 @@ def _check_format(has_params, has_consolidated, has_hf_weights) -> bool:
from huggingface_hub import HfApi

files = {s.rfilename for s in HfApi().model_info(cfg.model_path).siblings}
return _check_format(
has_params="params.json" in files,
has_consolidated=any(
f.startswith("consolidated") and f.endswith(".safetensors")
for f in files
),
has_hf_weights=any(
f.startswith("model") and f.endswith(".safetensors") and "/" not in f
for f in files
),
)
except Exception:
return False
# Hub unreachable (offline, rate-limited): judge from the local snapshot.
snapshot_dir = find_local_repo_dir(cfg.model_path, cfg.revision)

@ch-wan ch-wan Oct 10, 2026

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[suggestion] The new fallback passes cfg.revision into find_local_repo_dir, but that helper only resolves a revision when it is falsy (None → read refs/main) or when snapshots/<revision> already exists as a directory (commit SHA). A branch or tag string such as "main" or "v1.0" is truthy, so the helper looks for snapshots/main (which HF cache does not create) and returns None, and the fallback silently returns False — same broken offline behavior the PR is fixing. Default revision=None (the CI case) is fine; explicit --revision main / tag while Hub is unreachable is not.

Suggestion: Before treating a miss as fatal, resolve non-SHA revisions through the cache refs (e.g. try refs/<revision> / refs/main then snapshots/<sha>), or call an HF helper that already does that (snapshot_download(..., local_files_only=True) / equivalent). Alternatively, if you only intend to support the default-revision offline path, pass revision=None here and note that branch/tag offline detection remains best-effort.

@alisonshao alisonshao Oct 10, 2026

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is fixed in pr 43229. find_local_repo_dir now reads refs/ for any revision

if snapshot_dir is None:
return False
files = set(os.listdir(snapshot_dir))
return _check_format(
has_params="params.json" in files,
has_consolidated=any(
f.startswith("consolidated") and f.endswith(".safetensors") for f in files
),
has_hf_weights=any(
f.startswith("model") and f.endswith(".safetensors") and "/" not in f
for f in files
),
)
Loading