Repository navigation
fix(ci): resolve flashinfer-jit-cache index to the nearest published CUDA tag - #2086
Merged
Merged
Conversation
slin1237
requested review from
CatherineSue,
XinyueZhang369,
gongwei-130 and
key4ng
as code owners
August 10, 2026 22:08
📝 WalkthroughSummary by CodeRabbit
WalkthroughThe SGLang and vLLM CI installation scripts now find the nearest published FlashInfer CUDA wheel index within the same CUDA major version before installing the JIT cache. ChangesFlashInfer CUDA index fallback
Estimated code review effort: 2 (Simple) | ~10 minutes Possibly related PRs
Suggested labels: Suggested reviewers: 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
|
👋 The PR description doesn't fully follow
Please update the PR description so reviewers have the context they need. |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@scripts/ci_install_sglang.sh`:
- Around line 56-62: Both installers currently run the fallback after any failed
uv pip install, masking transient and other errors. Update the installation
logic in scripts/ci_install_sglang.sh lines 56-62 and scripts/ci_install_vllm.sh
lines 90-96 to use one validated policy: only try the base version for
FLASHINFER_VERSION values matching .postN and confirmed missing distributions;
propagate network, authentication, metadata, non-post, and release-candidate
failures without retrying the same specification.
🪄 Autofix
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro Plus
Run ID: d781487d-23a9-46d1-a0fa-4f0c36170f0e
📒 Files selected for processing (2)
scripts/ci_install_sglang.shscripts/ci_install_vllm.sh
…CUDA tag Every vLLM e2e lane dies in setup since torch resolved to 2.13.0+cu132: the install scripts derive the flashinfer wheel index from torch.version.cuda, but flashinfer hosts one index per CUDA tag and its newest is cu130 - https://flashinfer.ai/whl/cu132 404s, and uv reports a missing index as "flashinfer-jit-cache was not found in the package registry" (first hit: main run 31430981291 at a600833; the .post3 in that message was a red herring - 0.6.16.post3 exists on cu130). CUDA minor versions are ABI-compatible, so probe the index URL and walk down to the nearest published tag within the same major (cu132 -> cu130 today; a same-major floor stops the walk from crossing into cu12x). Applied to both ci_install_vllm.sh and ci_install_sglang.sh - sglang's torch is still cu130 so its lanes pass today, but it carries the same latent assumption. Verified against the live index: the walk resolves 132 -> 130 and leaves 130 untouched. Signed-off-by: Simo Lin <25425177+slin1237@users.noreply.github.com>
slin1237
force-pushed
the
fix/flashinfer-jit-cache-post-release
branch
from
August 10, 2026 23:00
e710920 to
cfe9468
Compare
slin1237
changed the title
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
Comment on lines
+95
to
+98
| while [ "${CUDA_TAG}" -gt "${CUDA_MAJOR_FLOOR}" ] \ | ||
| && ! curl -sfo /dev/null "https://flashinfer.ai/whl/cu${CUDA_TAG}/flashinfer-jit-cache/"; do | ||
| CUDA_TAG=$((CUDA_TAG - 1)) | ||
| done |
There was a problem hiding this comment.
🟡 Nit: The curl probes have no timeout. If flashinfer.ai is unresponsive (not down — just slow), each loop iteration blocks on curl's default unlimited timeout, and the walk could stall CI until the job-level timeout kills it. Consider adding --connect-timeout 5 --max-time 10 to bound each probe. Same applies to the identical loop in scripts/ci_install_sglang.sh:61-64.
Suggested change
| while [ "${CUDA_TAG}" -gt "${CUDA_MAJOR_FLOOR}" ] \ | |
| && ! curl -sfo /dev/null "https://flashinfer.ai/whl/cu${CUDA_TAG}/flashinfer-jit-cache/"; do | |
| CUDA_TAG=$((CUDA_TAG - 1)) | |
| done | |
| CUDA_MAJOR_FLOOR=$((CUDA_TAG / 10 * 10)) | |
| while [ "${CUDA_TAG}" -gt "${CUDA_MAJOR_FLOOR}" ] \ | |
| && ! curl -sfo /dev/null --connect-timeout 5 --max-time 10 "https://flashinfer.ai/whl/cu${CUDA_TAG}/flashinfer-jit-cache/"; do | |
| CUDA_TAG=$((CUDA_TAG - 1)) |
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@scripts/ci_install_sglang.sh`:
- Around line 61-65: Update the CUDA index-selection loops in
scripts/ci_install_sglang.sh lines 61-65 and scripts/ci_install_vllm.sh lines
94-98 to validate that each candidate index contains a compatible
flashinfer-jit-cache==${FLASHINFER_VERSION} wheel for the runner platform,
rather than accepting any successful HTTP response. Preserve the fallback
decrement behavior, and fail explicitly when no compatible CUDA index is found.
- Around line 61-64: Update the FlashInfer probe loops in
scripts/ci_install_sglang.sh lines 61-64 and scripts/ci_install_vllm.sh lines
95-98 so both use connection and total-operation timeouts and capture HTTP
status separately from curl transport errors. Decrement the CUDA tag only when
the probe returns HTTP 404, exit immediately for other HTTP statuses or
transport failures, and ensure the CUDA_MAJOR_FLOOR validation still runs after
all eligible tags are exhausted.
🪄 Autofix
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro Plus
Run ID: 07710358-e3be-4081-9779-93e8b952ef08
📒 Files selected for processing (2)
scripts/ci_install_sglang.shscripts/ci_install_vllm.sh
Comment on lines
+61
to
+64
| while [ "${CUDA_TAG}" -gt "${CUDA_MAJOR_FLOOR}" ] \ | ||
| && ! curl -sfo /dev/null "https://flashinfer.ai/whl/cu${CUDA_TAG}/flashinfer-jit-cache/"; do | ||
| CUDA_TAG=$((CUDA_TAG - 1)) | ||
| done |
There was a problem hiding this comment.
🩺 Stability & Availability | 🟠 Major | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
set -eu
printf '%s\n' '--- target files and line ranges ---'
for f in scripts/ci_install_sglang.sh scripts/ci_install_vllm.sh; do
printf '\n### %s\n' "$f"
wc -l "$f"
sed -n '35,115p' "$f"
done
printf '%s\n' '--- curl and timeout usage in target scripts ---'
rg -n -C 2 'curl|timeout|CURL|set -|CUDA_TAG|CUDA_MAJOR_FLOOR' \
scripts/ci_install_sglang.sh scripts/ci_install_vllm.sh
printf '%s\n' '--- references and likely tests ---'
rg -n -C 2 'ci_install_sglang|ci_install_vllm|flashinfer-jit-cache|CUDA_MAJOR_FLOOR|CUDA_TAG' \
. --glob '!node_modules' --glob '!dist' --glob '!build' || trueRepository: smg-project/smg
Length of output: 16635
🌐 Web query:
curl man page --fail HTTP 400 errors --connect-timeout --max-time default timeout behavior
💡 Result:
The curl command-line utility behaves as follows regarding the requested options and behaviors: --fail (-f): When this option is used, curl will exit with error code 22 if an HTTP server returns an error response code of 400 or greater [1][2]. By default, curl does not consider HTTP response codes to indicate failure and will output the server's response body regardless of the status code [1][3]. If you wish to see the error response body while still triggering an error exit, use --fail-with-body instead [4][1]. --connect-timeout: This option limits the time allowed for the connection phase (including DNS resolution and TCP/TLS/QUIC handshakes) [5][6]. If the connection is not established within the specified time, curl exits with error code 28 [5]. If this option is not set, curl relies on the default built-in timeout, which is typically 300 seconds (5 minutes) [7][6]. --max-time (-m): This option sets the maximum time, in seconds, allowed for the entire operation [8][9]. If this time is exceeded, curl exits with error code 28 [8]. By default, curl has no maximum time limit for an operation and will wait indefinitely for a server response unless a connection timeout is triggered or the server closes the connection [8][10]. In summary, if no timeout options are explicitly provided, curl generally waits indefinitely for data once a connection is established, though individual operations may still be subject to system-level or network-level limits [8][10].
Citations:
- 1: https://curl.se/docs/manpage.html
- 2: https://man.archlinux.org/man/curl.1
- 3: https://manpages.debian.org/curl(1)
- 4: https://man7.org/linux/man-pages/man1/curl.1.html
- 5: https://github.com/curl/everything-curl/blob/master/usingcurl/timeouts.md
- 6: https://curl.se/libcurl/c/CURLOPT_CONNECTTIMEOUT.html
- 7: https://github.com/curl/curl/blob/500ea908/lib/connect.h
- 8: https://everything.curl.dev/usingcurl/timeouts.html
- 9: https://github.com/curl/curl/blob/c37405cb/docs/cmdline-opts/max-time.md
- 10: curl -m / --max-time : not working. Connection timeout default MAX to 15mins. curl/curl#4562
🏁 Script executed:
#!/bin/bash
set -eu
python3 - <<'PY'
from pathlib import Path
files = [
Path("scripts/ci_install_sglang.sh"),
Path("scripts/ci_install_vllm.sh"),
]
for path in files:
text = path.read_text()
marker = 'while [ "${CUDA_TAG}" -gt "${CUDA_MAJOR_FLOOR}" ]'
start = text.index(marker)
end = text.index("\n done" if path.name == "ci_install_sglang.sh" else "\n done", start)
loop = text[start:end + len("\n done")]
print(f"{path}:")
print(loop)
print()
def selected_tag(initial, probe_results):
floor = initial // 10 * 10
tag = initial
i = 0
while tag > floor and not probe_results[i]:
tag -= 1
i += 1
return tag, i
cases = {
"404 then 200": [False, True],
"500 then 200": [False, True],
"transport failure then 200": [False, True],
"all failures": [False, False],
"initial 130 (floor, no probe)": [],
}
for name, results in cases.items():
initial = 132 if name != "initial 130 (floor, no probe)" else 130
print(name, "=>", selected_tag(initial, results))
PYRepository: smg-project/smg
Length of output: 487
🏁 Script executed:
#!/bin/bash
set -eu
python3 - <<'PY'
from pathlib import Path
files = [
Path("scripts/ci_install_sglang.sh"),
Path("scripts/ci_install_vllm.sh"),
]
for path in files:
text = path.read_text()
marker = 'while [ "${CUDA_TAG}" -gt "${CUDA_MAJOR_FLOOR}" ]'
start = text.index(marker)
end = text.index("\n done", start)
print(f"{path}:")
print(text[start:end + len("\n done")])
print()
def selected_tag(initial, probe_results):
floor = initial // 10 * 10
tag = initial
i = 0
while tag > floor and not probe_results[i]:
tag -= 1
i += 1
return tag, i
cases = {
"404 then 200": [False, True],
"500 then 200": [False, True],
"transport failure then 200": [False, True],
"all failures": [False, False],
"initial 130 (floor, no probe)": [],
}
for name, results in cases.items():
initial = 132 if name != "initial 130 (floor, no probe)" else 130
print(name, "=>", selected_tag(initial, results))
PYRepository: smg-project/smg
Length of output: 487
🏁 Script executed:
#!/bin/bash
set -eu
python3 - <<'PY'
from pathlib import Path
for path in (Path("scripts/ci_install_sglang.sh"), Path("scripts/ci_install_vllm.sh")):
lines = path.read_text().splitlines()
start = next(i for i, line in enumerate(lines)
if line.lstrip().startswith('while [ "${CUDA_TAG}"'))
end = next(i for i in range(start, len(lines)) if lines[i].strip() == "done")
print(f"{path}:{start + 1}-{end + 1}")
print("\n".join(lines[start:end + 1]))
print()
def selected_tag(initial, probe_results):
floor = initial // 10 * 10
tag = initial
i = 0
while tag > floor and not probe_results[i]:
tag -= 1
i += 1
return tag, i
for name, initial, results in (
("404 then 200", 132, [False, True]),
("500 then 200", 132, [False, True]),
("transport failure then 200", 132, [False, True]),
("all failures", 132, [False, False]),
("initial floor", 130, []),
):
print(f"{name}: selected cu{selected_tag(initial, results)[0]}, probes={selected_tag(initial, results)[1]}")
PYRepository: smg-project/smg
Length of output: 819
🏁 Script executed:
#!/bin/bash
set -u
for tag in 132 131 130; do
url="https://flashinfer.ai/whl/cu${tag}/flashinfer-jit-cache/"
result=$(curl -sS -o /dev/null --connect-timeout 5 --max-time 15 \
-w 'http=%{http_code} exit=%{exitcode}' "$url" 2>&1) || true
printf 'cu%s: %s\n' "$tag" "$result"
doneRepository: smg-project/smg
Length of output: 222
🏁 Script executed:
#!/bin/bash
set -eu
python3 - <<'PY'
from pathlib import Path
for path in (Path("scripts/ci_install_sglang.sh"), Path("scripts/ci_install_vllm.sh")):
lines = path.read_text().splitlines()
start = next(i for i, line in enumerate(lines)
if line.lstrip().startswith('while [ "${CUDA_TAG}"'))
end = next(i for i in range(start, len(lines)) if lines[i].strip() == "done")
print(f"{path}:{start + 1}-{end + 1}")
print("\n".join(lines[start:end + 1]))
print()
def selected_tag(initial, probe_results):
floor = initial // 10 * 10
tag = initial
i = 0
while tag > floor and not probe_results[i]:
tag -= 1
i += 1
return tag, i
for name, initial, results in (
("404 then 200", 132, [False, True]),
("500 then 200", 132, [False, True]),
("transport failure then 200", 132, [False, True]),
("all failures", 132, [False, False]),
("initial floor", 130, []),
):
tag, probes = selected_tag(initial, results)
print(f"{name}: selected cu{tag}, probes={probes}")
PYRepository: smg-project/smg
Length of output: 819
🔴 Important: Make both FlashInfer index probes status-aware and bounded.
curl -sfo returns nonzero for 404, 5xx, and transport failures. The ! condition treats each result as a missing index, so transient failures can select an unintended lower CUDA tag. The loop also skips CUDA_MAJOR_FLOOR validation after all probes fail. Add connection and total-operation timeouts. Decrement only for HTTP 404. Exit on other HTTP or transport failures in scripts/ci_install_sglang.sh and scripts/ci_install_vllm.sh.
📍 Affects 2 files
scripts/ci_install_sglang.sh#L61-L64(this comment)scripts/ci_install_vllm.sh#L95-L98
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@scripts/ci_install_sglang.sh` around lines 61 - 64, Update the FlashInfer
probe loops in scripts/ci_install_sglang.sh lines 61-64 and
scripts/ci_install_vllm.sh lines 95-98 so both use connection and
total-operation timeouts and capture HTTP status separately from curl transport
errors. Decrement the CUDA tag only when the probe returns HTTP 404, exit
immediately for other HTTP statuses or transport failures, and ensure the
CUDA_MAJOR_FLOOR validation still runs after all eligible tags are exhausted.
Source: Coding guidelines
Comment on lines
+61
to
+65
| while [ "${CUDA_TAG}" -gt "${CUDA_MAJOR_FLOOR}" ] \ | ||
| && ! curl -sfo /dev/null "https://flashinfer.ai/whl/cu${CUDA_TAG}/flashinfer-jit-cache/"; do | ||
| CUDA_TAG=$((CUDA_TAG - 1)) | ||
| done | ||
| CU_VERSION="cu${CUDA_TAG}" |
There was a problem hiding this comment.
🗄️ Data Integrity & Integration | 🟠 Major | 🏗️ Heavy lift
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
set -eu
printf '%s\n' '--- target files ---'
for f in scripts/ci_install_sglang.sh scripts/ci_install_vllm.sh scripts/update_whl_index.py; do
if [ -f "$f" ]; then
printf '\n### %s (%s lines)\n' "$f" "$(wc -l < "$f")"
case "$f" in
scripts/update_whl_index.py) sed -n '1,240p' "$f" ;;
*) sed -n '1,150p' "$f" ;;
esac
else
printf 'MISSING: %s\n' "$f"
fi
done
printf '\n--- relevant references ---\n'
rg -n -C 3 'CUDA_TAG|CU_VERSION|flashinfer|update_whl_index|curl' scripts .github tests 2>/dev/null | head -400 || trueRepository: smg-project/smg
Length of output: 44448
🏁 Script executed:
#!/bin/bash
set -eu
printf '%s\n' '--- changed files ---'
git diff --name-only 2>/dev/null || true
git status --short 2>/dev/null || true
printf '\n--- test and CI references ---\n'
rg -n -C 3 'ci_install_sglang|ci_install_vllm|CUDA_MAJOR_FLOOR|flashinfer-jit-cache|flashinfer' \
-g '*test*' -g '*.yml' -g '*.yaml' -g '*.sh' -g '*.py' . 2>/dev/null | head -400 || trueRepository: smg-project/smg
Length of output: 9204
🏁 Script executed:
#!/bin/bash
set -u
python3 - <<'PY'
from pathlib import Path
for name in ("scripts/ci_install_sglang.sh", "scripts/ci_install_vllm.sh"):
text = Path(name).read_text()
print(f"{name}:")
for line in text.splitlines():
if "curl -sfo /dev/null" in line or "uv pip install \"flashinfer-jit-cache" in line:
print(" " + line.strip())
PY
printf '\n--- live FlashInfer index probes ---\n'
for tag in cu129 cu130 cu131 cu132; do
url="https://flashinfer.ai/whl/${tag}/flashinfer-jit-cache/"
body="$(mktemp)"
code="$(curl -L --connect-timeout 10 --max-time 30 -sS -o "$body" -w '%{http_code}' "$url" 2>/tmp/flashinfer-curl.err || true)"
printf '%s status=%s bytes=%s\n' "$tag" "$code" "$(wc -c < "$body")"
if [ "$code" = 200 ]; then
grep -Eio 'href="[^"]+flashinfer[^"]+\.whl[^"]*"' "$body" | head -10 || true
grep -Eio 'flashinfer-jit-cache[^"< ]*' "$body" | head -10 || true
fi
rm -f "$body"
done
if [ -s /tmp/flashinfer-curl.err ]; then
printf '\n--- curl diagnostics ---\n'
cat /tmp/flashinfer-curl.err
fiRepository: smg-project/smg
Length of output: 5165
🏁 Script executed:
#!/bin/bash
set -eu
python3 - <<'PY'
import re
from pathlib import Path
def extract(path):
s = Path(path).read_text()
probe = re.search(r'while \[ "\$\{CUDA_TAG\}" > "\$\{CUDA_MAJOR_FLOOR\}" \].*?done', s, re.S)
print(f"\n{path}\n{probe.group(0) if probe else 'loop not found'}")
extract("scripts/ci_install_sglang.sh")
extract("scripts/ci_install_vllm.sh")
PY
printf '\n--- deterministic loop behavior with mocked curl ---\n'
tmp="$(mktemp -d)"
trap 'rm -rf "$tmp"' EXIT
cat >"$tmp/mock-curl" <<'SH'
#!/bin/sh
# Return success while writing an empty response, which models an empty 200 page.
exit 0
SH
chmod +x "$tmp/mock-curl"
for start in 132 131; do
CUDA_TAG="$start"
CUDA_MAJOR_FLOOR=$((CUDA_TAG / 10 * 10))
iterations=0
while [ "$CUDA_TAG" -gt "$CUDA_MAJOR_FLOOR" ] \
&& ! "$tmp/mock-curl" -sfo /dev/null "https://example.invalid/cu${CUDA_TAG}/flashinfer-jit-cache/"; do
CUDA_TAG=$((CUDA_TAG - 1))
iterations=$((iterations + 1))
done
printf 'start=%s => selected=cu%s iterations=%s\n' "$start" "$CUDA_TAG" "$iterations"
done
printf '\n--- empty-index behavior in update_whl_index.py ---\n'
tmprepo="$(mktemp -d)"
tmpwheels="$(mktemp -d)"
python3 scripts/update_whl_index.py \
--package flashinfer-jit-cache --cuda 130 --release-tag test-release \
--wheel-dir "$tmpwheels" --whl-repo-dir "$tmprepo"
find "$tmprepo" -type f -print -exec sh -c 'printf " bytes="; wc -c < "$1"; cat "$1"' sh {} \;Repository: smg-project/smg
Length of output: 691
🏁 Script executed:
#!/bin/bash
set -u
printf '%s\n' '--- versions and wheel tags in live package indexes ---'
for tag in cu129 cu130; do
body="$(mktemp)"
code="$(curl -L --connect-timeout 10 --max-time 30 -sS -o "$body" -w '%{http_code}' \
"https://flashinfer.ai/whl/${tag}/flashinfer-jit-cache/" 2>/tmp/flashinfer-curl.err || true)"
printf '\n%s status=%s bytes=%s\n' "$tag" "$code" "$(wc -c < "$body")"
if [ "$code" = 200 ]; then
python3 - "$body" <<'PY'
import re, sys
from collections import defaultdict
text = open(sys.argv[1], encoding="utf-8").read()
links = re.findall(r'href="([^"]+flashinfer_jit_cache-[^"]+\.whl)(?:#[^"]*)?"', text)
by_version = defaultdict(list)
for link in links:
name = link.rsplit("/", 1)[-1].split("#", 1)[0]
# filename: flashinfer_jit_cache-VERSION+CUDA-python-abi-platform.whl
m = re.match(r"flashinfer_jit_cache-([^+]+)\+[^-]+-(cp[^-]+)-([^-]+)-(.+)\.whl$", name)
if m:
version, py, abi, platform = m.groups()
by_version[version].append((py, abi, platform))
else:
print("UNPARSED", name)
for version in sorted(by_version):
print(version, sorted(set(by_version[version])))
PY
fi
rm -f "$body"
done
printf '\n--- source evidence for empty package pages ---\n'
nl -ba scripts/update_whl_index.py | sed -n '45,85p'
printf '\n--- shell-only reproduction of the probe invariant ---\n'
tmp="$(mktemp -d)"
trap 'rm -rf "$tmp"' EXIT
cat >"$tmp/curl" <<'SH'
#!/bin/sh
# Simulate curl -sfo against an empty HTTP 200 page.
: > "$3"
exit 0
SH
chmod +x "$tmp/curl"
for start in 132 131; do
CUDA_TAG="$start"
CUDA_MAJOR_FLOOR=$((CUDA_TAG / 10 * 10))
calls=0
while [ "$CUDA_TAG" -gt "$CUDA_MAJOR_FLOOR" ] \
&& ! "$tmp/curl" -sfo /dev/null "https://flashinfer.ai/whl/cu${CUDA_TAG}/flashinfer-jit-cache/"; do
CUDA_TAG=$((CUDA_TAG - 1))
calls=$((calls + 1))
done
printf 'empty-200 start=cu%s selected=cu%s fallback_steps=%s\n' \
"$start" "$CUDA_TAG" "$calls"
done
printf '\n--- probe failure at every minor index ---\n'
cat >"$tmp/curl" <<'SH'
#!/bin/sh
exit 6
SH
chmod +x "$tmp/curl"
for start in 132 131; do
CUDA_TAG="$start"
CUDA_MAJOR_FLOOR=$((CUDA_TAG / 10 * 10))
calls=0
while [ "$CUDA_TAG" -gt "$CUDA_MAJOR_FLOOR" ] \
&& ! "$tmp/curl" -sfo /dev/null "https://flashinfer.ai/whl/cu${CUDA_TAG}/flashinfer-jit-cache/"; do
CUDA_TAG=$((CUDA_TAG - 1))
calls=$((calls + 1))
done
printf 'all-fail start=cu%s selected=cu%s fallback_steps=%s\n' \
"$start" "$CUDA_TAG" "$calls"
doneRepository: smg-project/smg
Length of output: 11680
Validate the requested FlashInfer wheel before selecting a CUDA index.
Both loops accept any successful package-index response. A 200 response can contain no wheel, or no wheel for ${FLASHINFER_VERSION} and the runner platform. Select an index only when it contains a compatible flashinfer-jit-cache==${FLASHINFER_VERSION} wheel. Fail explicitly when no compatible index exists.
Apply this to scripts/ci_install_sglang.sh#L61-L65 and scripts/ci_install_vllm.sh#L94-L98.
📍 Affects 2 files
scripts/ci_install_sglang.sh#L61-L65(this comment)scripts/ci_install_vllm.sh#L94-L98
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@scripts/ci_install_sglang.sh` around lines 61 - 65, Update the CUDA
index-selection loops in scripts/ci_install_sglang.sh lines 61-65 and
scripts/ci_install_vllm.sh lines 94-98 to validate that each candidate index
contains a compatible flashinfer-jit-cache==${FLASHINFER_VERSION} wheel for the
runner platform, rather than accepting any successful HTTP response. Preserve
the fallback decrement behavior, and fail explicitly when no compatible CUDA
index is found.
Source: Coding guidelines
2 of 4 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Every vLLM e2e lane dies in Setup (~1m40s) on main and on this PR's first revision:
Real cause (first revision of this PR diagnosed it wrong)
torch now resolves to
2.13.0+cu132in the vLLM install. The scripts derive the flashinfer wheel index fromtorch.version.cuda, but flashinfer hosts one index per CUDA tag and its newest iscu130—https://flashinfer.ai/whl/cu132404s, and uv reports a missing index as "package not found in the registry". The.post3in the error was a red herring:0.6.16.post3exists on the cu130 index (verified live). This PR's first revision added a version fallback for that red herring; both its installs failed identically because the index itself was absent.Fix
CUDA minor versions are ABI-compatible, so probe the index URL and walk down to the nearest published tag within the same major:
Applied to both
ci_install_vllm.shandci_install_sglang.sh— sglang's torch is still cu130 so its lanes pass today, but it carries the same latent assumption. The major floor stops the walk from crossing into cu12x (a different ABI).Verified against the live index: 132 → 130; 130 stays 130.
Verification
The vLLM lanes on this PR exercise the cu132→cu130 walk end-to-end; the sglang lanes prove no regression on the already-working path.