Fix install one-liner 404 on NVIDIA hosts without a CUDA toolkit - #1267
Conversation
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (2)
🚧 Files skipped from review as they are similar to previous changes (2)
📝 WalkthroughWalkthroughThe installer now tolerates failed CUDA toolkit and library probes, falls back to ChangesCUDA detection and asset selection
Estimated code review effort: 2 (Simple) | ~10 minutes Mergeability Score: 🔵 Low · up to The installer now selects CUDA 12 or 13 archives on NVIDIA hosts without a toolkit, but affected Linux systems may still install successfully and then fail at runtime if the matching CUDA libraries are unavailable. The PR is mergeable with explicit owner awareness and follow-up on runtime library bundling or linkage. Possibly related issues
Suggested reviewers: 🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches 💡 1📝 Generate docstrings 💡
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
e4eeaa0 to
84001ee
Compare
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 3
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@install.sh`:
- Around line 381-384: Update the CUDA-version error message in the aarch64
install path to recommend only --flavor cpu, removing the unsupported --flavor
vulkan suggestion while preserving the existing toolkit guidance and error
behavior.
- Around line 381-386: Align the installer and packaging contract by adding
explicit CUDA 12 and CUDA 13 lanes in check_package_release_assets, setting
deterministic MESH_CUDA_VERSION values for x86_64, aarch64, and Orin checks, and
updating fixture/release exact-name assertions to use versioned cuda-12 or
cuda-13 assets. Update release documentation to describe the versioned asset
names and supported CUDA lanes.
- Around line 353-362: Remove the nvidia-smi fallback that assigns ver from the
driver-reported CUDA version. When toolkit probes are empty, only select a CUDA
lane after verifying matching-major loader-path evidence for libcudart,
libcublas, and libcublasLt, or use an existing verified self-contained bundle
path; do not treat driver_max_major as runtime evidence.
🪄 Autofix
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
| # Inference-only hosts carry an NVIDIA driver but no CUDA toolkit, so none of | ||
| # the probes above find anything. The driver still advertises the highest CUDA | ||
| # it supports in the nvidia-smi header ("CUDA Version: 13.0"); use that as an | ||
| # upper bound and clamp it to a CUDA lane we actually publish. | ||
| if [[ -z "$ver" ]] && command -v nvidia-smi >/dev/null 2>&1; then | ||
| ver="$(nvidia-smi 2>/dev/null | grep -oE 'CUDA Version: *[0-9]+' | grep -oE '[0-9]+' | head -n 1)" | ||
| if [[ -n "$ver" ]] && (( ver > 13 )); then | ||
| ver=13 | ||
| fi | ||
| fi |
There was a problem hiding this comment.
🩺 Stability & Availability | 🟠 Major | 🏗️ Heavy lift
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
set -euo pipefail
rg -n -C 4 \
'driver_max_major|libcudart|libcublas|libcublasLt|native-runtimes|cuda-12|cuda-13'Repository: Mesh-LLM/mesh-llm
Length of output: 155
🏁 Script executed:
#!/bin/bash
set -euo pipefail
printf '%s\n' '--- install.sh locations ---'
git ls-files | grep -E '(^|/)install\.sh$' || true
printf '%s\n' '--- install.sh relevant symbols ---'
rg -n -C 6 'nvidia-smi|driver_max_major|CUDA Version|cuda-[0-9]+|native|runtime|libcudart|libcublas' install.sh 2>/dev/null || true
printf '%s\n' '--- repository references ---'
rg -n -C 3 'driver_max_major|libcudart|libcublas|libcublasLt|native-runtimes|cuda-12|cuda-13' . --glob '!target/**' --glob '!node_modules/**' 2>/dev/null || trueRepository: Mesh-LLM/mesh-llm
Length of output: 50376
🏁 Script executed:
#!/bin/bash
set -euo pipefail
printf '%s\n' '--- install.sh CUDA detection and asset selection ---'
sed -n '300,380p' install.sh
sed -n '520,610p' install.sh
printf '%s\n' '--- hardware profile implementation ---'
sed -n '280,390p' crates/mesh-llm-hardware-profile/src/lib.rs
sed -n '390,475p' crates/mesh-llm-hardware-profile/src/lib.rs
printf '%s\n' '--- resolver/runtime manifest references ---'
rg -n -C 5 'driver_max_major|toolkit_majors|bundled|bundle|cuda|cudart|cublas|windows|target_os' crates --glob '*.rs' | head -n 500
printf '%s\n' '--- focused tests ---'
sed -n '820,900p' crates/mesh-llm-hardware-profile/src/lib.rs
rg -n -C 5 'driver_max_major|toolkit_majors|cuda.*selection|selection.*cuda|runtime.*compatible|bundled' crates --glob '*test*' --glob '*.rs' | head -n 500Repository: Mesh-LLM/mesh-llm
Length of output: 50373
🏁 Script executed:
#!/bin/bash
set -euo pipefail
printf '%s\n' '--- complete install asset selection ---'
sed -n '360,455p' install.sh
rg -n -C 8 'asset_name|detect_cuda_major|preferred_asset|recommended_flavor|INSTALL_FLAVOR' install.sh
printf '%s\n' '--- runtime-related Rust files ---'
git ls-files 'crates/**/*.rs' | while read -r file; do
if rg -q 'HostCudaProfile|driver_max_major|toolkit_majors|RuntimeSelection|NativeRuntime|runtime.*manifest|manifest.*runtime' "$file"; then
printf '%s\n' "$file"
fi
done
printf '%s\n' '--- CUDA profile consumers ---'
rg -n -C 12 'driver_max_major|toolkit_majors|HostCudaProfile|cuda.*compatible|compatible.*cuda|runtime.*candidate|candidate.*runtime' \
crates/mesh-llm-hardware-profile crates --glob '*.rs' \
--glob '!crates/mesh-llm-hardware-profile/src/lib.rs' | head -n 700
printf '%s\n' '--- packaging and runtime documentation ---'
rg -n -C 5 'bundl(e|ed).*CUDA|CUDA.*bundl|libcudart|libcublas|libcublasLt|host-linked|host linked|Windows.*CUDA|CUDA.*Windows' \
docs README.md scripts crates --glob '!*.lock' | head -n 500Repository: Mesh-LLM/mesh-llm
Length of output: 50374
🏁 Script executed:
#!/bin/bash
set -euo pipefail
printf '%s\n' '--- resolver CUDA evaluation and bundling ---'
sed -n '400,530p' crates/mesh-llm-native-runtime/src/resolver.rs
sed -n '700,840p' crates/mesh-llm-native-runtime/src/resolver.rs
printf '%s\n' '--- runtime manifest CUDA requirements ---'
rg -n -C 8 'CudaRuntimeRequirements|bundled_libraries|library|cuda_toolkit|toolkit_major|runtime.*requirements' \
crates/mesh-llm-native-runtime scripts docs README.md --glob '*.rs' --glob '*.sh' --glob '*.md' \
| head -n 700
printf '%s\n' '--- Linux CUDA package construction ---'
rg -n -C 10 'cudart|cublas|cublasLt|CUDA.*runtime|runtime.*CUDA|MESH_LLM_CUDA_TOOLKIT_MAJOR|backend.*cuda' \
scripts/package-native-runtime.sh scripts/package-release.sh docs/cuda-release-lanes.md \
docs/design/NATIVE_RUNTIMES.md docs/SDK.md README.md | head -n 700
printf '%s\n' '--- focused static behavioral verifier ---'
python3 - <<'PY'
from pathlib import Path
install = Path("install.sh").read_text()
resolver = Path("crates/mesh-llm-native-runtime/src/resolver.rs").read_text()
host = Path("crates/mesh-llm-native-runtime/src/host.rs").read_text()
checks = {
"installer uses nvidia-smi fallback": 'nvidia-smi 2>/dev/null | grep -oE \'CUDA Version:' in install,
"installer emits CUDA-major Linux archive": 'unknown-linux-gnu-cuda-%s.tar.gz' in install,
"resolver rejects host-linked CUDA without toolkit": 'CudaToolkitNotDetected { required }' in resolver,
"resolver checks complete bundled CUDA set": all(x in resolver for x in ['"cudart"', '"cublas"', '"cublasLt"']),
"driver bound is separate from toolkit evidence": 'pub driver_max_major' in host and 'pub toolkit_majors' in host,
}
for name, ok in checks.items():
print(f"{name}: {'yes' if ok else 'no'}")
if not all(checks.values()):
raise SystemExit(1)
PYRepository: Mesh-LLM/mesh-llm
Length of output: 50373
🏁 Script executed:
#!/bin/bash
set -euo pipefail
printf '%s\n' '--- CUDA release-lane contract ---'
cat -n docs/cuda-release-lanes.md | sed -n '1,180p'
printf '%s\n' '--- release archive composition ---'
rg -n -C 12 'cuda-[0-9]|unknown-linux-gnu-cuda|PRECOMPOSED_PRODUCT|product-manifest|native-runtimes|bundle' \
scripts/package-release.sh scripts/package-release.ps1 scripts/ci-compose-product-input.sh \
scripts/build-development-product.sh scripts/build-release.sh 2>/dev/null | head -n 900
printf '%s\n' '--- installer archive validation and setup handoff ---'
sed -n '620,850p' install.sh
printf '%s\n' '--- source-level contract verifier ---'
python3 - <<'PY'
from pathlib import Path
install = Path("install.sh").read_text()
resolver = Path("crates/mesh-llm-native-runtime/src/resolver.rs").read_text()
host = Path("crates/mesh-llm-native-runtime/src/host.rs").read_text()
needles = {
"nvidia-smi fallback in installer": "nvidia-smi 2>/dev/null" in install,
"CUDA archive selected from detected major": "unknown-linux-gnu-cuda-%s.tar.gz" in install,
"installer probes only libcudart": "libcudart.so.*" in install and "libcublas.so.*" not in install,
"resolver requires host toolkit for nonbundled runtime": "CudaToolkitNotDetected { required }" in resolver,
"resolver treats driver maximum separately": "driver_max_major" in host and "toolkit_majors" in host,
"resolver requires cudart/cublas/cublasLt for bundling": all(token in resolver for token in ("cudart", "cublas", "cublas_lt")),
}
for label, result in needles.items():
print(f"{label}: {result}")
PYRepository: Mesh-LLM/mesh-llm
Length of output: 50375
🏁 Script executed:
#!/bin/bash
set -euo pipefail
printf '%s\n' '--- release asset naming and flavor mapping ---'
sed -n '35,190p' scripts/package-release.sh
rg -n -C 5 'cuda-12|cuda-13|cuda-blackwell|unknown-linux-gnu-cuda|MESH_LLM_INSTALL_FLAVOR|RELEASE_FLAVOR' .github Justfile scripts install.sh docs \
--glob '!*.lock' | head -n 700
printf '%s\n' '--- native CUDA runtime package manifest generation ---'
rg -n -C 10 'manifest|libraries|CUDA|cudart|cublas|cublasLt|link' scripts/package-native-runtime.sh | head -n 700
sed -n '430,700p' scripts/package-native-runtime.sh
printf '%s\n' '--- runtime selection outcomes for driver-only profiles ---'
sed -n '840,940p' crates/mesh-llm-native-runtime/src/resolver.rs
rg -n -C 8 'select_native_runtime_from_artifacts|CandidateRejection|CudaToolkitNotDetected|fallback.*cpu|no compatible' \
crates/mesh-llm-native-runtime crates/mesh-llm-commands --glob '*.rs' | head -n 800Repository: Mesh-LLM/mesh-llm
Length of output: 50376
🏁 Script executed:
#!/bin/bash
set -euo pipefail
printf '%s\n' '--- runtime package library list and CUDA linkage ---'
sed -n '500,690p' scripts/package-native-runtime.sh
sed -n '300,430p' crates/llama-quant-ffi/build.rs
printf '%s\n' '--- deterministic driver-only scenario probe ---'
python3 - <<'PY'
from pathlib import Path
# Model only the documented branch conditions. Do not execute repository code.
def installer_cuda_major(nvcc_major, libcudart_majors, nvidia_smi_major):
if nvcc_major:
return nvcc_major
if libcudart_majors:
return max(libcudart_majors)
return min(nvidia_smi_major, 13) if nvidia_smi_major else None
def resolver_accepts_linux_host_linked(required, toolkit_majors, driver_max):
if driver_max is not None and required > driver_max:
return False
return required in toolkit_majors
selected = installer_cuda_major(None, set(), 13)
accepted = resolver_accepts_linux_host_linked(selected, set(), 13)
print(f"driver-only host: installer_major={selected}, linux_host_linked_runtime_accepted={accepted}")
assert selected == 13
assert accepted is False
install = Path("install.sh").read_text()
resolver = Path("crates/mesh-llm-native-runtime/src/resolver.rs").read_text()
assert "nvidia-smi 2>/dev/null" in install
assert "unknown-linux-gnu-cuda-%s.tar.gz" in install
assert "CudaToolkitNotDetected { required }" in resolver
print("source branches match the driver-only scenario")
PYRepository: Mesh-LLM/mesh-llm
Length of output: 10888
Do not use nvidia-smi as CUDA runtime evidence.
When toolkit probes are empty, this fallback can select a Linux CUDA archive from the driver upper bound. Linux runtimes require matching-major loader-path evidence for libcudart, libcublas, and libcublasLt; driver_max_major does not provide that evidence. Require all three libraries, or select a verified self-contained bundle.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@install.sh` around lines 353 - 362, Remove the nvidia-smi fallback that
assigns ver from the driver-reported CUDA version. When toolkit probes are
empty, only select a CUDA lane after verifying matching-major loader-path
evidence for libcudart, libcublas, and libcublasLt, or use an existing verified
self-contained bundle path; do not treat driver_max_major as runtime evidence.
Sources: Learnings, MCP tools
84001ee to
3176a14
Compare
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@scripts/tests/test_install_sh.py`:
- Line 85: Make CUDA detection tests independent of host toolkit state by adding
an injectable probe-root or equivalent test-only isolation mechanism to
detect_cuda_major. Apply it at scripts/tests/test_install_sh.py lines 85-85 so
the no-evidence case cannot inspect host CUDA, and at lines 146-164 so detection
uses the supplied nvidia-smi output rather than a host library version.
- Around line 127-168: Update detect_cuda_major and its test expectations so
nvidia-smi driver headers are treated only as a compatibility upper bound, not
sufficient CUDA runtime evidence. Require matching-major libcudart, libcublas,
and libcublasLt evidence before selecting a host-linked CUDA archive, or select
an archive that bundles those dependencies; driver-only cases must not return a
CUDA major.
🪄 Autofix
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro Plus
Run ID: dcd72d1f-3009-4b26-8606-d69784577959
📒 Files selected for processing (2)
install.shscripts/tests/test_install_sh.py
🚧 Files skipped from review as they are similar to previous changes (1)
- install.sh
Inference-only Linux hosts carry an NVIDIA driver but no CUDA toolkit, so detect_cuda_major found nothing (it probed only nvcc, /usr/local/cuda*, and libcudart in ldconfig) and asset_name fell back to the legacy bare -cuda archive name, which releases no longer publish. The install one-liner 404'd. Fall back to the CUDA major advertised in the nvidia-smi header, clamped to a published lane, and drop the legacy fallback in favour of an actionable error so a missing lane never becomes a mystery 404. Refs #1220 Co-authored-by: Michael Neale <14976+michaelneale@users.noreply.github.com> Signed-off-by: Michael Neale <14976+michaelneale@users.noreply.github.com> Co-authored-by: Michael Neale <14976+michaelneale@users.noreply.github.com> Signed-off-by: Michael Neale <14976+michaelneale@users.noreply.github.com> Co-authored-by: Michael Neale <14976+michaelneale@users.noreply.github.com> Signed-off-by: Michael Neale <14976+michaelneale@users.noreply.github.com>
3176a14 to
8cbc998
Compare
|
Thanks — two of these were right and are now fixed; one I am deliberately not addressing here. Fixed: aarch64 remediation suggested an unavailable flavor. Correct, Fixed: tests were not isolated from host toolkit state. Also correct — the Not fixed here, deliberately: using The real defect you are pointing at is that Linux CUDA runtimes are host-linked while the Windows ones bundle their CUDA DLLs. That is a genuine bug, it is the second half of #1220, and it needs a fix in packaging rather than in archive-name selection. This PR is scoped to the 404: the fallback it replaces could never resolve at all. After this change a driver-only host gets the correct archive and then fails at runtime load with a clear error — worse-shaped than a bundled runtime would be, better than a 404, and the packaging fix is tracked separately. Third comment (release-asset contract / |
1 participant
What this fixes
The documented install one-liner now works on Linux hosts that have an NVIDIA driver but no CUDA toolkit — the normal shape for an inference-only GPU box.
Before, it failed with a 404:
After, it selects the right lane and installs.
Why it happened
detect_cuda_majorprobed three things —nvcc --version,/usr/local/cuda*/.../libcudart.so.*, andlibcudartinldconfig -p. All three are toolkit artifacts, so a driver-only host returned empty.asset_namethen fell back to the legacy bare namemesh-llm-<arch>-unknown-linux-gnu-cuda.tar.gz, which releases stopped publishing when the CUDA lanes were split into-cuda-12/-cuda-13. The v0.75.1 release publishes only the split names, so that fallback could never resolve — on x86_64 or aarch64.This is the installer-side counterpart to #1195, which separated driver bound from installed toolkit in host runtime selection. Same underlying confusion (driver ≠ toolkit), different code path: #1195 fixed picking the native runtime after install, this fixes picking the archive to download in the first place. It is not a platform-support limitation — the correct archive exists and is published, the installer just asked for a name that does not.
What changed
nvidia-smiis present, take the CUDA major from its header (CUDA Version: 13.0). That is the driver's maximum supported CUDA, which is the correct upper bound for choosing a lane, and clamp it to the newest published lane.-cudafallback on both the x86_64 and aarch64 branches. If no supported major can be determined, fail with an actionable message instead of fabricating an archive name that cannot exist:Validation
bash -n install.shpasses.detect_cuda_majorexercised against a stubbednvidia-smion a machine with no CUDA toolkit at all, covering every branch of the new path:nvidia-smioutputCUDA Version: 13.013CUDA Version: 14.2(future driver)13(clamped to newest published lane)CUDA Version: 11.8(too old)nvidia-smion PATHRelease-asset names confirmed against v0.75.1 with
gh release view: only-cuda-12and-cuda-13are published for bothx86_64andaarch64, no bare-cuda.Partially addresses #1220 — the installer half. Deliberately not using a closing keyword: #1220 also reports a runtime-linkage failure that this does not fix, so the issue should stay open until that half is tracked separately.
Not covered here: the reporter also hit a second, separate problem — after installing, the Linux cuda13 native runtime links
libcudart.so.13/libcublas.so.13without bundling them, while the Windows runtime ships its own CUDA DLLs. That needs its own issue and fix.Summary by CodeRabbit
Update after first CI run
CI caught something my local checks did not, and it was a real bug rather than a stale assertion.
scripts/tests/test_install_sh.pyasserted the legacy bare-cudaname that this PR removes, so that assertion is updated to require the refusal instead. But running the suite locally then exposed a genuine defect in my own change:install.shruns underset -euo pipefail, and a non-matchinggrepinside$(...)aborts the whole script. On a host where the newnvidia-smiprobe found no version, the installer would have died mid-detection instead of reaching the actionable error. Everydetect_cuda_majorprobe pipeline now ends in|| true, which also hardens the two pre-existingnvccandldconfigprobes that had the same latent exposure.Test coverage added:
test_detect_cuda_major_falls_back_to_nvidia_smi_driver_version— stubs a driver-only host (nonvcc, noldconfig) across13.0→13,12.4→12,14.2→13(clamped),11.8→ empty.test_asset_name_uses_detected_cuda_major_lane— the-cuda-12/-cuda-13lane name is built correctly on bothx86_64andaarch64.test_release_target_helpers_keep_linux_aarch64_flavor_surface— now asserts refusal plus the actionable stderr, and asserts the legacy name is never emitted.python3 -m unittest scripts.tests.test_install_sh→ 26 passed.