Skip to content

Fix install one-liner 404 on NVIDIA hosts without a CUDA toolkit #1267

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
michaelneale merged 1 commit into from
Aug 12, 2026
Merged

Fix install one-liner 404 on NVIDIA hosts without a CUDA toolkit #1267

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
35 changes: 24 additions & 11 deletions install.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -336,19 +336,30 @@ recommended_flavor() {
detect_cuda_major() {
local ver=""
if command -v nvcc >/dev/null 2>&1; then
ver="$(nvcc --version 2>/dev/null | grep -oE 'release [0-9]+' | awk '{print $2}' | head -n 1)"
ver="$(nvcc --version 2>/dev/null | grep -oE 'release [0-9]+' | awk '{print $2}' | head -n 1 || true)"
fi
if [[ -z "$ver" ]]; then
local lib
for lib in /usr/local/cuda*/targets/*/lib/libcudart.so.* /usr/local/cuda*/targets/*/lib/stubs/libcudart.so.*; do
local probe_root="${MESH_LLM_TEST_CUDA_PROBE_ROOT:-}"
for lib in "$probe_root"/usr/local/cuda*/targets/*/lib/libcudart.so.* "$probe_root"/usr/local/cuda*/targets/*/lib/stubs/libcudart.so.*; do
if [[ -f "$lib" ]]; then
ver="$(basename "$lib" | grep -oE 'libcudart\.so\.[0-9]+' | awk -F. '{print $3}' | head -n 1)"
break
fi
done
fi
if [[ -z "$ver" ]]; then
ver="$(ldconfig -p 2>/dev/null | grep -oE 'libcudart\.so\.[0-9]+' | awk -F. '{print $3}' | sort -rn | head -n 1)"
ver="$(ldconfig -p 2>/dev/null | grep -oE 'libcudart\.so\.[0-9]+' | awk -F. '{print $3}' | sort -rn | head -n 1 || true)"
fi
# Inference-only hosts carry an NVIDIA driver but no CUDA toolkit, so none of
# the probes above find anything. The driver still advertises the highest CUDA
# it supports in the nvidia-smi header ("CUDA Version: 13.0"); use that as an
# upper bound and clamp it to a CUDA lane we actually publish.
if [[ -z "$ver" ]] && command -v nvidia-smi >/dev/null 2>&1; then
ver="$(nvidia-smi 2>/dev/null | grep -oE 'CUDA Version: *[0-9]+' | grep -oE '[0-9]+' | head -n 1 || true)"
if [[ -n "$ver" ]] && (( ver > 13 )); then
ver=13
fi
fi
Comment on lines +354 to +363

@coderabbitai coderabbitai Bot Aug 12, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🩺 Stability & Availability | 🟠 Major | 🏗️ Heavy lift

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
set -euo pipefail

rg -n -C 4 \
  'driver_max_major|libcudart|libcublas|libcublasLt|native-runtimes|cuda-12|cuda-13'

Repository: Mesh-LLM/mesh-llm

Length of output: 155

🏁 Script executed:

#!/bin/bash
set -euo pipefail

printf '%s\n' '--- install.sh locations ---'
git ls-files | grep -E '(^|/)install\.sh$' || true

printf '%s\n' '--- install.sh relevant symbols ---'
rg -n -C 6 'nvidia-smi|driver_max_major|CUDA Version|cuda-[0-9]+|native|runtime|libcudart|libcublas' install.sh 2>/dev/null || true

printf '%s\n' '--- repository references ---'
rg -n -C 3 'driver_max_major|libcudart|libcublas|libcublasLt|native-runtimes|cuda-12|cuda-13' . --glob '!target/**' --glob '!node_modules/**' 2>/dev/null || true

Repository: Mesh-LLM/mesh-llm

Length of output: 50376

🏁 Script executed:

#!/bin/bash
set -euo pipefail

printf '%s\n' '--- install.sh CUDA detection and asset selection ---'
sed -n '300,380p' install.sh
sed -n '520,610p' install.sh

printf '%s\n' '--- hardware profile implementation ---'
sed -n '280,390p' crates/mesh-llm-hardware-profile/src/lib.rs
sed -n '390,475p' crates/mesh-llm-hardware-profile/src/lib.rs

printf '%s\n' '--- resolver/runtime manifest references ---'
rg -n -C 5 'driver_max_major|toolkit_majors|bundled|bundle|cuda|cudart|cublas|windows|target_os' crates --glob '*.rs' | head -n 500

printf '%s\n' '--- focused tests ---'
sed -n '820,900p' crates/mesh-llm-hardware-profile/src/lib.rs
rg -n -C 5 'driver_max_major|toolkit_majors|cuda.*selection|selection.*cuda|runtime.*compatible|bundled' crates --glob '*test*' --glob '*.rs' | head -n 500

Repository: Mesh-LLM/mesh-llm

Length of output: 50373

🏁 Script executed:

#!/bin/bash
set -euo pipefail

printf '%s\n' '--- complete install asset selection ---'
sed -n '360,455p' install.sh
rg -n -C 8 'asset_name|detect_cuda_major|preferred_asset|recommended_flavor|INSTALL_FLAVOR' install.sh

printf '%s\n' '--- runtime-related Rust files ---'
git ls-files 'crates/**/*.rs' | while read -r file; do
    if rg -q 'HostCudaProfile|driver_max_major|toolkit_majors|RuntimeSelection|NativeRuntime|runtime.*manifest|manifest.*runtime' "$file"; then
        printf '%s\n' "$file"
    fi
done

printf '%s\n' '--- CUDA profile consumers ---'
rg -n -C 12 'driver_max_major|toolkit_majors|HostCudaProfile|cuda.*compatible|compatible.*cuda|runtime.*candidate|candidate.*runtime' \
  crates/mesh-llm-hardware-profile crates --glob '*.rs' \
  --glob '!crates/mesh-llm-hardware-profile/src/lib.rs' | head -n 700

printf '%s\n' '--- packaging and runtime documentation ---'
rg -n -C 5 'bundl(e|ed).*CUDA|CUDA.*bundl|libcudart|libcublas|libcublasLt|host-linked|host linked|Windows.*CUDA|CUDA.*Windows' \
  docs README.md scripts crates --glob '!*.lock' | head -n 500

Repository: Mesh-LLM/mesh-llm

Length of output: 50374

🏁 Script executed:

#!/bin/bash
set -euo pipefail

printf '%s\n' '--- resolver CUDA evaluation and bundling ---'
sed -n '400,530p' crates/mesh-llm-native-runtime/src/resolver.rs
sed -n '700,840p' crates/mesh-llm-native-runtime/src/resolver.rs

printf '%s\n' '--- runtime manifest CUDA requirements ---'
rg -n -C 8 'CudaRuntimeRequirements|bundled_libraries|library|cuda_toolkit|toolkit_major|runtime.*requirements' \
  crates/mesh-llm-native-runtime scripts docs README.md --glob '*.rs' --glob '*.sh' --glob '*.md' \
  | head -n 700

printf '%s\n' '--- Linux CUDA package construction ---'
rg -n -C 10 'cudart|cublas|cublasLt|CUDA.*runtime|runtime.*CUDA|MESH_LLM_CUDA_TOOLKIT_MAJOR|backend.*cuda' \
  scripts/package-native-runtime.sh scripts/package-release.sh docs/cuda-release-lanes.md \
  docs/design/NATIVE_RUNTIMES.md docs/SDK.md README.md | head -n 700

printf '%s\n' '--- focused static behavioral verifier ---'
python3 - <<'PY'
from pathlib import Path

install = Path("install.sh").read_text()
resolver = Path("crates/mesh-llm-native-runtime/src/resolver.rs").read_text()
host = Path("crates/mesh-llm-native-runtime/src/host.rs").read_text()

checks = {
    "installer uses nvidia-smi fallback": 'nvidia-smi 2>/dev/null | grep -oE \'CUDA Version:' in install,
    "installer emits CUDA-major Linux archive": 'unknown-linux-gnu-cuda-%s.tar.gz' in install,
    "resolver rejects host-linked CUDA without toolkit": 'CudaToolkitNotDetected { required }' in resolver,
    "resolver checks complete bundled CUDA set": all(x in resolver for x in ['"cudart"', '"cublas"', '"cublasLt"']),
    "driver bound is separate from toolkit evidence": 'pub driver_max_major' in host and 'pub toolkit_majors' in host,
}
for name, ok in checks.items():
    print(f"{name}: {'yes' if ok else 'no'}")
if not all(checks.values()):
    raise SystemExit(1)
PY

Repository: Mesh-LLM/mesh-llm

Length of output: 50373

🏁 Script executed:

#!/bin/bash
set -euo pipefail

printf '%s\n' '--- CUDA release-lane contract ---'
cat -n docs/cuda-release-lanes.md | sed -n '1,180p'

printf '%s\n' '--- release archive composition ---'
rg -n -C 12 'cuda-[0-9]|unknown-linux-gnu-cuda|PRECOMPOSED_PRODUCT|product-manifest|native-runtimes|bundle' \
  scripts/package-release.sh scripts/package-release.ps1 scripts/ci-compose-product-input.sh \
  scripts/build-development-product.sh scripts/build-release.sh 2>/dev/null | head -n 900

printf '%s\n' '--- installer archive validation and setup handoff ---'
sed -n '620,850p' install.sh

printf '%s\n' '--- source-level contract verifier ---'
python3 - <<'PY'
from pathlib import Path
install = Path("install.sh").read_text()
resolver = Path("crates/mesh-llm-native-runtime/src/resolver.rs").read_text()
host = Path("crates/mesh-llm-native-runtime/src/host.rs").read_text()

needles = {
    "nvidia-smi fallback in installer": "nvidia-smi 2>/dev/null" in install,
    "CUDA archive selected from detected major": "unknown-linux-gnu-cuda-%s.tar.gz" in install,
    "installer probes only libcudart": "libcudart.so.*" in install and "libcublas.so.*" not in install,
    "resolver requires host toolkit for nonbundled runtime": "CudaToolkitNotDetected { required }" in resolver,
    "resolver treats driver maximum separately": "driver_max_major" in host and "toolkit_majors" in host,
    "resolver requires cudart/cublas/cublasLt for bundling": all(token in resolver for token in ("cudart", "cublas", "cublas_lt")),
}
for label, result in needles.items():
    print(f"{label}: {result}")
PY

Repository: Mesh-LLM/mesh-llm

Length of output: 50375

🏁 Script executed:

#!/bin/bash
set -euo pipefail

printf '%s\n' '--- release asset naming and flavor mapping ---'
sed -n '35,190p' scripts/package-release.sh
rg -n -C 5 'cuda-12|cuda-13|cuda-blackwell|unknown-linux-gnu-cuda|MESH_LLM_INSTALL_FLAVOR|RELEASE_FLAVOR' .github Justfile scripts install.sh docs \
  --glob '!*.lock' | head -n 700

printf '%s\n' '--- native CUDA runtime package manifest generation ---'
rg -n -C 10 'manifest|libraries|CUDA|cudart|cublas|cublasLt|link' scripts/package-native-runtime.sh | head -n 700
sed -n '430,700p' scripts/package-native-runtime.sh

printf '%s\n' '--- runtime selection outcomes for driver-only profiles ---'
sed -n '840,940p' crates/mesh-llm-native-runtime/src/resolver.rs
rg -n -C 8 'select_native_runtime_from_artifacts|CandidateRejection|CudaToolkitNotDetected|fallback.*cpu|no compatible' \
  crates/mesh-llm-native-runtime crates/mesh-llm-commands --glob '*.rs' | head -n 800

Repository: Mesh-LLM/mesh-llm

Length of output: 50376

🏁 Script executed:

#!/bin/bash
set -euo pipefail

printf '%s\n' '--- runtime package library list and CUDA linkage ---'
sed -n '500,690p' scripts/package-native-runtime.sh
sed -n '300,430p' crates/llama-quant-ffi/build.rs

printf '%s\n' '--- deterministic driver-only scenario probe ---'
python3 - <<'PY'
from pathlib import Path

# Model only the documented branch conditions. Do not execute repository code.
def installer_cuda_major(nvcc_major, libcudart_majors, nvidia_smi_major):
    if nvcc_major:
        return nvcc_major
    if libcudart_majors:
        return max(libcudart_majors)
    return min(nvidia_smi_major, 13) if nvidia_smi_major else None

def resolver_accepts_linux_host_linked(required, toolkit_majors, driver_max):
    if driver_max is not None and required > driver_max:
        return False
    return required in toolkit_majors

selected = installer_cuda_major(None, set(), 13)
accepted = resolver_accepts_linux_host_linked(selected, set(), 13)
print(f"driver-only host: installer_major={selected}, linux_host_linked_runtime_accepted={accepted}")
assert selected == 13
assert accepted is False

install = Path("install.sh").read_text()
resolver = Path("crates/mesh-llm-native-runtime/src/resolver.rs").read_text()
assert "nvidia-smi 2>/dev/null" in install
assert "unknown-linux-gnu-cuda-%s.tar.gz" in install
assert "CudaToolkitNotDetected { required }" in resolver
print("source branches match the driver-only scenario")
PY

Repository: Mesh-LLM/mesh-llm

Length of output: 10888

Do not use nvidia-smi as CUDA runtime evidence.

When toolkit probes are empty, this fallback can select a Linux CUDA archive from the driver upper bound. Linux runtimes require matching-major loader-path evidence for libcudart, libcublas, and libcublasLt; driver_max_major does not provide that evidence. Require all three libraries, or select a verified self-contained bundle.

🤖 Prompt for AI Agents 
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@install.sh` around lines 353 - 362, Remove the nvidia-smi fallback that
assigns ver from the driver-reported CUDA version. When toolkit probes are
empty, only select a CUDA lane after verifying matching-major loader-path
evidence for libcudart, libcublas, and libcublasLt, or use an existing verified
self-contained bundle path; do not treat driver_max_major as runtime evidence.

Sources: Learnings, MCP tools

case "$ver" in
12|13) printf '%s\n' "$ver" ;;
Expand All @@ -368,11 +379,12 @@ asset_name() {
cuda)
local cuda_major
cuda_major="$(detect_cuda_major)"
if [[ -n "$cuda_major" ]]; then
printf 'mesh-llm-aarch64-unknown-linux-gnu-cuda-%s.tar.gz\n' "$cuda_major"
else
printf 'mesh-llm-aarch64-unknown-linux-gnu-cuda.tar.gz\n'
if [[ -z "$cuda_major" ]]; then
echo "error: detected an NVIDIA GPU but could not determine a supported CUDA major version (expected 12 or 13)." >&2
echo " install a CUDA toolkit, or re-run with --flavor cpu to use a non-CUDA build." >&2
return 1
Comment thread
coderabbitai[bot] marked this conversation as resolved.
fi
printf 'mesh-llm-aarch64-unknown-linux-gnu-cuda-%s.tar.gz\n' "$cuda_major"
Comment thread
coderabbitai[bot] marked this conversation as resolved.
;;
*) echo "error: unsupported aarch64 flavor '$flavor'" >&2; return 1 ;;
esac
Expand All @@ -383,11 +395,12 @@ asset_name() {
cuda)
local cuda_major
cuda_major="$(detect_cuda_major)"
if [[ -n "$cuda_major" ]]; then
printf 'mesh-llm-x86_64-unknown-linux-gnu-cuda-%s.tar.gz\n' "$cuda_major"
else
printf 'mesh-llm-x86_64-unknown-linux-gnu-cuda.tar.gz\n'
if [[ -z "$cuda_major" ]]; then
echo "error: detected an NVIDIA GPU but could not determine a supported CUDA major version (expected 12 or 13)." >&2
echo " install a CUDA toolkit, or re-run with --flavor cpu (or --flavor vulkan) to use a non-CUDA build." >&2
return 1
fi
printf 'mesh-llm-x86_64-unknown-linux-gnu-cuda-%s.tar.gz\n' "$cuda_major"
;;
rocm) printf 'mesh-llm-x86_64-unknown-linux-gnu-rocm.tar.gz\n' ;;
vulkan) printf 'mesh-llm-x86_64-unknown-linux-gnu-vulkan.tar.gz\n' ;;
Expand Down
88 changes: 86 additions & 2 deletions scripts/tests/test_install_sh.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -78,11 +78,13 @@ def test_release_target_helpers_keep_linux_aarch64_flavor_surface(self) -> None:
"""
export MESH_LLM_TEST_UNAME_S=Linux
export MESH_LLM_TEST_UNAME_M=aarch64
# No CUDA evidence of any kind on this host.
detect_cuda_major() { printf '\\n'; }
printf 'support=%s\\n' "$(platform_support_status)"
printf 'flavors=%s\\n' "$(supported_flavors)"
printf 'recommended=%s\\n' "$(recommended_flavor)"
printf 'cpu=%s\\n' "$(asset_name cpu)"
printf 'cuda=%s\\n' "$(asset_name cuda)"
printf 'cuda=%s\\n' "$(asset_name cuda || true)"
Comment thread
coderabbitai[bot] marked this conversation as resolved.
""",
)

Expand All @@ -91,7 +93,86 @@ def test_release_target_helpers_keep_linux_aarch64_flavor_surface(self) -> None:
self.assertIn("flavors=cuda cpu", result.stdout)
self.assertIn("recommended=cpu", result.stdout)
self.assertIn("cpu=mesh-llm-aarch64-unknown-linux-gnu.tar.gz", result.stdout)
self.assertIn("cuda=mesh-llm-aarch64-unknown-linux-gnu-cuda.tar.gz", result.stdout)
# With no CUDA evidence at all there is no publishable cuda asset name.
# Releases stopped publishing the legacy bare -cuda archive when the
# lanes split into -cuda-12/-cuda-13, so asset_name must refuse rather
# than emit a name that always 404s.
self.assertIn("cuda=\n", result.stdout)
self.assertNotIn("mesh-llm-aarch64-unknown-linux-gnu-cuda.tar.gz", result.stdout)
self.assertIn("could not determine a supported CUDA major version", result.stderr)
self.assertIn("--flavor cpu", result.stderr)
self.assertNotIn("--flavor vulkan", result.stderr)

def test_asset_name_uses_detected_cuda_major_lane(self) -> None:
for arch in ("aarch64", "x86_64"):
with self.subTest(arch=arch):
with tempfile.TemporaryDirectory() as tmp:
tmp_path = Path(tmp)
install_dir = tmp_path / "bin"
install_dir.mkdir()

result = self._run_helper(
tmp_path,
install_dir,
f"""
export MESH_LLM_TEST_UNAME_S=Linux
export MESH_LLM_TEST_UNAME_M={arch}
detect_cuda_major() {{ printf '13\\n'; }}
asset_name cuda
""",
)

self.assertEqual(result.returncode, 0, result.stderr)
self.assertEqual(
result.stdout.strip(),
f"mesh-llm-{arch}-unknown-linux-gnu-cuda-13.tar.gz",
)

def test_detect_cuda_major_falls_back_to_nvidia_smi_driver_version(self) -> None:
# Inference-only hosts have a driver but no toolkit, so every toolkit
# probe comes up empty and only the nvidia-smi header carries a version.
cases = (
("CUDA Version: 13.0", "13"),
("CUDA Version: 12.4", "12"),
# A driver newer than any lane we publish clamps to the newest lane.
("CUDA Version: 14.2", "13"),
# A driver too old for any published lane yields nothing.
("CUDA Version: 11.8", ""),
)
for header, expected in cases:
with self.subTest(header=header):
with tempfile.TemporaryDirectory() as tmp:
tmp_path = Path(tmp)
install_dir = tmp_path / "bin"
install_dir.mkdir()
probe_root = tmp_path / "cuda-probe-root"
probe_root.mkdir()
wrappers = tmp_path / "wrappers"
wrappers.mkdir()
for absent in ("nvcc", "ldconfig"):
stub = wrappers / absent
stub.write_text("#!/usr/bin/env bash\nexit 1\n", encoding="utf-8")
stub.chmod(0o755)
nvidia_smi = wrappers / "nvidia-smi"
nvidia_smi.write_text(
"#!/usr/bin/env bash\n"
f"printf '%s\\n' '| NVIDIA-SMI 595.84 Driver Version: 595.84 {header} |'\n",
encoding="utf-8",
)
nvidia_smi.chmod(0o755)

result = self._run_helper(
tmp_path,
install_dir,
f"""
export PATH={shlex_quote(str(wrappers))}:$PATH
export MESH_LLM_TEST_CUDA_PROBE_ROOT={shlex_quote(str(probe_root))}
detect_cuda_major
""",
)

self.assertEqual(result.returncode, 0, result.stderr)
self.assertEqual(result.stdout.strip(), expected)
Comment thread
coderabbitai[bot] marked this conversation as resolved.

def test_release_target_helpers_recommend_cuda_on_jetson_orin(self) -> None:
with tempfile.TemporaryDirectory() as tmp:
Expand Down Expand Up @@ -151,6 +232,8 @@ def test_detect_cuda_major_reads_ldconfig_libcudart_version(self) -> None:
install_dir.mkdir()
wrappers = tmp_path / "wrappers"
wrappers.mkdir()
probe_root = tmp_path / "cuda-probe-root"
probe_root.mkdir()
ldconfig = wrappers / "ldconfig"
ldconfig.write_text(
"#!/usr/bin/env bash\n"
Expand All @@ -164,6 +247,7 @@ def test_detect_cuda_major_reads_ldconfig_libcudart_version(self) -> None:
install_dir,
f"""
export PATH={shlex_quote(str(wrappers))}:$PATH
export MESH_LLM_TEST_CUDA_PROBE_ROOT={shlex_quote(str(probe_root))}
detect_cuda_major
""",
)
Expand Down
Loading