Skip to content

Allow Windows setup to complete without NVIDIA GPU #4399

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
danielhanchen wants to merge 11 commits into from
Closed

Allow Windows setup to complete without NVIDIA GPU #4399

Changes from 8 commits
Commits
Show all changes
11 commits
Select commit Hold shift + click to select a range
ad9d622
Allow Windows setup to complete without NVIDIA GPU
danielhanchen Mar 18, 2026
a7d4e88
Show pip progress for PyTorch download on Windows
danielhanchen Mar 18, 2026
c53f529
Guard CUDA env re-sanitization behind GPU check in llama.cpp build
danielhanchen Mar 18, 2026
be61a34
Fix cmake not found on Windows after winget install
danielhanchen Mar 18, 2026
c2f1222
Show cmake errors on failure and retry CUDA VS integration with eleva…
danielhanchen Mar 18, 2026
4f1e9df
Address reviewer feedback: cmake PATH persistence, stale cache, torch…
danielhanchen Mar 18, 2026
7125772
Remove extra CUDA cmake flags to align Windows with Linux build
danielhanchen Mar 18, 2026
8035e9c
Address reviewer round 2: GPU probe fallback, Triton check, stale bin…
danielhanchen Mar 18, 2026
ce80c3f
Merge branch 'main' into fix/windows-setup-no-gpu
danielhanchen Mar 18, 2026
69d99a4
Fix nvidia-smi PATH persistence and cmake requirement for CPU-only
danielhanchen Mar 18, 2026
e717042
Restore Unsloth CUDA tuning flags in the GPU llama.cpp configure
danielhanchen Jul 6, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
237 changes: 181 additions & 56 deletions studio/setup.ps1
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@
Always installs Node.js if needed. When running from pip install:
skips frontend build (already bundled). When running from git repo:
full setup including frontend build.
Requires an NVIDIA GPU -- CPU-only machines are not supported.
Supports NVIDIA GPU (full training + inference) and CPU-only (GGUF chat mode).
.NOTES
Usage: powershell -ExecutionPolicy Bypass -File setup.ps1
#>
Expand Down Expand Up @@ -251,23 +251,44 @@ Write-Host "+==============================================+" -ForegroundColor G
# ==========================================================================

# ============================================
# 1a. GPU requirement check
# 1a. GPU detection
# ============================================
$HasNvidiaSmi = $false
try {
nvidia-smi 2>&1 | Out-Null
if ($LASTEXITCODE -eq 0) { $HasNvidiaSmi = $true }
} catch {}
# Fallback: nvidia-smi may not be on PATH even though a GPU + driver exist.
# Check the default install location and the Windows driver store.
if (-not $HasNvidiaSmi) {
$nvSmiDefaults = @(
"$env:ProgramFiles\NVIDIA Corporation\NVSMI\nvidia-smi.exe",
"$env:SystemRoot\System32\nvidia-smi.exe"
)
foreach ($p in $nvSmiDefaults) {
if (Test-Path $p) {
try {
& $p 2>&1 | Out-Null
if ($LASTEXITCODE -eq 0) {
$nvSmiDir = Split-Path $p -Parent
$env:Path = "$nvSmiDir;$env:Path"
$HasNvidiaSmi = $true

@chatgpt-codex-connector chatgpt-codex-connector Bot Mar 18, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Persist fallback nvidia-smi path before env refreshes

When fallback GPU detection finds nvidia-smi.exe, this branch only prepends $nvSmiDir to the current process PATH. Later steps call Refresh-Environment, which rebuilds PATH from Machine/User registry values and drops that temporary entry, so subsequent nvidia-smi calls can fail even though $HasNvidiaSmi is still true. In that scenario CUDA driver capability detection is skipped and setup may install an unconstrained latest CUDA toolkit that is incompatible with the installed driver.

Useful? React with 👍 / 👎.

@danielhanchen danielhanchen Jul 6, 2026

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The fallback stores the absolute path in $NvidiaSmiExe at line 288 and every later nvidia-smi call uses that at lines 111, 175 and 459, so it survives Refresh-Environment by construction.

Write-Host " Found nvidia-smi at $nvSmiDir (added to PATH)" -ForegroundColor Gray
break
}
} catch {}
}
}
}
if (-not $HasNvidiaSmi) {
Write-Host ""
Write-Host "[ERROR] Unsloth Studio requires an NVIDIA GPU." -ForegroundColor Red
Write-Host " CPU-only machines are not supported." -ForegroundColor Red
Write-Host "[WARN] No NVIDIA GPU detected. Studio will run in chat-only (GGUF) mode." -ForegroundColor Yellow
Write-Host " Training and GPU inference require an NVIDIA GPU with drivers installed." -ForegroundColor Yellow
Write-Host " https://www.nvidia.com/Download/index.aspx" -ForegroundColor Yellow
Write-Host ""
Write-Host " If you have an NVIDIA GPU, ensure the driver is installed:" -ForegroundColor Yellow
Write-Host " https://www.nvidia.com/Download/index.aspx" -ForegroundColor Yellow
exit 1
} else {
Write-Host "[OK] NVIDIA GPU detected" -ForegroundColor Green
}
Write-Host "[OK] NVIDIA GPU detected" -ForegroundColor Green

# ============================================
# 1a.5. Windows Long Paths (required for deep node_modules / Python paths)
Expand Down Expand Up @@ -341,6 +362,30 @@ if (-not $HasCmake) {
$HasCmake = $null -ne (Get-Command cmake -ErrorAction SilentlyContinue)
} catch { }
}
# winget may succeed but cmake isn't on PATH yet (MSI PATH changes need a
# new shell). Try the default install location as a fallback.
if (-not $HasCmake) {
$cmakeDefaults = @(
"$env:ProgramFiles\CMake\bin",
"${env:ProgramFiles(x86)}\CMake\bin",
"$env:LOCALAPPDATA\CMake\bin"
)
foreach ($d in $cmakeDefaults) {
if (Test-Path (Join-Path $d "cmake.exe")) {
$env:Path = "$d;$env:Path"

@chatgpt-codex-connector chatgpt-codex-connector Bot Mar 18, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Persist fallback CMake path before later refreshes

When fallback discovery finds cmake.exe, this branch only prepends $d to the current process PATH. Later Refresh-Environment calls rebuild PATH from Machine+User registry values, which drops this temporary entry if the installer did not persist PATH, and Phase 4 then takes the new -not $HasCmakeForBuild skip path instead of building llama.cpp. On fresh setups where CMake is present but not yet registered in PATH, setup can complete without llama-server, leaving GGUF inference/export unavailable until a rerun; persist the fallback path (User/Machine) or reapply the fallback before the build check.

Useful? React with 👍 / 👎.

@danielhanchen danielhanchen Jul 6, 2026

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The fallback persists $d to the User PATH registry at lines 390-393 right after prepending it to the process PATH, so Refresh-Environment rebuilds PATH with it intact and the Phase 4 cmake check still finds it.

# Persist to user PATH so Refresh-Environment does not drop it later
$userPath = [Environment]::GetEnvironmentVariable('Path', 'User')
if (-not $userPath -or $userPath -notlike "*$d*") {
[Environment]::SetEnvironmentVariable('Path', "$d;$userPath", 'User')
}
$HasCmake = $null -ne (Get-Command cmake -ErrorAction SilentlyContinue)
if ($HasCmake) {
Write-Host " Found cmake at $d (added to PATH)" -ForegroundColor Gray
break
}
}
}
}
if ($HasCmake) {
Write-Host "[OK] CMake installed" -ForegroundColor Green
} else {
Expand Down Expand Up @@ -389,6 +434,7 @@ if ($vsResult) {
# ============================================
# 1e. CUDA Toolkit (nvcc for llama.cpp build + env vars)
# ============================================
if ($HasNvidiaSmi) {

@chatgpt-codex-connector chatgpt-codex-connector Bot Mar 18, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Guard CUDA env rewrites when skipping toolkit detection

Wrapping the CUDA toolkit section in if ($HasNvidiaSmi) means CPU-only runs never initialize $CudaToolkitRoot, but Phase 4 still unconditionally uses it (e.g., Split-Path $CudaToolkitRoot -Leaf and SetEnvironmentVariable('CudaToolkitDir', "$CudaToolkitRoot\", ...) around lines 1028-1034). On Windows CPU-only setups this produces PowerShell errors and can leave CudaToolkitDir set to \, which may poison subsequent cmake runs in the same session; the CUDA env rewrite block should be gated behind the same GPU check.

Useful? React with 👍 / 👎.

@danielhanchen danielhanchen Jul 6, 2026

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On the current head the Phase 4 CUDA env rewrites are gated behind if ($HasNvidiaSmi -and $CudaToolkitRoot) at line 1137 and the cmake CUDA flags behind if ($HasNvidiaSmi -and $NvccPath) at line 1206, so $CudaToolkitRoot is never dereferenced and CudaToolkitDir is never set on a CPU-only box.

# IMPORTANT: The CUDA Toolkit version must be <= the max CUDA version the
# NVIDIA driver supports. nvidia-smi reports this as "CUDA Version: X.Y".
# If we install a toolkit newer than the driver supports, llama-server will
Expand Down Expand Up @@ -624,11 +670,24 @@ if ($VsInstallPath -and $CudaToolkitRoot) {
Copy-Item "$cudaExtras\*" $vsCustomizations -Force -ErrorAction Stop
Write-Host " [OK] CUDA VS integration files installed" -ForegroundColor Green
} catch {
Write-Host " [WARN] Could not copy CUDA VS integration files (may need admin)" -ForegroundColor Yellow
Write-Host " Manual fix: copy contents of" -ForegroundColor Yellow
Write-Host " $cudaExtras" -ForegroundColor Cyan
Write-Host " into:" -ForegroundColor Yellow
Write-Host " $vsCustomizations" -ForegroundColor Cyan
# Direct copy failed (needs admin). Try elevated copy via Start-Process.
try {
$copyCmd = "Copy-Item '$cudaExtras\*' '$vsCustomizations' -Force"
Start-Process powershell -ArgumentList "-NoProfile -Command $copyCmd" -Verb RunAs -Wait -ErrorAction Stop
$hasTargetsRetry = Get-ChildItem $vsCustomizations -Filter "CUDA *.targets" -ErrorAction SilentlyContinue
if ($hasTargetsRetry) {
Write-Host " [OK] CUDA VS integration files installed (elevated)" -ForegroundColor Green
} else {
throw "Copy did not produce .targets files"
}
} catch {
Write-Host " [WARN] Could not copy CUDA VS integration files" -ForegroundColor Yellow
Write-Host " The llama.cpp build may fail with 'No CUDA toolset found'." -ForegroundColor Yellow
Write-Host " Manual fix: copy contents of" -ForegroundColor Yellow
Write-Host " $cudaExtras" -ForegroundColor Cyan
Write-Host " into:" -ForegroundColor Yellow
Write-Host " $vsCustomizations" -ForegroundColor Cyan
}
}
}
}
Expand All @@ -643,6 +702,9 @@ Write-Host " CudaToolkitDir = $CudaToolkitRoot\" -ForegroundColor Gray
if (-not $CudaArch) {
Write-Host " [WARN] Could not detect compute capability -- cmake will use defaults" -ForegroundColor Yellow
}
} else {
Write-Host "[SKIP] CUDA Toolkit -- no NVIDIA GPU detected" -ForegroundColor Yellow
}

# ============================================
# 1f. Node.js / npm (skip if pip-installed -- only needed for frontend build)
Expand Down Expand Up @@ -880,14 +942,32 @@ $env:TORCHINDUCTOR_CACHE_DIR = $TorchCacheDir
[Environment]::SetEnvironmentVariable('TORCHINDUCTOR_CACHE_DIR', $TorchCacheDir, 'User')
Write-Host "[OK] TORCHINDUCTOR_CACHE_DIR set to $TorchCacheDir (avoids MAX_PATH issues)" -ForegroundColor Green

$CuTag = Get-PytorchCudaTag
Write-Host " Installing PyTorch with CUDA support ($CuTag)..." -ForegroundColor Cyan
pip install torch torchvision torchaudio --index-url "https://download.pytorch.org/whl/$CuTag" 2>&1 | Out-Null
if ($HasNvidiaSmi) {
$CuTag = Get-PytorchCudaTag
Write-Host " Installing PyTorch with CUDA support ($CuTag)..." -ForegroundColor Cyan
Write-Host " (This download is ~2.8 GB -- may take a few minutes)" -ForegroundColor Gray
pip install torch torchvision torchaudio --index-url "https://download.pytorch.org/whl/$CuTag"

@chatgpt-codex-connector chatgpt-codex-connector Bot Jul 6, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Reinstall CUDA torch when switching from CPU-only

When setup is rerun after a prior CPU-only install and the user now has an NVIDIA GPU/driver, the script reuses the existing venv and this plain pip install will report the already-installed CPU torch wheel as satisfying the requirement instead of replacing it from the CUDA index. The rest of the GPU path can build CUDA llama.cpp successfully, but the backend's detect_hardware() still gates full mode on torch.cuda.is_available(), so Studio remains chat-only despite the GPU; force an upgrade/reinstall or detect CPU wheels before skipping the CUDA wheel install.

Useful? React with 👍 / 👎.

if ($LASTEXITCODE -ne 0) {
Write-Host "[FAILED] PyTorch CUDA install failed (exit code $LASTEXITCODE)" -ForegroundColor Red
exit 1
}

# Install Triton for Windows (enables torch.compile — without it training can hang)
Write-Host " Installing Triton for Windows..." -ForegroundColor Cyan
pip install "triton-windows<3.7" 2>&1 | Out-Null
Write-Host "[OK] Triton for Windows installed (enables torch.compile)" -ForegroundColor Green
# Install Triton for Windows (enables torch.compile -- without it training can hang)
Write-Host " Installing Triton for Windows..." -ForegroundColor Cyan
pip install "triton-windows<3.7" 2>&1 | Out-Null
if ($LASTEXITCODE -ne 0) {
Write-Host "[WARN] Triton install failed -- torch.compile may not work" -ForegroundColor Yellow
} else {
Write-Host "[OK] Triton for Windows installed (enables torch.compile)" -ForegroundColor Green
}
} else {
Write-Host " Installing PyTorch (CPU-only)..." -ForegroundColor Cyan
pip install torch torchvision torchaudio
if ($LASTEXITCODE -ne 0) {
Write-Host "[FAILED] PyTorch install failed (exit code $LASTEXITCODE)" -ForegroundColor Red
exit 1
}
}

# Ordered heavy dependency installation — shared cross-platform script
Write-Host " Running ordered dependency installation..." -ForegroundColor Cyan
Expand Down Expand Up @@ -982,12 +1062,40 @@ $LlamaCppDir = Join-Path $UnslothHome "llama.cpp"
$BuildDir = Join-Path $LlamaCppDir "build"
$LlamaServerBin = Join-Path $BuildDir "bin\Release\llama-server.exe"

$HasCmakeForBuild = $null -ne (Get-Command cmake -ErrorAction SilentlyContinue)

# Check if existing llama-server matches current GPU mode. A CUDA-built binary
# on a now-CPU-only machine (or vice versa) needs to be rebuilt.
$NeedRebuild = $false
if (Test-Path $LlamaServerBin) {
$CmakeCacheFile = Join-Path $BuildDir "CMakeCache.txt"
if (Test-Path $CmakeCacheFile) {
$cachedCuda = Select-String -Path $CmakeCacheFile -Pattern 'GGML_CUDA:BOOL=ON' -Quiet
if ($HasNvidiaSmi -and -not $cachedCuda) {
Write-Host " Existing llama-server is CPU-only but GPU is available -- rebuilding" -ForegroundColor Yellow
$NeedRebuild = $true
} elseif (-not $HasNvidiaSmi -and $cachedCuda) {
Write-Host " Existing llama-server was built with CUDA but no GPU detected -- rebuilding" -ForegroundColor Yellow
$NeedRebuild = $true
}
}
}

if ((Test-Path $LlamaServerBin) -and -not $NeedRebuild) {
Write-Host ""
Write-Host "[OK] llama-server already exists at $LlamaServerBin" -ForegroundColor Green
} elseif (-not $HasCmakeForBuild) {
Comment on lines 1083 to +1100

@chatgpt-codex-connector chatgpt-codex-connector Bot Mar 18, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Rebuild llama-server when switching to CPU-only mode

This early-return path skips the new CPU reconfigure logic (GGML_CUDA=OFF + cache cleanup) whenever llama-server.exe already exists, so a host that previously built with CUDA will keep using the stale GPU build even after nvidia-smi is absent. In that CPU-only rerun, setup reports success but never produces a deterministic CPU-only binary, which can leave GGUF chat mode broken or dependent on leftover CUDA runtime files.

Useful? React with 👍 / 👎.

@danielhanchen danielhanchen Jul 6, 2026

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The early return is gated by $NeedRebuild, which inspects CMakeCache.txt for GGML_CUDA:BOOL=ON and forces a rebuild when a CUDA build is found on a now CPU-only host at lines 1082-1097, so the stale GPU binary is replaced by a deterministic CPU-only one.

Write-Host ""
Write-Host "[SKIP] llama-server build -- cmake not available" -ForegroundColor Yellow
Write-Host " GGUF inference and export will not be available." -ForegroundColor Yellow
Write-Host " Install CMake from https://cmake.org/download/ and re-run setup." -ForegroundColor Yellow
} else {
Write-Host ""
Write-Host "Building llama.cpp with CUDA support..." -ForegroundColor Cyan
if ($HasNvidiaSmi) {
Write-Host "Building llama.cpp with CUDA support..." -ForegroundColor Cyan
} else {
Write-Host "Building llama.cpp (CPU-only, no NVIDIA GPU detected)..." -ForegroundColor Cyan
}
Write-Host " This typically takes 5-10 minutes on first build." -ForegroundColor Gray
Write-Host ""

Expand All @@ -1007,17 +1115,19 @@ if (Test-Path $LlamaServerBin) {
# Re-sanitize CUDA_PATH_V* vars — Refresh-Environment (called during
# Node/Python installs above) may have repopulated conflicting versioned
# vars from the Machine registry.
$cudaPathVars2 = @([Environment]::GetEnvironmentVariables('Process').Keys | Where-Object { $_ -match '^CUDA_PATH_V' })
foreach ($v2 in $cudaPathVars2) {
[Environment]::SetEnvironmentVariable($v2, $null, 'Process')
}
$tkDirName2 = Split-Path $CudaToolkitRoot -Leaf
if ($tkDirName2 -match '^v(\d+)\.(\d+)') {
[Environment]::SetEnvironmentVariable("CUDA_PATH_V$($Matches[1])_$($Matches[2])", $CudaToolkitRoot, 'Process')
if ($HasNvidiaSmi -and $CudaToolkitRoot) {
$cudaPathVars2 = @([Environment]::GetEnvironmentVariables('Process').Keys | Where-Object { $_ -match '^CUDA_PATH_V' })
foreach ($v2 in $cudaPathVars2) {
[Environment]::SetEnvironmentVariable($v2, $null, 'Process')
}
$tkDirName2 = Split-Path $CudaToolkitRoot -Leaf
if ($tkDirName2 -match '^v(\d+)\.(\d+)') {
[Environment]::SetEnvironmentVariable("CUDA_PATH_V$($Matches[1])_$($Matches[2])", $CudaToolkitRoot, 'Process')
}
# Also re-assert CUDA_PATH and CudaToolkitDir in case they were overwritten
[Environment]::SetEnvironmentVariable('CUDA_PATH', $CudaToolkitRoot, 'Process')
[Environment]::SetEnvironmentVariable('CudaToolkitDir', "$CudaToolkitRoot\", 'Process')
}
# Also re-assert CUDA_PATH and CudaToolkitDir in case they were overwritten
[Environment]::SetEnvironmentVariable('CUDA_PATH', $CudaToolkitRoot, 'Process')
[Environment]::SetEnvironmentVariable('CudaToolkitDir', "$CudaToolkitRoot\", 'Process')

# -- Step A: Clone or pull llama.cpp --

Expand All @@ -1037,7 +1147,14 @@ if (Test-Path $LlamaServerBin) {
}
}

# -- Step B: cmake configure (CUDA + Unsloth flags) --
# -- Step B: cmake configure --
# Clean stale CMake cache to prevent previous CUDA settings from leaking
# into a CPU-only rebuild (or vice versa).
$CmakeCacheFile = Join-Path $BuildDir "CMakeCache.txt"
if (Test-Path $CmakeCacheFile) {
Remove-Item -Recurse -Force $BuildDir
}

if ($BuildOk) {
Write-Host ""
Write-Host "--- cmake configure ---" -ForegroundColor Cyan
Expand Down Expand Up @@ -1066,37 +1183,45 @@ if (Test-Path $LlamaServerBin) {
$CmakeArgs += '-DLLAMA_CURL=OFF'
}
$CmakeArgs += '-DCMAKE_EXE_LINKER_FLAGS=/NODEFAULTLIB:LIBCMT'
# CUDA flags (Unsloth-aligned)
$CmakeArgs += '-DGGML_CUDA=ON'
$CmakeArgs += "-DCUDAToolkit_ROOT=$CudaToolkitRoot"
$CmakeArgs += "-DCUDA_TOOLKIT_ROOT_DIR=$CudaToolkitRoot"
$CmakeArgs += "-DCMAKE_CUDA_COMPILER=$NvccPath"
$CmakeArgs += '-DGGML_CUDA_FA_ALL_QUANTS=ON'
$CmakeArgs += '-DGGML_CUDA_F16=OFF'
$CmakeArgs += '-DGGML_CUDA_GRAPHS=OFF'
$CmakeArgs += '-DGGML_CUDA_FORCE_CUBLAS=OFF'
$CmakeArgs += '-DGGML_CUDA_PEER_MAX_BATCH_SIZE=8192'
if ($CudaArch) {
# Validate nvcc actually supports this architecture
if (Test-NvccArchSupport -NvccExe $NvccPath -Arch $CudaArch) {
$CmakeArgs += "-DCMAKE_CUDA_ARCHITECTURES=$CudaArch"
} else {
# GPU arch too new for this toolkit — fall back to highest supported.
# PTX forward-compatibility will JIT-compile for the actual GPU at runtime.
$maxArch = Get-NvccMaxArch -NvccExe $NvccPath
if ($maxArch) {
$CmakeArgs += "-DCMAKE_CUDA_ARCHITECTURES=$maxArch"
Write-Host " [WARN] GPU is sm_$CudaArch but nvcc only supports up to sm_$maxArch" -ForegroundColor Yellow
Write-Host " Building with sm_$maxArch (PTX will JIT for your GPU at runtime)" -ForegroundColor Yellow
# CUDA flags -- only if GPU available, otherwise explicitly disable
if ($HasNvidiaSmi -and $NvccPath) {
$CmakeArgs += '-DGGML_CUDA=ON'
$CmakeArgs += "-DCUDAToolkit_ROOT=$CudaToolkitRoot"
$CmakeArgs += "-DCUDA_TOOLKIT_ROOT_DIR=$CudaToolkitRoot"
$CmakeArgs += "-DCMAKE_CUDA_COMPILER=$NvccPath"
Comment thread
danielhanchen marked this conversation as resolved.
if ($CudaArch) {
# Validate nvcc actually supports this architecture
if (Test-NvccArchSupport -NvccExe $NvccPath -Arch $CudaArch) {
$CmakeArgs += "-DCMAKE_CUDA_ARCHITECTURES=$CudaArch"
} else {
# GPU arch too new for this toolkit -- fall back to highest supported.
# PTX forward-compatibility will JIT-compile for the actual GPU at runtime.
$maxArch = Get-NvccMaxArch -NvccExe $NvccPath
if ($maxArch) {
$CmakeArgs += "-DCMAKE_CUDA_ARCHITECTURES=$maxArch"
Write-Host " [WARN] GPU is sm_$CudaArch but nvcc only supports up to sm_$maxArch" -ForegroundColor Yellow
Write-Host " Building with sm_$maxArch (PTX will JIT for your GPU at runtime)" -ForegroundColor Yellow
}
# else: omit flag entirely, let cmake pick defaults
}
# else: omit flag entirely, let cmake pick defaults
}
Comment thread
danielhanchen marked this conversation as resolved.
} else {
$CmakeArgs += '-DGGML_CUDA=OFF'
}

cmake @CmakeArgs 2>&1 | Out-Null
$cmakeOutput = cmake @CmakeArgs 2>&1 | Out-String
if ($LASTEXITCODE -ne 0) {
$BuildOk = $false
$FailedStep = "cmake configure"
Write-Host $cmakeOutput -ForegroundColor Red
if ($cmakeOutput -match 'No CUDA toolset found|CUDA_TOOLKIT_ROOT_DIR|nvcc') {
Write-Host ""
Write-Host " Hint: CUDA VS integration may be missing. Try running as admin:" -ForegroundColor Yellow
Write-Host " Copy contents of:" -ForegroundColor Yellow
Write-Host " <CUDA_PATH>\extras\visual_studio_integration\MSBuildExtensions" -ForegroundColor Yellow
Write-Host " into:" -ForegroundColor Yellow
Write-Host " <VS_PATH>\MSBuild\Microsoft\VC\v170\BuildCustomizations" -ForegroundColor Yellow
}
}
}

Expand Down
Loading