Fix Windows binary crashing (STATUS_ILLEGAL_INSTRUCTION) on CPUs without AVX-512 - #829
Conversation
The Windows native build configured llama.cpp/ggml without GGML_NATIVE=OFF, unlike the Linux and macOS builds in scripts/build-llama.sh. On the windows-2025 release runner GGML_NATIVE then defaults ON, so ggml is compiled for the runner's CPU. Those runners can expose AVX-512, so the shipped binary contains AVX-512 instructions. On consumer desktops without AVX-512 (e.g. Intel Alder/Raptor Lake i5/i7) the process crashes with STATUS_ILLEGAL_INSTRUCTION (0xC000001D) the moment a ggml compute kernel runs during inference. Pin GGML_NATIVE=OFF and select a portable AVX2 baseline (AVX2 implies F16C and FMA on MSVC), with AVX-512 and BMI2 explicitly off, so the emitted instruction set is independent of the build host and runs on every x86-64 CPU since ~2013. Confirmed against the current shipped release mesh-llm.exe: disassembly shows 8257 zmm-register and 820 opmask-register AVX-512 instructions baked into the binary with no runtime guard, which is exactly what faults on a non-AVX-512 CPU. With GGML_AVX512=OFF and a fixed AVX2 baseline, ggml-cpu's CMake emits only the AVX2/FMA/F16C codepaths, so no AVX-512 instructions are generated.
📝 WalkthroughWalkthroughPins GGML CPU feature flags for Windows builds to an AVX/AVX2 baseline (disables GGML_NATIVE, AVX-512, BMI2) and replaces PowerShell Get-FileHash usage with a new Get-Sha256Hex helper that computes SHA-256 via .NET for checksum sidecar files. ChangesWindows Build Configuration
Packaging checksum
🎯 3 (Moderate) | ⏱️ ~20 minutes 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
|
this is still a WIP - I have a PC at home I will try this with |
…Hash The Windows release bundling step (package-release.ps1, invoked as `powershell -NoProfile -File ...`) failed with CommandNotFoundException for Get-FileHash on the windows-2025 runner, because module autoloading of Microsoft.PowerShell.Utility does not reliably resolve under that invocation. This aborted release-bundle-windows before the bundle and its .sha256 sidecars were produced. Compute the SHA-256 with the .NET System.Security.Cryptography API directly, which is always available regardless of module autoloading. Verified on Windows PowerShell 5.1 that the digest matches Get-FileHash.
088a043 to
5f478a2
Compare
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@scripts/package-release.ps1`:
- Around line 175-177: Set the checksum sidecar file to a deterministic ASCII
encoding to avoid BOM/leading bytes differences across PowerShell versions: when
writing the string composed of $hash and $name with Set-Content (the block using
Get-Sha256Hex, $hash, $Path, $name), add the explicit -Encoding ASCII (or
System.Text.Encoding::ASCII) parameter to Set-Content so the written file is
plain ASCII and Linux tools like awk '{print $1}' correctly extract the digest.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro Plus
Run ID: 3ee6a479-27df-4a33-b45e-d4282cf8a398
📒 Files selected for processing (1)
scripts/package-release.ps1
| $hash = Get-Sha256Hex $Path | ||
| $name = Split-Path -Leaf $Path | ||
| Set-Content -Path "$Path.sha256" -Value "$hash $name" -NoNewline |
There was a problem hiding this comment.
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
# Read-only verification: confirm sidecar writer has no explicit encoding
# and confirm downstream parser reads first whitespace token via awk.
set -euo pipefail
echo "== Sidecar write implementation =="
rg -n -C2 'Set-Content\s+-Path\s+"\$Path\.sha256"' scripts/package-release.ps1
echo
echo "== Downstream sidecar token parser =="
rg -n -C2 "awk '\\{print \\$1\\}'" scripts/verify-native-sdk-package.sh
echo
echo "== Linux sidecar formatting contract =="
rg -n -C3 "printf '%s %s\\\\n'" scripts/package-release.shRepository: Mesh-LLM/mesh-llm
Length of output: 354
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo "== Sidecar write implementation (package-release.ps1) =="
rg -n -C3 'function\s+New-ChecksumSidecar|Set-Content\s+-Path\s+"\$Path\.sha256"' scripts/package-release.ps1
echo
echo "== Downstream sidecar token parser (verify-native-sdk-package.sh) =="
rg -n -C3 'awk\s*\x27\{\s*print\s*\$1\s*\}\x27|awk\s*\x27\{print\s*\$1\s*\}\x27|awk\s*\x27\{print\s*\$1' scripts/verify-native-sdk-package.sh || true
echo
echo "== Downstream regex/consume sites (install.ps1) =="
rg -n -C3 '\.sha256|checksum|sha256' install.ps1
echo
echo "== Linux sidecar formatting contract (package-release.sh) =="
rg -n -C3 "printf\s+'%s\s+%s\\\\n'|printf\s+'%s %s\\\\n'|\.sha256" scripts/package-release.shRepository: Mesh-LLM/mesh-llm
Length of output: 3053
🌐 Web query:
PowerShell Set-Content default encoding UTF-16LE BOM behavior -NoNewline
💡 Result:
The behavior of the Set-Content cmdlet, including the impact of the -NoNewline parameter and default encoding, depends significantly on whether you are using Windows PowerShell (v5.1 and earlier) or PowerShell (v6+). The -NoNewline parameter itself does not influence character encoding or the addition of a Byte Order Mark (BOM) [1][2]. It strictly instructs the cmdlet to omit the trailing newline (line break) that is otherwise appended to the output by default [1][2][3]. Regarding character encoding and BOM behavior: 1. PowerShell (v6 and higher): The default encoding for Set-Content is utf8NoBOM [1][2]. Because it defaults to a BOM-less encoding, it does not add a BOM unless you explicitly specify -Encoding utf8BOM [4][1][5]. 2. Windows PowerShell (v5.1 and earlier): Set-Content defaults to the system's active ANSI legacy code page (often referred to as Default) [4][6][7]. In this version, Windows PowerShell behaves differently with Unicode encodings: if you explicitly set -Encoding Unicode (which is UTF-16LE), it will include a BOM [4][8]. Notably, Windows PowerShell v5.1 does not provide a native -Encoding option to produce BOM-less UTF-8; specifying -Encoding UTF8 in these versions will always include a BOM [9][5]. In summary, the -NoNewline switch only affects the presence of the terminal line break [1]. The presence of a BOM is determined entirely by the -Encoding parameter and the version of PowerShell being used [4][10][11]. To ensure consistent behavior across all versions of PowerShell, it is recommended to specify the encoding explicitly—for example, using -Encoding utf8NoBOM in PowerShell 7+ [4][11].
Citations:
- 1: https://learn.microsoft.com/en-us/powershell/module/microsoft.powershell.management/set-content?view=powershell-7.6
- 2: https://learn.microsoft.com/en-us/powershell/module/microsoft.powershell.management/set-content?view=powershell-7.5
- 3: https://learn.microsoft.com/en-us/archive/blogs/jmanning/powershell-gotcha-of-the-day-set-content-adds-newlines-by-default
- 4: https://learn.microsoft.com/en-us/powershell/module/microsoft.powershell.core/about/about_character_encoding?view=powershell-7.6
- 5: https://stackoverflow.com/questions/66553344/powershell-set-content-replace-word-and-encoding-utf8-without-bom
- 6: https://github.com/MicrosoftDocs/PowerShell-Docs/blob/main/reference/7.5/Microsoft.PowerShell.Core/About/about_Character_Encoding.md
- 7: Set-Content / Add-Content help topics incorrectly claim that the default character encoding is ASCII - it is "ANSI" MicrosoftDocs/PowerShell-Docs#1483
- 8: https://github.com/MicrosoftDocs/PowerShell-Docs/blob/main/reference/7.4/Microsoft.PowerShell.Core/About/about_Character_Encoding.md
- 9: https://stackoverflow.com/questions/77987322/powershell-5-1-output-to-a-text-file-with-out-file-set-content-and-utf8
- 10: https://shellmap.eversources.app/cmd/set-content
- 11: https://www.commandinline.com/powershell-out-file-vs-set-content/
🌐 Web query:
GNU awk behavior when input file contains NUL bytes UTF-16LE
💡 Result:
GNU awk (gawk) does not support UTF-16LE input natively [1][2]. Because UTF-16LE encodes characters using two bytes (or more) and frequently includes NUL bytes (0x00) in the byte stream, it is incompatible with gawk's processing model [1][2]. Gawk operates on C-style strings internally, where the NUL byte is treated as a string terminator [3][4]. Consequently, when gawk encounters a NUL byte in the input, it effectively treats it as the end of the string or record, causing data corruption, truncated input, or failures in pattern matching and field splitting [3][4][1]. While you can force gawk to treat input as raw bytes using the --characters-as-bytes (-b) option or by setting the locale to "C", this does not resolve the encoding mismatch [5][6][7]. Even in "byte mode," gawk will see the UTF-16LE stream as a sequence of individual bytes, meaning standard field separators (like a comma or pipe) will not be recognized because they are encoded as two bytes (e.g., 0x2C 0x00) rather than a single byte [1][2]. The standard and most reliable way to process UTF-16LE files with gawk is to convert the input to a format gawk understands (such as UTF-8) before processing [1][2]. This is typically done using the iconv utility: iconv -f UTF-16LE -t UTF-8 input.file | gawk -F'|' '{... }' If you need to output the data back into UTF-16LE, you can pipe the result of your awk command back through iconv:... | iconv -f UTF-8 -t UTF-16LE > output.file [2]
Citations:
- 1: https://stackoverflow.com/questions/75253663/simple-pattern-match-with-a-field-and-a-variable-does-not-seem-to-work-in-gawk-a
- 2: https://unix.stackexchange.com/questions/338789/centos7-awk-supporting-utf16le
- 3: https://www.gnu.org/software/gawk/manual/html_node/gawk-split-records.html
- 4: https://unix.stackexchange.com/questions/564109/what-is-posix-awks-stance-on-null-byte-in-variables-printf
- 5: https://www.gnu.org/software/gawk/manual/html_node/Options.html
- 6: https://man.archlinux.org/man/gawk.1.en.txt
- 7: https://lists.gnu.org/r/bug-gawk/2018-08/msg00041.html
Specify checksum sidecar encoding for deterministic Linux verification (awk '{print $1}')
Set-Content in scripts/package-release.ps1 has no explicit -Encoding, so the output encoding/leading bytes vary by PowerShell version. The Linux verifier extracts the expected digest using the first whitespace-delimited token (awk '{print $1}'), so the sidecar should be written as plain ASCII (no BOM/extra leading bytes).
Suggested fix
function New-ChecksumSidecar {
param([string]$Path)
$hash = Get-Sha256Hex $Path
$name = Split-Path -Leaf $Path
- Set-Content -Path "$Path.sha256" -Value "$hash $name" -NoNewline
+ Set-Content -Path "$Path.sha256" -Value "$hash $name" -NoNewline -Encoding Ascii
}🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@scripts/package-release.ps1` around lines 175 - 177, Set the checksum sidecar
file to a deterministic ASCII encoding to avoid BOM/leading bytes differences
across PowerShell versions: when writing the string composed of $hash and $name
with Set-Content (the block using Get-Sha256Hex, $hash, $Path, $name), add the
explicit -Encoding ASCII (or System.Text.Encoding::ASCII) parameter to
Set-Content so the written file is plain ASCII and Linux tools like awk '{print
$1}' correctly extract the digest.
ndizazzo
left a comment
ndizazzo
left a comment
There was a problem hiding this comment.
Looks clear! nothing out of whack to my eye
2 participants
What this fixes
Windows users on CPUs without AVX-512 — which includes all Intel consumer
desktops from 11th–14th gen (Alder/Raptor Lake), e.g. the i5/i7-12xxx/13xxx —
got an immediate hard crash (
STATUS_ILLEGAL_INSTRUCTION,0xC000001D) themoment inference actually ran. The binary would load a model, then die at
compute-graph / KV-cache allocation. After this change the Windows release
binaries run on those machines.
Root cause
scripts/build-windows.ps1did not setGGML_NATIVE, while the Linux/macOSpath in
scripts/build-llama.shdoes setGGML_NATIVE=OFF. WithGGML_NATIVEunset and the build not cross-compiling, llama.cpp defaults
GGML_NATIVE=ON,so ggml-cpu was compiled for the build runner's CPU. The
windows-2025runners expose AVX-512, so the shipped binary contained AVX-512 instructions
with no runtime CPU-feature guard. Disassembly of the released
mesh-llm.exeshowed 8,257 AVX-512 (zmm) instructions baked into thecompute path.
The fix
scripts/build-windows.ps1: pinGGML_NATIVE=OFF(matching Linux/macOS) andselect a portable AVX2 baseline (
GGML_AVX2=ON,GGML_AVX512=OFF,GGML_BMI2=OFF). AVX2/FMA/F16C are supported by every x86-64 CPU since ~2013.scripts/package-release.ps1: compute release-archive SHA-256 checksums viathe .NET
SHA256API instead ofGet-FileHash. Underpowershell -NoProfileon the runners,
Get-FileHashfailed to autoload (CommandNotFoundException),which broke release bundling and prevented
.sha256sidecars from beinggenerated. (This is the same sidecar the installer fix in Fix Windows install on PowerShell 5.1 (irm | iex and missing checksum sidecar) #828 consumes.)
Validation
Built a canary CUDA bundle for sm_89 from this branch and ran it on a real
RTX 4060 + Intel i5-12400F (no AVX-512):
0xC000001Dright afterload_tensors: CPU_Mapped model buffer size./arch:AVX2(
GGML_AVX2;GGML_FMA;GGML_F16C, no AVX-512), loads the model onto the GPU,reaches
mesh-llm runtime ready, and returns a real chat completion(
"Hello! How can I help you today?") with the process still alive after..sha256sidecars (digest matches the archive).
Notes
a large share of Windows desktops, not an edge case.
local_capacity_gb=0.0(VRAM-based) and refuses to run even a 0.5 GB modellocally, forcing a split. CPU nodes should size local capacity from system RAM.
Summary by CodeRabbit
Bug Fixes
Chores