Skip to content

fix(desktop): enable Windows mesh-llm builds and address Compute Share startup/MeshLLM debug logging/non-image models trying to parse image input - #3223

Open
stevepresley wants to merge 14 commits into
block:mainfrom
stevepresley:fix/windows-mesh-llm-2836-upstream
Open

Conversation

@stevepresley

@stevepresley stevepresley commented Jul 27, 2026

Copy link
Copy Markdown

fix: enable Windows mesh-llm desktop builds

Closes #2836
Closes #3300

Summary

  • Enable the mesh-llm feature for Windows release and canary desktop builds.
  • Bundle required Windows MeshLLM native runtime DLL dependencies as Tauri resources and register their directory at runtime.
  • Avoid Windows stack overflows during mesh start and managed-agent start by running heavy startup work on larger-stack OS threads.
  • Make Compute Share startup more resilient/idempotent and fix stale progress UI behavior.
  • Add opt-in MeshLLM diagnostic logging controls in Compute → Advanced.
  • Add relay-mesh protection so prior tool-result images are not replayed into text-only shared-compute LLM requests.

Validation

Validated locally on Windows 11:

  • cargo fmt --manifest-path desktop/src-tauri/Cargo.toml --all
  • cargo check --manifest-path desktop/src-tauri/Cargo.toml --features mesh-llm
  • pnpm typecheck
  • cargo check -p buzz-agent
  • cargo build -p buzz-agent --release --target x86_64-pc-windows-msvc
  • bash scripts/bundle-sidecars.sh x86_64-pc-windows-msvc
  • pnpm tauri build --target x86_64-pc-windows-msvc --bundles nsis --features mesh-llm --config "{\"bundle\":{\"createUpdaterArtifacts\":false}}"

Manual validation:

  • Installed the NSIS build on Windows 11.
  • Verified Settings → Compute no longer shows the mesh-llm feature-stub behavior.
  • Verified Compute Share starts and serves requests.
  • Verified Windows MeshLLM native runtime dependency loading works with bundled MinGW DLLs.
  • Verified managed buzz-agent starts from the installed app.
  • Verified relay-mesh LLM calls no longer fail with the text-only model media-input 422 caused by replayed tool-result images.
  • Verified MeshLLM diagnostic logging toggle works at runtime.

Related/out of scope findings

Windows relay-mesh validation also reproduced existing ACP/agent delivery behavior where a model may produce Activity/final text without publishing a DM/channel message, and failed ACP turns can be retried later. That is out of scope for this Windows mesh-llm packaging/runtime PR.

Refs #2698
Refs #2421
Refs #2681

@stevepresley
stevepresley requested a review from a team as a code owner July 27, 2026 22:08
@michaelneale

Copy link
Copy Markdown
Contributor

@stevepresley thanks - that is a big change and not all windows related is it?

@stevepresley

stevepresley commented Jul 28, 2026

Copy link
Copy Markdown
Author

@stevepresley thanks - that is a big change and not all windows related is it?

It started as mostly Windows related, but these three items are cross-platform.

  • Make Compute Share startup more resilient/idempotent and fix stale progress UI behavior.
  • Add opt-in MeshLLM diagnostic logging controls in Compute → Advanced.
  • Add relay-mesh protection so prior tool-result images are not replayed into text-only shared-compute LLM requests.

I'll update the title

@stevepresley stevepresley changed the title fix(desktop): enable Windows mesh-llm builds fix(desktop): enable Windows mesh-llm builds and address Compute Share startup/MeshLLM debug logging/non-image models trying to parse image input Jul 28, 2026
@stevepresley

stevepresley commented Jul 28, 2026

Copy link
Copy Markdown
Author

@michaelneale opened #3300 to track the 3 other issues fixed in this PR. Let me know if you want me to split it out into a second PR

Signed-off-by: stevepresley <github@stevepresley.net>
…m-2836-upstream

Signed-off-by: stevepresley <github@stevepresley.net>

# Conflicts:
#	desktop/src-tauri/src/commands/mesh_llm.rs
#	desktop/src/features/mesh-compute/ui/MeshComputeSettingsCard.tsx
Signed-off-by: stevepresley <github@stevepresley.net>
@stevepresley
stevepresley force-pushed the fix/windows-mesh-llm-2836-upstream branch from dcd0ed9 to c35bbda Compare July 31, 2026 18:30
Signed-off-by: stevepresley <github@stevepresley.net>
@stevepresley
stevepresley force-pushed the fix/windows-mesh-llm-2836-upstream branch from 073672b to b478dad Compare July 31, 2026 18:44
tlongwell-block added a commit that referenced this pull request Aug 3, 2026
…#4524)

## Summary

Official Linux desktop packages (`.deb` / AppImage) are built without
`--features mesh-llm`, so they ship the `mesh_llm_stubs` backend and
Settings → Compute always fails with `mesh-llm feature not enabled`.
This PR adds the feature flag to the two Linux build commands:

- `release.yml` → `release-linux` job
- `linux-canary.yml` → canary build

That's the whole diff — 2 lines. Fixes #3788 (Linux); see also #3841
(dup with UI-gating PR #3914) and the Windows twin #2836/#3223.

## Why no native prebuild step (unlike the macOS job)

The macOS job carries Metal llama prebuild/cache steps from #798. Linux
doesn't need an equivalent:

- `mesh-llm-host-runtime` is compiled with `dynamic-native-runtime` and
installs the recommended runtime on first use (verified by sha256
checksum over HTTPS; upstream's signature verification path is not yet
implemented — default policy is `RequireChecksum`, per
`mesh-llm-runtime-install/src/lib.rs`)
(`desktop/src-tauri/src/mesh_llm/mod.rs` —
`initialize_mesh_native_runtime`), so release builds work on clean
machines without bundling llama.cpp.
- Upstream publishes Linux x86_64/aarch64 runtime bundles for the pinned
`v0.74.0` line, and `scripts/ensure-mesh-native-runtime.sh` already maps
`meshllm-native-runtime-linux-x86_64-cpu` / `linux-aarch64-cpu` for
local/e2e use.
- The unmerged branch `micn/mesh-node-download` (`96f29417a`) treats
even the macOS prebuild steps as removable dead weight for the same
reason.

## Background

The omission is historical drift, not a decision: Linux packaging
predates the mesh feature flag (#693), mesh became opt-in for
build-cost/reliability reasons (#823, #1183), and #1221 re-enabled it
for releases by editing only the macOS build line. `release-linux` and
the later `linux-canary` copy were never revisited.

The mesh shutdown hard-exit/relaunch path is gated `all(mesh-llm,
target_os = "macos")` because ggml/Metal destructors abort on macOS;
ordinary mesh shutdown (`shutdown_mesh_runtime`) is cross-platform, so
Linux falls through to the generic path.

## Validation

- [x] `./bin/cargo check --manifest-path desktop/src-tauri/Cargo.toml
--features mesh-llm` green at base `2c0ac2467` (feature graph compiles
at the pinned v0.74.0 line)
- [ ] Linux canary run with this change: AppImage/.deb build succeeds
and binary contains real `mesh_llm` symbols (not `mesh_llm_stubs`)
- [ ] Installed package: cold-start → Settings → Compute → runtime
download → serve → clean shutdown

The last two need a Linux run/host. **Note (from review):**
`linux-canary.yml` is `workflow_dispatch`-only and its `Require main`
step rejects non-main refs, so the canary cannot run on this branch
pre-merge — and `.github/workflows/**` matches no ci.yml paths-filter,
so this PR's own CI does not exercise the changed lines. Validation
sequencing is therefore merge → dispatch linux-canary on main →
live-package pass, with a trivial 2-line revert as the escape hatch.

Signed-off-by: npub1qyvc0c5kl4gqv2fd97fsk46tu378sqgy35vc83rvgfwne90sel7s0ed67d <011987e296fd5006292d2f930b574be47c7801048d1983c46c425d3c95f0cffd@buzz.block.builderlab.xyz>
Co-authored-by: npub1qyvc0c5kl4gqv2fd97fsk46tu378sqgy35vc83rvgfwne90sel7s0ed67d <011987e296fd5006292d2f930b574be47c7801048d1983c46c425d3c95f0cffd@buzz.block.builderlab.xyz>
Signed-off-by: stevepresley <github@stevepresley.net>
@jds1-ai

jds1-ai commented Aug 9, 2026

Copy link
Copy Markdown

Built and tested this PR on Windows 11 / RTX 5060 Ti. The feature side works. Two things worth
adding: one small CI blocker in the PR itself, and one blocker that sits entirely upstream of Buzz.

The build is correct — verified against a control

Built fix/windows-mesh-llm-2836-upstream merged onto current main, via windows-canary.yml.
Scanning the resulting buzz-desktop.exe for ASCII strings, with the stock 0.5.8 release binary as
a negative control:

string this PR's build stock 0.5.8
mesh-llm feature not enabled 0 1
mesh_llm_stubs 0 1
mesh-llm-host-runtime 813 0
iroh 445 0

Binary 147.9 MB vs 86.5 MB. Settings → Compute renders a real panel, detects the GPU, and populates
the model catalogue — no stub error. #2836's symptom is resolved by this PR.

1. CI blocker: windows-latest no longer ships libgomp-1.dll

scripts/bundle-windows-mesh-runtime-deps.ps1 fails on a stock runner. The job gets through sidecar
compilation and then:

Bundled libgcc_s_seh-1.dll from C:\Program Files\Git\mingw64\bin\...
Bundled libstdc++-6.dll     from C:\Program Files\Git\mingw64\bin\...
Exception: scripts\bundle-windows-mesh-runtime-deps.ps1:31
  Missing Windows MeshLLM runtime dependency: libgomp-1.dll
##[error]Process completed with exit code 1

Runner image is now windows-2025-vs2026. None of the first three search directories
(C:\msys64\mingw64\bin, C:\msys64\ucrt64\bin, C:\ProgramData\mingw64\mingw64\bin) are
populated on it, so the script falls through to Git for Windows' MinGW — which carries three of the
four DLLs but not the OpenMP runtime. The C:\ProgramData\mingw64\mingw64\bin entry in the search
list suggests this was written against an older image that installed MinGW via Chocolatey there.

The script fails loudly rather than shipping a partial bundle, which is the right behaviour.

We worked around it with a step before the bundler that installs the toolchain and then asserts
all four are resolvable:

- name: Provide MinGW runtime DLLs for the bundler
  shell: pwsh
  run: |
    $need = 'libgcc_s_seh-1.dll','libstdc++-6.dll','libgomp-1.dll','libwinpthread-1.dll'
    $dirs = 'C:\msys64\mingw64\bin','C:\msys64\ucrt64\bin',
            'C:\ProgramData\mingw64\mingw64\bin','C:\Program Files\Git\mingw64\bin'
    function Get-Missing { $need | Where-Object { $n=$_; -not ($dirs | Where-Object { Test-Path (Join-Path $_ $n) }) } }
    if (Get-Missing) { & C:\msys64\usr\bin\pacman.exe -Sy --noconfirm --needed mingw-w64-x86_64-gcc }
    if (Get-Missing) { choco install mingw -y --no-progress | Out-Null }
    $still = Get-Missing
    if ($still) { throw "MinGW runtime still unavailable: $($still -join ', ')" }

MSYS2's pacman resolved it on the runner (missing before: libgomp-1.dll, then supplied). With
that step the canary builds clean and produces the NSIS installer.

2. Bundled DLLs replace the runtime's own copies

Worth knowing regardless of the above: after the app runs, the libstdc++-6.dll inside
%LOCALAPPDATA%\mesh-llm\native-runtimes\<ver>\<runtime-id>\lib is byte-identical to the copy in
resources/mesh-llm/windows-x86_64, where before it matched the one shipped in the runtime archive
(different SHA-256 and mtime). So the bundled dependencies don't merely sit earlier in the search
order — they end up replacing what the runtime shipped with itself.

In our case it changed nothing, because both builds are missing the same symbols (see below). But if
a future runtime archive ships a correct libstdc++-6.dll, this would silently substitute a
different one for it.

3. The remaining blocker is upstream of Buzz

With the build correct and the panel working, enabling Share this machine still fails:

mesh native runtime failed to install or load:
load native runtime meshllm-native-runtime-windows-x86_64-vulkan ...: LoadLibraryExW failed

Two independent upstream causes, both measured against freshly downloaded MeshLLM archives with no
Buzz involved:

(a) The MinGW Windows runtimes don't load. ggml.dll imports 54 symbols from libstdc++-6.dll
and 5 are absent from the copy shipped in the same archive — std::codecvt_utf8_utf16<wchar_t> /
std::codecvt<wchar_t,char,mbstate_t> members. LoadLibraryExW returns
ERROR_PROC_NOT_FOUND (127). Reproduces on vulkan and cpu, at v0.74.0 and v0.75.0. The
MSVC-built cuda12 artifact is unaffected. Filing separately with MeshLLM.

(b) No Blackwell-capable Windows CUDA runtime exists. native-runtimes.json gives
windows-x86_64-cuda12 gpu_arches = 75,80,86,87,89,90; sm_120 appears only on
linux-x86_64-cuda13. On an RTX 50-series card the resolver correctly rejects cuda12 and falls
through to the broken vulkan.

Net: on Windows + Blackwell, Share Compute cannot start a node even with this PR merged. On
sm 75–90 hardware the cuda12 path looks viable, provided cudart64_12 / cublas64_12 /
cublasLt64_12 are available — they are not on a machine with current (CUDA 13) drivers, and adding
them made all five CUDA libraries load for us.

None of this is an argument against merging — the PR does what it says. It is an argument for
expecting #2836 reports to continue from RTX 50-series users afterwards, and possibly for surfacing
the resolver's rejection reason in the UI: at verbose logging the app reports only that the chosen
runtime failed to load, never that a different one was evaluated and rejected, which makes this
considerably harder to diagnose than it needs to be.

Minor

  • Opening Settings → Compute and toggling sharing spawns briefly visible console windows.
  • The model catalogue reports vram=32.558606745 on a 16 GB card, and the panel shows
    "32 GB AI memory", so recommendations are sized against roughly double the real VRAM.

Happy to test builds on Windows 11 / RTX 5060 Ti.

jds1-ai added a commit to jds1-ai/buzz that referenced this pull request Aug 10, 2026
…wn bootstrap

Our 08-09 fork run installed the MinGW runtime DLLs before the bundler. PR block#3223
now bootstraps them itself (pacman -> choco -> assert). Leaving our step in would
let that bootstrap borrow strength from ours and pass without being exercised, so
it is removed.

Replaced with a read-only probe that installs nothing and records which of the
four DLLs the stock windows-latest runner actually has, so a bundler PASS can be
attributed to the bootstrap rather than to a runner that already had them.

Signed-off-by: jds1-ai <jds1-ai@users.noreply.github.com>

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@stevepresley

Copy link
Copy Markdown
Author

Thanks @jds1-ai - made the following updates:

  • Fixed Windows canary/CI DLL bundling failure:

    • Bundler now bootstraps missing MinGW runtime DLLs via MSYS2 pacman, then Chocolatey fallback.
    • Still fails loudly if required DLLs remain unavailable.
  • Avoided replacing MeshLLM runtime archive DLLs:

    • Buzz no longer overwrites DLLs already present in the downloaded MeshLLM runtime lib directory.
    • Bundled DLLs are only copied when missing.
  • Fixed Windows catalog VRAM over-reporting:

    • Settings → Compute catalog now uses dedicated GPU VRAM on Windows instead of MeshLLM’s survey value that includes system RAM offload.
    • A 16 GB card should no longer display/recommend as ~32 GB AI memory.
  • Reduced/removed console window flashes during catalog probing:

    • Windows catalog probes for nvidia-smi / PowerShell now run with hidden child-process flags.

Validation performed:

  • cargo fmt
  • git diff --check
  • PowerShell script parse check
  • cargo check --features mesh-llm from desktop/src-tauri

Give it another go and let me know how it works for you. I'm trying to get another box with an nVidia card setup to test locally as well.

@jds1-ai

jds1-ai commented Aug 10, 2026

Copy link
Copy Markdown

Tested 2b16d630 on Windows 11 / RTX 5060 Ti. Thanks for picking up the CI blocker — one result,
one observation about a second hunk in the same commit, and one note about coverage.

1. The bundler bootstrap works on a stock runner — confirmed

I re-ran the canary in a fork with our own MinGW-install workaround removed, so the bootstrap in
bundle-windows-mesh-runtime-deps.ps1 is what's actually under test rather than something already
satisfied for it. The only fork edit is the github.repository job guard.

To make the result attributable I added a read-only probe before the bundler that installs nothing
and records the stock runner's pre-state — otherwise a runner that happened to ship the four DLLs
would let the bootstrap "pass" without executing anything:

runner image: win25-vs2026 / 20260803.193.1
dir C:\msys64\mingw64\bin                         exists=True
dir C:\msys64\ucrt64\bin                          exists=True
dir C:\ProgramData\mingw64\mingw64\bin            exists=False
dir C:\Program Files\Git\mingw64\bin              exists=True
pacman present: True
choco present:  True
PRESENT  libgcc_s_seh-1.dll  <-  C:\Program Files\Git\mingw64\bin
PRESENT  libstdc++-6.dll     <-  C:\Program Files\Git\mingw64\bin
MISSING  libgomp-1.dll
PRESENT  libwinpthread-1.dll <-  C:\Program Files\Git\mingw64\bin
PROBE_MISSING_BEFORE=libgomp-1.dll

So the runner really was missing libgomp-1.dll — the bootstrap had something to do, and the
result is attributable to it. Your script then:

Missing Windows MeshLLM runtime dependencies before bootstrap: libgomp-1.dll
... pacman installs mingw-w64-x86_64-gcc (487.68 MiB, gcc 16.2.0)
Bundled libgcc_s_seh-1.dll from C:\msys64\mingw64\bin\libgcc_s_seh-1.dll
Bundled libstdc++-6.dll     from C:\msys64\mingw64\bin\libstdc++-6.dll
Bundled libgomp-1.dll       from C:\msys64\mingw64\bin\libgomp-1.dll
Bundled libwinpthread-1.dll from C:\msys64\mingw64\bin\libwinpthread-1.dll

PASS on the MSYS2 branch; the choco fallback was not exercised (and so remains untested by this
run). Whole job ~22 min against a warm Cargo cache, comfortably inside the 60-minute timeout;
pacman added roughly 30 s. This is the same fork pipeline that previously failed at exactly this
step, with our own MinGW workaround removed, so it's a real before/after.

To be precise about what this rests on: the verdict comes from the bundler step's own log, not
from the artifact — see §5, the canary's uploaded artifact turned out to be stale. The bootstrap
demonstrably ran and did its work; that part is unaffected.

One incidental effect worth knowing: because C:\msys64\mingw64\bin sorts first in the candidate
list, once pacman runs all four DLLs come from MSYS2 GCC 16.2.0 rather than from Git for
Windows. The bundled libstdc++-6.dll is therefore a different, newer build than the one a
developer bundling locally off Git for Windows would get. That interacts with §2 below, and it also
means the bundle's contents now depend on which branch of the bootstrap fired — worth pinning if
reproducibility matters.

(Minor: pacman -Sy without -Su is the documented MSYS2 partial-upgrade hazard. It worked here,
and on a throwaway runner the blast radius is nil, but -Syu is the safer spelling.)

2. mesh_llm.rs: the new skip-if-present guard makes the bundle inert on the MinGW runtimes

The same commit changes the per-DLL copy in prepare_windows_mesh_runtime_dependencies from an
unconditional std::fs::copy(src, dst) to:

if dst.is_file() {
    append_mesh_debug_log(app, format!("windows mesh runtime dependency already present; not replacing file={}", dst.display()));
    continue;
}

Measured on this box, against the extracted runtime tree and a pre-run backup of it:

  • dst is the runtime's own …\native-runtimes\<ver>\meshllm-native-runtime-windows-x86_64-<flavor>\lib\.

  • The MinGW archives already ship all four of libgcc_s_seh-1.dll, libstdc++-6.dll,
    libgomp-1.dll, libwinpthread-1.dll.

  • Under the previous unconditional copy, all four were in fact overwritten. SHA-256, pristine
    archive vs the same files after a build of this PR ran:

    DLL pristine (archive) after run (bundled)
    libstdc++-6.dll 3529d11c… 887c21db…
    libgcc_s_seh-1.dll 278f1101… b37c1770…
    libgomp-1.dll ae3939c3… acf25eee…
    libwinpthread-1.dll 1bbd639c… d54ed5ba…

The MeshLLM diagnostic log added by this PR confirms it directly — this is the app's own output on
every sharing attempt, for all four DLLs and for both installed runtimes:

windows mesh runtime dependency already present; not replacing
  file=…\native-runtimes\0.74.0\meshllm-native-runtime-windows-x86_64-vulkan\lib\libstdc++-6.dll
windows mesh runtime dependency already present; not replacing
  file=…\native-runtimes\0.74.0\meshllm-native-runtime-windows-x86_64-cuda12\lib\libstdc++-6.dll
prepared windows mesh runtime dependency dirs=…\Buzz\resources\mesh-llm\windows-x86_64;
  …\meshllm-native-runtime-windows-x86_64-cuda12\lib;…-vulkan\lib

Note it skips for cuda12 as well, whose archive ships none of these four — because an earlier
build of this PR, with the unconditional copy, already put them there. That's the sharper form of
the issue: the guard is sticky.
It isn't only "the MinGW archive already occupies those
filenames"; it's that once any version has populated them by any means, no later version can
refresh them. A machine that has run today's build will keep today's DLLs indefinitely, even if a
future Buzz bundles corrected ones — the upgrade silently no-ops. That's the part I'd weigh, more
than the current-release behaviour.

To be clear about impact: this breaks nothing today. Every libstdc++-6.dll I have — the
archive's, the bundled one, and Git for Windows' — is missing the same five codecvt exports, so
the substitution was never fixing the 127. And skip-if-present does cure a real hazard I'd flagged
earlier in this thread: the old behaviour would silently clobber a correct archive DLL if a future
release shipped one.

The two halves of 2b16d630 pull against each other, though. Per §1, the bootstrap changes
where the bundled DLLs come from — after pacman fires, libstdc++-6.dll is MSYS2 GCC 16.2.0's
rather than Git for Windows'. I checked whether that newer build helps: parsing its export table, it
is also missing all five symbols (7058 exports), so it would not have fixed the 127 even if it
had been applied. That's why I'm confident the skip costs nothing today.

The structural point is what remains: one half of the commit changes what goes into the bundle, and
the other half guarantees it can never reach a runtime directory that has already been populated. If
a future MeshLLM release ships a corrected libstdc++-6.dll the skip is right; if the intended cure
for the 127 was ever "bundle a good one", the skip closes that door — and closes it hardest on the
machines that already ran an affected build.

The question is just whether "archive is authoritative, the bundle only fills gaps" is the intended
semantics. If it is, this is right and worth a comment saying so. If the bundle was meant to be able
to repair a bad archive DLL, this removes that ability on exactly the two runtimes that are
currently broken — and the debug line reads "already present; not replacing", which is accurate but
easy to skim past when diagnosing a load failure.

3. Nothing in this PR's CI exercises the script

The checks on 2b16d630 are zizmor, Semgrep OSS, and DCO Check. windows-canary.yml is
workflow_dispatch:-only and release.yml runs on release, so bundle-windows-mesh-runtime-deps.ps1
— including the bootstrap just added to it — runs on no check in this PR. A green PR here
carries no signal about the thing that was fixed. Not necessarily worth restructuring CI over, but
worth knowing that the fix is currently only validated by someone running the canary by hand, which
is what this comment is.

4. 7e2644d4 (dedicated VRAM) — confirmed fixed in an installed build; the fallback is not

Confirmed end-to-end. I installed a canary built from 7e2644d4 (verified by hash that it was
actually that build — see §5) on the 16 GB RTX 5060 Ti / 32 GB RAM machine. Settings → Compute now
reads:

Recommended for this machine (NVIDIA GeForce RTX 5060 Ti, 16 GB AI memory).

That's the ~32 GB over-report I raised, fixed, and the recommendation is sized sensibly against it
(a 2.7 GB model, "Fits well"). Marker scan confirms it's a real mesh build rather than the stub:
mesh_llm_stubs 0, mesh-llm-host-runtime 813, iroh 445, 147.9 MB vs stock 0.5.8's 86.5 MB.

The premise in your code comment ("a 16 GB card with 32 GB system RAM reads as ~32 GB") is this
machine exactly, which is why I reported it. Running the two probes the patch uses:

> nvidia-smi --query-gpu=name,memory.total --format=csv,noheader,nounits
NVIDIA GeForce RTX 5060 Ti, 16311

> Get-CimInstance Win32_VideoController | Select-Object Name,AdapterRAM | ConvertTo-Json -Compress
{"Name":"NVIDIA GeForce RTX 5060 Ti","AdapterRAM":4293918720}

The nvidia-smi path gives 16311 MiB and format_rated_capacity renders "16 GB" — correct, and it
fixes the ~32 GB over-report I reported.

The Win32_VideoController fallback reports 4 GB for the same 16 GB card. AdapterRAM is a
uint32, so it saturates at 4293918720 bytes (4095 MiB) and cannot represent any card above 4 GB.
This is a long-standing WMI limitation, not something specific to this machine. So whenever
nvidia-smi isn't usable — not on PATH, non-NVIDIA hardware, or windows_command_output
returning None because the exit status isn't success — the catalog silently swaps a ~2x
over-report for a 4x under-report, and the picker starts recommending models far smaller than the
card can hold. It fails quietly and plausibly, which is the awkward part: 4 GB is a believable
number for a GPU, so nothing looks wrong.

Worth noting that the accompanying unit test can't catch this. Its fixture is:

r#"[{"Name":"GPU A","AdapterRAM":8589934592},{"Name":"GPU B","AdapterRAM":"4294967296"}]"#

Both values are unreachable in practice — 4294967296 is exactly one byte past u32::MAX, so
AdapterRAM can never return either. The parser is being tested against input the data source
cannot produce, so it passes regardless of the saturation behaviour. If the fallback is kept, a
fixture of 4293918720 for a large card would be the one that bites.

If DXGI is available to you, IDXGIAdapter::GetDesc / DedicatedVideoMemory is the usual
replacement — it's 64-bit and doesn't need a child process. Failing that, it may be better to return
no VRAM figure at all than a saturated one, since "unknown" degrades more honestly than "4 GB".

Second, smaller point on both paths: they sum VRAM across adapters. parse_nvidia_smi_catalog_output
accumulates total over every row, and the controller path does .map(|(_, bytes)| *bytes).sum().
For a catalog that ranks what will fit, that reproduces the class of error this commit fixes — two
16 GB cards would present as 32 GB and attract a model that fits in neither one alone, absent layer
splitting. summarize_gpu_names already special-cases multi-GPU for display ("N GPUs"), so the
multi-adapter case is clearly in scope. Max-per-adapter is likely the safer figure for
recommendations, with the sum kept for any pooled-placement view. Flagging this one as reasoning
from the code rather than a measurement — this machine is single-GPU, so unlike the points above I
have not actually observed the two-card behaviour.

The console-window half is fixed for the case you described, but not for enabling sharing.
Opening Settings → Compute is now clean — no flashes, and a process watcher confirms the catalog's
probe (nvidia-smi --query-gpu=name,memory.total …, i.e. your windows_nvidia_smi_gpus) spawns
with no conhost attached. That also settles which path produced the 16 GB: the primary one, not
the Win32_VideoController fallback.

Toggling Share this machine still flickers, repeatedly. Watching child processes, the sharing
path spawns all of these with a conhost.exe attached — which is what the visible window is:

nvidia-smi -L                                                   + conhost
nvidia-smi --query-gpu=index,compute_cap --format=csv,noheader  + conhost
nvidia-smi                                                      + conhost
vulkaninfo                                                      + conhost
powershell -NoProfile -Command "Get-CimInstance Win32_VideoController | Select-Object -ExpandProperty Name"  + conhost

That last one is not your call site — yours is Select-Object Name,AdapterRAM | ConvertTo-Json -Compress. This is the upstream mesh_llm_system::hardware::survey(), which your commit bypasses
for the catalog but which the runtime-resolution path still calls. The set repeats while sharing
retries, which is why it reads as several flickers rather than one — the diagnostic log shows the
frontend polling mesh_node_status every ~4 s and the dependency-preparation block re-running
alongside it, so the survey is not a one-shot at toggle time.

So the fix is real and correctly scoped to what your comment claims; the remaining flashes come from
the upstream survey rather than from catalog.rs. Flagging it mainly so the symptom isn't reported
back at you as "still broken" after this merges.

5. Separate pre-existing bug: windows-canary.yml can upload a stale installer

This one isn't yours — it's in the canary workflow on main, and it matters here because the canary
is how people are going to validate this PR by hand.

Three consecutive green runs on my fork uploaded the same binary:

run head makensis produced uploaded sha256
4 957b5fab Buzz_0.5.9-test.4 test.4 4ccf4da0…
5 34859e0c Buzz_0.5.9-test.5 test.4 4ccf4da0…
6 eaf9aa9f Buzz_0.5.9-test.6 test.4 4ccf4da0…

Byte-identical, 68,095,414 bytes each. Runs 5 and 6 compiled their own installers and then shipped
run 4's. Two things combine:

  1. The Cargo cache path: exclusion
    !desktop/src-tauri/target/**/release/bundle
    
    removes the directory entry but not its contents — @actions/glob needs a trailing /** — so
    bundle/nsis/*.exe from a previous run is restored into the workspace.
  2. Locate NSIS installer then does
    EXE=$(find "$BUNDLE_DIR/nsis" -name '*.exe' -type f | head -1)
    and takes whichever entry comes first, with no check that it's the one this run built.

So the canary can hand you a green artifact that does not contain the commit you dispatched it for,
with nothing in the run to indicate it. I nearly drew a conclusion about §4 from one of these before
hashing it — the artifact's ID and timestamp are legitimately new each run even when the bytes are
recycled, so only the hash distinguishes them.

What worked in my fork, if it's useful: delete bundle/nsis after the cache restore so exactly one
installer can exist, and have the locate step assert the filename carries the version the run
derived, erroring on more than one match rather than picking. Fixing the glob to
!desktop/src-tauri/target/**/release/bundle/** would address the cache half, but the assertion is
worth having regardless — it's the part that turns a silent wrong answer into a failed step. Happy
to open this as its own issue or PR against main rather than tangling it with yours.

Still outside this PR

With this build installed, enabling sharing on the 5060 Ti fails as expected:

mesh native runtime failed to install or load: load native runtime
meshllm-native-runtime-windows-x86_64-vulkan from
C:\Users\<user>\AppData\Local\mesh-llm\native-runtimes\0.74.0\meshllm-native-runtime-windows-x86_64-vulkan:
LoadLibraryExW failed

That confirms the resolver rejects windows-x86_64-cuda12 on sm_120 and falls through to
vulkan, rather than attempting cuda12 and failing there. I also re-checked whether bundling a
newer libstdc++ could rescue the Vulkan path: after the bootstrap runs, the DLL Buzz bundles is
MSYS2 GCC 16.2.0's, and parsing its export table it is also missing all five symbols (7058
exports). So no available libstdc++-6.dll on this machine satisfies ggml.dll — which, usefully
for §2, means the skip-if-present guard costs nothing today.

Unchanged from my earlier comment, and not an argument against merging: sm_120 is published only
for linux-x86_64-cuda13, so on RTX 50-series the resolver correctly rejects windows-x86_64-cuda12
(gpu_arches = 75,80,86,87,89,90) and falls through to the MinGW runtimes, which don't load. A
reporter on the upstream issue has also now measured that the coordinator excludes v0.74.0 peers
from stage-splits on stage_protocol_generation, so pinning back isn't a workaround either. Expect
#2836 reports to continue from Blackwell users after this merges.

@jds1-ai

jds1-ai commented Aug 10, 2026

Copy link
Copy Markdown

Our comments crossed — mine went up about twenty minutes after yours and doesn't reference it, sorry
about that. It does answer "give it another go", though: everything in your list is tested above on
Windows 11 / RTX 5060 Ti. Short version — bootstrap PASS on a stock runner (MSYS2 branch; the
Chocolatey fallback still hasn't been exercised by anything), VRAM now reads 16 GB, and catalog
probing is flash-free.

One thing worth separating out, because your description and the code differ slightly:

Buzz no longer overwrites DLLs already present in the downloaded MeshLLM runtime lib directory.

That's the right intent, and it settles the question I raised — but dst.is_file() can't distinguish
a DLL the archive shipped from one Buzz itself copied there on an earlier run. The
diagnostic log shows the consequence: it skips all four in the cuda12 lib directory, and that
archive ships none of them. They're only there because a previous build of this PR, with the
unconditional copy, put them there.

So the guard reads as "never replace the archive's files" but behaves as "never replace anything,
including our own". Once Buzz has populated a runtime directory, no later Buzz can update it — a
future release bundling corrected DLLs would silently no-op on every machine that ran an affected
build. If you want the stated semantics, the discriminator has to be provenance rather than
presence: track what Buzz wrote (a sidecar manifest, or compare against the bundled resource's hash
and replace only on a match to a previous bundled version), so archive-provided files stay
untouched while Buzz's own stay updatable.

Not urgent for this PR — as noted above, no libstdc++-6.dll available on this machine satisfies
ggml.dll anyway, including the MSYS2 gcc 16.2.0 one the bootstrap now bundles, so nothing is being
lost today. It's the upgrade path I'd want on the record.

On validation: cargo fmt / git diff --check / parse check / cargo check --features mesh-llm are
all static, and per §3 above nothing in this PR's CI executes the bundler — so the canary run is
currently the only thing that does. Worth knowing that the canary has its own stale-artifact bug
(§5) which will hand you a previous run's installer; I'd fix that before trusting a canary build to
tell you whether a change worked.

I'm trying to get another box with an nVidia card setup to test locally as well.

Happy to keep covering that in the meantime — this machine is a 16 GB Blackwell (RTX 5060 Ti, sm_120)
on Windows 11, which is also the configuration that hits the runtime blocker, so it exercises both
the working paths and the broken one. Push a revision and I'll build and test it the same way.

@jds1-ai

jds1-ai commented Aug 11, 2026

Copy link
Copy Markdown

Two things — one on 3aaea807 which landed while I was writing, one correcting a claim of mine
earlier in this thread.

3aaea807 is the right diagnosis, but it makes the 4 GB problem live rather than fixing it

Good catch on the serde mapping: rename_all = "PascalCase" turns adapter_ram into AdapterRam,
which never matches WMI's AdapterRAM. Before that commit the field always deserialized to None,
so parse_one dropped every controller and the fallback returned an empty list — effectively no
VRAM figure at all.

With the rename in place the fallback now genuinely reads the value, which is exactly the case I
measured above: on this 16 GB card WMI returns AdapterRAM = 4293918720 (4095 MiB), because it's a
uint32 and saturates just under 4 GiB. So the path that previously produced nothing will now
produce a confident 4 GB for any card above that. The saturation point in my earlier comment
still stands, and this commit is what makes it reachable.

Worth knowing how it got through, because it bears on §3 of my earlier comment: the unit test
shipped alongside it asserts

vec![("GPU A".to_string(), 8_589_934_592), ("GPU B".to_string(), 4_294_967_296)]

which the pre-3aaea807 code could not have produced — it would have returned an empty vec, so that
test must have been failing. Your validation list was cargo fmt / git diff --check / parse check
/ cargo check --features mesh-llm, and cargo check doesn't run tests; the PR's own checks are
zizmor, Semgrep OSS and DCO Check. So nothing in the loop would have surfaced it. A cargo test -p buzz-desktop --features mesh-llm locally would have.

And once you do run it, note both fixture values are above u32::MAX4294967296 is exactly one
byte past it — so AdapterRAM can never return either, and the test still can't catch the
saturation. 4293918720 for a large card is the fixture that bites.

Correcting myself: there is a working Windows runtime, just not a GPU one

I said there was no working Windows path at all. That's wrong for CPU. Upstream rebuilt the Windows
cpu runtime with MSVC between mesh-llm v0.74.0 and v0.75.0, and it loads clean — verified
by import table, not filename:

archive ggml.dll imports toolchain load
v0.74.0 vulkan libgcc_s_seh-1, libstdc++-6 MinGW 127
v0.75.0 vulkan libgcc_s_seh-1, libstdc++-6 MinGW 127
v0.75.0 cpu msvcp140, vcruntime140, vcruntime140_1 MSVC ok

The switch is cpu-only; I re-downloaded v0.75.0 vulkan to confirm. Details and a matching
retraction upstream: Mesh-LLM/mesh-llm#1215.

What it does and doesn't change here:

  • GPU story unchanged. vulkan is still MinGW and still fails, sm_120 is still published only
    for linux-x86_64-cuda13, so everything I reported about Windows + RTX 50-series stands.
  • A CPU participant looks reachable, but not from here. Cargo.toml pins all six mesh crates to
    v0.74.0, and skippy_abi differs across versions (0.1.32 / 0.1.35) and is matched exactly, so
    SDK and runtime have to move as a pair. Deliberate work, not a bump — and whether CPU-only sharing
    is worth offering is a product call.
  • One interaction: the v0.75.0 cpu archive ships no MinGW runtime DLLs at all (it carries
    vcomp140.dll and drops the lib prefix), so for that runtime
    bundle-windows-mesh-runtime-deps.ps1 would have nothing to do. The bundler stays necessary for
    vulkan.

Not asking you to widen this PR — it does what it says and I'd merge it. Flagging it because "no
Windows path exists" was my framing and it turns out to be true only of the GPU path.

jds1-ai added a commit to jds1-ai/buzz that referenced this pull request Aug 11, 2026
…wn bootstrap

Our 08-09 fork run installed the MinGW runtime DLLs before the bundler. PR block#3223
now bootstraps them itself (pacman -> choco -> assert). Leaving our step in would
let that bootstrap borrow strength from ours and pass without being exercised, so
it is removed.

Replaced with a read-only probe that installs nothing and records which of the
four DLLs the stock windows-latest runner actually has, so a bundler PASS can be
attributed to the bootstrap rather than to a runner that already had them.

Signed-off-by: jds1-ai <jds1-ai@users.noreply.github.com>

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@jds1-ai

jds1-ai commented Aug 11, 2026

Copy link
Copy Markdown

Short status from the Windows 11 / RTX 5060 Ti (Blackwell, sm_120) rig, plus two heads-ups.

d7fabfae — right call, and the bug it removes is real here

source reading
Win32_VideoController.AdapterRAM 4,293,918,720 B = 4.00 GB
actual dedicated VRAM ~15.93 GB (16,311 MiB)

The uint32 saturation I flagged on 3aaea807 reproduces exactly. Moving to
DXGI_ADAPTER_DESC1.DedicatedVideoMemory removes the class rather than patching it, since that
field is 64-bit. ModelFit::Unknown for the zero-VRAM case is a good addition too — an unreadable
probe no longer masquerades as TooLarge.

One fairness correction to my earlier comment: the old code tried nvidia-smi first and only
fell back to WMI, and nvidia-smi resolves here — so this box read 16 GB either way. The 4 GB
path would have bitten machines with no nvidia-smi on PATH. Narrower blast radius than I implied.

Tonight's run — runtime blocker unchanged

Launched the canary and hit Share Compute. Four attempts, identical:

DesktopMeshRuntime::start returned error: mesh native runtime failed to install or load:
load native runtime meshllm-native-runtime-windows-x86_64-vulkan ...: LoadLibraryExW failed

Catalog side is healthy (entries=37, VRAM read correctly) — it's purely the native runtime load.

This isn't something your PR can fix, and I'd rather it wasn't judged on it. sm_120 cubins
ship only for linux-x86_64-cuda13 at v0.74.0, so the resolver correctly declines cuda12 and falls
to Vulkan, which is MinGW-built and missing the codecvt exports ggml.dll imports. I've confirmed
no libstdc++-6.dll on this machine satisfies it (including MSYS2 gcc 16.2.0 — 7058 exports, the
char32_t/char16_t instantiations all present, only wchar_t absent), so no bundling strategy
cures it. That's an upstream mesh-llm packaging matter, not a Buzz one.

⚠️ On 2b16d630's skip guard, visible again on a cold launch: "already present; not replacing" now
fires for all four DLLs in the cuda12 lib dir too — an archive shipping none of them — because
an earlier build's unconditional copy created those names. The skip is sticky: once any version
writes them, no later version refreshes them. Still reads as provenance-vs-presence rather than a
bug, but it's now observable from a clean start.

Two heads-ups

  1. The PR is CONFLICTING / DIRTY against main. One file:
    MeshComputeSettingsCard.tsxmain wrapped that section in
    <SettingsOptionGroup title="Sharing"> while the branch still has the flat
    <div className="space-y-5">. Your delta there (diagnostic-logging switch, its two effects, the
    handler, the defaultShareModelFromCatalog import) is additive and drops into the new structure
    cleanly. Restyle collision, not logic.

  2. Still no Rust test job on this PR — checks are DCO Check / Semgrep OSS / zizmor. The
    catalog tests in d7fabfae are good ones (zero-VRAM, and max-adapter rather than sum), but
    nothing in CI runs them, so a regression there would land green. windows-canary.yml is still
    workflow_dispatch:-only.

On the CI fix

I'd raised the stale-installer bug and carried my own fix. Yours is better and I've dropped mine
for it
— you fixed the root cause I missed, that the cache exclusion needs release/bundle/**
rather than release/bundle (the glob drops the directory entry, not its contents), on both cache
steps. Mine only purged nsis and left the broken exclusion in place. My fork is rebased onto
d7fabfae with my commit gone.

Happy to run the canary on the new head whenever useful — this box stays available as a
Blackwell/Windows test rig.

jds1-ai added a commit to jds1-ai/buzz that referenced this pull request Aug 11, 2026
…t they can go red

PR block#3223 adds Rust unit tests to desktop/src-tauri/src/mesh_llm/catalog.rs and
none has ever executed. `mod mesh_llm` is #[cfg(feature = "mesh-llm")], while
ci.yml's desktop job runs `just desktop-tauri-test` = `cargo test --workspace`
with no features -- so the tests are never compiled, and a test that is not
compiled cannot fail.

Four steps, each two-sided so the job cannot pass by not running:
A. reproduce CI's command and assert ZERO mesh_llm tests collected (fails loudly
   if the premise is wrong, and fails if it collects no tests at all)
B. add --features mesh-llm and assert the tests now exist
C. run them, require green
D. watch-red: break the zero-VRAM branch d7fabfa fixed and require the suite to
   go RED, then restore and assert the tree is clean

Without D, C is a tautology.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@stevepresley

Copy link
Copy Markdown
Author

Short status from the Windows 11 / RTX 5060 Ti (Blackwell, sm_120) rig, plus two heads-ups.

d7fabfae — right call, and the bug it removes is real here

source reading
Win32_VideoController.AdapterRAM 4,293,918,720 B = 4.00 GB
actual dedicated VRAM ~15.93 GB (16,311 MiB)
The uint32 saturation I flagged on 3aaea807 reproduces exactly. Moving to DXGI_ADAPTER_DESC1.DedicatedVideoMemory removes the class rather than patching it, since that field is 64-bit. ModelFit::Unknown for the zero-VRAM case is a good addition too — an unreadable probe no longer masquerades as TooLarge.

One fairness correction to my earlier comment: the old code tried nvidia-smi first and only fell back to WMI, and nvidia-smi resolves here — so this box read 16 GB either way. The 4 GB path would have bitten machines with no nvidia-smi on PATH. Narrower blast radius than I implied.

Tonight's run — runtime blocker unchanged

Launched the canary and hit Share Compute. Four attempts, identical:

DesktopMeshRuntime::start returned error: mesh native runtime failed to install or load:
load native runtime meshllm-native-runtime-windows-x86_64-vulkan ...: LoadLibraryExW failed

Catalog side is healthy (entries=37, VRAM read correctly) — it's purely the native runtime load.

This isn't something your PR can fix, and I'd rather it wasn't judged on it. sm_120 cubins ship only for linux-x86_64-cuda13 at v0.74.0, so the resolver correctly declines cuda12 and falls to Vulkan, which is MinGW-built and missing the codecvt exports ggml.dll imports. I've confirmed no libstdc++-6.dll on this machine satisfies it (including MSYS2 gcc 16.2.0 — 7058 exports, the char32_t/char16_t instantiations all present, only wchar_t absent), so no bundling strategy cures it. That's an upstream mesh-llm packaging matter, not a Buzz one.

⚠️ On 2b16d630's skip guard, visible again on a cold launch: "already present; not replacing" now fires for all four DLLs in the cuda12 lib dir too — an archive shipping none of them — because an earlier build's unconditional copy created those names. The skip is sticky: once any version writes them, no later version refreshes them. Still reads as provenance-vs-presence rather than a bug, but it's now observable from a clean start.

Two heads-ups

  1. The PR is CONFLICTING / DIRTY against main. One file:
    MeshComputeSettingsCard.tsxmain wrapped that section in
    <SettingsOptionGroup title="Sharing"> while the branch still has the flat
    <div className="space-y-5">. Your delta there (diagnostic-logging switch, its two effects, the
    handler, the defaultShareModelFromCatalog import) is additive and drops into the new structure
    cleanly. Restyle collision, not logic.
  2. Still no Rust test job on this PR — checks are DCO Check / Semgrep OSS / zizmor. The
    catalog tests in d7fabfae are good ones (zero-VRAM, and max-adapter rather than sum), but
    nothing in CI runs them, so a regression there would land green. windows-canary.yml is still
    workflow_dispatch:-only.

On the CI fix

I'd raised the stale-installer bug and carried my own fix. Yours is better and I've dropped mine for it — you fixed the root cause I missed, that the cache exclusion needs release/bundle/** rather than release/bundle (the glob drops the directory entry, not its contents), on both cache steps. Mine only purged nsis and left the broken exclusion in place. My fork is rebased onto d7fabfae with my commit gone.

Happy to run the canary on the new head whenever useful — this box stays available as a Blackwell/Windows test rig.

@jds1-ai My agent has made a complete mess of this latest set of commits and has broken the build, expanded the scope to try and band-aid things.. let me try to sort this out before you run it through CI again. I don't see a way to change this to Draft like I can with an Issue. UGH.

@jds1-ai

jds1-ai commented Aug 12, 2026

Copy link
Copy Markdown

Understood on the build state — not touching CI, and this isn't a verdict on the PR. Agent churn aside, we got a Windows NVIDIA box from LoadLibraryExW to a loaded CUDA runtime and a started node tonight, and the exact steps are below so you can reproduce or discard them.

Different rig from the one in this thread so far: Windows 11 / RTX 4070 Ti SUPER (Ada, sm_89) — i.e. an arch that is in the published Windows cuda12 list, unlike the Blackwell box. That turned out to matter.


TL;DR

Two independent upstream mesh-llm bugs, the first hiding the second. Neither is yours to fix.

  1. The CUDA runtime is rejected because the driver is too NEW. Backward compatibility isn't considered.
  2. The Windows cuda12 artifact ships none of the CUDA runtime DLLs it imports, so even once selected it can't load.

With both worked around, the runtime loads and the node starts. ⚠️ It has not served a single token — see the caveat at the bottom. I'm not claiming Share Compute works on Windows.


🔴 Before anything else: the installed-runtime cache invalidates configuration experiments

This wasted three of my runs, so it's first.

mesh-llm-host-runtime/src/system/native_runtime.rs @ v0.74.0:

fn resolve_installed_native_runtime_plan(cache, profile, ...) {
    let installed = cache.installed()?;
    if installed.is_empty() { return Ok(None); }   // only THEN download/install
    let manifest = NativeRuntimeReleaseManifest {
        artifacts: installed.iter().map(|r| r.manifest.runtime.clone()).collect(),  // INSTALLED ONLY
    };
    select_native_runtime_from_artifacts(&manifest.artifacts, profile, ...)
}

Once any runtime is installed, the candidate set is exactly what's on disk. Vulkan lands on the first toggle, so every later start ranks a one-element setcuda12 is never a candidate and no environment override can change the outcome. Nothing in the UI, the logs, or the error text tells you this is happening.

I confirmed my override reached the process and treated that as proof the control had landed. It hadn't — a variable that reaches a process which never consults it is exactly as dead as one that never arrived. I published a false "refuted" on that basis and had to withdraw it.

So: clear the cache before testing anything about selection.


Step by step, to the working state

# 1. CLEAR THE INSTALLED-RUNTIME CACHE. Without this, every step below is a no-op.
Get-ChildItem "$env:LOCALAPPDATA\mesh-llm\native-runtimes\0.74.0" -Directory |
  ForEach-Object { Move-Item $_.FullName "<somewhere-safe>\$($_.Name)" }

# 2. Launch with the toolkit-major override. It is read in-process, BEFORE nvidia-smi.
$env:MESH_LLM_CUDA_TOOLKIT_MAJORS = '12'
Start-Process "$env:LOCALAPPDATA\Buzz\buzz-desktop.exe"

# 3. Toggle Share Compute.
#    -> downloads and selects ...-windows-x86_64-cuda12  (219 MB; vulkan is 22 MB,
#       so download size alone tells you which arm you are in)
#    -> STILL fails LoadLibraryExW, now on the cuda12 runtime

# 4. Supply the CUDA runtime DLLs the artifact does not ship.
#    Any CUDA 12 build works as a donor; I used the ones shipped alongside another
#    local inference app.
$dst = "$env:LOCALAPPDATA\mesh-llm\native-runtimes\0.74.0\meshllm-native-runtime-windows-x86_64-cuda12\lib"
"cudart64_12.dll","cublas64_12.dll","cublasLt64_12.dll" |
  ForEach-Object { Copy-Item "<cuda12-donor-dir>\$_" "$dst\$_" -Force }

# 5. Relaunch (override still set) and toggle again.
#    -> "starting DesktopMeshRuntime" with NO error line after it.
#    -> node starts, ports bind: 9337 (ingress), 3131 (console)
#    -> UI: "Sharing <model> with relay members."

Every prior attempt threw LoadLibraryExW failed within ~0.7–1.3 s of the starting DesktopMeshRuntime line. After step 4, that line is followed by nothing.


Bug 1 — a CUDA 13 driver is judged incompatible with a CUDA 12 build

mesh-llm-native-runtime/src/resolver.rsevaluate_cuda_requirements:

if !cuda.toolkit_majors.contains(&requirements.toolkit_major) {
    reasons.push(CandidateRejection::CudaToolkitMajorMismatch { required: *toolkit_major });
}

Exact set membership. On this box nvidia-smi prints CUDA UMD Version: 13.3 and no plain CUDA Version: line; cuda_majors_from_nvidia_smi_output reads the UMD marker, giving toolkit_majors = {13}. The artifact declares toolkit_major: 12. {13}.contains(12) is false ⇒ rejected.

CUDA is backward compatible — a 13 driver runs CUDA 12 binaries. The test should be driver major ≥ artifact major. As written it excludes every machine on a current driver, on any OS, for any listed arch.

Controlled A/B, cache cleared on both arms:

Arm MESH_LLM_CUDA_TOOLKIT_MAJORS Selected
control unset …-vulkan (22 MB)
treatment 12 …-cuda12 (219 MB)

⚠️ This does not explain Blackwell, and shouldn't be reported as if it did. The arch check is separate and sm_89 passes it. On sm_120 the rejection is correct — that arch genuinely isn't in gpu_arches at v0.74.0. This box is a supported card wrongly excluded; the sm_120 box is an unsupported card correctly excluded. Two different problems that produce the identical error string.


Bug 2 — the Windows cuda12 artifact is not self-contained

ggml-cuda.dll (228 MB) imports:  cudart64_12.dll   <- not shipped, not on the box
                                 cublas64_12.dll   <- not shipped, not on the box
                                 cuda.dll          <- driver API (nvcuda.dll), present

The package contains all 7 of its declared libraries and zero CUDA redistributables. On a driver-only machine — any ordinary user, no CUDA toolkit — the load cannot succeed.

Corroboration: linux-x86_64-cuda12 is 656 MB; windows-x86_64-cuda12 is 219 MB. The ~437 MB gap is about the size of the omitted redistributables.

Proven rather than inferred: copying the three DLLs in made it load.

Worth noting the two Windows artifacts are built differently despite declaring the same x86_64-pc-windows-msvc triple — cuda12 is genuinely MSVC (llama.dll, mtmd.dll, no lib prefix, no MinGW DLLs), while vulkan is MinGW. That's consistent with the codecvt export problem being specific to the MinGW artifacts.

This is the same class already being tracked upstream for the cpu runtime (ERROR_MOD_NOT_FOUND on llama-common.dll) — missing dependency, different DLL. Difference here is there's a proven fix.


⚠️ What this does NOT show

It loads and starts. It has served zero tokens. The node reported state=Running mode=Serve health=Ok and the UI showed "Sharing <model> with relay members · N requests served this session" — but the model was never downloaded (no file over 100 MB anywhere in the runtime, cache or app dirs; VRAM never moved off idle), and that request counter tracks the UI's own 4-second status polling, not inference.

I'd treat the "requests served" figure and health=Ok as unreliable until a node has actually fetched weights. Not filing that as a bug yet — this box also isn't a member of the relay it was pointed at, and a node that can't join may simply never be told to fetch anything, so the download path hasn't had a fair test.


Suggestion, worth more than any of the above

evaluate_candidates builds a Vec<CandidateEvaluation> where every declined candidate carries typed rejection_reasonsCudaProfileMissing, CudaToolkitMajorMismatch { required }, CudaGpuArchUnsupported { supported }, and three more. None of it is logged anywhere.

Checked three ways: not in either file the diagnostic-logging toggle writes (they're byte-identical copies of one stream, not two sinks); not on stderr even with RUST_LOG=trace and stderr redirected to a file; and the selection path contains no logging macros and no tracing import at all.

NativeRuntimeResolver::evaluate() is already public and returns exactly this. One log line per declined candidate would have turned a multi-hour code read into a one-minute answer, and it's the difference between users being able to report these themselves or not.

@jds1-ai

jds1-ai commented Aug 12, 2026

Copy link
Copy Markdown

Follow-up to my last comment, from a Windows 11 / RTX 4070 Ti SUPER (Ada, sm_89) box — a listed arch, unlike the Blackwell rig.

Result first: Share Compute works on this PR, on real NVIDIA silicon, once the branch is merged onto desktop-v0.5.10. Served a token from a GPU-resident model with no workarounds. Details and the one bug that is still yours below.


1. Run against the current head (c238f1cd, mesh-llm v0.74.0) — two blockers

Both are upstream mesh-llm, neither is yours.

Selection. evaluate_cuda_requirements tested cuda.toolkit_majors.contains(required) — exact membership. nvidia-smi on this box reports CUDA UMD Version: 13.3 and no plain CUDA Version: line, so toolkit_majors = {13}; the artifact declares toolkit_major: 12; rejected, fell through to the MinGW vulkan runtime, LoadLibraryExW failed.

Controlled A/B, runtime cache cleared on both arms:

MESH_LLM_CUDA_TOOLKIT_MAJORS selected
unset …-vulkan (22 MB)
12 …-cuda12 (219 MB)

Packaging. With cuda12 selected it still failed: ggml-cuda.dll imports cudart64_12.dll and cublas64_12.dll, and the v0.74.0 Windows artifact shipped neither. Copying cudart64_12.dll, cublas64_12.dll, cublasLt64_12.dll into its lib\ made it load.

⚠️ Method note that cost me three runs: resolve_installed_native_runtime_plan builds its candidate manifest from cache.installed() only. Once any runtime is on disk the candidate set is one element, so no environment override changes anything until you clear %LOCALAPPDATA%\mesh-llm\native-runtimes\. I confirmed the variable reached the process and wrongly read that as a landed control.

2. desktop-v0.5.10 published mid-session — it moves the pin

0.5.10 bumps mesh-llm v0.74.0v0.75.1 across all seven mesh crates. 0.5.9 did not (same v0.74.0 as the 0.5.5 base), which is why merging forward was not worth doing before.

What v0.75.1 changes, read from source and release assets:

  • Selection fixed. New evaluate_cuda_toolkit_major() compares against driver_max_major in the correct direction, doc comment stating "a CUDA 13 driver with a CUDA 12 toolkit still selects the cuda12 runtime", plus a regression test newer_driver_with_older_toolkit_selects_installed_toolkit_runtime. New rejections: CudaToolkitMajorAboveDriver, CudaToolkitNotDetected, CudaToolkitMajorMismatch { required, installed }.
  • Packaging fixed. windows-x86_64-cuda12 209.3 MB → 792.5 MB; manifest now declares lib/cudart64_12.dll, lib/cublas64_12.dll, lib/cublasLt64_12.dll (11 libraries, was 7). New artifact_bundles_cuda_runtime() skips the host-toolkit requirement when those three are present. gpu_arches gained "61".

Upstream 0.5.10 still builds Windows without --features mesh-llm (macOS and Linux have it) and does not ship scripts/bundle-windows-mesh-runtime-deps.ps1. So stock 0.5.10 has no Share Compute on Windows — your PR is still what enables it.

3. Merge and build

Merged desktop-v0.5.10 into the branch head. One conflict, as you predicted: MeshComputeSettingsCard.tsx — upstream's <SettingsOptionGroup title="Sharing"> vs the branch's flat <div className="space-y-5">. Resolved to upstream's structure, re-applied the one additive element (the diagnostic-logging switch) inside upstream's Advanced block. The diagnostic state, handler and meshDebugLog calls merged cleanly outside the conflict. Everything else auto-merged, including commands/mesh_llm.rs and the Cargo.toml pin.

Windows canary build succeeded. Installed clean: no prior Buzz, no %LOCALAPPDATA%\mesh-llm, no app data, no environment overrides set.

4. Result

First toggle failed — see §5. Second toggle:

starting DesktopMeshRuntime
DesktopMeshRuntime::start returned ok

Selected and loaded meshllm-native-runtime-windows-x86_64-cuda12 under native-runtimes/0.75.1/, with no cache clearing, no MESH_LLM_CUDA_TOOLKIT_MAJORS, and no hand-copied DLLs. Both v0.74.0 blockers are gone.

Inference through the mesh ingress:

POST 127.0.0.1:9337/v1/chat/completions
  model unsloth/Qwen3.5-4B-GGUF:Q4_K_M   "What is 12 x 12? Reply with just the number."
-> HTTP 200, 1.46 s, finish_reason=stop, content="144", completion_tokens=127

GPU-resident: 4969 MiB VRAM, 69% utilisation during generation.

⚠️ Probe caveat for anyone repeating this: at max_tokens: 24 the same model returned empty content with finish_reason: length — it is a reasoning model and spent the budget before emitting content. A content-validating probe with a small budget reads a healthy model as dead.

5. One bug that IS yours: first start after a runtime download always fails

prepare_windows_mesh_runtime_dependencies() runs before the native runtime is installed. On the first-ever start after a download, the newly-created runtime lib\ directory is therefore never passed to AddDllDirectory, so its DLLs cannot resolve each other:

failed to load native runtime library ...\meshllm-native-runtime-windows-x86_64-cuda12\lib/cublas64_12.dll:
LoadLibraryExW failed (OS error 126: The specified module could not be found.)

The DLL is present and its own imports (KERNEL32.dll, cublasLt64_12.dll) are both present in that same directory — it is a search-path problem, not a missing file.

Second toggle, from the log:

02:34:13  DesktopMeshRuntime::start returned error      <- no cuda12 libdir registration before it
02:35:24  copied windows mesh runtime dependency  [x4 into cuda12\lib]
02:35:24  registered windows DLL directory        [cuda12\lib]
02:35:24  starting DesktopMeshRuntime
02:35:35  DesktopMeshRuntime::start returned ok

It self-heals on retry, which makes it easy to misattribute to whatever else you changed between attempts. Fix is ordering: register the runtime's lib directory after install and before load, or re-run the prepare step once the install completes.

Same shape explains the v0.74.0 vulkan runs: first toggle downloaded and failed, later toggles logged already present; not replacing — vulkan just kept failing afterwards for the separate codecvt reason.

6. Still open upstream

evaluate_candidates computes typed rejection_reasons for every declined candidate and logs none of them. At v0.75.1 resolver.rs is 1203 lines with zero logging macros. Checked three ways: not in either file the diagnostic-logging toggle writes (they are byte-identical copies of one stream), not on stderr with RUST_LOG=trace and stderr redirected, and no tracing import in the selection path. NativeRuntimeResolver::evaluate() is public and returns exactly this. One log line per declined candidate would have replaced most of the work above.


Bottom line: merge desktop-v0.5.10 into this branch and the Windows runtime blocker is gone on a supported NVIDIA arch. Happy to re-run anything on this box.

Signed-off-by: stevepresley <github@stevepresley.net>
Signed-off-by: stevepresley <github@stevepresley.net>
(cherry picked from commit 2b16d63)
Signed-off-by: stevepresley <github@stevepresley.net>
(cherry picked from commit 7e2644d)
Signed-off-by: stevepresley <github@stevepresley.net>
(cherry picked from commit 3aaea80)
Signed-off-by: stevepresley <github@stevepresley.net>
(cherry picked from commit d7fabfa)
Signed-off-by: stevepresley <github@stevepresley.net>
@stevepresley
stevepresley force-pushed the fix/windows-mesh-llm-2836-upstream branch from d7fabfa to d133908 Compare August 12, 2026 19:14
@stevepresley

stevepresley commented Aug 12, 2026

Copy link
Copy Markdown
Author

5. One bug that IS yours: first start after a runtime download always fails

prepare_windows_mesh_runtime_dependencies() runs before the native runtime is installed. On the first-ever start after a download, the newly-created runtime lib\ directory is therefore never passed to AddDllDirectory, so its DLLs cannot resolve each other:

failed to load native runtime library ...\meshllm-native-runtime-windows-x86_64-cuda12\lib/cublas64_12.dll:
LoadLibraryExW failed (OS error 126: The specified module could not be found.)

The DLL is present and its own imports (KERNEL32.dll, cublasLt64_12.dll) are both present in that same directory — it is a search-path problem, not a missing file.

Second toggle, from the log:

02:34:13  DesktopMeshRuntime::start returned error      <- no cuda12 libdir registration before it
02:35:24  copied windows mesh runtime dependency  [x4 into cuda12\lib]
02:35:24  registered windows DLL directory        [cuda12\lib]
02:35:24  starting DesktopMeshRuntime
02:35:35  DesktopMeshRuntime::start returned ok

It self-heals on retry, which makes it easy to misattribute to whatever else you changed between attempts. Fix is ordering: register the runtime's lib directory after install and before load, or re-run the prepare step once the install completes.

Same shape explains the v0.74.0 vulkan runs: first toggle downloaded and failed, later toggles logged already present; not replacing — vulkan just kept failing afterwards for the separate codecvt reason.

Hi @jds1-ai - got this cleaned up! Item 5 is addressed in:

  d133908f3 fix(windows): retry MeshLLM start after DLL registration

What it does:

  • Wraps Share Compute DesktopMeshRuntime::start(...).
  • If start fails with Windows loader errors:
    • LoadLibraryExW failed
    • or OS error 126
  • Buzz runs prepare_windows_mesh_runtime_dependencies(...) again.
  • Then it retries DesktopMeshRuntime::start(...) once.

So the sequence becomes:

  prepare before start
  start downloads/extracts runtime
  start fails because new runtime lib dir was not registered yet
  prepare again, now sees/registers the new lib dir
  retry start once

There is still a 30s-2m delay between "Checking status.. " and the mesh-llm agent being online (depending if a cold load, or a "hot" load where the model is already cached from prior launch. I also purposely did not do anything with the UI to make this clearer in that process as part of this PR. I also checked that the cold-launch “disarmed” checkpoint works correctly, and after a successful launch, the write enabled: true persists and the agent launches automatically (with the expected delay) on subsequent launches of Buzz after that.

Let me know if it works on your box as well!

…m-2836-upstream

Signed-off-by: stevepresley <github@stevepresley.net>

# Conflicts:
#	desktop/src/features/mesh-compute/ui/MeshComputeSettingsCard.tsx
@jds1-ai

jds1-ai commented Aug 13, 2026

Copy link
Copy Markdown

Ran d133908f on the Blackwell box (Win 11, RTX 5060 Ti, sm_120). Your retry fires exactly as designed — and it cannot fix this failure, because the error is 127, not 126. Root cause below; it's a Buzz packaging bug and the fix looks small.

Build: your head eefb212e built with --features mesh-llm (0.5.11-test.12). Clean install: uninstalled, deleted %LOCALAPPDATA%\mesh-llm and app data, no MESH_LLM_* overrides, nothing on PATH. mesh-llm v0.75.1.

Steps and results

# config outcome
1 as shipped ggml.dllOS error 127 (specified procedure could not be found)
2 bundled MinGW DLLs removed ggml-base.dllOS error 126 — see the trap below
3 bundled DLLs replaced with the runtime's own build DesktopMeshRuntime::start returned ok, state=Running mode=Serve

Only one variable changed between 1 and 3.

Root cause

Buzz bundles its own MinGW runtime in resources\mesh-llm\windows-x86_64 and registers that directory in the DLL search path. Windows resolves by base name, so the mesh-llm runtime's binaries bind against Buzz's libstdc++-6.dll, which is a different GCC build:

libstdc++-6.dll   buzz=7058 exports   runtime=6854   251 present only in the runtime's copy

ggml.dll imports 5 of those, all codecvt:

_ZNKSt25__codecvt_utf8_utf16_baseIwE5do_inER9_MbstatetPKcS4_RS4_PwS6_RS6_
_ZNKSt25__codecvt_utf8_utf16_baseIwE6do_outER9_MbstatetPKwS4_RS4_PcS6_RS6_
_ZNKSt25__codecvt_utf8_utf16_baseIwE9do_lengthER9_MbstatetPKcS4_y
_ZNKSt25__codecvt_utf8_utf16_baseIwE10do_unshiftER9_MbstatetPcS3_RS3_
_ZNSt7codecvtIwc9_MbstatetEC2Ey

16 such imports in total across ggml.dll, libllama-common.dll and libllama.dll. Missing export, not missing module — which is why AddDllDirectory can't reach it.

Your code comment already has the right rule — "Runtime archives are authoritative for DLLs they ship; Buzz's bundle only fills gaps" — the copy logic honours it, the search order doesn't.

Suggested fix: register the selected runtime's lib dir ahead of the bundle, or skip bundling any DLL the archive already ships.

⚠️ Two traps worth knowing

Don't just delete the bundle. prepare_windows_mesh_runtime_dependencies returns early when dependency_dirs is empty (windows mesh runtime dependency resources not found) and then registers nothing at all — including the runtime's own lib dir. That's config 2 above: the error moved 127 → 126, which looks like progress and isn't.

126 vs 127 are different problems. Your matcher retries on LoadLibraryExW failed || OS error 126, so 127 also triggers it. Harmless (~2 s), but a retry can never fix a missing export — might be worth splitting them so the message can say which.

Credit where due: on the first-ever start the cache is empty, so the pre-start prepare has nothing to enumerate, and your retry is what registered the runtime lib dir at all. It's doing real work here — it's just downstream of the shadowing.

Not bugs, for the record

  • cuda12 correctly declined on this card. windows-x86_64-cuda12 declares gpu_arches ["61","75","80","86","87","89","90"]; sm_120 isn't there and Windows ships no cuda13 artifact. Correct behaviour, not a defect.
  • Serving not demonstrated here. Node reaches Running/Serve with the model installed, but GPU stays flat, /v1/models is [], /health 503, and the ingress returns model_not_found. Looks like mesh-side serving rather than the local ingress, so I'm not reporting it as a defect — just not claiming it works.
  • mesh_serving_usage reported an honest requests_served=0 throughout.

Next

Running the same clean-room script now on a 4070 Ti SUPER (sm_89), where cuda12 is a legitimate candidate — results to follow.

@jds1-ai

jds1-ai commented Aug 13, 2026

Copy link
Copy Markdown

Follow-up to the Blackwell comment above, from the 4070 Ti SUPER (Ada, sm_89) box — the promised
sm_89 run, where cuda12 is a legitimate candidate.

Result: d133908f works here end to end. The retry recovered a real OS error 126, and the node
served real tokens from the GPU.
And the two boxes turn out to be consistent — the shadowing bug
above can't reach this path, for a concrete reason.

Same clean-room: uninstalled, deleted %LOCALAPPDATA%\mesh-llm and app data, no MESH_LLM_*
overrides. Built your head eefb212e (0.5.11-test.12), mesh-llm v0.75.1, diagnostic logging on
before the first toggle, one toggle.

The retry doing its job

18:57:29.799  prepared windows mesh runtime dependency
              dirs=…\Buzz\resources\mesh-llm\windows-x86_64        <- bundled dir only
18:57:29.824  starting DesktopMeshRuntime
18:57:50.477  DesktopMeshRuntime::start load failed; refreshing Windows DLL directories
              before one retry: … cublas64_12.dll: LoadLibraryExW failed (OS error 126)
18:57:50.481  registered windows DLL directory  …-cuda12\lib  ok=true
18:58:01.677  DesktopMeshRuntime::start returned ok

11.2 s, single toggle, no workarounds. This is the genuine 126 case — a missing module, which
is exactly what re-registering the directory fixes. Matches the read above that on a first-ever start
the pre-start prepare has nothing to enumerate and the retry is what registers the runtime lib dir
at all.

Serving confirmed here

The Blackwell comment couldn't demonstrate serving, so to close that gap on this arch:

POST 127.0.0.1:9337/v1/chat/completions
  model unsloth/gemma-4-E4B-it-GGUF:Q4_K_M
-> HTTP 200, 0.42 s, finish_reason=stop, content="144"

Three round-trips total, all HTTP 200 with real content. GPU-resident at 4496 MiB and 59%
utilisation during generation, so it is genuinely on the card. Log names the backend
(…-x86_64-cuda12); catalog read the card correctly (entries=39 … vram_gb=16.84). First inference
ran against a cold isolated CUDA_CACHE_PATH, which ended at 2 files / 0.02 MB — no PTX-JIT on this
arch.

Why this box escapes the libstdc++ shadowing

The root cause above is real, and it looks confined to the MinGW-built runtimes. The cuda12
runtime is MSVC-built and has no MinGW dependency to shadow:

binary (cuda12 archive) MinGW imports MSVC imports
ggml.dll none MSVCP140, VCRUNTIME140, api-ms-win-crt
ggml-base.dll none + vcomp140
llama.dll, llama-common.dll, mtmd.dll none same

ggml.dll here contains zero __codecvt_utf8_utf16_base references — the exact symbols missing
on the Blackwell path. And the four MinGW DLLs that end up in …-cuda12\lib are byte-identical to
Buzz's bundled copies
(sha256), i.e. placed there by prepare, not shipped by the archive.

So on this arch Buzz's bundle is inert: nothing in the cuda12 runtime links it either way. That's
consistent with the suggested fix above — for cuda12 the archive ships none of those DLLs, so
skipping the copy would be a no-op here and wouldn't regress this path.

Persistence and auto-launch

Quit, relaunched, touched nothing: enabled: true persisted, the node came back up on its own and
served. startOnNextLaunch was false throughout, so enabled is what drives the restore.

One small thing in case it saves time later: restore_mesh_sharing doesn't write to the diagnostic
log, so the auto-start produced only the launch-time DLL-prep lines and I first read it as not having
started — confirmed it by probing the ingress and getting a token back. A line on entry/exit there
would make this path verifiable from outside, which matters a bit more now that the retry wrapper is
wired into it. (Checked both sinks — %TEMP%\buzz-mesh-debug.log and the app-data copy are one
stream written twice, so nothing was landing elsewhere.)

Warm-start check

On a later start with the runtime already cached, both directories were registered up front, start
succeeded first try, and the retry line is absent — so it only engages in the case it's meant for.


Also confirmed desktop-v0.5.11 still builds Windows with --features default and ships no bundle
script, so this PR remains the path to Share Compute on Windows.

Net from the pair of boxes: on a listed arch this works today, and the outstanding item is the
MinGW search-order issue on the runtimes that need it. Happy to re-run anything on either rig.

Signed-off-by: stevepresley <github@stevepresley.net>
@stevepresley

Copy link
Copy Markdown
Author

Hi @jds1-ai - Update ready for review.

What changed:

  • Narrowed the Windows MeshLLM DLL search path to prefer only the current native runtime version (0.75.1) ahead of Buzz’s bundled fallback DLL directory.
  • Stale cached native runtime versions are no longer registered ahead of the active runtime, avoiding DLL shadowing from older MeshLLM runtimes.
  • Kept Buzz’s bundled MinGW DLLs as fallback only, so native runtime-provided DLLs remain authoritative.
  • Left GPU SDK discovery unchanged after the fallback path.

Validation performed locally on Windows:

  • Targeted Rust tests passed.
  • git diff --check passed.
  • cargo check --features mesh-llm passed.
  • Built NSIS installer with:
    • pnpm tauri build --target x86_64-pc-windows-msvc --bundles nsis --features mesh-llm --config src-tauri/tauri.windows.conf.json
  • Smoke-tested installer:
    • Share Compute starts.
    • Runtime registers only current 0.75.1 runtime lib before fallback/ROCm dirs.
    • Agent successfully responds through MeshLLM without crashing.

@jds1-ai

jds1-ai commented Aug 15, 2026

Copy link
Copy Markdown

Blackwell result for 9bec128b: the shadowing fix works. First time Share Compute has started on this box from a clean install with no workarounds — my 08-13 run needed your bundled DLLs swapped by hand, this one needed nothing.

Build under test: your head 9bec128b with --features mesh-llm (0.5.11-test.13), mesh-llm v0.75.1. Clean room: uninstalled, deleted %LOCALAPPDATA%\Buzz, %APPDATA%\xyz.block.buzz.app and %LOCALAPPDATA%\mesh-llm, no MESH_LLM_* overrides, nothing on PATH. Windows 11, RTX 5060 Ti (sm_120).

Control first, so the pass is attributable

The bundle this build shipped is still the bad one, so the shadowing condition was fully present:

resources\mesh-llm\windows-x86_64\libstdc++-6.dll
  total exports 7058 | codecvt 267 | utf8_utf16 44
  ggml.dll's 5 wchar_t codecvt symbols: 0/5 present

CI took the pacman branch (MSYS2 gcc 16.2.0-3, bundled from C:\msys64\mingw64\bin) and logged Missing Windows MeshLLM runtime dependencies before bootstrap: libgomp-1.dll, so the bootstrap did real work. Between my 08-13 failure and this pass, the only thing that changed is the search order.

Step 1 — cold start, first ever toggle

15:18:06.145  start load failed ... ggml.dll: LoadLibraryExW failed (OS error 127)
15:18:06.147  already present; not replacing   x4   (the vulkan archive's own DLLs)
15:18:06.150  registered windows DLL directory  ...-vulkan\lib   ok=true
15:18:06.150  prepared ... dirs=<runtime vulkan lib>;<Buzz resources>
15:18:16.761  DesktopMeshRuntime::start returned ok

On a cold cache the pre-start prepare has no 0.75.1 directory to enumerate, so the first attempt still registers the bundle only and still hits 127. d133908f's retry is what recovers it — it re-runs prepare, which only then can see the runtime lib dir. Worth knowing that the retry is not made redundant by this change; on a first-ever start it is load-bearing.

Step 2 — warm start, runtime already cached

15:21:39.675  registered windows DLL directory  ...-vulkan\lib                    ok=true
15:21:39.676  registered windows DLL directory  ...\Buzz\resources\mesh-llm\...   ok=true

No LoadLibraryExW, no 127, no retry line. The ordering applies up front and the load simply works. This is the cell that isolates your change.

One defect worth flagging

WINDOWS_MESH_NATIVE_RUNTIME_VERSION = "0.75.1" duplicates the Cargo.toml pin with nothing tying the two together. When the pin next moves, windows_mesh_native_runtime_lib_dirs_from matches no directory, runtime_lib_dirs comes back empty, and registration degrades to bundle-first — silently re-opening this exact bug, with no error and no log line naming the cause. Both new tests pass "0.75.1" in literally, so neither would catch it. Deriving it from the SDK, or taking the highest version directory present, would close it.

What I could not show

The node reaches Running / Serve / health=Ok on unsloth/gemma-4-E4B-it-GGUF:Q4_K_M, but it is not serving:

probe result
GPU 767–1005 MiB, 1% — flat
/v1/models 200 {"data":[],"object":"list"}
/health 503
POST /v1/chat/completions 404
model bytes on disk zero — no .gguf anywhere; %LOCALAPPDATA%\mesh-llm holds only native-runtimes

So the node advertises a specific model with health=Ok while none of it exists on disk. Because this ran from a verified clean room, "downloaded on an earlier run" is ruled out by construction. I have not diagnosed it and I'm not claiming downloads are broken — but the status/health mismatch is real and reproducible. It is unrelated to this PR.

Two smaller notes: mesh_serving_usage stayed at an honest requests_served=0 throughout, and restore_mesh_sharing still writes nothing to the diagnostic log — the warm auto-start was invisible there and I had to probe the ingress to confirm it had happened.

Scope: this is the Blackwell (sm_120) box only. Re-running the same clean room on the 4070 Ti SUPER (sm_89, the cuda12 path) now and will follow up with that result shortly.

@jds1-ai

jds1-ai commented Aug 15, 2026

Copy link
Copy Markdown

sm_89 / cuda12 result for 9bec128b: the ordering fix works, and serving is proven on this path. Companion to the Blackwell (sm_120) comment above — same clean room, same build, different runtime flavour.

Build under test: your head 9bec128b built with --features mesh-llm as 0.5.11-test.13, mesh-llm v0.75.1. Windows 11, RTX 4070 Ti SUPER (sm_89). The only delta from your head is a CI ref guard — no product code (verified with gh api .../compare).

Step 1 — clean room, verified by read-back

Uninstalled 0.5.11-test.12, then removed %LOCALAPPDATA%\mesh-llm, %APPDATA%\xyz.block.buzz.app, %LOCALAPPDATA%\xyz.block.buzz.app, %LOCALAPPDATA%\Buzz, and moved both diagnostic logs aside — the %TEMP% one survives an uninstall, so it has to be moved rather than assumed cleared. Every path re-tested for absence afterwards. No MESH_LLM_* overrides. Diagnostic logging enabled before the first toggle.

Step 2 — cold start, first toggle

15:59:42.822  prepared ... dirs=<Buzz resources>                       <- runtime dir does not exist yet
15:59:58.130  start load failed ... cublas64_12.dll: LoadLibraryExW failed (OS error 126)
15:59:58.131  copied ... x4 MinGW DLLs into ...-cuda12\lib
15:59:58.135  registered windows DLL directory  ...-cuda12\lib  ok=true
15:59:58.135  prepared ... dirs=<0.75.1 cuda12 lib>;<Buzz resources>   <- runtime FIRST
16:00:08.998  DesktopMeshRuntime::start returned ok

Failure→running 10.9 s (11.2 s on eefb212e). This matches the Blackwell finding that d133908f's retry is load-bearing on a first-ever start: the pre-start prepare has no 0.75.1 directory to enumerate yet. The failure here is 126 (missing module, cublas64_12.dll), not the 127 (missing export) seen on the vulkan path — consistent with cuda12 being MSVC-built, so the MinGW libstdc++ shadowing vector cannot reach it.

Serving proven, not inferred: POST 127.0.0.1:9337/v1/chat/completions → HTTP 200, content="144", 0.41 s, 4783 MiB GPU-resident, model echoed back as unsloth/gemma-4-E4B-it-GGUF:Q4_K_M.

Step 3 — warm start

Relaunch with the runtime already cached: dirs=<0.75.1 cuda12 lib>;<Buzz resources>, and zero LoadLibraryExW or retry lines. The ordering applies up front.

Step 4 — the stale-version claim, tested directly

A clean wipe cannot test "stale cached versions are no longer registered ahead of the active runtime": a clean box only ever has 0.75.1, so that branch has nothing to act on and would pass a test that structurally cannot fail. Both new unit tests pass "0.75.1" in literally, so they don't close it either. I seeded a stale runtime shaped exactly as the old walk matched it —

%LOCALAPPDATA%\mesh-llm\native-runtimes\0.74.0\meshllm-native-runtime-windows-x86_64-cuda12\lib\ + the four MinGW DLLs

— then ran both builds against byte-identical state (the runtime cache and mesh-sharing.json both survive an uninstall, so the control is exact):

build prepared … dirs=
eefb212e (test.12) <Buzz resources> ; <0.74.0 lib> ; <0.75.1 lib>
9bec128b (test.13) <0.75.1 lib> ; <Buzz resources> — zero mentions of 0.74.0

The old build registers the stale runtime ahead of the current one — the shadowing condition reproduced on demand — and the new build excludes it. That makes the pass attributable rather than assumed.

Defect — concur, independently hit

WINDOWS_MESH_NATIVE_RUNTIME_VERSION = "0.75.1" duplicates the Cargo.toml pin with nothing tying the two together; when the pin moves, windows_mesh_native_runtime_lib_dirs_from matches no directory and runtime_lib_dirs comes back empty. Two things to add to what's already been said: the retry cannot rescue that case — it re-runs the same prepare, which finds nothing on the second pass either — and the empty result emits no log line, so it degrades silently. Taking the highest version directory present, or deriving it from the SDK, closes it.

Roadblocks, and one limit on this result

  • No CI artifact exists for 9bec128b. Every block/buzz run on this branch is action_required (fork PR), so I built it in our own fork's Windows Canary. Not a blocker for us, but this PR currently can't be verified from upstream CI alone.
  • This run does not test downloads. I wiped %LOCALAPPDATA%\mesh-llm but not %LOCALAPPDATA%\huggingface. The served weights are dated 2026-08-11, four days before this run, so the download path was never exercised here and I can neither corroborate nor refute the Blackwell zero-bytes finding.
  • Relevant to that finding: on this box the model is not under %LOCALAPPDATA%\mesh-llm, which holds only native-runtimes and skippy-package-integrity. It is at %LOCALAPPDATA%\huggingface\hub\models--unsloth--gemma-4-E4B-it-GGUF\snapshots\<rev>\gemma-4-E4B-it-Q4_K_M.gguf (4747 MB), with a layered copy under models--meshllm--gemma-4-E4B-it-Q4_K_M-layers (93 files). Worth re-checking the Blackwell box against those paths before treating "no bytes on disk" as settled.
  • restore_mesh_sharing still writes nothing to the diagnostic log — the warm auto-start was invisible there and had to be confirmed by probing the ingress.

Scope: 4070 Ti SUPER (sm_89), cuda12 path, Windows 11. Ordering fix and retry both verified; download path not exercised on this box.

Signed-off-by: stevepresley <github@stevepresley.net>
@stevepresley

Copy link
Copy Markdown
Author

Hi @jds1-ai

Updated this PR with the Windows MeshLLM runtime fixes from local verification:

  • Bundles the fallback MinGW runtime DLLs required by MeshLLM on Windows.
  • Registers the active MeshLLM native runtime lib directory before Buzz’s bundled fallback DLL directory, so stale/fallback libstdc++-6.dll/related DLLs cannot shadow the runtime
    archive’s DLLs.
  • Registers Buzz’s fallback DLLs before vendor GPU SDK paths, while still adding installed ROCm/CUDA SDK DLL directories for backend dependencies.
  • Limits native runtime DLL scanning to the current pinned MeshLLM runtime version and now derives that version from mesh_llm_host_runtime::VERSION instead of hardcoding 0.75.1.
  • Keeps archive-provided native runtime DLLs authoritative: Buzz only fills missing fallback DLLs and does not overwrite runtime archive files.
  • Uses DXGI dedicated video memory for the Windows model catalog/recommendation path so AMD/NVIDIA/Intel adapters report VRAM correctly without relying on AdapterRAM/nvidia-smi.

Local validation completed on Windows:

  • cargo test --features mesh-llm windows_mesh --lib
  • cargo test --features mesh-llm mesh_runtime_load_error_retry_only_matches_windows_loader_failures --lib
  • git diff --check
  • cargo check --features mesh-llm
  • NSIS installer build with --features mesh-llm
  • Installer smoke test on AMD/ROCm: MeshLLM runtime starts, DLL registration order is correct, Share Compute reaches MeshLLM without the previous LoadLibraryExW / OS error 126/127 failures.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

3 participants