ollama converge: wait for it before the first start, read the binary's own version, keep ROCm (#1511)

[gen16k]

Fixes #1511

Why

The daemon's start-up converge of the bundled ollama had four problems. All were found by reading the code; a hardware check follows below.

1. It read the wrong version. ollama --version asks whatever server listens on OLLAMA_HOST's default, 127.0.0.1:11434. waired's engine runs on its own port, so the server that answers there is the user's own Ollama if they have one. ParseEngineVersion preferred the server's line. On a host whose own Ollama runs a different release, the bundled engine read as off the pin, and the converge downloaded the pin (about 1.4 GB) again on every daemon start.
2. It dropped ROCm. NewOllamaConvergeDeps installed without an overlay decision, and the promotion replaces lib/ wholesale. On the apt path, an AMD host lost ROCm at every pin move.
3. It swapped bin/ and lib/ under a running ollama serve.
   • The daemon spawns an off-pin engine as normal: ExpectedVersion only guards adopting an orphan.
   • ollama launches its runners from its own directory on every model load. A load during the swap found lib/ half gone. A load after it ran new runners under an old server.
   • On Windows, lib/ was deleted before the locked, running ollama.exe failed to go, and nothing was restored.
4. Two converges shared one staging directory. On the apt path the postinst restarts the daemon, and install.sh then runs the CLI's converge; both used <BaseDir>/.stage.

Change

• The binary's own version wins (internal/hardware/profiler.go). ollama v0.34.2's versionHandler prints the server line, then Warning: client version is X whenever the two differ or no server answers. The client line now wins.
• One overlay answer (setup.OllamaROCmOverlayWanted(goos, profile, gpuMode)):
  • used by the CLI install on Linux and Windows, and by the daemon;
  • WAIRED_OLLAMA_GPU_MODE is still read on Windows only.
• The daemon passes the overlay answer. NewOllamaConvergeDeps takes it, and the daemon starts the converge right after the profiler is built. That is still ahead of anything that can fail startInferenceSubsystem, so a host whose engine choice errors is still converged.
• The first start waits for the converge (OllamaConfig.StartGate, wired to engineStartGate, which the converge's ollama pass releases however it ends):
  • nothing is serving at daemon start, so nothing is interrupted;
  • a host already at the pin waits one --version;
  • a wait ended by Stop or Park spawns nothing and counts as no strike (ErrEngineStartHeld);
  • the first wait logs engine start is waiting for the bundled engine update to finish once.
• Installs are serialised (OllamaInstaller.Lock):
  • an flock on <BaseDir>/.install.lock, on Linux and macOS;
  • Windows needs none, because install.ps1 stops the service before it converges; the file says why;
  • ConvergeOllama decides unlocked, then takes the lock and probes again;
  • runtimes install ollama and runtimes upgrade ollama take it too.
• The promotion can be undone. The previous install is moved aside and moved back on any failure, instead of being deleted first. Windows allows renaming a running executable, so this step works there too.
• Docs:
  • decision 20260921/1937, which partly supersedes 20260816/2243 (links in both directions; the guard passes);
  • getting-started/update.mdx (en/ja): one sentence saying local inference starts after a background engine update has downloaded.

User-facing wording added (draft, for owner review)

• CLI, while another install holds the lock: Another engine install is running on this computer; waiting for it to finish...
• docs-site update page: the sentence above.

Tests

• TestParseOllamaVersion_TheBinarysOwnVersionWins. This inverts TestParseOllamaVersion_ServerLineWins. The old fixture put the lines in an order ollama never prints; the new one uses v0.34.2's real order.
• TestOllamaROCmOverlayWanted: GOOS × host shape × mode, including an AMD iGPU that is set aside, the Windows Strix Halo arm, and the mode being read on Windows only.
• TestOllamaAdapter_StartGate:
  • nothing spawns while the gate is closed, the state reads starting, and there is exactly one spawn once it opens;
  • a Stop during the wait spawns nothing and costs no strike.
• TestStartFailureIsEvidence_AHeldStartIsNot.
• TestEngineStartGate: passes through once the update is done, logs the wait once, and a Stop ends the wait.
• TestConvergeOllama_ProbesAgainUnderTheLock: events run probe → lock → probe → unlock with no install; a host at the pin never locks.
• TestOllamaInstallerLock_ExcludesASecondHolder (Unix).
• TestPromoteStagedInstall_AFailureRestoresThePreviousInstall: each of the four renames fails in turn, for shared and exclusive destination directories, and the old tree comes back intact.
• TestRuntimesInstallOllama_TakesTheInstallLock (Linux).
• Mutations, each caught:
  • removing the gate check fails both StartGate cases;
  • preferring the server line again fails the parse test.
• Not covered by a unit test: that the daemon's deps really carry the overlay answer. That is one argument, and the answer itself is table-tested. The fleet has no Linux AMD dGPU; its measurement issue is measure: AMD discrete GPUs on Linux and Windows — detection, gfx target, host key, and which backend the engine picks #1495.
• Checks run:
  • go test ./... (exit 0);
  • -race on the gate tests;
  • GOOS=windows|darwin vet and go test -c for internal/runtime, cmd/waired and cmd/waired-agent;
  • golangci-lint: 0 issues on Linux. With GOOS=windows, the only hit is the existing oom_score_other.go;
  • decision-log guard.

Not in this PR

• Version-warning wording. The warning still suggests waired runtimes install ollama alongside restarting. With the gate and the rollback-capable promotion, that is no longer unsafe, so its wording is left alone.
• Remaining overlaps, recorded in the decision:
  • a hand-run waired runtimes install ollama on a host whose engine is running;
  • the few seconds in macOS darwin_update before the agent restarts.

Verification on hardware

To follow:

• Linux (RTX PRO 4000): a foreign ollama serve of another release on 11434 no longer triggers a re-download; the first start waits for the converge.
• Windows Strix Halo reference host and macOS: the gate and the start.

Results will be added here.

🤖 Generated with Claude Code

https://claude.ai/code/session_01BviBGHJqCeM88hxCRYrtAg

[github-actions]

📘 Docs preview — the preview channel for this PR has been deleted now that it is closed.

[gen16k]

Hardware check after merge, 2026-09-21. The build was main plus #1514; each host moved from its installed ollama to the 0.34.2 pin at start.

• Start waits for the converge. engine start is waiting for the bundled engine update to finish was logged, then bundled engine converged to the pin, and only then did the engine start:
  • Linux (RTX PRO 4000): 0.33.3 → 0.34.2
  • Windows Strix Halo: 0.33.3 → 0.34.2, 45 s
  • Mac mini M4: 0.33.3 → 0.34.2, 8 s
• The version comes from the binary itself (C0). On Linux, a stand-in "user's own Ollama" answered /api/version on 11434 with 0.33.3. The bundled ollama --version then printed both lines (ollama version is 0.33.3 / Warning: client version is 0.34.2), and the stand-in received 4 requests. The daemon read 0.34.2 and did not download again (live=0.34.2 pinned=0.34.2, bin mtime unchanged).
• Seen on the Mac only. On the first start right after the move, ollama's own GPU discovery hit its 30 s watchdog, and the engine treated the host as CPU-only for that run. The loaded model still ran on Metal. The second start found Metal in 0.3 s. ollama: check the engine's own GPU discovery against waired's prediction at start (#1513) #1514 reports this (engine_discovery_error), and it will be filed separately.

Every host was restored to its previous build and engine afterwards. — #1361-L107