diff --git a/docs/manage-sandboxes/update-sandboxes.mdx b/docs/manage-sandboxes/update-sandboxes.mdx index f7d43e32028..47ee13ac6ff 100644 --- a/docs/manage-sandboxes/update-sandboxes.mdx +++ b/docs/manage-sandboxes/update-sandboxes.mdx @@ -70,11 +70,25 @@ If a support workflow asks you to pass the maintained tag explicitly, clear any curl -fsSL https://www.nvidia.com/nemoclaw.sh | NEMOCLAW_INSTALL_REF= NEMOCLAW_INSTALL_TAG=lkg bash ``` -Before upgrade work, the installer prepares the current NemoClaw CLI without replacing OpenShell and requires a fresh backup of every registered sandbox. +During the automatic upgrade path, the installer prepares the current NemoClaw CLI without replacing OpenShell and requires a fresh backup of every registered sandbox. If any sandbox is skipped or fails, the installer exits before it changes the gateway. After backup, the installer requires any existing OpenShell executable it will use to report a version, then compares that version with the supported range declared by the prepared current source. It retires the running gateway before replacing an out-of-range OpenShell installation, keeps the gateway when the installed version is supported, and stops without retiring it when the installed version or supported range cannot be validated. -If the installed OpenShell release cannot retire its gateway through a supported lifecycle command or the verified NemoClaw-owned gateway PID, the installer also stops after backup with the sandbox backups preserved. +On Linux, failed OpenShell lifecycle commands cause the installer to check a verified NemoClaw-managed gateway PID file for any configured gateway port. +For the default gateway on port `8080`, the installer first checks a verified active `nemoclaw-openshell-gateway.service`, then checks the PID file. +The service path requires an active `nemoclaw-openshell-gateway.service` with a trusted unit path, NemoClaw-managed marker, file owner, and `ExecStart` binary. +After the service or PID-file check confirms the gateway process is stopped, the installer tries to remove the selected OpenShell registration. +If registration removal fails, the installer warns that onboarding will replace the stale registration. +If neither fallback can verify and stop the process, the installer stops after backup, preserves every sandbox backup, and reports the failed operation. +After the automatic path retires an out-of-range gateway, it forces installation of the OpenShell version pinned by the prepared source before sandbox recovery. +This mandatory installation applies to source and managed install modes and cannot remain deferred after gateway retirement. +If the forced installation fails, the installer does not stage a gateway service or start sandbox recovery, preserves the backups, and reports how to retry. + +For any registered-sandbox upgrade that you already prepared manually, set `NEMOCLAW_OPENSHELL_UPGRADE_PREPARED=1` only after you back up every registered sandbox and retire the gateway. +This environment variable asserts that you completed those steps, so the installer skips the repeated backup and gateway-retirement phase before it checks whether OpenShell is installed or whether its version is in range. +It reuses the latest backups, forces installation of the OpenShell version pinned by the prepared source, and starts recovery only after that installation succeeds. +If the forced installation fails, rerun the installer with `NEMOCLAW_OPENSHELL_UPGRADE_PREPARED=1` to reuse the prepared state and retry the OpenShell installation. +Outside these forced paths, an in-range source checkout preserves an existing OpenShell installation and installs OpenShell only when it is missing. After the host CLI and OpenShell update, the installer runs `$$nemoclaw upgrade-sandboxes --auto` to reconcile the existing sandboxes. diff --git a/docs/reference/commands.mdx b/docs/reference/commands.mdx index da62ab7dfcc..b92ed904675 100644 --- a/docs/reference/commands.mdx +++ b/docs/reference/commands.mdx @@ -507,17 +507,26 @@ To make the installer abort instead of continuing, set `NEMOCLAW_SINGLE_SESSION= NEMOCLAW_SINGLE_SESSION=1 curl -fsSL https://www.nvidia.com/nemoclaw.sh | bash ``` -When existing sandboxes were created with OpenShell earlier than `0.0.37`, the installer prompts before running the new automatic gateway upgrade path. -For scripted installs, set `NEMOCLAW_ACCEPT_EXPERIMENTAL_OPENSHELL_UPGRADE=1` to allow the installer to prepare the current CLI without replacing OpenShell, back up every registered sandbox with the current state manifest, retire an installed gateway whose OpenShell version is outside the current release's supported range, install the supported OpenShell release, and recover the existing sandboxes. +When existing sandboxes were created with OpenShell earlier than `0.0.37`, the installer prompts before running the automatic gateway upgrade path. +For scripted installs, set `NEMOCLAW_ACCEPT_EXPERIMENTAL_OPENSHELL_UPGRADE=1` to allow the automatic path to prepare the current CLI without replacing OpenShell, back up every registered sandbox with the current state manifest, retire an installed gateway whose OpenShell version is outside the current release's supported range, install the supported OpenShell release, and recover the existing sandboxes. The installer reads that supported range from the prepared current source and stops without retiring the gateway if the installed version is unknown or the range is missing or invalid. When the installed OpenShell version is already supported, the installer keeps the running gateway through the host update. -If an out-of-range gateway cannot be retired through a supported lifecycle command or the verified NemoClaw-owned gateway PID, the installer stops after backup with the sandbox backups preserved. +On Linux, if installed OpenShell lifecycle commands cannot retire the gateway, the installer checks a verified NemoClaw-managed gateway PID file for any configured gateway port. +For the default gateway on port `8080`, the installer first checks a verified active `nemoclaw-openshell-gateway.service`, then checks the PID file. +After either fallback confirms the gateway process is stopped, the installer tries to remove the selected OpenShell registration and warns if onboarding must replace a stale registration. +If neither fallback can verify and stop the process, the installer stops after backup with every sandbox backup preserved. If any registered sandbox cannot be backed up, the installer aborts before it changes the gateway. +After the automatic path retires an out-of-range gateway, it forces installation of the OpenShell version pinned by the prepared source before recovery. +This mandatory installation applies to source and managed install modes and cannot remain deferred after gateway retirement. +If the forced installation fails, the installer does not stage a gateway service or start recovery, preserves the backups, and tells you to rerun with `NEMOCLAW_OPENSHELL_UPGRADE_PREPARED=1`. When the registry contains a pre-fingerprint OpenClaw or Hermes entry with no recorded custom-image evidence, an interactive install asks you to confirm that the listed sandbox used a NemoClaw-managed image. For a non-interactive install, set `NEMOCLAW_CONFIRM_LEGACY_MANAGED_RECREATE` to the exact JSON array of names printed by the installer, such as `["my-assistant","preserve-hermes"]`, only after verifying every named sandbox used a managed image. The confirmation permits those legacy entries to recover onto the current managed image, but it does not override recorded custom-image evidence. After successful recovery, the installer skips generic onboarding. -For a manually prepared upgrade, set `NEMOCLAW_OPENSHELL_UPGRADE_PREPARED=1` only after preserving every registered sandbox and retiring the old gateway. +For any registered-sandbox upgrade that you already prepared manually, set `NEMOCLAW_OPENSHELL_UPGRADE_PREPARED=1` only after backing up every registered sandbox and retiring the old gateway. +This environment variable asserts that those steps are complete, so the installer skips the repeated backup and gateway-retirement phase before it checks whether OpenShell is installed or whether its version is in range. +It reuses the latest backups, forces the pinned OpenShell installation, and starts recovery only after that installation succeeds. +If the installation fails, rerun with `NEMOCLAW_OPENSHELL_UPGRADE_PREPARED=1` to reuse the prepared state and retry the OpenShell installation. Prepared backup recovery for a legacy sandbox restores only the managed state directory recorded in its validated manifest, such as `/sandbox/.openclaw` or `/sandbox/.hermes`. diff --git a/scripts/install.sh b/scripts/install.sh index 85eac792b3c..08cd26112b8 100755 --- a/scripts/install.sh +++ b/scripts/install.sh @@ -1131,6 +1131,7 @@ ONBOARD_RAN=false _CLI_PATH="" _NEMOCLAW_CLI_INSTALL_PREPARED=false _NEMOCLAW_CLI_INSTALL_MODE="" +_OPENSHELL_INSTALL_REQUIRED_BEFORE_RECOVERY=false _PREEXISTING_SANDBOX_COUNT=0 _PREEXISTING_SANDBOX_RECOVERY_RAN=false # #6520: set when the automatic recovery pass exited 0 but skipped recorded @@ -1327,10 +1328,14 @@ upstream_openshell_gateway_user_service_installed() { || [[ -f /lib/systemd/user/openshell-gateway.service ]] } -resolve_upstream_openshell_gateway_bin_for_service() { - local exec_start gateway_bin +resolve_openshell_gateway_bin_for_user_service() { + local service_name="${1:-}" exec_start gateway_bin local -a gateway_bins=() - exec_start="$(systemctl --user show openshell-gateway.service --property=ExecStart --value 2>/dev/null)" \ + case "$service_name" in + openshell-gateway.service | "${NEMOCLAW_GATEWAY_SERVICE_NAME}.service") ;; + *) return 1 ;; + esac + exec_start="$(systemctl --user show "$service_name" --property=ExecStart --value 2>/dev/null)" \ || return 1 while IFS= read -r gateway_bin; do gateway_bins+=("$gateway_bin") @@ -1346,6 +1351,10 @@ resolve_upstream_openshell_gateway_bin_for_service() { printf '%s\n' "$gateway_bin" } +resolve_upstream_openshell_gateway_bin_for_service() { + resolve_openshell_gateway_bin_for_user_service openshell-gateway.service +} + openshell_binary_version() { local binary="${1:-}" version_output [[ -x "$binary" ]] || return 1 @@ -1603,7 +1612,9 @@ maybe_install_openshell_during_install() { return 0 fi fi - spin "Installing OpenShell CLI" bash "${NEMOCLAW_SOURCE_ROOT}/scripts/install-openshell.sh" + if ! spin "Installing OpenShell CLI" bash "${NEMOCLAW_SOURCE_ROOT}/scripts/install-openshell.sh"; then + return 1 + fi prefer_user_local_openshell install_nemoclaw_openshell_gateway_user_service } @@ -2077,11 +2088,37 @@ restore_managed_source_lockfile() { finish_nemoclaw_install() { # A backup-preparation pass defers OpenShell but still prepares the CLI. The # later install pass completes only the OpenShell policy for that source mode. + # Once an out-of-range gateway has been retired, install its replacement + # before recovery can restart the service, including from a source checkout. case "${_NEMOCLAW_CLI_INSTALL_MODE:-}" in - source) maybe_install_openshell_during_install if-missing ;; - managed) maybe_install_openshell_during_install force ;; + source | managed) ;; *) error "The prepared ${_CLI_DISPLAY} CLI has no installation mode." ;; esac + if [[ "${_OPENSHELL_INSTALL_REQUIRED_BEFORE_RECOVERY:-false}" == true ]]; then + local old_defer="${NEMOCLAW_DEFER_OPENSHELL_INSTALL:-}" + local defer_was_set="${NEMOCLAW_DEFER_OPENSHELL_INSTALL+1}" + local defer_declaration="" defer_was_exported=false + if [[ -n "$defer_was_set" ]]; then + defer_declaration="$(declare -p NEMOCLAW_DEFER_OPENSHELL_INSTALL 2>/dev/null || true)" + [[ "$defer_declaration" == declare\ -x* ]] && defer_was_exported=true + fi + local openshell_install_status=0 + unset NEMOCLAW_DEFER_OPENSHELL_INSTALL + maybe_install_openshell_during_install force || openshell_install_status=$? + if [[ -n "$defer_was_set" ]]; then + NEMOCLAW_DEFER_OPENSHELL_INSTALL="$old_defer" + [[ "$defer_was_exported" == true ]] && export NEMOCLAW_DEFER_OPENSHELL_INSTALL + fi + if [[ "$openshell_install_status" -ne 0 ]]; then + error "Could not install the OpenShell version pinned by the prepared source after retiring the gateway. The installer preserved the sandbox backups and did not start recovery. Rerun the installer with NEMOCLAW_OPENSHELL_UPGRADE_PREPARED=1 to reuse the prepared upgrade state and retry the OpenShell install." + fi + _OPENSHELL_INSTALL_REQUIRED_BEFORE_RECOVERY=false + else + case "${_NEMOCLAW_CLI_INSTALL_MODE:-}" in + source) maybe_install_openshell_during_install if-missing ;; + managed) maybe_install_openshell_during_install force ;; + esac + fi refresh_path ensure_nemoclaw_shim || true } @@ -2430,14 +2467,20 @@ resolve_existing_cli_runner() { } prepare_current_cli_for_preupgrade_backup() { - local old_defer="${NEMOCLAW_DEFER_OPENSHELL_INSTALL:-__unset__}" + local old_defer="${NEMOCLAW_DEFER_OPENSHELL_INSTALL:-}" + local defer_was_set="${NEMOCLAW_DEFER_OPENSHELL_INSTALL+1}" + local defer_declaration="" defer_was_exported=false + if [[ -n "$defer_was_set" ]]; then + defer_declaration="$(declare -p NEMOCLAW_DEFER_OPENSHELL_INSTALL 2>/dev/null || true)" + [[ "$defer_declaration" == declare\ -x* ]] && defer_was_exported=true + fi info "Preparing current ${_CLI_DISPLAY} CLI for pre-upgrade backup…" export NEMOCLAW_DEFER_OPENSHELL_INSTALL=1 install_nemoclaw - if [[ "$old_defer" == "__unset__" ]]; then - unset NEMOCLAW_DEFER_OPENSHELL_INSTALL - else - export NEMOCLAW_DEFER_OPENSHELL_INSTALL="$old_defer" + unset NEMOCLAW_DEFER_OPENSHELL_INSTALL + if [[ -n "$defer_was_set" ]]; then + NEMOCLAW_DEFER_OPENSHELL_INSTALL="$old_defer" + [[ "$defer_was_exported" == true ]] && export NEMOCLAW_DEFER_OPENSHELL_INSTALL fi verify_nemoclaw } @@ -2800,6 +2843,40 @@ stop_legacy_openshell_gateway_process() { rm -f "$pid_file" } +stop_nemoclaw_openshell_gateway_user_service() { + [ "$(uname -s)" = "Linux" ] || return 1 + + local gateway_port service_name service_path fragment_path gateway_bin + gateway_port="$(resolve_nemoclaw_gateway_port)" || return 1 + [ "$gateway_port" -eq 8080 ] || return 1 + command_exists systemctl || return 1 + + service_name="${NEMOCLAW_GATEWAY_SERVICE_NAME}.service" + service_path="$(openshell_user_config_home)/systemd/user/${service_name}" + [ -f "$service_path" ] || return 1 + if [ -L "$service_path" ] || ! [ -O "$service_path" ]; then + error "Refusing to retire the OpenShell gateway from an untrusted user service: ${service_path}" + fi + is_nemoclaw_openshell_gateway_user_service "$service_path" \ + || error "Refusing to retire the OpenShell gateway from a non-NemoClaw user service: ${service_path}" + systemctl --user is-active --quiet "$service_name" 2>/dev/null || return 1 + + fragment_path="$(systemctl --user show "$service_name" --property=FragmentPath --value 2>/dev/null)" \ + || return 1 + [ "$fragment_path" = "$service_path" ] \ + || error "Refusing to retire the OpenShell gateway because the active user service does not match ${service_path}." + gateway_bin="$(resolve_openshell_gateway_bin_for_user_service "$service_name")" \ + || return 1 + trusted_openshell_gateway_bin_for_service "$gateway_bin" \ + || error "Refusing to retire an OpenShell gateway user service with an untrusted binary: ${gateway_bin}" + + systemctl --user stop "$service_name" \ + || error "Could not stop the trusted NemoClaw OpenShell gateway user service. Run 'systemctl --user status ${service_name}' for details." + systemctl --user is-active --quiet "$service_name" 2>/dev/null \ + && error "The trusted NemoClaw OpenShell gateway user service remained active after the stop command." + return 0 +} + preinstall_backup_and_retire_legacy_gateway() { local reg_file gateway_name reg_file="$(nemoclaw_state_dir)/sandboxes.json" @@ -2816,6 +2893,16 @@ preinstall_backup_and_retire_legacy_gateway() { _PREEXISTING_SANDBOX_COUNT="$sandbox_count" [ "$sandbox_count" -gt 0 ] 2>/dev/null || return 0 require_openshell_compatible_sandbox_names "$reg_file" + if truthy_env "${NEMOCLAW_OPENSHELL_UPGRADE_PREPARED:-}"; then + if [[ "${NEMOCLAW_SINGLE_SESSION:-}" == "1" ]]; then + error "Aborting — NEMOCLAW_SINGLE_SESSION is set. Destroy existing sessions with '${_CLI_BIN} destroy' before reinstalling." + fi + confirm_legacy_managed_image_recovery "$reg_file" + info "Using manually prepared OpenShell gateway upgrade state." + _OPENSHELL_INSTALL_REQUIRED_BEFORE_RECOVERY=true + export NEMOCLAW_RESTORE_LATEST_BACKUP_ON_RECREATE=1 + return 0 + fi if ! command_exists openshell; then # NemoClaw v0.0.55's OpenShell 0.0.44 layout could install this binary # without persisting ~/.local/bin on PATH. Retain this fallback while direct @@ -2831,13 +2918,6 @@ preinstall_backup_and_retire_legacy_gateway() { local old_openshell_version="" old_openshell_version="$(installed_openshell_version || true)" - if legacy_openshell_gateway_upgrade_needed "$old_openshell_version" && truthy_env "${NEMOCLAW_OPENSHELL_UPGRADE_PREPARED:-}"; then - confirm_legacy_managed_image_recovery "$reg_file" - info "Using manually prepared OpenShell gateway upgrade state." - export NEMOCLAW_RESTORE_LATEST_BACKUP_ON_RECREATE=1 - return 0 - fi - if legacy_openshell_gateway_upgrade_needed "$old_openshell_version"; then if ! confirm_experimental_openshell_gateway_upgrade "$sandbox_count" "$old_openshell_version"; then return 0 @@ -2873,17 +2953,20 @@ preinstall_backup_and_retire_legacy_gateway() { if [ "$gateway_name" = "nemoclaw" ]; then openshell gateway destroy -g "$gateway_name" >/dev/null 2>&1 \ || openshell gateway destroy >/dev/null 2>&1 \ - || { stop_legacy_openshell_gateway_process \ + || { { stop_nemoclaw_openshell_gateway_user_service \ + || stop_legacy_openshell_gateway_process; } \ && { openshell gateway remove "$gateway_name" >/dev/null 2>&1 \ || warn "The legacy gateway process stopped, but its OpenShell registration could not be removed; onboarding will replace the stale registration."; }; } \ - || error "Could not retire the legacy OpenShell gateway after backup. The installer stopped with the sandbox backups preserved." + || error "Could not retire the legacy OpenShell gateway after backup. Installed OpenShell lifecycle commands failed, and no trusted active user service or PID-file gateway could be stopped. The installer stopped with the sandbox backups preserved." else openshell gateway destroy -g "$gateway_name" >/dev/null 2>&1 \ - || { stop_legacy_openshell_gateway_process \ + || { { stop_nemoclaw_openshell_gateway_user_service \ + || stop_legacy_openshell_gateway_process; } \ && { openshell gateway remove "$gateway_name" >/dev/null 2>&1 \ || warn "Legacy gateway ${gateway_name} stopped, but its OpenShell registration could not be removed; onboarding will replace only that stale registration."; }; } \ - || error "Could not retire legacy gateway ${gateway_name} after backup. The installer stopped with the sandbox backups preserved." + || error "Could not retire legacy gateway ${gateway_name} after backup. Installed OpenShell lifecycle commands failed, and no trusted active user service or PID-file gateway could be stopped. The installer stopped with the sandbox backups preserved." fi + _OPENSHELL_INSTALL_REQUIRED_BEFORE_RECOVERY=true fi } diff --git a/test/install-openshell-gateway-service.test.ts b/test/install-openshell-gateway-service.test.ts index f14b15912ec..5a21b756151 100644 --- a/test/install-openshell-gateway-service.test.ts +++ b/test/install-openshell-gateway-service.test.ts @@ -76,6 +76,47 @@ function stageService(home: string, gatewayBin: string, env: NodeJS.ProcessEnv = ); } +function writeSystemctlStub( + home: string, + unitPath: string, + gatewayBin: string, + options: { failedMetadataProperty?: "ExecStart" | "FragmentPath"; fragmentPath?: string } = {}, +) { + const bin = path.join(home, "systemctl-bin"); + const log = path.join(home, "systemctl.log"); + const active = path.join(home, "gateway-service.active"); + fs.mkdirSync(bin, { recursive: true }); + fs.writeFileSync(active, "active\n"); + writeExecutable( + path.join(bin, "systemctl"), + [ + "#!/usr/bin/env bash", + `printf '%s\\n' "$*" >> ${JSON.stringify(log)}`, + 'case "$*" in', + ' "--user is-active --quiet nemoclaw-openshell-gateway.service")', + ` test -f ${JSON.stringify(active)}`, + " ;;", + ' "--user show nemoclaw-openshell-gateway.service --property=FragmentPath --value")', + options.failedMetadataProperty === "FragmentPath" + ? " exit 98" + : ` printf '%s\\n' ${JSON.stringify(options.fragmentPath ?? unitPath)}`, + " ;;", + ' "--user show nemoclaw-openshell-gateway.service --property=ExecStart --value")', + options.failedMetadataProperty === "ExecStart" + ? " exit 98" + : ` printf '{ path=%s ; argv[]=%s ; ignore_errors=no ; }\\n' ${JSON.stringify(gatewayBin)} ${JSON.stringify(gatewayBin)}`, + " ;;", + ' "--user stop nemoclaw-openshell-gateway.service")', + ` rm -f ${JSON.stringify(active)}`, + " ;;", + " *) exit 97 ;;", + "esac", + "", + ].join("\n"), + ); + return { bin, log }; +} + describe("install.sh OpenShell gateway service", () => { it.each([ "user-local", @@ -230,6 +271,115 @@ describe("install.sh OpenShell gateway service", () => { expect(result.stdout).not.toContain(conventionalGatewayBin); }); + it("stops an active trusted NemoClaw gateway user service during upgrade retirement (#8800)", () => { + const home = makeTempRoot(); + const gatewayBin = userGatewayBin(home); + const staged = stageService(home, gatewayBin); + const unitPath = servicePath(home); + const systemctl = writeSystemctlStub(home, unitPath, gatewayBin); + + expect(staged.status, staged.stdout + staged.stderr).toBe(0); + + const result = runInstallHelper(home, "stop_nemoclaw_openshell_gateway_user_service", { + PATH: `${systemctl.bin}:${path.dirname(process.execPath)}:${TEST_SYSTEM_PATH}`, + }); + + expect(result.status, result.stdout + result.stderr).toBe(0); + expect(fs.readFileSync(systemctl.log, "utf-8").trim().split(/\r?\n/)).toEqual([ + "--user is-active --quiet nemoclaw-openshell-gateway.service", + "--user show nemoclaw-openshell-gateway.service --property=FragmentPath --value", + "--user show nemoclaw-openshell-gateway.service --property=ExecStart --value", + "--user stop nemoclaw-openshell-gateway.service", + "--user is-active --quiet nemoclaw-openshell-gateway.service", + ]); + }); + + it.each([ + "FragmentPath", + "ExecStart", + ] as const)("returns control for the PID-file fallback when %s service metadata is unavailable (#8800)", (failedMetadataProperty) => { + const home = makeTempRoot(); + const gatewayBin = userGatewayBin(home); + const staged = stageService(home, gatewayBin); + const systemctl = writeSystemctlStub(home, servicePath(home), gatewayBin, { + failedMetadataProperty, + }); + + expect(staged.status, staged.stdout + staged.stderr).toBe(0); + + const result = runInstallHelper( + home, + "stop_nemoclaw_openshell_gateway_user_service || printf 'pid-file-fallback\\n'", + { PATH: `${systemctl.bin}:${path.dirname(process.execPath)}:${TEST_SYSTEM_PATH}` }, + ); + const calls = fs.readFileSync(systemctl.log, "utf-8"); + + expect(result.status, result.stdout + result.stderr).toBe(0); + expect(calls).toContain( + `--user show nemoclaw-openshell-gateway.service --property=${failedMetadataProperty} --value`, + ); + expect(result.stdout).toContain("pid-file-fallback"); + expect(calls).not.toContain("--user stop nemoclaw-openshell-gateway.service"); + }); + + it("does not stop a user service whose active fragment differs from the trusted unit (#8800)", () => { + const home = makeTempRoot(); + const gatewayBin = userGatewayBin(home); + const staged = stageService(home, gatewayBin); + const systemctl = writeSystemctlStub(home, servicePath(home), gatewayBin, { + fragmentPath: path.join(home, "foreign.service"), + }); + + expect(staged.status, staged.stdout + staged.stderr).toBe(0); + + const result = runInstallHelper(home, "stop_nemoclaw_openshell_gateway_user_service", { + PATH: `${systemctl.bin}:${path.dirname(process.execPath)}:${TEST_SYSTEM_PATH}`, + }); + const calls = fs.readFileSync(systemctl.log, "utf-8"); + + expect(result.status).not.toBe(0); + expect(result.stderr).toContain("active user service does not match"); + expect(calls).not.toContain("--user stop nemoclaw-openshell-gateway.service"); + }); + + it("does not stop a trusted unit whose active command uses an untrusted binary (#8800)", () => { + const home = makeTempRoot(); + const gatewayBin = userGatewayBin(home); + const foreignGatewayBin = path.join(home, "foreign", "openshell-gateway"); + fs.mkdirSync(path.dirname(foreignGatewayBin), { recursive: true }); + writeExecutable(foreignGatewayBin, "#!/usr/bin/env bash\nexit 0\n"); + const staged = stageService(home, gatewayBin); + const systemctl = writeSystemctlStub(home, servicePath(home), foreignGatewayBin); + + expect(staged.status, staged.stdout + staged.stderr).toBe(0); + + const result = runInstallHelper(home, "stop_nemoclaw_openshell_gateway_user_service", { + PATH: `${systemctl.bin}:${path.dirname(process.execPath)}:${TEST_SYSTEM_PATH}`, + }); + const calls = fs.readFileSync(systemctl.log, "utf-8"); + + expect(result.status).not.toBe(0); + expect(result.stderr).toContain("user service with an untrusted binary"); + expect(calls).not.toContain("--user stop nemoclaw-openshell-gateway.service"); + }); + + it("does not inspect the default gateway user service for a custom gateway port (#8800)", () => { + const home = makeTempRoot(); + const gatewayBin = userGatewayBin(home); + const staged = stageService(home, gatewayBin); + const systemctl = writeSystemctlStub(home, servicePath(home), gatewayBin); + + expect(staged.status, staged.stdout + staged.stderr).toBe(0); + + const result = runInstallHelper(home, "stop_nemoclaw_openshell_gateway_user_service", { + NEMOCLAW_GATEWAY_PORT: "18080", + PATH: `${systemctl.bin}:${path.dirname(process.execPath)}:${TEST_SYSTEM_PATH}`, + }); + + expect(result.status).not.toBe(0); + expect(fs.existsSync(systemctl.log)).toBe(false); + }); + it("does not overwrite a foreign unit at the NemoClaw path (#6903)", () => { const home = makeTempRoot(); const unitPath = servicePath(home); diff --git a/test/install-openshell-upgrade-prompt.test.ts b/test/install-openshell-upgrade-prompt.test.ts index c4224ff22e8..04c587bfc79 100644 --- a/test/install-openshell-upgrade-prompt.test.ts +++ b/test/install-openshell-upgrade-prompt.test.ts @@ -106,9 +106,13 @@ function runPreinstallUpgradeGuard( currentCliAvailable?: boolean; currentMaxOpenshellVersion?: string; currentMinOpenshellVersion?: string; + finishDeferAsPlain?: boolean; + finishInstallMode?: "managed" | "source" | "unset"; + finishPreparedInstallSucceeds?: boolean; gatewayDestroySucceeds?: boolean; gatewayProcessStopSucceeds?: boolean; gatewayRemoveSucceeds?: boolean; + gatewayServiceStopSucceeds?: boolean; hasOldCli?: boolean; openshellOnPath?: boolean; openshellVersion?: string; @@ -142,6 +146,10 @@ function runPreinstallUpgradeGuard( const gatewayDestroySucceeds = options.gatewayDestroySucceeds === true ? "1" : "0"; const gatewayProcessStopSucceeds = options.gatewayProcessStopSucceeds === false ? "0" : "1"; const gatewayRemoveSucceeds = options.gatewayRemoveSucceeds === false ? "0" : "1"; + const gatewayServiceStopSucceeds = options.gatewayServiceStopSucceeds === true ? "1" : "0"; + const finishDeferAsPlain = options.finishDeferAsPlain === true ? "1" : "0"; + const finishInstallMode = options.finishInstallMode ?? ""; + const finishPreparedInstallSucceeds = options.finishPreparedInstallSucceeds === false ? "0" : "1"; const openshellVersionCommandFails = options.openshellVersionCommandFails === true ? "1" : "0"; const installedOpenshellVersionOverride = options.openshellVersionCommandFails === true @@ -210,6 +218,10 @@ exit 0 printf 'gateway process-stop\n' >> "${openshellLog}" [ "${gatewayProcessStopSucceeds}" = "1" ] } + stop_nemoclaw_openshell_gateway_user_service() { + printf 'gateway service-stop\n' >> "${openshellLog}" + [ "${gatewayServiceStopSucceeds}" = "1" ] + } resolve_existing_cli_runner() { ${resolveCli}; } prepare_current_cli_for_preupgrade_backup() { printf 'prepare-current\\n' >> "${cliLog}" @@ -218,7 +230,29 @@ exit 0 _CLI_PATH="${currentCli}" return 0 } + if [ "${finishDeferAsPlain}" = "1" ] && [[ -n "\${NEMOCLAW_DEFER_OPENSHELL_INSTALL+1}" ]]; then + export -n NEMOCLAW_DEFER_OPENSHELL_INSTALL + fi preinstall_backup_and_retire_legacy_gateway + if [ -n "${finishInstallMode}" ]; then + if [ "${finishInstallMode}" = "unset" ]; then + unset _NEMOCLAW_CLI_INSTALL_MODE + else + _NEMOCLAW_CLI_INSTALL_MODE="${finishInstallMode}" + fi + maybe_install_openshell_during_install() { + printf 'openshell install-mode %s defer=%s\n' "$1" "\${NEMOCLAW_DEFER_OPENSHELL_INSTALL:-}" >> "${openshellLog}" + [ "${finishPreparedInstallSucceeds}" = "1" ] + } + refresh_path() { :; } + ensure_nemoclaw_shim() { :; } + finish_nemoclaw_install + fi + printf 'DEFER=%s\\n' "\${NEMOCLAW_DEFER_OPENSHELL_INSTALL:-}" + case "$(declare -p NEMOCLAW_DEFER_OPENSHELL_INSTALL 2>/dev/null || true)" in + "declare -x "*) printf 'DEFER_EXPORTED=1\\n' ;; + *) printf 'DEFER_EXPORTED=0\\n' ;; + esac printf 'RESTORE=%s\\n' "\${NEMOCLAW_RESTORE_LATEST_BACKUP_ON_RECREATE:-}" printf 'CONFIRMED_NAMES=%s\\n' "\${_LEGACY_MANAGED_RECOVERY_NAMES_JSON:-}" `; @@ -234,6 +268,7 @@ exit 0 "NEMOCLAW_NON_INTERACTIVE", "NEMOCLAW_ACCEPT_EXPERIMENTAL_OPENSHELL_UPGRADE", "NEMOCLAW_CONFIRM_LEGACY_MANAGED_RECREATE", + "NEMOCLAW_DEFER_OPENSHELL_INSTALL", "NEMOCLAW_OPENSHELL_BIN", "NEMOCLAW_OPENSHELL_UPGRADE_PREPARED", "XDG_BIN_HOME", @@ -435,6 +470,85 @@ require_reportable_openshell_version`, }, ); + it("does not stage the gateway service after a forced OpenShell install fails (#8800)", () => { + const tmp = fs.mkdtempSync(path.join(os.tmpdir(), "nemoclaw-openshell-install-failure-")); + const sideEffectLog = path.join(tmp, "side-effects.log"); + const result = spawnSync( + "bash", + [ + "-c", + `source "${INSTALLER_PAYLOAD}" >/dev/null 2>&1 +NEMOCLAW_SOURCE_ROOT="${tmp}" +spin() { return 7; } +prefer_user_local_openshell() { printf 'preferred\\n' >>"${sideEffectLog}"; } +install_nemoclaw_openshell_gateway_user_service() { printf 'service-staged\\n' >>"${sideEffectLog}"; } +maybe_install_openshell_during_install force`, + ], + { + encoding: "utf-8", + env: { ...process.env, NEMOCLAW_DEFER_OPENSHELL_INSTALL: "" }, + }, + ); + + expect(result.status).not.toBe(0); + expect(fs.existsSync(sideEffectLog)).toBe(false); + }); + + it.each([ + { + expectedExported: "0", + expectedSet: "", + expectedValue: "", + initialState: "unset", + setup: "unset NEMOCLAW_DEFER_OPENSHELL_INSTALL", + }, + { + expectedExported: "0", + expectedSet: "1", + expectedValue: "plain", + initialState: "plain", + setup: "NEMOCLAW_DEFER_OPENSHELL_INSTALL=plain", + }, + { + expectedExported: "1", + expectedSet: "1", + expectedValue: "exported", + initialState: "exported", + setup: "export NEMOCLAW_DEFER_OPENSHELL_INSTALL=exported", + }, + ])("restores an $initialState deferral variable after CLI backup preparation (#8800)", ({ + expectedExported, + expectedSet, + expectedValue, + setup, + }) => { + const result = spawnSync( + "bash", + [ + "-c", + `source "${INSTALLER_PAYLOAD}" >/dev/null 2>&1 +info() { :; } +install_nemoclaw() { [ "\${NEMOCLAW_DEFER_OPENSHELL_INSTALL:-}" = "1" ]; } +verify_nemoclaw() { :; } +_CLI_DISPLAY=NemoClaw +${setup} +prepare_current_cli_for_preupgrade_backup +printf 'DEFER_SET=%s\\n' "\${NEMOCLAW_DEFER_OPENSHELL_INSTALL+1}" +printf 'DEFER_VALUE=%s\\n' "\${NEMOCLAW_DEFER_OPENSHELL_INSTALL:-}" +case "$(declare -p NEMOCLAW_DEFER_OPENSHELL_INSTALL 2>/dev/null || true)" in + "declare -x "*) printf 'DEFER_EXPORTED=1\\n' ;; + *) printf 'DEFER_EXPORTED=0\\n' ;; +esac`, + ], + { encoding: "utf-8", env: process.env }, + ); + + expect(result.status, result.stdout + result.stderr).toBe(0); + expect(result.stdout).toContain(`DEFER_SET=${expectedSet}\n`); + expect(result.stdout).toContain(`DEFER_VALUE=${expectedValue}\n`); + expect(result.stdout).toContain(`DEFER_EXPORTED=${expectedExported}\n`); + }); + it("aborts non-interactive legacy gateway upgrades without explicit opt-in", () => { const { result, cliLog, openshellLog } = runPreinstallUpgradeGuard({ NON_INTERACTIVE: "1", @@ -672,6 +786,7 @@ require_reportable_openshell_version`, const { result, cliLog, openshellLog } = runPreinstallUpgradeGuard( { NON_INTERACTIVE: "1" }, { + finishInstallMode: "source", hasOldCli: false, openshellVersion: "0.0.85", registryJson: @@ -681,6 +796,33 @@ require_reportable_openshell_version`, expect(result.status).toBe(0); expect(cliLog.split(/\r?\n/)).toContain("current:backup-all"); + expect(openshellLog).toBe("openshell install-mode if-missing defer=\n"); + }); + + it("forces OpenShell installation for a managed CLI install mode (#8800)", () => { + const { result, openshellLog } = runPreinstallUpgradeGuard( + { NON_INTERACTIVE: "1" }, + { + finishInstallMode: "managed", + registryJson: '{"sandboxes":{}}', + }, + ); + + expect(result.status, result.stdout + result.stderr).toBe(0); + expect(openshellLog).toBe("openshell install-mode force defer=\n"); + }); + + it("rejects a prepared CLI that has no install mode (#8800)", () => { + const { result, openshellLog } = runPreinstallUpgradeGuard( + { NON_INTERACTIVE: "1" }, + { + finishInstallMode: "unset", + registryJson: '{"sandboxes":{}}', + }, + ); + + expect(result.status).not.toBe(0); + expect(result.stdout + result.stderr).toContain("has no installation mode"); expect(openshellLog).toBe(""); }); @@ -700,6 +842,90 @@ require_reportable_openshell_version`, expect(openshellLog).toContain("gateway remove nemoclaw"); }); + it("retires an OpenShell 0.0.85 user-service gateway without a PID file before installing 0.0.101 (#8800)", () => { + const { result, cliLog, openshellLog } = runPreinstallUpgradeGuard( + { NEMOCLAW_DEFER_OPENSHELL_INSTALL: "1", NON_INTERACTIVE: "1" }, + { + currentMaxOpenshellVersion: "0.0.101", + currentMinOpenshellVersion: "0.0.101", + finishInstallMode: "source", + gatewayDestroySucceeds: false, + gatewayProcessStopSucceeds: false, + gatewayServiceStopSucceeds: true, + hasOldCli: false, + openshellVersion: "0.0.85", + registryJson: + '{"sandboxes":{"alpha":{"name":"alpha","nemoclawVersion":"0.0.105","fromDockerfile":false}}}', + }, + ); + + expect(result.status).toBe(0); + expect(cliLog.split(/\r?\n/)).toContain("current:backup-all"); + expect(openshellLog.split(/\r?\n/)).toEqual( + expect.arrayContaining([ + "gateway destroy -g nemoclaw", + "gateway destroy", + "gateway service-stop", + "gateway remove nemoclaw", + "openshell install-mode force defer=", + ]), + ); + expect(openshellLog.indexOf("gateway remove nemoclaw")).toBeLessThan( + openshellLog.indexOf("openshell install-mode force defer="), + ); + expect(result.stdout).toContain("DEFER=1"); + expect(result.stdout).toContain("DEFER_EXPORTED=1"); + expect(openshellLog).not.toContain("gateway process-stop"); + }); + + it("restores a plain OpenShell installation deferral as a plain variable (#8800)", () => { + const { result, openshellLog } = runPreinstallUpgradeGuard( + { + NEMOCLAW_CONFIRM_LEGACY_MANAGED_RECREATE: '["alpha"]', + NEMOCLAW_DEFER_OPENSHELL_INSTALL: "1", + NEMOCLAW_OPENSHELL_UPGRADE_PREPARED: "1", + NON_INTERACTIVE: "1", + }, + { + finishDeferAsPlain: true, + finishInstallMode: "source", + hasOldCli: false, + openshellVersion: "0.0.85", + }, + ); + + expect(result.status, result.stdout + result.stderr).toBe(0); + expect(result.stdout).toContain("DEFER=1"); + expect(result.stdout).toContain("DEFER_EXPORTED=0"); + expect(openshellLog).toBe("openshell install-mode force defer=\n"); + }); + + it("preserves prepared backups when the required OpenShell install fails after retirement (#8800)", () => { + const { result, cliLog, openshellLog } = runPreinstallUpgradeGuard( + { NON_INTERACTIVE: "1" }, + { + currentMaxOpenshellVersion: "0.0.101", + currentMinOpenshellVersion: "0.0.101", + finishPreparedInstallSucceeds: false, + finishInstallMode: "source", + gatewayDestroySucceeds: false, + gatewayProcessStopSucceeds: false, + gatewayServiceStopSucceeds: true, + hasOldCli: false, + openshellVersion: "0.0.85", + registryJson: + '{"sandboxes":{"alpha":{"name":"alpha","nemoclawVersion":"0.0.105","fromDockerfile":false}}}', + }, + ); + + expect(result.status).not.toBe(0); + expect(result.stdout + result.stderr).toContain("preserved the sandbox backups"); + expect(result.stdout + result.stderr).toContain("did not start recovery"); + expect(result.stdout + result.stderr).toContain("NEMOCLAW_OPENSHELL_UPGRADE_PREPARED=1"); + expect(cliLog.split(/\r?\n/)).toContain("current:backup-all"); + expect(openshellLog).toContain("openshell install-mode force defer="); + }); + it("fails closed before gateway retirement when the supported range is invalid", () => { const { result, cliLog, openshellLog } = runPreinstallUpgradeGuard( { @@ -761,7 +987,7 @@ require_reportable_openshell_version`, expect(openshellLog).not.toContain("gateway remove nemoclaw"); }); - it("fails closed after backup when no gateway retirement verb succeeds", () => { + it("preserves the backup when neither a trusted service nor a PID can retire the gateway (#8800)", () => { const { result, cliLog, openshellLog } = runPreinstallUpgradeGuard( { NON_INTERACTIVE: "1" }, { @@ -784,9 +1010,11 @@ require_reportable_openshell_version`, expect.arrayContaining([ "gateway destroy -g nemoclaw", "gateway destroy", + "gateway service-stop", "gateway process-stop", ]), ); + expect(openshellLog).not.toContain("gateway remove nemoclaw"); }); it("rejects a managed-image confirmation that is not a JSON name array (#6114)", () => { @@ -1086,14 +1314,20 @@ require_reportable_openshell_version`, expect(openshellLog).toContain("gateway remove nemoclaw"); }); - it("continues after the user manually prepared the old gateway state", () => { + it("forces OpenShell installation from a manually prepared gateway upgrade state (#8800)", () => { const { result, cliLog, openshellLog } = runPreinstallUpgradeGuard( { NON_INTERACTIVE: "1", NEMOCLAW_OPENSHELL_UPGRADE_PREPARED: "1", NEMOCLAW_CONFIRM_LEGACY_MANAGED_RECREATE: '["alpha"]', }, - { hasOldCli: false }, + { + currentMaxOpenshellVersion: "0.0.101", + currentMinOpenshellVersion: "0.0.101", + finishInstallMode: "source", + hasOldCli: false, + openshellVersion: "0.0.85", + }, ); expect(result.status).toBe(0); @@ -1101,6 +1335,29 @@ require_reportable_openshell_version`, expect(result.stdout).toContain("RESTORE=1"); expect(result.stdout).toContain('CONFIRMED_NAMES=["alpha"]'); expect(cliLog).toBe(""); - expect(openshellLog).toBe(""); + expect(openshellLog).toBe("openshell install-mode force defer=\n"); + }); + + it("reuses prepared backups when a failed install left no OpenShell executable (#8800)", () => { + const { result, cliLog, openshellLog } = runPreinstallUpgradeGuard( + { + NON_INTERACTIVE: "1", + NEMOCLAW_OPENSHELL_UPGRADE_PREPARED: "1", + NEMOCLAW_CONFIRM_LEGACY_MANAGED_RECREATE: '["alpha"]', + }, + { + finishInstallMode: "source", + hasOldCli: false, + openshellOnPath: false, + registryJson: + '{"sandboxes":{"alpha":{"name":"alpha","nemoclawVersion":"0.0.105","fromDockerfile":false}}}', + }, + ); + + expect(result.status).toBe(0); + expect(result.stdout).toContain("Using manually prepared OpenShell gateway upgrade state"); + expect(result.stdout).toContain("RESTORE=1"); + expect(cliLog).toBe(""); + expect(openshellLog).toBe("openshell install-mode force defer=\n"); }); });