Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 16 additions & 2 deletions docs/manage-sandboxes/update-sandboxes.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -70,11 +70,25 @@ If a support workflow asks you to pass the maintained tag explicitly, clear any
curl -fsSL https://www.nvidia.com/nemoclaw.sh | NEMOCLAW_INSTALL_REF= NEMOCLAW_INSTALL_TAG=lkg bash
```

Before upgrade work, the installer prepares the current NemoClaw CLI without replacing OpenShell and requires a fresh backup of every registered sandbox.
During the automatic upgrade path, the installer prepares the current NemoClaw CLI without replacing OpenShell and requires a fresh backup of every registered sandbox.
If any sandbox is skipped or fails, the installer exits before it changes the gateway.
After backup, the installer requires any existing OpenShell executable it will use to report a version, then compares that version with the supported range declared by the prepared current source.
It retires the running gateway before replacing an out-of-range OpenShell installation, keeps the gateway when the installed version is supported, and stops without retiring it when the installed version or supported range cannot be validated.
If the installed OpenShell release cannot retire its gateway through a supported lifecycle command or the verified NemoClaw-owned gateway PID, the installer also stops after backup with the sandbox backups preserved.
On Linux, failed OpenShell lifecycle commands cause the installer to check a verified NemoClaw-managed gateway PID file for any configured gateway port.
For the default gateway on port `8080`, the installer first checks a verified active `nemoclaw-openshell-gateway.service`, then checks the PID file.
The service path requires an active `nemoclaw-openshell-gateway.service` with a trusted unit path, NemoClaw-managed marker, file owner, and `ExecStart` binary.
After the service or PID-file check confirms the gateway process is stopped, the installer tries to remove the selected OpenShell registration.
If registration removal fails, the installer warns that onboarding will replace the stale registration.
If neither fallback can verify and stop the process, the installer stops after backup, preserves every sandbox backup, and reports the failed operation.
After the automatic path retires an out-of-range gateway, it forces installation of the OpenShell version pinned by the prepared source before sandbox recovery.
This mandatory installation applies to source and managed install modes and cannot remain deferred after gateway retirement.
If the forced installation fails, the installer does not stage a gateway service or start sandbox recovery, preserves the backups, and reports how to retry.

For any registered-sandbox upgrade that you already prepared manually, set `NEMOCLAW_OPENSHELL_UPGRADE_PREPARED=1` only after you back up every registered sandbox and retire the gateway.
This environment variable asserts that you completed those steps, so the installer skips the repeated backup and gateway-retirement phase before it checks whether OpenShell is installed or whether its version is in range.
It reuses the latest backups, forces installation of the OpenShell version pinned by the prepared source, and starts recovery only after that installation succeeds.
If the forced installation fails, rerun the installer with `NEMOCLAW_OPENSHELL_UPGRADE_PREPARED=1` to reuse the prepared state and retry the OpenShell installation.
Outside these forced paths, an in-range source checkout preserves an existing OpenShell installation and installs OpenShell only when it is missing.

After the host CLI and OpenShell update, the installer runs `$$nemoclaw upgrade-sandboxes --auto` to reconcile the existing sandboxes.

Expand Down
17 changes: 13 additions & 4 deletions docs/reference/commands.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -507,17 +507,26 @@ To make the installer abort instead of continuing, set `NEMOCLAW_SINGLE_SESSION=
NEMOCLAW_SINGLE_SESSION=1 curl -fsSL https://www.nvidia.com/nemoclaw.sh | bash
```

When existing sandboxes were created with OpenShell earlier than `0.0.37`, the installer prompts before running the new automatic gateway upgrade path.
For scripted installs, set `NEMOCLAW_ACCEPT_EXPERIMENTAL_OPENSHELL_UPGRADE=1` to allow the installer to prepare the current CLI without replacing OpenShell, back up every registered sandbox with the current state manifest, retire an installed gateway whose OpenShell version is outside the current release's supported range, install the supported OpenShell release, and recover the existing sandboxes.
When existing sandboxes were created with OpenShell earlier than `0.0.37`, the installer prompts before running the automatic gateway upgrade path.
For scripted installs, set `NEMOCLAW_ACCEPT_EXPERIMENTAL_OPENSHELL_UPGRADE=1` to allow the automatic path to prepare the current CLI without replacing OpenShell, back up every registered sandbox with the current state manifest, retire an installed gateway whose OpenShell version is outside the current release's supported range, install the supported OpenShell release, and recover the existing sandboxes.
The installer reads that supported range from the prepared current source and stops without retiring the gateway if the installed version is unknown or the range is missing or invalid.
When the installed OpenShell version is already supported, the installer keeps the running gateway through the host update.
If an out-of-range gateway cannot be retired through a supported lifecycle command or the verified NemoClaw-owned gateway PID, the installer stops after backup with the sandbox backups preserved.
On Linux, if installed OpenShell lifecycle commands cannot retire the gateway, the installer checks a verified NemoClaw-managed gateway PID file for any configured gateway port.
For the default gateway on port `8080`, the installer first checks a verified active `nemoclaw-openshell-gateway.service`, then checks the PID file.
After either fallback confirms the gateway process is stopped, the installer tries to remove the selected OpenShell registration and warns if onboarding must replace a stale registration.
If neither fallback can verify and stop the process, the installer stops after backup with every sandbox backup preserved.
If any registered sandbox cannot be backed up, the installer aborts before it changes the gateway.
After the automatic path retires an out-of-range gateway, it forces installation of the OpenShell version pinned by the prepared source before recovery.
This mandatory installation applies to source and managed install modes and cannot remain deferred after gateway retirement.
If the forced installation fails, the installer does not stage a gateway service or start recovery, preserves the backups, and tells you to rerun with `NEMOCLAW_OPENSHELL_UPGRADE_PREPARED=1`.
When the registry contains a pre-fingerprint OpenClaw or Hermes entry with no recorded custom-image evidence, an interactive install asks you to confirm that the listed sandbox used a NemoClaw-managed image.
For a non-interactive install, set `NEMOCLAW_CONFIRM_LEGACY_MANAGED_RECREATE` to the exact JSON array of names printed by the installer, such as `["my-assistant","preserve-hermes"]`, only after verifying every named sandbox used a managed image.
The confirmation permits those legacy entries to recover onto the current managed image, but it does not override recorded custom-image evidence.
After successful recovery, the installer skips generic onboarding.
For a manually prepared upgrade, set `NEMOCLAW_OPENSHELL_UPGRADE_PREPARED=1` only after preserving every registered sandbox and retiring the old gateway.
For any registered-sandbox upgrade that you already prepared manually, set `NEMOCLAW_OPENSHELL_UPGRADE_PREPARED=1` only after backing up every registered sandbox and retiring the old gateway.
This environment variable asserts that those steps are complete, so the installer skips the repeated backup and gateway-retirement phase before it checks whether OpenShell is installed or whether its version is in range.
It reuses the latest backups, forces the pinned OpenShell installation, and starts recovery only after that installation succeeds.
If the installation fails, rerun with `NEMOCLAW_OPENSHELL_UPGRADE_PREPARED=1` to reuse the prepared state and retry the OpenShell installation.

<Warning title="Legacy Upgrade Recovery Scope">
Prepared backup recovery for a legacy sandbox restores only the managed state directory recorded in its validated manifest, such as `/sandbox/.openclaw` or `/sandbox/.hermes`.
Expand Down
127 changes: 105 additions & 22 deletions scripts/install.sh
Original file line number Diff line number Diff line change
Expand Up @@ -1131,6 +1131,7 @@ ONBOARD_RAN=false
_CLI_PATH=""
_NEMOCLAW_CLI_INSTALL_PREPARED=false
_NEMOCLAW_CLI_INSTALL_MODE=""
_OPENSHELL_INSTALL_REQUIRED_BEFORE_RECOVERY=false
_PREEXISTING_SANDBOX_COUNT=0
_PREEXISTING_SANDBOX_RECOVERY_RAN=false
# #6520: set when the automatic recovery pass exited 0 but skipped recorded
Expand Down Expand Up @@ -1327,10 +1328,14 @@ upstream_openshell_gateway_user_service_installed() {
|| [[ -f /lib/systemd/user/openshell-gateway.service ]]
}

resolve_upstream_openshell_gateway_bin_for_service() {
local exec_start gateway_bin
resolve_openshell_gateway_bin_for_user_service() {
local service_name="${1:-}" exec_start gateway_bin
local -a gateway_bins=()
exec_start="$(systemctl --user show openshell-gateway.service --property=ExecStart --value 2>/dev/null)" \
case "$service_name" in
openshell-gateway.service | "${NEMOCLAW_GATEWAY_SERVICE_NAME}.service") ;;
*) return 1 ;;
esac
exec_start="$(systemctl --user show "$service_name" --property=ExecStart --value 2>/dev/null)" \
|| return 1
while IFS= read -r gateway_bin; do
gateway_bins+=("$gateway_bin")
Expand All @@ -1346,6 +1351,10 @@ resolve_upstream_openshell_gateway_bin_for_service() {
printf '%s\n' "$gateway_bin"
}

resolve_upstream_openshell_gateway_bin_for_service() {
resolve_openshell_gateway_bin_for_user_service openshell-gateway.service
}

openshell_binary_version() {
local binary="${1:-}" version_output
[[ -x "$binary" ]] || return 1
Expand Down Expand Up @@ -1603,7 +1612,9 @@ maybe_install_openshell_during_install() {
return 0
fi
fi
spin "Installing OpenShell CLI" bash "${NEMOCLAW_SOURCE_ROOT}/scripts/install-openshell.sh"
if ! spin "Installing OpenShell CLI" bash "${NEMOCLAW_SOURCE_ROOT}/scripts/install-openshell.sh"; then
return 1
fi
prefer_user_local_openshell
install_nemoclaw_openshell_gateway_user_service
}
Expand Down Expand Up @@ -2077,11 +2088,37 @@ restore_managed_source_lockfile() {
finish_nemoclaw_install() {
# A backup-preparation pass defers OpenShell but still prepares the CLI. The
# later install pass completes only the OpenShell policy for that source mode.
# Once an out-of-range gateway has been retired, install its replacement
# before recovery can restart the service, including from a source checkout.
case "${_NEMOCLAW_CLI_INSTALL_MODE:-}" in
source) maybe_install_openshell_during_install if-missing ;;
managed) maybe_install_openshell_during_install force ;;
source | managed) ;;
*) error "The prepared ${_CLI_DISPLAY} CLI has no installation mode." ;;
esac
if [[ "${_OPENSHELL_INSTALL_REQUIRED_BEFORE_RECOVERY:-false}" == true ]]; then
local old_defer="${NEMOCLAW_DEFER_OPENSHELL_INSTALL:-}"
local defer_was_set="${NEMOCLAW_DEFER_OPENSHELL_INSTALL+1}"
local defer_declaration="" defer_was_exported=false
if [[ -n "$defer_was_set" ]]; then
defer_declaration="$(declare -p NEMOCLAW_DEFER_OPENSHELL_INSTALL 2>/dev/null || true)"
[[ "$defer_declaration" == declare\ -x* ]] && defer_was_exported=true
fi
local openshell_install_status=0
unset NEMOCLAW_DEFER_OPENSHELL_INSTALL
maybe_install_openshell_during_install force || openshell_install_status=$?
if [[ -n "$defer_was_set" ]]; then
NEMOCLAW_DEFER_OPENSHELL_INSTALL="$old_defer"
[[ "$defer_was_exported" == true ]] && export NEMOCLAW_DEFER_OPENSHELL_INSTALL
fi
if [[ "$openshell_install_status" -ne 0 ]]; then
error "Could not install the OpenShell version pinned by the prepared source after retiring the gateway. The installer preserved the sandbox backups and did not start recovery. Rerun the installer with NEMOCLAW_OPENSHELL_UPGRADE_PREPARED=1 to reuse the prepared upgrade state and retry the OpenShell install."
fi
_OPENSHELL_INSTALL_REQUIRED_BEFORE_RECOVERY=false
else
case "${_NEMOCLAW_CLI_INSTALL_MODE:-}" in
source) maybe_install_openshell_during_install if-missing ;;
managed) maybe_install_openshell_during_install force ;;
esac
fi
refresh_path
ensure_nemoclaw_shim || true
}
Expand Down Expand Up @@ -2430,14 +2467,20 @@ resolve_existing_cli_runner() {
}

prepare_current_cli_for_preupgrade_backup() {
local old_defer="${NEMOCLAW_DEFER_OPENSHELL_INSTALL:-__unset__}"
local old_defer="${NEMOCLAW_DEFER_OPENSHELL_INSTALL:-}"
local defer_was_set="${NEMOCLAW_DEFER_OPENSHELL_INSTALL+1}"
local defer_declaration="" defer_was_exported=false
if [[ -n "$defer_was_set" ]]; then
defer_declaration="$(declare -p NEMOCLAW_DEFER_OPENSHELL_INSTALL 2>/dev/null || true)"
[[ "$defer_declaration" == declare\ -x* ]] && defer_was_exported=true
fi
info "Preparing current ${_CLI_DISPLAY} CLI for pre-upgrade backup…"
export NEMOCLAW_DEFER_OPENSHELL_INSTALL=1
install_nemoclaw
if [[ "$old_defer" == "__unset__" ]]; then
unset NEMOCLAW_DEFER_OPENSHELL_INSTALL
else
export NEMOCLAW_DEFER_OPENSHELL_INSTALL="$old_defer"
unset NEMOCLAW_DEFER_OPENSHELL_INSTALL
if [[ -n "$defer_was_set" ]]; then
NEMOCLAW_DEFER_OPENSHELL_INSTALL="$old_defer"
[[ "$defer_was_exported" == true ]] && export NEMOCLAW_DEFER_OPENSHELL_INSTALL
fi
verify_nemoclaw
}
Expand Down Expand Up @@ -2800,6 +2843,40 @@ stop_legacy_openshell_gateway_process() {
rm -f "$pid_file"
}

stop_nemoclaw_openshell_gateway_user_service() {
[ "$(uname -s)" = "Linux" ] || return 1

local gateway_port service_name service_path fragment_path gateway_bin
gateway_port="$(resolve_nemoclaw_gateway_port)" || return 1
[ "$gateway_port" -eq 8080 ] || return 1
command_exists systemctl || return 1

service_name="${NEMOCLAW_GATEWAY_SERVICE_NAME}.service"
service_path="$(openshell_user_config_home)/systemd/user/${service_name}"
[ -f "$service_path" ] || return 1
if [ -L "$service_path" ] || ! [ -O "$service_path" ]; then
error "Refusing to retire the OpenShell gateway from an untrusted user service: ${service_path}"
fi
is_nemoclaw_openshell_gateway_user_service "$service_path" \
|| error "Refusing to retire the OpenShell gateway from a non-NemoClaw user service: ${service_path}"
systemctl --user is-active --quiet "$service_name" 2>/dev/null || return 1

fragment_path="$(systemctl --user show "$service_name" --property=FragmentPath --value 2>/dev/null)" \
|| return 1
[ "$fragment_path" = "$service_path" ] \
|| error "Refusing to retire the OpenShell gateway because the active user service does not match ${service_path}."
gateway_bin="$(resolve_openshell_gateway_bin_for_user_service "$service_name")" \
|| return 1
trusted_openshell_gateway_bin_for_service "$gateway_bin" \
|| error "Refusing to retire an OpenShell gateway user service with an untrusted binary: ${gateway_bin}"
Comment thread
coderabbitai[bot] marked this conversation as resolved.

systemctl --user stop "$service_name" \
|| error "Could not stop the trusted NemoClaw OpenShell gateway user service. Run 'systemctl --user status ${service_name}' for details."
systemctl --user is-active --quiet "$service_name" 2>/dev/null \
&& error "The trusted NemoClaw OpenShell gateway user service remained active after the stop command."
return 0
}

preinstall_backup_and_retire_legacy_gateway() {
local reg_file gateway_name
reg_file="$(nemoclaw_state_dir)/sandboxes.json"
Expand All @@ -2816,6 +2893,16 @@ preinstall_backup_and_retire_legacy_gateway() {
_PREEXISTING_SANDBOX_COUNT="$sandbox_count"
[ "$sandbox_count" -gt 0 ] 2>/dev/null || return 0
require_openshell_compatible_sandbox_names "$reg_file"
if truthy_env "${NEMOCLAW_OPENSHELL_UPGRADE_PREPARED:-}"; then
if [[ "${NEMOCLAW_SINGLE_SESSION:-}" == "1" ]]; then
error "Aborting — NEMOCLAW_SINGLE_SESSION is set. Destroy existing sessions with '${_CLI_BIN} <name> destroy' before reinstalling."
fi
confirm_legacy_managed_image_recovery "$reg_file"
info "Using manually prepared OpenShell gateway upgrade state."
_OPENSHELL_INSTALL_REQUIRED_BEFORE_RECOVERY=true
export NEMOCLAW_RESTORE_LATEST_BACKUP_ON_RECREATE=1
return 0
fi
if ! command_exists openshell; then
# NemoClaw v0.0.55's OpenShell 0.0.44 layout could install this binary
# without persisting ~/.local/bin on PATH. Retain this fallback while direct
Expand All @@ -2831,13 +2918,6 @@ preinstall_backup_and_retire_legacy_gateway() {

local old_openshell_version=""
old_openshell_version="$(installed_openshell_version || true)"
if legacy_openshell_gateway_upgrade_needed "$old_openshell_version" && truthy_env "${NEMOCLAW_OPENSHELL_UPGRADE_PREPARED:-}"; then
confirm_legacy_managed_image_recovery "$reg_file"
info "Using manually prepared OpenShell gateway upgrade state."
export NEMOCLAW_RESTORE_LATEST_BACKUP_ON_RECREATE=1
return 0
fi

if legacy_openshell_gateway_upgrade_needed "$old_openshell_version"; then
if ! confirm_experimental_openshell_gateway_upgrade "$sandbox_count" "$old_openshell_version"; then
return 0
Expand Down Expand Up @@ -2873,17 +2953,20 @@ preinstall_backup_and_retire_legacy_gateway() {
if [ "$gateway_name" = "nemoclaw" ]; then
openshell gateway destroy -g "$gateway_name" >/dev/null 2>&1 \
|| openshell gateway destroy >/dev/null 2>&1 \
|| { stop_legacy_openshell_gateway_process \
|| { { stop_nemoclaw_openshell_gateway_user_service \
|| stop_legacy_openshell_gateway_process; } \
&& { openshell gateway remove "$gateway_name" >/dev/null 2>&1 \
|| warn "The legacy gateway process stopped, but its OpenShell registration could not be removed; onboarding will replace the stale registration."; }; } \
|| error "Could not retire the legacy OpenShell gateway after backup. The installer stopped with the sandbox backups preserved."
|| error "Could not retire the legacy OpenShell gateway after backup. Installed OpenShell lifecycle commands failed, and no trusted active user service or PID-file gateway could be stopped. The installer stopped with the sandbox backups preserved."
else
openshell gateway destroy -g "$gateway_name" >/dev/null 2>&1 \
|| { stop_legacy_openshell_gateway_process \
|| { { stop_nemoclaw_openshell_gateway_user_service \
|| stop_legacy_openshell_gateway_process; } \
&& { openshell gateway remove "$gateway_name" >/dev/null 2>&1 \
|| warn "Legacy gateway ${gateway_name} stopped, but its OpenShell registration could not be removed; onboarding will replace only that stale registration."; }; } \
|| error "Could not retire legacy gateway ${gateway_name} after backup. The installer stopped with the sandbox backups preserved."
|| error "Could not retire legacy gateway ${gateway_name} after backup. Installed OpenShell lifecycle commands failed, and no trusted active user service or PID-file gateway could be stopped. The installer stopped with the sandbox backups preserved."
fi
_OPENSHELL_INSTALL_REQUIRED_BEFORE_RECOVERY=true
fi
}

Expand Down
Loading
Loading