diff --git a/docs/deployment/gateway-lifecycle-authority.mdx b/docs/deployment/gateway-lifecycle-authority.mdx index dd8708fc4c4..b5f3c513fd6 100644 --- a/docs/deployment/gateway-lifecycle-authority.mdx +++ b/docs/deployment/gateway-lifecycle-authority.mdx @@ -29,7 +29,15 @@ Choose the mode that matches the component responsible for process lifecycle ope | `externally-supervised` | A declared platform supervisor | Validates and attaches to the running gateway without managing its process. | When nothing is declared, NemoClaw keeps its existing managed behavior. -It uses the packaged gateway user service when that service is installed and otherwise manages a standalone gateway. +On Linux, the installer selects a packaged gateway user service only after the systemd user manager reports the effective unit and executable identity. +The installer verifies that identity and the gateway version before NemoClaw uses the service. +It does not parse a static package unit as the effective service identity. + +If the systemd user manager reports a recognized unavailable condition, the installer can keep the existing standalone gateway lifecycle. +This fallback requires no `.wants`, `.requires`, or `.upholds` activation link for either gateway service in the standard systemd user unit paths. +The installer stops when `SYSTEMD_UNIT_PATH` overrides those paths because it cannot verify the activation state. +An activation path stops installation until you restore the user manager or resolve the competing service through its owner. +Other service query errors, malformed metadata, untrusted paths, executable failures, and version mismatches stop installation. ## Declare the mode diff --git a/docs/reference/architecture.mdx b/docs/reference/architecture.mdx index 18111c1c495..e1195810d7c 100644 --- a/docs/reference/architecture.mdx +++ b/docs/reference/architecture.mdx @@ -86,22 +86,27 @@ Before adoption, NemoClaw asks systemd for the unit's effective `ExecStart`. When systemd returns the service identity, NemoClaw requires trusted package paths for both the unit and its effective binary. It then checks the effective binary's version against the blueprint range (`min_openshell_version` through `max_openshell_version`). This check prevents a supported user-local OpenShell install from hiding an out-of-range binary selected by the package unit's absolute `ExecStart`. -NemoClaw declines a known version below the minimum or a known stable version above the maximum. -An above-maximum development build remains eligible when `NEMOCLAW_OPENSHELL_CHANNEL=dev`, and an undetermined version also remains eligible. -When the systemd identity query fails, the package unit also remains eligible for later startup inspection. -NemoClaw declines the package unit when systemd positively identifies an untrusted unit or effective binary. -For a version rejection, NemoClaw reports the version once. -After either rejection, it selects an existing marked NemoClaw user service when available. -Otherwise, onboarding continues with the standalone gateway fallback. +NemoClaw rejects a known version below the minimum or a known stable version above the maximum. +An above-maximum development build remains eligible when `NEMOCLAW_OPENSHELL_CHANNEL=dev`. +An undetermined version or an unsupported version stops package-unit adoption. +When the systemd identity query reports a recognized unavailable user manager, NemoClaw checks the standard user unit paths before standalone fallback. +It checks `.wants`, `.requires`, and `.upholds` activation links for both gateway service names. +An activation path blocks fallback because its service can later claim port `8080`. +An active `SYSTEMD_UNIT_PATH` override also blocks fallback because NemoClaw cannot verify its activation state. +Other identity-query failures stop package-unit adoption. +NemoClaw stops when systemd identifies an untrusted unit or effective binary. +For a version rejection, NemoClaw reports the version once and blocks another gateway lifecycle. +Identity and path-trust failures also block another gateway lifecycle. After `systemctl --user daemon-reload` and systemd identity validation, startup checks the effective binary's version again before any service-state mutation. If a known version is now unsupported, startup does not stop, enable, or restart the package service. -Onboarding continues with the standalone fallback. +Onboarding stops instead of starting a standalone gateway. For tarball installs, the NemoClaw installer stages a marked user-level `nemoclaw-openshell-gateway.service` based on the upstream unit. Onboarding validates and reuses a healthy selected service. It enables or restarts the service when startup or verified runtime drift requires it, then checks gateway health. The marked service generates the local OpenShell mTLS bundle and reads `$XDG_CONFIG_HOME/openshell/gateway.env`, or `~/.config/openshell/gateway.env` when `XDG_CONFIG_HOME` is not absolute. NemoClaw preserves unrelated environment entries, accepts `DOCKER_HOST` only for an absolute local `unix://` socket, and refuses foreign or symlinked managed files. -If the selected systemd user service fails inspection, startup, or its health check, NemoClaw prints its `journalctl` log command and attempts the standalone fallback. +For ordinary startup or health failures, NemoClaw prints the `journalctl` log command and attempts the standalone fallback. +Unknown inspection, trust, and version failures stop onboarding. On Apple Silicon macOS, Homebrew makes the official OpenShell formula authoritative. The installer stages the formula and onboarding starts its `openshell` service. diff --git a/docs/reference/troubleshooting.mdx b/docs/reference/troubleshooting.mdx index 8758d05372f..464ca5bfb81 100644 --- a/docs/reference/troubleshooting.mdx +++ b/docs/reference/troubleshooting.mdx @@ -571,6 +571,39 @@ curl -fsSL https://www.nvidia.com/nemoclaw.sh | bash The next installer run must continue past the OpenShell installation step without reporting a version mismatch. +### Installer Reports That the Systemd User Manager Is Unavailable + +On Linux, an OpenShell package can install `/usr/lib/systemd/user/openshell-gateway.service` on a host without a reachable systemd user manager. +The service query can then return this diagnostic: + +```text +Failed to connect to bus: No medium found +``` + +The installer accepts only recognized user-manager-unavailable diagnostics for the standalone gateway fallback. +It checks `.wants`, `.requires`, and `.upholds` links in the standard systemd user unit paths. +The installer keeps the standalone lifecycle only when neither gateway service has an activation path that can later claim port `8080`. +The installer also stops when `SYSTEMD_UNIT_PATH` overrides the standard paths. +The installer does not parse, modify, or remove a package or foreign unit to make this decision. + +If an activation path exists, the installer stops because the service can start later and compete for port `8080`. +Restore the systemd user manager, then inspect both possible services: + +```bash +systemctl --user status openshell-gateway.service +systemctl --user is-enabled openshell-gateway.service +systemctl --user status nemoclaw-openshell-gateway.service +systemctl --user is-enabled nemoclaw-openshell-gateway.service +``` + +Resolve the competing service through its package or platform owner. +Do not delete an activation link or edit a unit file by hand. +Rerun the installer only after the owner confirms that no enabled user service can claim port `8080`. + +Unknown service query errors remain fatal. +The installer also stops for malformed effective metadata, an untrusted unit or executable path, an executable failure, or a gateway version mismatch. +Follow the reported condition instead of forcing the standalone fallback. + ### Sandbox build fails during OpenClaw plugin install diff --git a/scripts/install.sh b/scripts/install.sh index b556ed411d3..c1536746a4a 100755 --- a/scripts/install.sh +++ b/scripts/install.sh @@ -1320,6 +1320,8 @@ prefer_user_local_openshell() { NEMOCLAW_GATEWAY_SERVICE_MARKER="NEMOCLAW_MANAGED_OPENSHELL_GATEWAY=1" NEMOCLAW_GATEWAY_SERVICE_MARKER_LINE="# ${NEMOCLAW_GATEWAY_SERVICE_MARKER}" NEMOCLAW_GATEWAY_SERVICE_NAME="nemoclaw-openshell-gateway" +UPSTREAM_OPENSHELL_GATEWAY_SERVICE_BIN="" +UPSTREAM_OPENSHELL_GATEWAY_SERVICE_ERROR="" upstream_openshell_gateway_user_service_installed() { [[ "$(uname -s)" == "Linux" ]] || return 1 @@ -1342,8 +1344,7 @@ resolve_openshell_gateway_bin_for_user_service() { done < <( printf '%s\n' "$exec_start" \ | grep -oE 'path=[^ ;}]+' \ - | sed 's/^path=//' \ - | sort -u + | sed 's/^path=//' ) [[ "${#gateway_bins[@]}" -eq 1 ]] || return 1 gateway_bin="${gateway_bins[0]}" @@ -1351,8 +1352,127 @@ resolve_openshell_gateway_bin_for_user_service() { printf '%s\n' "$gateway_bin" } +systemd_user_manager_unavailable_diagnostic() { + local diagnostic="${1:-}" line recognized=0 + while IFS= read -r line; do + line="${line%$'\r'}" + case "$line" in + "") ;; + "Failed to connect to bus: No medium found" | \ + "Failed to connect to bus: Host is down" | \ + "Failed to connect to bus: No such file or directory" | \ + "System has not been booted with systemd as init system (PID 1). Can't operate." | \ + "XDG_RUNTIME_DIR is not set in the environment." | \ + "Failed to connect to bus: \$DBUS_SESSION_BUS_ADDRESS and \$XDG_RUNTIME_DIR not defined (consider using --machine=@.host --user to connect to bus of other user)") + recognized=1 + ;; + *) return 1 ;; + esac + done <<<"$diagnostic" + [[ "$recognized" -eq 1 ]] +} + +trusted_upstream_openshell_gateway_unit_for_service() { + case "${1:-}" in + /usr/local/lib/systemd/user/openshell-gateway.service | \ + /usr/lib/systemd/user/openshell-gateway.service | \ + /lib/systemd/user/openshell-gateway.service) + return 0 + ;; + *) + return 1 + ;; + esac +} + +trusted_upstream_openshell_gateway_bin_for_service() { + case "${1:-}" in + /usr/local/bin/openshell-gateway | /usr/bin/openshell-gateway) + return 0 + ;; + *) + return 1 + ;; + esac +} + +inspect_upstream_openshell_gateway_user_service() { + local service_output service_status line fragment_path="" exec_start="" gateway_bin + local fragment_count=0 exec_start_count=0 + local -a gateway_bins=() + UPSTREAM_OPENSHELL_GATEWAY_SERVICE_BIN="" + UPSTREAM_OPENSHELL_GATEWAY_SERVICE_ERROR="" + + if service_output="$(LC_ALL=C systemctl --user show openshell-gateway.service \ + --property=FragmentPath --property=ExecStart 2>&1)"; then + : + else + service_status=$? + UPSTREAM_OPENSHELL_GATEWAY_SERVICE_ERROR="systemctl --user show openshell-gateway.service failed: ${service_output:-exit ${service_status}}" + if systemd_user_manager_unavailable_diagnostic "$service_output"; then + return 2 + fi + return 1 + fi + + while IFS= read -r line; do + line="${line%$'\r'}" + case "$line" in + FragmentPath=*) + fragment_path="${line#FragmentPath=}" + fragment_count=$((fragment_count + 1)) + ;; + ExecStart=*) + exec_start="${line#ExecStart=}" + exec_start_count=$((exec_start_count + 1)) + ;; + "") ;; + *) + UPSTREAM_OPENSHELL_GATEWAY_SERVICE_ERROR="The effective upstream OpenShell gateway service returned unexpected metadata." + return 1 + ;; + esac + done <<<"$service_output" + + if [[ "$fragment_count" -ne 1 || "$exec_start_count" -ne 1 ]]; then + UPSTREAM_OPENSHELL_GATEWAY_SERVICE_ERROR="The effective upstream OpenShell gateway service did not return one FragmentPath and one ExecStart value." + return 1 + fi + if ! trusted_upstream_openshell_gateway_unit_for_service "$fragment_path"; then + UPSTREAM_OPENSHELL_GATEWAY_SERVICE_ERROR="The effective upstream OpenShell gateway unit path is not trusted: ${fragment_path:-}" + return 1 + fi + + while IFS= read -r gateway_bin; do + gateway_bins+=("$gateway_bin") + done < <( + printf '%s\n' "$exec_start" \ + | grep -oE 'path=[^ ;}]+' \ + | sed 's/^path=//' + ) + if [[ "${#gateway_bins[@]}" -ne 1 ]]; then + UPSTREAM_OPENSHELL_GATEWAY_SERVICE_ERROR="The effective upstream OpenShell gateway service did not return one executable path." + return 1 + fi + gateway_bin="${gateway_bins[0]}" + if ! trusted_upstream_openshell_gateway_bin_for_service "$gateway_bin"; then + UPSTREAM_OPENSHELL_GATEWAY_SERVICE_ERROR="The effective upstream OpenShell gateway executable path is not trusted: $gateway_bin" + return 1 + fi + if [[ ! -x "$gateway_bin" ]]; then + UPSTREAM_OPENSHELL_GATEWAY_SERVICE_ERROR="The effective upstream OpenShell gateway executable is unavailable: $gateway_bin" + return 1 + fi + + UPSTREAM_OPENSHELL_GATEWAY_SERVICE_BIN="$gateway_bin" +} + resolve_upstream_openshell_gateway_bin_for_service() { - resolve_openshell_gateway_bin_for_user_service openshell-gateway.service + if inspect_upstream_openshell_gateway_user_service; then + printf '%s\n' "$UPSTREAM_OPENSHELL_GATEWAY_SERVICE_BIN" + else + return $? + fi } openshell_binary_version() { @@ -1363,14 +1483,21 @@ openshell_binary_version() { } require_compatible_upstream_openshell_gateway_service() { - local nemoclaw_gateway_bin upstream_gateway_bin nemoclaw_version upstream_version + local nemoclaw_gateway_bin upstream_gateway_bin nemoclaw_version upstream_version inspect_status + if inspect_upstream_openshell_gateway_user_service; then + upstream_gateway_bin="$UPSTREAM_OPENSHELL_GATEWAY_SERVICE_BIN" + else + inspect_status=$? + if [[ "$inspect_status" -eq 2 ]]; then + return 2 + fi + error "Could not inspect the effective upstream OpenShell gateway user service. ${UPSTREAM_OPENSHELL_GATEWAY_SERVICE_ERROR} Repair that OpenShell installation, then rerun the installer." + fi nemoclaw_gateway_bin="$(resolve_openshell_gateway_bin_for_service)" \ || error "Could not locate the NemoClaw OpenShell gateway binary before checking the existing upstream service." if ! trusted_openshell_gateway_bin_for_service "$nemoclaw_gateway_bin"; then error "OpenShell gateway user service binary path is not a trusted install path: $nemoclaw_gateway_bin" fi - upstream_gateway_bin="$(resolve_upstream_openshell_gateway_bin_for_service)" \ - || error "Could not locate the gateway binary used by the existing upstream OpenShell user service. Remove or repair that OpenShell installation, then rerun the installer." nemoclaw_version="$(openshell_binary_version "$nemoclaw_gateway_bin")" \ || error "Could not determine the NemoClaw OpenShell gateway version at $nemoclaw_gateway_bin." upstream_version="$(openshell_binary_version "$upstream_gateway_bin")" \ @@ -1431,6 +1558,85 @@ openshell_user_config_home() { fi } +enabled_openshell_gateway_user_service_activation_path() { + local user_config_home user_data_home runtime_dir unit_root activation_dir service_name activation_path + local config_dirs data_dirs directory + local -a unit_roots=() + if [[ -n "${SYSTEMD_UNIT_PATH:-}" ]]; then + printf 'SYSTEMD_UNIT_PATH=%q\n' "$SYSTEMD_UNIT_PATH" + return 2 + fi + user_config_home="$(openshell_user_config_home)" + user_data_home="${XDG_DATA_HOME:-${HOME}/.local/share}" + if [[ "$user_data_home" != /* ]]; then + printf '%s\n' "$user_data_home" + return 2 + fi + unit_roots+=( + "${user_config_home}/systemd/user" + "${user_config_home}/systemd/user.control" + "${user_data_home%/}/systemd/user" + "/etc/systemd/user" + "/run/systemd/user" + "/usr/local/lib/systemd/user" + "/usr/lib/systemd/user" + "/lib/systemd/user" + ) + config_dirs="${XDG_CONFIG_DIRS:-/etc/xdg}" + data_dirs="${XDG_DATA_DIRS:-/usr/local/share:/usr/share}" + local IFS=: + for directory in $config_dirs $data_dirs; do + [[ -n "$directory" ]] || continue + if [[ "$directory" != /* ]]; then + printf '%s\n' "$directory" + return 2 + fi + unit_roots+=("${directory%/}/systemd/user") + done + runtime_dir="${XDG_RUNTIME_DIR:-}" + if [[ "$runtime_dir" != /* && "${UID:-}" =~ ^[0-9]+$ ]]; then + runtime_dir="/run/user/${UID}" + fi + if [[ "$runtime_dir" == /* ]]; then + unit_roots+=( + "${runtime_dir%/}/systemd/user.control" + "${runtime_dir%/}/systemd/transient" + "${runtime_dir%/}/systemd/generator.early" + "${runtime_dir%/}/systemd/user" + "${runtime_dir%/}/systemd/generator" + "${runtime_dir%/}/systemd/generator.late" + ) + fi + + for unit_root in "${unit_roots[@]}"; do + if [[ -e "$unit_root" || -L "$unit_root" ]]; then + if [[ ! -d "$unit_root" || ! -r "$unit_root" || ! -x "$unit_root" ]]; then + printf '%s\n' "$unit_root" + return 2 + fi + fi + for activation_dir in "$unit_root"/*.wants "$unit_root"/*.requires "$unit_root"/*.upholds; do + if [[ -L "$activation_dir" && ! -d "$activation_dir" ]]; then + printf '%s\n' "$activation_dir" + return 2 + fi + [[ -d "$activation_dir" ]] || continue + if [[ ! -r "$activation_dir" || ! -x "$activation_dir" ]]; then + printf '%s\n' "$activation_dir" + return 2 + fi + for service_name in openshell-gateway "${NEMOCLAW_GATEWAY_SERVICE_NAME}"; do + activation_path="${activation_dir}/${service_name}.service" + if [[ -e "$activation_path" || -L "$activation_path" ]]; then + printf '%s\n' "$activation_path" + return 0 + fi + done + done + done + return 1 +} + install_nemoclaw_openshell_gateway_user_service() { [[ "$(uname -s)" == "Linux" ]] || return 0 [[ "$(resolve_nemoclaw_gateway_port)" -eq 8080 ]] || return 0 @@ -1448,8 +1654,25 @@ install_nemoclaw_openshell_gateway_user_service() { if [[ -f "$service_path" ]] && ! is_nemoclaw_openshell_gateway_user_service "$service_path"; then error "Refusing to replace non-NemoClaw OpenShell gateway user service: $service_path" fi - require_compatible_upstream_openshell_gateway_service - info "OpenShell upstream gateway user service is staged; onboarding will select and start it." + local compatibility_status activation_path activation_status + if require_compatible_upstream_openshell_gateway_service; then + info "OpenShell upstream gateway user service is staged; onboarding will select and start it." + return 0 + else + compatibility_status=$? + fi + if [[ "$compatibility_status" -ne 2 ]]; then + error "Could not determine whether the effective upstream OpenShell gateway user service is compatible." + fi + if activation_path="$(enabled_openshell_gateway_user_service_activation_path)"; then + error "The systemd user manager is unavailable, but $activation_path can activate a gateway user service that can later claim port 8080. Restore the systemd user manager and inspect or disable that service before rerunning NemoClaw. The installer did not change the unit or activation path." + else + activation_status=$? + if [[ "$activation_status" -eq 2 ]]; then + error "The systemd user manager is unavailable, and the installer could not inspect OpenShell gateway activation configuration at $activation_path. Restore the default unit search path or access to that location, then rerun NemoClaw." + fi + fi + warn "The systemd user manager is unavailable. No enabled OpenShell gateway user service activation path was found, so onboarding will keep the existing standalone gateway on port 8080." return 0 fi diff --git a/src/lib/onboard/docker-driver-gateway-env-deb-override.test.ts b/src/lib/onboard/docker-driver-gateway-env-deb-override.test.ts index 245f8f80abc..305fb5d8c28 100644 --- a/src/lib/onboard/docker-driver-gateway-env-deb-override.test.ts +++ b/src/lib/onboard/docker-driver-gateway-env-deb-override.test.ts @@ -16,6 +16,22 @@ function homeEnv(home: string, xdgConfigHome = ""): NodeJS.ProcessEnv { return { HOME: home, XDG_CONFIG_HOME: xdgConfigHome } as NodeJS.ProcessEnv; } +function trustedPackageServiceOptions(home: string) { + return { + env: homeEnv(home), + getUpstreamGatewayVersion: () => "openshell-gateway 0.0.85", + getUpstreamGatewayVersionBounds: () => ({ max: "0.0.85", min: "0.0.85" }), + platform: "linux" as const, + spawnSyncImpl: () => ({ + status: 0, + stdout: [ + "FragmentPath=/usr/lib/systemd/user/openshell-gateway.service", + "ExecStart={ path=/usr/bin/openshell-gateway ; argv[]=/usr/bin/openshell-gateway ; }", + ].join("\n"), + }), + }; +} + describe("buildDockerGatewayDebEnvFile", () => { it("replaces all managed gateway env keys and preserves unrelated values", () => { const next = buildDockerGatewayDebEnvFile( @@ -138,7 +154,7 @@ describe("writeDockerGatewayDebEnvOverride", () => { () => ({ OPENSHELL_BIND_ADDRESS: "127.0.0.1", }), - { env: homeEnv(tempHome), platform: "linux" }, + trustedPackageServiceOptions(tempHome), ); const envFileContent = fs.readFileSync(envFile, "utf-8"); diff --git a/src/lib/onboard/docker-driver-gateway-env.test.ts b/src/lib/onboard/docker-driver-gateway-env.test.ts index a625b916113..fb9f9ef95ad 100644 --- a/src/lib/onboard/docker-driver-gateway-env.test.ts +++ b/src/lib/onboard/docker-driver-gateway-env.test.ts @@ -18,6 +18,22 @@ function homeEnv(home: string, xdgConfigHome = ""): NodeJS.ProcessEnv { return { HOME: home, XDG_CONFIG_HOME: xdgConfigHome } as NodeJS.ProcessEnv; } +function trustedPackageServiceOptions(home: string) { + return { + env: homeEnv(home), + getUpstreamGatewayVersion: () => "openshell-gateway 0.0.85", + getUpstreamGatewayVersionBounds: () => ({ max: "0.0.85", min: "0.0.85" }), + platform: "linux" as const, + spawnSyncImpl: () => ({ + status: 0, + stdout: [ + "FragmentPath=/usr/lib/systemd/user/openshell-gateway.service", + "ExecStart={ path=/usr/bin/openshell-gateway ; argv[]=/usr/bin/openshell-gateway ; }", + ].join("\n"), + }), + }; +} + describe("buildDockerDriverGatewayEnv", () => { it("sets Docker-driver gateway networking from NemoClaw configuration", () => { const env = buildDockerDriverGatewayEnv({ @@ -208,7 +224,7 @@ describe("writeDockerGatewayDebEnvOverride", () => { () => ({ OPENSHELL_BIND_ADDRESS: "127.0.0.1", }), - { env: homeEnv(tempHome), platform: "linux" }, + trustedPackageServiceOptions(tempHome), ); const envFileContent = fs.readFileSync(envFile, "utf-8"); @@ -253,7 +269,7 @@ describe("writeDockerGatewayDebEnvOverride", () => { () => ({ OPENSHELL_BIND_ADDRESS: "127.0.0.1", }), - { env: homeEnv(tempHome), platform: "linux" }, + trustedPackageServiceOptions(tempHome), ), ).toThrow("regular file changed during validation"); @@ -304,7 +320,7 @@ describe("writeDockerGatewayDebEnvOverride", () => { () => ({ OPENSHELL_BIND_ADDRESS: "127.0.0.1", }), - { env: homeEnv(tempHome), platform: "linux" }, + trustedPackageServiceOptions(tempHome), ); expect(wrote).toBe(true); diff --git a/src/lib/onboard/docker-driver-gateway-service-version-gate.test.ts b/src/lib/onboard/docker-driver-gateway-service-version-gate.test.ts index 029a47f422d..8dd7de43023 100644 --- a/src/lib/onboard/docker-driver-gateway-service-version-gate.test.ts +++ b/src/lib/onboard/docker-driver-gateway-service-version-gate.test.ts @@ -67,28 +67,35 @@ describe("package-managed gateway version gate (#8094)", () => { ); }); - it("adopts the package gateway when its version cannot be determined", () => { - // Pre-#8094 behaviour: only decline on positive evidence of a bad version, - // so an unreadable binary never turns a working host into a failing one. + it("declines the package gateway when its version cannot be determined (#8926)", () => { const verdict = checkUpstreamGatewayVersion( PACKAGE_BINARY, resolveOptions("ignored", { getUpstreamGatewayVersion: () => null }), ); - expect(verdict.supported).toBe(true); + expect(verdict).toMatchObject({ + binaryPath: PACKAGE_BINARY, + supported: false, + version: null, + }); }); - it("adopts when the effective package gateway binary cannot be resolved", () => { + it("declines when the effective package gateway binary cannot be resolved (#8926)", () => { const verdict = checkUpstreamGatewayVersion(null, resolveOptions("0.0.91")); - expect(verdict.supported).toBe(true); + expect(verdict).toMatchObject({ + binaryPath: "", + supported: false, + version: null, + }); }); - it("stops reporting a package unit whose gateway is out of window", () => { + it("blocks a package unit whose gateway is out of window (#8926)", () => { const warn = vi.fn(); - // The unit file is present, so the pre-fix resolver adopted it outright. - expect(hasOpenShellGatewayUserService(resolveOptions("0.0.91", { warn }))).toBe(false); + expect(() => hasOpenShellGatewayUserService(resolveOptions("0.0.91", { warn }))).toThrow( + "outside the maximum 0.0.85", + ); expect(warn).toHaveBeenCalledWith(expect.stringContaining("0.0.91")); }); @@ -96,44 +103,44 @@ describe("package-managed gateway version gate (#8094)", () => { expect(hasOpenShellGatewayUserService(resolveOptions("0.0.85"))).toBe(true); }); - it("falls back to the NemoClaw-managed unit when the package gateway is rejected", () => { + it("blocks the NemoClaw-managed fallback when the package gateway is rejected (#8926)", () => { const home = "/home/tester"; const nemoclawUnit = getNemoclawOpenShellGatewayUserServicePath(home, {}); - const resolved = hasOpenShellGatewayUserService( - resolveOptions("0.0.91", { - home, - env: {}, - existsSync: (p: string) => packageOnly(p) || p === nemoclawUnit, - lstatSync: (() => ({ isSymbolicLink: () => false })) as never, - readFileSync: () => NEMOCLAW_OPENSHELL_GATEWAY_USER_SERVICE_MARKER_LINE, - }), - ); - - expect(resolved).toBe(true); + expect(() => + hasOpenShellGatewayUserService( + resolveOptions("0.0.91", { + home, + env: {}, + existsSync: (p: string) => packageOnly(p) || p === nemoclawUnit, + lstatSync: (() => ({ isSymbolicLink: () => false })) as never, + readFileSync: () => NEMOCLAW_OPENSHELL_GATEWAY_USER_SERVICE_MARKER_LINE, + }), + ), + ).toThrow("outside the maximum 0.0.85"); }); - it("falls back to the NemoClaw-managed unit when the package service identity is untrusted", () => { + it("blocks the NemoClaw-managed fallback when the package service identity is untrusted (#8926)", () => { const home = "/home/tester"; const nemoclawUnit = getNemoclawOpenShellGatewayUserServicePath(home, {}); const getUpstreamGatewayVersion = vi.fn(() => "0.0.85"); - const resolved = hasOpenShellGatewayUserService( - resolveOptions("ignored", { - home, - env: {}, - existsSync: (p: string) => packageOnly(p) || p === nemoclawUnit, - getUpstreamGatewayVersion, - lstatSync: (() => ({ isSymbolicLink: () => false })) as never, - readFileSync: () => NEMOCLAW_OPENSHELL_GATEWAY_USER_SERVICE_MARKER_LINE, - spawnSyncImpl: () => ({ - status: 0, - stdout: trustedShowOutput("/opt/foreign/openshell-gateway"), + expect(() => + hasOpenShellGatewayUserService( + resolveOptions("ignored", { + home, + env: {}, + existsSync: (p: string) => packageOnly(p) || p === nemoclawUnit, + getUpstreamGatewayVersion, + lstatSync: (() => ({ isSymbolicLink: () => false })) as never, + readFileSync: () => NEMOCLAW_OPENSHELL_GATEWAY_USER_SERVICE_MARKER_LINE, + spawnSyncImpl: () => ({ + status: 0, + stdout: trustedShowOutput("/opt/foreign/openshell-gateway"), + }), }), - }), - ); - - expect(resolved).toBe(true); + ), + ).toThrow("trusted OpenShell gateway"); expect(getUpstreamGatewayVersion).not.toHaveBeenCalled(); }); @@ -141,9 +148,9 @@ describe("package-managed gateway version gate (#8094)", () => { const warn = vi.fn(); const options = resolveOptions("0.0.91", { warn }); - hasOpenShellGatewayUserService(options); - hasOpenShellGatewayUserService(options); - hasOpenShellGatewayUserService(options); + for (let attempt = 0; attempt < 3; attempt += 1) { + expect(() => hasOpenShellGatewayUserService(options)).toThrow("outside the maximum 0.0.85"); + } expect(warn).toHaveBeenCalledTimes(1); }); @@ -152,17 +159,19 @@ describe("package-managed gateway version gate (#8094)", () => { const suppressedWarn = vi.fn(); const laterWarn = vi.fn(); - expect( + expect(() => hasOpenShellGatewayUserService( resolveOptions("0.0.91", { suppressUnsupportedVersionWarning: true, warn: suppressedWarn, }), ), - ).toBe(false); + ).toThrow(); expect(suppressedWarn).not.toHaveBeenCalled(); - hasOpenShellGatewayUserService(resolveOptions("0.0.91", { warn: laterWarn })); + expect(() => + hasOpenShellGatewayUserService(resolveOptions("0.0.91", { warn: laterWarn })), + ).toThrow(); expect(laterWarn).toHaveBeenCalledOnce(); }); @@ -177,7 +186,7 @@ describe("package-managed gateway version gate (#8094)", () => { it("checks the effective ExecStart when both package binaries exist", () => { const getUpstreamGatewayVersion = vi.fn(() => "0.0.91"); - expect( + expect(() => hasOpenShellGatewayUserService( resolveOptions("ignored", { existsSync: (p: string) => @@ -187,7 +196,7 @@ describe("package-managed gateway version gate (#8094)", () => { spawnSyncImpl: () => ({ status: 0, stdout: trustedShowOutput(PACKAGE_BINARY) }), }), ), - ).toBe(false); + ).toThrow("outside the maximum 0.0.85"); expect(getUpstreamGatewayVersion).toHaveBeenCalledWith(PACKAGE_BINARY); }); @@ -227,9 +236,10 @@ describe("package-managed gateway version gate (#8094)", () => { ); expect(result).toMatchObject({ - attempted: false, + attempted: true, reason: expect.stringContaining("0.0.91"), serviceName: "openshell-gateway", + standaloneFallbackBlocked: true, started: false, }); expect(getUpstreamGatewayVersion).toHaveBeenCalledTimes(2); diff --git a/src/lib/onboard/docker-driver-gateway-service.test.ts b/src/lib/onboard/docker-driver-gateway-service.test.ts index d6b72dc9f70..459d899c43e 100644 --- a/src/lib/onboard/docker-driver-gateway-service.test.ts +++ b/src/lib/onboard/docker-driver-gateway-service.test.ts @@ -1,6 +1,8 @@ // SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. // SPDX-License-Identifier: Apache-2.0 +import path from "node:path"; + import { describe, expect, it, vi } from "vitest"; import { createVirtualClock } from "./__test-helpers__/virtual-clock"; @@ -13,7 +15,9 @@ import { getTrustedActiveOpenShellGatewayUserServiceIdentity, getTrustedActiveOpenShellGatewayUserServicePid, hasOpenShellGatewayUserService, + NEMOCLAW_OPENSHELL_GATEWAY_USER_SERVICE, NEMOCLAW_OPENSHELL_GATEWAY_USER_SERVICE_MARKER, + OPENSHELL_GATEWAY_USER_SERVICE, OpenShellGatewayServiceTrustError, type SpawnSyncLikeResult, startOpenShellGatewayUserService, @@ -87,6 +91,12 @@ function nonSymlinkStat(): never { return { isSymbolicLink: () => false } as never; } +function throwErrno(message: string, code: string): never { + const error = new Error(message) as NodeJS.ErrnoException; + error.code = code; + throw error; +} + function systemdSpawn( events: string[], fragmentPath = "/lib/systemd/user/openshell-gateway.service", @@ -111,6 +121,8 @@ describe("docker-driver-gateway-service", () => { expect( hasOpenShellGatewayUserService({ existsSync: linuxExists, + getUpstreamGatewayVersion: () => "0.0.85", + getUpstreamGatewayVersionBounds: () => ({ max: "0.0.85", min: "0.0.85" }), platform: "linux", spawnSyncImpl: systemdSpawn([]), }), @@ -382,6 +394,8 @@ describe("docker-driver-gateway-service", () => { env: { HOME: "/home/nvidia" }, existsSync: (candidate) => candidate === servicePath || candidate === "/lib/systemd/user/openshell-gateway.service", + getUpstreamGatewayVersion: () => "0.0.85", + getUpstreamGatewayVersionBounds: () => ({ max: "0.0.85", min: "0.0.85" }), home: "/home/nvidia", lstatSync: nonSymlinkStat, platform: "linux", @@ -401,6 +415,8 @@ describe("docker-driver-gateway-service", () => { commandExists: () => true, env: {}, existsSync: (candidate) => candidate === "/lib/systemd/user/openshell-gateway.service", + getUpstreamGatewayVersion: () => "0.0.85", + getUpstreamGatewayVersionBounds: () => ({ max: "0.0.85", min: "0.0.85" }), platform: "linux", spawnSyncImpl: systemdSpawn(events), validatePortOwnerForServiceStart: () => { @@ -457,6 +473,8 @@ describe("docker-driver-gateway-service", () => { commandExists: () => true, env: {}, existsSync: (candidate) => candidate === "/lib/systemd/user/openshell-gateway.service", + getUpstreamGatewayVersion: () => "0.0.85", + getUpstreamGatewayVersionBounds: () => ({ max: "0.0.85", min: "0.0.85" }), platform: "linux", spawnSyncImpl: vi.fn((_command: string, args: string[]) => { if (args.includes(failedCommand)) { @@ -483,28 +501,24 @@ describe("docker-driver-gateway-service", () => { }); }); - it("declines an upstream systemd service with a foreign executable", () => { - const result = startOpenShellGatewayUserService({ - commandExists: () => true, - env: {}, - existsSync: (candidate) => candidate === "/lib/systemd/user/openshell-gateway.service", - platform: "linux", - spawnSyncImpl: () => - spawnResult( - 0, - "", - trustedShowOutput( - "/lib/systemd/user/openshell-gateway.service", - "/tmp/openshell-gateway", + it("blocks an upstream systemd service with a foreign executable (#8926)", () => { + expect(() => + startOpenShellGatewayUserService({ + commandExists: () => true, + env: {}, + existsSync: (candidate) => candidate === "/lib/systemd/user/openshell-gateway.service", + platform: "linux", + spawnSyncImpl: () => + spawnResult( + 0, + "", + trustedShowOutput( + "/lib/systemd/user/openshell-gateway.service", + "/tmp/openshell-gateway", + ), ), - ), - }); - - expect(result).toMatchObject({ - attempted: false, - reason: "service not installed", - started: false, - }); + }), + ).toThrow("trusted OpenShell gateway"); }); it("selects the managed service log command without service validation (#8104)", () => { @@ -650,9 +664,7 @@ describe("docker-driver-gateway-service", () => { ); }); - it("continues to standalone fallback when managed service cleanup fails (#8104)", async () => { - const warn = vi.spyOn(console, "warn").mockImplementation(() => undefined); - + it("blocks standalone fallback when managed service cleanup fails without permission (#8926)", async () => { await expect( startPackageManagedDockerDriverGateway({ clearDockerDriverGatewayRuntimeFiles: vi.fn(), @@ -672,6 +684,34 @@ describe("docker-driver-gateway-service", () => { }, verifySandboxBridgeGatewayReachableOrExit: vi.fn(), }), + ).rejects.toThrow("service manager unavailable"); + }); + + it("continues only when cleanup explicitly permits standalone fallback (#8926)", async () => { + const warn = vi.spyOn(console, "warn").mockImplementation(() => undefined); + + await expect( + startPackageManagedDockerDriverGateway({ + clearDockerDriverGatewayRuntimeFiles: vi.fn(), + exitOnFailure: false, + gatewayName: "nemoclaw", + hasOpenShellGatewayUserService: () => true, + registerDockerDriverGatewayEndpoint: vi.fn(), + runCaptureOpenshell: vi.fn(), + skipSandboxBridgeReachability: false, + startOpenShellGatewayUserService: () => ({ + attempted: true, + reason: "restart failed", + started: false, + }), + stopOpenShellGatewayUserService: () => ({ + attempted: true, + reason: "Failed to connect to bus: No medium found", + standaloneFallbackAllowed: true, + stopped: false, + }), + verifySandboxBridgeGatewayReachableOrExit: vi.fn(), + }), ).resolves.toBe(false); expect(warn.mock.calls.flat().join("\n")).toContain( "standalone startup will verify gateway port ownership", @@ -1021,6 +1061,7 @@ describe("docker-driver-gateway-service", () => { home, lstatSync: nonSymlinkStat, platform: "linux", + readdirSync: (() => []) as never, readFileSync: () => `# ${NEMOCLAW_OPENSHELL_GATEWAY_USER_SERVICE_MARKER}\n`, spawnSyncImpl: vi.fn(() => spawnResult(1, "Failed to connect to bus: No medium found")), }); @@ -1032,18 +1073,35 @@ describe("docker-driver-gateway-service", () => { }); }); - it("refuses standalone fallback when the systemd service can activate automatically", () => { + it.each([ + OPENSHELL_GATEWAY_USER_SERVICE, + NEMOCLAW_OPENSHELL_GATEWAY_USER_SERVICE, + ])("blocks standalone fallback when the %s service can activate automatically (#8926)", (activationService) => { const home = "/home/nvidia"; const servicePath = `${home}/.config/systemd/user/nemoclaw-openshell-gateway.service`; - const activationPath = `${home}/.config/systemd/user/default.target.wants/nemoclaw-openshell-gateway.service`; + const activationPath = `${home}/.config/systemd/user/default.target.wants/${activationService}.service`; const result = stopOpenShellGatewayUserService({ commandExists: (command) => command === "systemctl", env: { HOME: home }, - existsSync: (candidate) => candidate === servicePath || candidate === activationPath, + existsSync: (candidate) => candidate === servicePath, home, - lstatSync: nonSymlinkStat, + lstatSync: ((candidate: string) => ({ + isSymbolicLink: () => candidate === activationPath, + })) as never, platform: "linux", + readdirSync: ((root: string) => + root === path.dirname(path.dirname(activationPath)) + ? [ + { + isDirectory: () => true, + isSymbolicLink: () => false, + name: path.basename(path.dirname(activationPath)), + }, + ] + : root === path.dirname(activationPath) + ? [path.basename(activationPath)] + : []) as never, readFileSync: () => `# ${NEMOCLAW_OPENSHELL_GATEWAY_USER_SERVICE_MARKER}\n`, spawnSyncImpl: vi.fn(() => spawnResult(1, "Failed to connect to bus: No medium found")), }); @@ -1051,7 +1109,341 @@ describe("docker-driver-gateway-service", () => { expect(result).toMatchObject({ attempted: true, standaloneFallbackAllowed: false, + standaloneFallbackBlocked: true, + stopped: false, + }); + expect(result.reason).toContain("can later claim port 8080"); + }); + + it("blocks standalone fallback when the service query returns an unknown error (#8926)", () => { + const home = "/home/nvidia"; + const servicePath = `${home}/.config/systemd/user/nemoclaw-openshell-gateway.service`; + + const result = stopOpenShellGatewayUserService({ + commandExists: (command) => command === "systemctl", + env: { HOME: home }, + existsSync: (candidate) => candidate === servicePath, + home, + lstatSync: nonSymlinkStat, + platform: "linux", + readFileSync: () => `# ${NEMOCLAW_OPENSHELL_GATEWAY_USER_SERVICE_MARKER}\n`, + spawnSyncImpl: vi.fn(() => + spawnResult( + 1, + "Failed to connect to bus: No medium found\nFailed to connect to bus: Permission denied", + ), + ), + }); + + expect(result).toMatchObject({ + attempted: true, + standaloneFallbackAllowed: false, + standaloneFallbackBlocked: true, stopped: false, }); + expect(result.reason).toContain("Permission denied"); + }); + + it("blocks standalone fallback when systemctl splits known and unknown diagnostics (#8926)", () => { + const home = "/home/nvidia"; + const servicePath = `${home}/.config/systemd/user/nemoclaw-openshell-gateway.service`; + + const result = stopOpenShellGatewayUserService({ + commandExists: (command) => command === "systemctl", + env: { HOME: home }, + existsSync: (candidate) => candidate === servicePath, + home, + lstatSync: nonSymlinkStat, + platform: "linux", + readFileSync: () => `# ${NEMOCLAW_OPENSHELL_GATEWAY_USER_SERVICE_MARKER}\n`, + spawnSyncImpl: vi.fn(() => + spawnResult(1, "Failed to connect to bus: No medium found", "Permission denied"), + ), + }); + + expect(result).toMatchObject({ + standaloneFallbackAllowed: false, + standaloneFallbackBlocked: true, + }); + expect(result.reason).toContain("Permission denied"); + }); + + it("blocks standalone fallback when the systemctl query throws (#8926)", () => { + const home = "/home/nvidia"; + const servicePath = `${home}/.config/systemd/user/nemoclaw-openshell-gateway.service`; + + const result = stopOpenShellGatewayUserService({ + commandExists: (command) => command === "systemctl", + env: { HOME: home }, + existsSync: (candidate) => candidate === servicePath, + home, + lstatSync: nonSymlinkStat, + platform: "linux", + readFileSync: () => `# ${NEMOCLAW_OPENSHELL_GATEWAY_USER_SERVICE_MARKER}\n`, + spawnSyncImpl: vi.fn(() => { + throw new Error("systemctl invocation failed"); + }), + }); + + expect(result).toMatchObject({ + standaloneFallbackAllowed: false, + standaloneFallbackBlocked: true, + }); + expect(result.reason).toContain("systemctl invocation failed"); + }); + + it("rejects multiple effective gateway executables (#8926)", () => { + const output = [ + "FragmentPath=/usr/lib/systemd/user/openshell-gateway.service", + "ExecStart={ path=/usr/bin/openshell-gateway ; argv[]=/usr/bin/openshell-gateway ; }; { path=/tmp/foreign/openshell-gateway ; argv[]=/tmp/foreign/openshell-gateway ; }", + ].join("\n"); + + expect(() => + hasOpenShellGatewayUserService({ + existsSync: (candidate) => candidate === "/usr/lib/systemd/user/openshell-gateway.service", + platform: "linux", + spawnSyncImpl: () => spawnResult(0, "", output), + }), + ).toThrow("trusted OpenShell gateway"); + }); + + it.each([ + ["user data", "/home/nvidia/.local/share/systemd/user/session.target.wants"], + ["user runtime", "/run/user/1000/systemd/user/default.target.wants"], + ["user control", "/home/nvidia/.config/systemd/user.control/default.target.wants"], + ["runtime control", "/run/user/1000/systemd/user.control/default.target.requires"], + ["early generator", "/run/user/1000/systemd/generator.early/default.target.wants"], + ["generator", "/run/user/1000/systemd/generator/default.target.requires"], + ["late generator", "/run/user/1000/systemd/generator.late/default.target.wants"], + ["transient", "/run/user/1000/systemd/transient/default.target.requires"], + ["upheld", "/etc/systemd/user/default.target.upholds"], + ["global config", "/etc/systemd/user/default.target.requires"], + ["package data", "/usr/share/systemd/user/default.target.wants"], + ])("blocks fallback for an activation link in the %s root (#8926)", (_root, activationDirectory) => { + const home = "/home/nvidia"; + const servicePath = `${home}/.config/systemd/user/nemoclaw-openshell-gateway.service`; + const activationPath = `${activationDirectory}/openshell-gateway.service`; + + const result = stopOpenShellGatewayUserService({ + commandExists: (command) => command === "systemctl", + env: { HOME: home, XDG_RUNTIME_DIR: "/run/user/1000" }, + existsSync: (candidate) => candidate === servicePath, + home, + lstatSync: ((candidate: string) => ({ + isSymbolicLink: () => candidate === activationPath, + })) as never, + platform: "linux", + readdirSync: ((root: string) => + root === activationDirectory + ? [path.basename(activationPath)] + : root === path.dirname(activationDirectory) + ? [{ isDirectory: () => true, name: path.basename(activationDirectory) }] + : []) as never, + readFileSync: () => `# ${NEMOCLAW_OPENSHELL_GATEWAY_USER_SERVICE_MARKER}\n`, + spawnSyncImpl: vi.fn(() => spawnResult(1, "Failed to connect to bus: No medium found")), + }); + + expect(result).toMatchObject({ + standaloneFallbackAllowed: false, + standaloneFallbackBlocked: true, + }); + expect(result.reason).toContain(activationPath); + }); + + it("fails closed when SYSTEMD_UNIT_PATH overrides the user unit search path (#8926)", () => { + const home = "/home/nvidia"; + const servicePath = `${home}/.config/systemd/user/nemoclaw-openshell-gateway.service`; + + expect(() => + stopOpenShellGatewayUserService({ + commandExists: (command) => command === "systemctl", + env: { HOME: home, SYSTEMD_UNIT_PATH: "/opt/custom-systemd/user" }, + existsSync: (candidate) => candidate === servicePath, + home, + lstatSync: nonSymlinkStat, + platform: "linux", + readFileSync: () => `# ${NEMOCLAW_OPENSHELL_GATEWAY_USER_SERVICE_MARKER}\n`, + spawnSyncImpl: vi.fn(() => spawnResult(1, "Failed to connect to bus: No medium found")), + }), + ).toThrow("SYSTEMD_UNIT_PATH"); + }); + + it("blocks fallback when an activation root cannot be inspected (#8926)", () => { + const home = "/home/nvidia"; + const servicePath = `${home}/.config/systemd/user/nemoclaw-openshell-gateway.service`; + + expect(() => + stopOpenShellGatewayUserService({ + commandExists: (command) => command === "systemctl", + env: { HOME: home }, + existsSync: (candidate) => candidate === servicePath, + home, + lstatSync: ((candidate: string) => + candidate === servicePath + ? { isSymbolicLink: () => false } + : throwErrno( + candidate === `${home}/.local/share/systemd/user` ? "permission denied" : "missing", + candidate === `${home}/.local/share/systemd/user` ? "EACCES" : "ENOENT", + )) as never, + platform: "linux", + readdirSync: ((root: string) => + root === `${home}/.local/share/systemd/user` + ? throwErrno("permission denied", "EACCES") + : throwErrno("missing", "ENOENT")) as never, + readFileSync: () => `# ${NEMOCLAW_OPENSHELL_GATEWAY_USER_SERVICE_MARKER}\n`, + spawnSyncImpl: vi.fn(() => spawnResult(1, "Failed to connect to bus: No medium found")), + }), + ).toThrow("permission denied"); + }); + + it.each([ + [ + "spawn error", + { + error: new Error("Failed to connect to bus: No medium found"), + status: null, + }, + ], + [ + "missing exit status", + { + status: null, + stderr: "Failed to connect to bus: No medium found", + }, + ], + ])("blocks fallback for a %s with a known-looking diagnostic (#8926)", (_case, queryResult) => { + const home = "/home/nvidia"; + const servicePath = `${home}/.config/systemd/user/nemoclaw-openshell-gateway.service`; + + const result = stopOpenShellGatewayUserService({ + commandExists: (command) => command === "systemctl", + env: { HOME: home }, + existsSync: (candidate) => candidate === servicePath, + home, + lstatSync: nonSymlinkStat, + platform: "linux", + readFileSync: () => `# ${NEMOCLAW_OPENSHELL_GATEWAY_USER_SERVICE_MARKER}\n`, + spawnSyncImpl: vi.fn(() => queryResult), + }); + + expect(result).toMatchObject({ + standaloneFallbackAllowed: false, + standaloneFallbackBlocked: true, + }); + }); + + it("blocks fallback for a symlinked activation directory (#8926)", () => { + const home = "/home/nvidia"; + const servicePath = `${home}/.config/systemd/user/nemoclaw-openshell-gateway.service`; + const userRoot = `${home}/.config/systemd/user`; + const activationDirectory = `${userRoot}/default.target.wants`; + const activationPath = `${activationDirectory}/openshell-gateway.service`; + + const result = stopOpenShellGatewayUserService({ + commandExists: (command) => command === "systemctl", + env: { HOME: home }, + existsSync: (candidate) => candidate === servicePath, + home, + lstatSync: nonSymlinkStat, + platform: "linux", + readdirSync: ((root: string) => + root === userRoot + ? [ + { + isDirectory: () => false, + isSymbolicLink: () => true, + name: "default.target.wants", + }, + ] + : root === activationDirectory + ? ["openshell-gateway.service"] + : []) as never, + readFileSync: () => `# ${NEMOCLAW_OPENSHELL_GATEWAY_USER_SERVICE_MARKER}\n`, + spawnSyncImpl: vi.fn(() => spawnResult(1, "Failed to connect to bus: No medium found")), + }); + + expect(result).toMatchObject({ standaloneFallbackBlocked: true }); + expect(result.reason).toContain(activationPath); + }); + + it("blocks fallback for a dangling activation directory (#8926)", () => { + const home = "/home/nvidia"; + const servicePath = `${home}/.config/systemd/user/nemoclaw-openshell-gateway.service`; + const userRoot = `${home}/.config/systemd/user`; + + expect(() => + stopOpenShellGatewayUserService({ + commandExists: (command) => command === "systemctl", + env: { HOME: home }, + existsSync: (candidate) => candidate === servicePath, + home, + lstatSync: nonSymlinkStat, + platform: "linux", + readdirSync: ((root: string) => + root === userRoot + ? [ + { + isDirectory: () => false, + isSymbolicLink: () => true, + name: "default.target.wants", + }, + ] + : throwErrno("dangling activation directory", "ENOENT")) as never, + readFileSync: () => `# ${NEMOCLAW_OPENSHELL_GATEWAY_USER_SERVICE_MARKER}\n`, + spawnSyncImpl: vi.fn(() => spawnResult(1, "Failed to connect to bus: No medium found")), + }), + ).toThrow("dangling activation directory"); + }); + + it("blocks fallback for a dangling activation root (#8926)", () => { + const home = "/home/nvidia"; + const servicePath = `${home}/.config/systemd/user/nemoclaw-openshell-gateway.service`; + const userRoot = `${home}/.config/systemd/user`; + + expect(() => + stopOpenShellGatewayUserService({ + commandExists: (command) => command === "systemctl", + env: { HOME: home }, + existsSync: (candidate) => candidate === servicePath, + home, + lstatSync: ((candidate: string) => ({ + isSymbolicLink: () => candidate === userRoot, + })) as never, + platform: "linux", + readdirSync: ((root: string) => { + const error = new Error( + root === userRoot ? "dangling activation root" : "missing", + ) as NodeJS.ErrnoException; + error.code = "ENOENT"; + throw error; + }) as never, + readFileSync: () => `# ${NEMOCLAW_OPENSHELL_GATEWAY_USER_SERVICE_MARKER}\n`, + spawnSyncImpl: vi.fn(() => spawnResult(1, "Failed to connect to bus: No medium found")), + }), + ).toThrow("dangling activation root"); + }); + + it("does not classify a thrown known diagnostic as a manager result (#8926)", () => { + const home = "/home/nvidia"; + const servicePath = `${home}/.config/systemd/user/nemoclaw-openshell-gateway.service`; + + const result = stopOpenShellGatewayUserService({ + commandExists: (command) => command === "systemctl", + env: { HOME: home }, + existsSync: (candidate) => candidate === servicePath, + home, + lstatSync: nonSymlinkStat, + platform: "linux", + readFileSync: () => `# ${NEMOCLAW_OPENSHELL_GATEWAY_USER_SERVICE_MARKER}\n`, + spawnSyncImpl: vi.fn(() => { + throw new Error("Failed to connect to bus: No medium found"); + }), + }); + + expect(result).toMatchObject({ + standaloneFallbackAllowed: false, + standaloneFallbackBlocked: true, + }); + expect(result.reason).toContain("systemctl invocation error"); }); }); diff --git a/src/lib/onboard/docker-driver-gateway-service.ts b/src/lib/onboard/docker-driver-gateway-service.ts index ab305166d58..6464e07e43b 100644 --- a/src/lib/onboard/docker-driver-gateway-service.ts +++ b/src/lib/onboard/docker-driver-gateway-service.ts @@ -39,10 +39,11 @@ export interface OpenShellGatewayUserServiceOptions { getUpstreamGatewayVersionBounds?: () => UpstreamGatewayVersionBounds; home?: string; lstatSync?: typeof fs.lstatSync; + readdirSync?: typeof fs.readdirSync; platform?: NodeJS.Platform; - /** Sink for the one-shot notice emitted when a package unit is declined. */ + /** Sink for the one-shot notice emitted when a package unit version is rejected. */ warn?: (message: string) => void; - /** Keep observation-only callers from emitting or consuming the warning latch. */ + /** Keep observation-only callers from emitting or consuming the version-error warning latch. */ suppressUnsupportedVersionWarning?: boolean; preparePortForServiceStart?: () => void; prepareServiceEnv?: () => void; @@ -185,8 +186,8 @@ export function getOpenShellGatewayUserServiceBinaryPaths(): string[] { * cannot rewrite either. What NemoClaw does own is the choice of whether to * adopt that unit, so the version window is enforced at adoption time. * whyNotSourceFix: editing or masking a distro-owned unit would fight the - * package manager and break on the next package upgrade; declining to adopt - * it keeps NemoClaw's own managed unit as the single source of truth. + * package manager and break on the next package upgrade. NemoClaw stops so + * another lifecycle cannot compete with the package unit for port 8080. * regressionTest: docker-driver-gateway-service-version-gate.test.ts * removalCondition: remove once the upstream unit resolves its gateway binary * through PATH (or a NemoClaw-supplied override) so a supported user-local @@ -196,7 +197,7 @@ export type UpstreamGatewayVersionBounds = { min: string | null; max: string | n export type UpstreamGatewayVersionVerdict = | { supported: true } - | { supported: false; binaryPath: string; version: string; message: string }; + | { supported: false; binaryPath: string; version: string | null; message: string }; function defaultUpstreamGatewayVersionBounds(): UpstreamGatewayVersionBounds { return { min: getBlueprintMinOpenshellVersion(), max: getBlueprintMaxOpenshellVersion() }; @@ -221,14 +222,7 @@ function readUpstreamGatewayVersion( } } -/** - * Decide whether the package-managed gateway binary may be adopted. - * - * An undetermined version keeps the pre-#8094 behaviour (adopt): this gate only - * declines when NemoClaw positively knows the binary is out of the supported - * window, mirroring how `ensureOpenshellForOnboard` guards both of its version - * checks on a resolved version. - */ +/** Decide whether the package-managed gateway binary may be adopted. */ export function checkUpstreamGatewayVersion( binaryPath: string | null, opts: Pick< @@ -240,12 +234,30 @@ export function checkUpstreamGatewayVersion( | "spawnSyncImpl" > = {}, ): UpstreamGatewayVersionVerdict { - if (!binaryPath) return { supported: true }; + if (!binaryPath) { + return { + supported: false, + binaryPath: "", + version: null, + message: + " NemoClaw could not resolve the effective package-managed OpenShell gateway executable. " + + "Restore the OpenShell package, then retry.", + }; + } const readVersion = opts.getUpstreamGatewayVersion ?? ((p: string) => readUpstreamGatewayVersion(p, opts)); const versionOutput = readVersion(binaryPath); const version = /([0-9]+\.[0-9]+\.[0-9]+)/.exec(versionOutput ?? "")?.[1]; - if (!version) return { supported: true }; + if (!version) { + return { + supported: false, + binaryPath, + version: null, + message: + ` NemoClaw could not determine the package-managed OpenShell gateway version at ${binaryPath}. ` + + "Restore the OpenShell package, then retry.", + }; + } const bounds = (opts.getUpstreamGatewayVersionBounds ?? defaultUpstreamGatewayVersionBounds)(); const belowMin = Boolean(bounds.min) && !versionGte(version, bounds.min as string); const aboveMax = @@ -263,10 +275,9 @@ export function checkUpstreamGatewayVersion( binaryPath, version, message: - ` Ignoring the system OpenShell gateway service: ${binaryPath} is ${version}, ` + + ` Refusing the system OpenShell gateway service: ${binaryPath} is ${version}, ` + `outside the ${bound} supported by this NemoClaw release.\n` + - " NemoClaw will manage its own gateway service instead. To use the system service, " + - "install a supported OpenShell package or remove the existing one.", + " Install a supported OpenShell package or remove the existing package before retrying NemoClaw.", }; } @@ -346,18 +357,35 @@ function runCommand( command: string, args: string[], opts: Required>, -): { ok: boolean; reason?: string; stdout?: string } { - const result = opts.spawnSyncImpl(command, args, { - encoding: "utf-8", - env: opts.env, - stdio: ["ignore", "pipe", "pipe"], - } satisfies SpawnSyncOptions); - if (result.error) return { ok: false, reason: result.error.message }; +): { diagnostic?: string; ok: boolean; reason?: string; stdout?: string } { + let result: SpawnSyncLikeResult; + try { + result = opts.spawnSyncImpl(command, args, { + encoding: "utf-8", + env: opts.env, + stdio: ["ignore", "pipe", "pipe"], + } satisfies SpawnSyncOptions); + } catch (error) { + return { ok: false, reason: `${command} invocation error: ${formatError(error)}` }; + } + if (result.error) + return { ok: false, reason: `${command} execution error: ${result.error.message}` }; + if (result.status === null) { + const detail = [text(result.stderr).trim(), text(result.stdout).trim()] + .filter(Boolean) + .join("\n"); + return { + ok: false, + reason: `${command} ended without an exit status${detail ? `: ${detail}` : ""}`, + }; + } if (result.status !== 0) { + const diagnostics = [text(result.stderr).trim(), text(result.stdout).trim()].filter(Boolean); + const diagnostic = diagnostics.join("\n") || `exit ${String(result.status)}`; return { ok: false, - reason: - text(result.stderr).trim() || text(result.stdout).trim() || `exit ${String(result.status)}`, + diagnostic, + reason: diagnostic, }; } return { ok: true, stdout: text(result.stdout) }; @@ -367,7 +395,10 @@ function runSystemctlUser( args: string[], opts: Required>, ) { - return runCommand("systemctl", ["--user", ...args], opts); + return runCommand("systemctl", ["--user", ...args], { + ...opts, + env: { ...opts.env, LC_ALL: "C" }, + }); } function runBrew( @@ -495,9 +526,8 @@ function resolveOpenShellGatewayUserService( if (platform !== "linux") return null; if (hasUpstreamOpenShellGatewayUserService(opts)) { // The package unit hard-codes an absolute ExecStart, so a supported - // user-local build cannot override it. Adopting it while it runs an - // out-of-window gateway is how #8094 got a 0.0.85 CLI driving a 0.0.91 - // gateway; decline instead and let NemoClaw manage its own service. + // user-local build cannot override it. A version or identity failure must + // block fallback because an enabled package service can later claim 8080. const upstreamService: OpenShellGatewayUserServiceTarget = { logCommand: getSystemdGatewayLogCommand(OPENSHELL_GATEWAY_USER_SERVICE), manager: "systemd", @@ -511,20 +541,23 @@ function resolveOpenShellGatewayUserService( env, spawnSyncImpl: opts.spawnSyncImpl ?? spawnSync, }); - // A failed systemctl query leaves the version unknown and preserves the - // existing adoption behaviour. Positive evidence of a foreign unit or - // executable must fail closed and continue to the NemoClaw fallback. - if (identity.ok || !identity.trustFailure) { - const verdict = checkUpstreamGatewayVersion( - identity.ok ? identity.execStartPath : null, - opts, + if (!identity.ok) { + if (!identity.trustFailure && userManagerLooksUnavailable(identity.reason ?? "")) { + return upstreamService; + } + throw new OpenShellGatewayServiceTrustError( + `Could not verify the effective OpenShell gateway user service: ${identity.reason ?? "systemctl query failed"}`, ); + } + if (identity.ok) { + const verdict = checkUpstreamGatewayVersion(identity.execStartPath, opts); if (verdict.supported) { return upstreamService; } if (!opts.suppressUnsupportedVersionWarning) { warnUnsupportedUpstreamGateway(verdict, opts); } + throw new OpenShellGatewayServiceTrustError(verdict.message.trim()); } } @@ -559,56 +592,200 @@ export function hasOpenShellGatewayUserService( } function userManagerLooksUnavailable(reason: string): boolean { - return /Failed to connect to bus|No medium found|XDG_RUNTIME_DIR|System has not been booted|Host is down/i.test( - reason, + const diagnostics = reason + .split(/\r?\n/) + .map((line) => line.trim()) + .filter(Boolean); + return ( + diagnostics.length > 0 && + diagnostics.every( + (diagnostic) => + diagnostic === "Failed to connect to bus: No medium found" || + diagnostic === "Failed to connect to bus: Host is down" || + diagnostic === "Failed to connect to bus: No such file or directory" || + diagnostic === + "System has not been booted with systemd as init system (PID 1). Can't operate." || + diagnostic === "XDG_RUNTIME_DIR is not set in the environment." || + diagnostic === + "Failed to connect to bus: $DBUS_SESSION_BUS_ADDRESS and $XDG_RUNTIME_DIR not defined (consider using --machine=@.host --user to connect to bus of other user)", + ) ); } -function hasSystemdUserServiceActivationLink( +function findSystemdUserServiceActivationPath( service: OpenShellGatewayUserServiceTarget, home: string, env: NodeJS.ProcessEnv, existsSync: (filePath: string) => boolean, -): boolean { - if (service.manager !== "systemd") return false; - return existsSync( - path.join( - getOpenShellUserConfigHome(home, env), - "systemd", - "user", - "default.target.wants", - `${service.serviceName}.service`, + lstatSync: typeof fs.lstatSync, + readdirSync: typeof fs.readdirSync, +): string | null { + if (service.manager !== "systemd") return null; + if (env.SYSTEMD_UNIT_PATH?.trim()) { + throw new OpenShellGatewayServiceTrustError( + "SYSTEMD_UNIT_PATH overrides the systemd user unit search path, so NemoClaw cannot prove that no gateway service can activate.", + ); + } + const configDirectories = (env.XDG_CONFIG_DIRS?.trim() || "/etc/xdg").split(":").filter(Boolean); + const dataHome = env.XDG_DATA_HOME?.trim(); + const effectiveDataHome = + dataHome && path.isAbsolute(dataHome) + ? path.normalize(dataHome) + : path.join(home, ".local", "share"); + const configuredDataDirectories = (env.XDG_DATA_DIRS?.trim() || "/usr/local/share:/usr/share") + .split(":") + .filter(Boolean); + if ( + configDirectories.some((directory) => !path.isAbsolute(directory)) || + configuredDataDirectories.some((directory) => !path.isAbsolute(directory)) + ) { + throw new OpenShellGatewayServiceTrustError( + "XDG_CONFIG_DIRS or XDG_DATA_DIRS contains a relative path, so NemoClaw cannot inspect user service activation paths.", + ); + } + const roots = [ + path.join(getOpenShellUserConfigHome(home, env), "systemd", "user"), + path.join(getOpenShellUserConfigHome(home, env), "systemd", "user.control"), + path.join(effectiveDataHome, "systemd", "user"), + "/etc/systemd/user", + "/run/systemd/user", + "/usr/local/lib/systemd/user", + "/usr/lib/systemd/user", + "/lib/systemd/user", + ...configDirectories.map((directory) => + path.join(path.normalize(directory), "systemd", "user"), ), - ); + ...configuredDataDirectories.map((directory) => + path.join(path.normalize(directory), "systemd", "user"), + ), + ]; + const runtimeDir = env.XDG_RUNTIME_DIR?.trim(); + const effectiveRuntimeDir = + runtimeDir && path.isAbsolute(runtimeDir) + ? path.normalize(runtimeDir) + : typeof process.getuid === "function" + ? path.join("/run/user", String(process.getuid())) + : null; + if (effectiveRuntimeDir) { + roots.push( + path.join(effectiveRuntimeDir, "systemd", "user.control"), + path.join(effectiveRuntimeDir, "systemd", "transient"), + path.join(effectiveRuntimeDir, "systemd", "generator.early"), + path.join(effectiveRuntimeDir, "systemd", "user"), + path.join(effectiveRuntimeDir, "systemd", "generator"), + path.join(effectiveRuntimeDir, "systemd", "generator.late"), + ); + } + const serviceNames = [OPENSHELL_GATEWAY_USER_SERVICE, NEMOCLAW_OPENSHELL_GATEWAY_USER_SERVICE]; + for (const root of new Set(roots)) { + let targetDirectories: string[]; + try { + targetDirectories = readdirSync(root, { withFileTypes: true }) + .filter( + (entry) => + (entry.isDirectory() || entry.isSymbolicLink()) && + (entry.name.endsWith(".wants") || + entry.name.endsWith(".requires") || + entry.name.endsWith(".upholds")), + ) + .map((entry) => entry.name); + } catch (error) { + const code = + error && typeof error === "object" && "code" in error ? String(error.code) : null; + if (code === "ENOENT" || code === "ENOTDIR") { + try { + lstatSync(root); + } catch (statError) { + const statCode = + statError && typeof statError === "object" && "code" in statError + ? String(statError.code) + : null; + if (statCode === "ENOENT" || statCode === "ENOTDIR") continue; + throw new OpenShellGatewayServiceTrustError( + `Could not inspect OpenShell gateway user service root ${root}: ${formatError(statError)}`, + ); + } + } + throw new OpenShellGatewayServiceTrustError( + `Could not inspect OpenShell gateway user service root ${root}: ${formatError(error)}`, + ); + } + for (const targetDirectory of targetDirectories) { + const targetPath = path.join(root, targetDirectory); + let targetEntries: string[]; + try { + targetEntries = readdirSync(targetPath).map(String); + } catch (error) { + throw new OpenShellGatewayServiceTrustError( + `Could not inspect OpenShell gateway user service dependency directory ${targetPath}: ${formatError(error)}`, + ); + } + for (const serviceName of serviceNames) { + const candidate = path.join(root, targetDirectory, `${serviceName}.service`); + if (targetEntries.includes(`${serviceName}.service`)) return candidate; + if (existsSync(candidate)) return candidate; + try { + if (lstatSync(candidate).isSymbolicLink()) return candidate; + } catch (error) { + const code = + error && typeof error === "object" && "code" in error ? String(error.code) : null; + if (code === "ENOENT" || code === "ENOTDIR") continue; + throw new OpenShellGatewayServiceTrustError( + `Could not inspect OpenShell gateway user service activation path ${candidate}: ${formatError(error)}`, + ); + } + } + } + } + return null; } -function parseSystemctlShow(output: string): Record { - return Object.fromEntries( - output - .split(/\r?\n/) - .map((line) => { - const separator = line.indexOf("="); - return separator > 0 ? [line.slice(0, separator), line.slice(separator + 1).trim()] : null; - }) - .filter((entry): entry is [string, string] => entry !== null), - ); +function parseSystemctlShow( + output: string, + expectedProperties: readonly string[], +): Record | null { + const expected = new Set(expectedProperties); + const properties: Record = {}; + for (const line of output.split(/\r?\n/)) { + if (line === "") continue; + const separator = line.indexOf("="); + const property = separator > 0 ? line.slice(0, separator) : ""; + if (!expected.has(property) || Object.hasOwn(properties, property)) return null; + properties[property] = line.slice(separator + 1).trim(); + } + return expectedProperties.every((property) => Object.hasOwn(properties, property)) + ? properties + : null; } function extractSystemdExecStartPath(execStart: string): string | null { - const candidate = /(?:^|[\s;])path=([^\s;]+)/.exec(execStart)?.[1]?.trim(); - return candidate && path.isAbsolute(candidate) ? path.normalize(candidate) : null; + const candidates = Array.from( + execStart.matchAll(/(?:^|[\s;])path=([^\s;]+)/g), + (match) => match[1]?.trim() ?? "", + ); + if (candidates.length !== 1 || !path.isAbsolute(candidates[0])) return null; + return path.normalize(candidates[0]); } function validateSystemdServiceIdentity( service: OpenShellGatewayUserServiceTarget, opts: Required>, -): { ok: true; execStartPath: string } | { ok: false; reason?: string; trustFailure?: boolean } { +): + | { ok: true; execStartPath: string } + | { diagnostic?: string; ok: false; reason?: string; trustFailure?: boolean } { const result = runSystemctlUser( ["show", service.serviceName, "--property=FragmentPath", "--property=ExecStart"], opts, ); - if (!result.ok) return { ok: false, reason: result.reason }; - const properties = parseSystemctlShow(result.stdout ?? ""); + if (!result.ok) return { diagnostic: result.diagnostic, ok: false, reason: result.reason }; + const properties = parseSystemctlShow(result.stdout ?? "", ["FragmentPath", "ExecStart"]); + if (!properties) { + return { + ok: false, + reason: "service identity query returned invalid metadata", + trustFailure: true, + }; + } return validateSystemdServiceIdentityFromProperties(service, properties); } @@ -720,7 +897,13 @@ export function getTrustedActiveOpenShellGatewayUserServiceIdentity( { env, spawnSyncImpl }, ); if (!result.ok) return null; - const properties = parseSystemctlShow(result.stdout ?? ""); + const properties = parseSystemctlShow(result.stdout ?? "", [ + "FragmentPath", + "ExecStart", + "ActiveState", + "MainPID", + ]); + if (!properties) return null; const identity = validateSystemdServiceIdentityFromProperties(service, properties); if (properties.ActiveState !== "active" || !identity.ok) { return null; @@ -784,21 +967,6 @@ function serviceFailure( }; } -function serviceDeclined( - service: OpenShellGatewayUserServiceTarget, - reason: string, -): OpenShellGatewayUserServiceStartResult { - return { - attempted: false, - logCommand: service.logCommand, - manager: service.manager, - reason, - serviceName: service.serviceName, - started: false, - statusCommand: service.statusCommand, - }; -} - function runHook( hook: (() => void) | undefined, service: OpenShellGatewayUserServiceTarget, @@ -862,9 +1030,11 @@ export function startOpenShellGatewayUserService( const verdict = checkUpstreamGatewayVersion(identity.execStartPath, opts); if (!verdict.supported) { warnUnsupportedUpstreamGateway(verdict, opts); - return serviceDeclined( + const version = verdict.version ?? "unknown"; + return serviceFailure( service, - `package-managed gateway changed before startup: ${verdict.binaryPath} is ${verdict.version}`, + `package-managed gateway changed before startup: ${verdict.binaryPath} is ${version}`, + true, ); } } @@ -976,37 +1146,58 @@ export function stopOpenShellGatewayUserService( stopped: boolean, reason?: string, standaloneFallbackBlocked = false, - ): OpenShellGatewayUserServiceStopResult => ({ - attempted: true, - standaloneFallbackAllowed: - !stopped && - !standaloneFallbackBlocked && - service.manager === "systemd" && - userManagerLooksUnavailable(reason ?? "") && - !hasSystemdUserServiceActivationLink(service, home, env, existsSync), - manager: service.manager, - serviceName: service.serviceName, - ...(standaloneFallbackBlocked ? { standaloneFallbackBlocked: true } : {}), - statusCommand: service.statusCommand, - stopped, - ...(reason === undefined ? {} : { reason }), - }); + managerDiagnostic?: string, + ): OpenShellGatewayUserServiceStopResult => { + const userManagerUnavailable = + service.manager === "systemd" && userManagerLooksUnavailable(managerDiagnostic ?? ""); + const activationPath = userManagerUnavailable + ? findSystemdUserServiceActivationPath( + service, + home, + env, + existsSync, + opts.lstatSync ?? fs.lstatSync, + opts.readdirSync ?? fs.readdirSync, + ) + : null; + const fallbackBlocked = standaloneFallbackBlocked || activationPath !== null; + const reportedReason = activationPath + ? `${reason ?? "The systemd user manager is unavailable"}; ${activationPath} can activate a gateway user service that can later claim port 8080` + : reason; + return { + attempted: true, + standaloneFallbackAllowed: !stopped && !fallbackBlocked && userManagerUnavailable, + manager: service.manager, + serviceName: service.serviceName, + ...(fallbackBlocked ? { standaloneFallbackBlocked: true } : {}), + statusCommand: service.statusCommand, + stopped, + ...(reportedReason === undefined ? {} : { reason: reportedReason }), + }; + }; const command = stopServiceCommandName(service); if (!commandExists(command)) return describe(false, `${command} is not available`); if (service.manager === "systemd") { const identity = validateSystemdServiceIdentity(service, { env, spawnSyncImpl }); if (!identity.ok) { + const userManagerUnavailable = userManagerLooksUnavailable(identity.reason ?? ""); return describe( false, identity.reason ?? "service identity is invalid", - identity.trustFailure, + identity.trustFailure || !userManagerUnavailable, + identity.diagnostic, ); } } const stop = runStopService(service, { env, spawnSyncImpl }); if (stop.ok) return describe(true); const prefix = service.manager === "homebrew" ? "brew services stop" : "systemctl --user stop"; - return describe(false, `${prefix} ${service.serviceName} failed: ${stop.reason}`); + return describe( + false, + `${prefix} ${service.serviceName} failed: ${stop.reason}`, + false, + stop.diagnostic, + ); } export async function startPackageManagedDockerDriverGateway({ @@ -1038,6 +1229,11 @@ export async function startPackageManagedDockerDriverGateway({ stopped.reason ?? "managed service identity is not trusted", ); } + if (stopped.attempted && !stopped.stopped && !stopped.standaloneFallbackAllowed) { + throw new OpenShellGatewayServiceTrustError( + stopped.reason ?? "managed service cleanup did not explicitly allow standalone fallback", + ); + } if (stopped.attempted && !stopped.stopped) { const detail = stopped.reason ? ` (${stopped.reason})` : ""; console.warn( @@ -1045,11 +1241,14 @@ export async function startPackageManagedDockerDriverGateway({ ); } } catch (error) { - if (error instanceof OpenShellGatewayServiceTrustError && exitOnFailure) process.exit(1); - if (error instanceof OpenShellGatewayServiceTrustError) throw error; - console.warn( - ` OpenShell gateway managed service cleanup failed (${formatError(error)}); standalone startup will verify gateway port ownership.`, - ); + const failure = + error instanceof OpenShellGatewayServiceTrustError + ? error + : new OpenShellGatewayServiceTrustError( + `OpenShell gateway managed service cleanup failed: ${formatError(error)}`, + ); + if (exitOnFailure) process.exit(1); + throw failure; } }; try { diff --git a/test/install-openshell-gateway-service.test.ts b/test/install-openshell-gateway-service.test.ts index 5a21b756151..c975a0b49b6 100644 --- a/test/install-openshell-gateway-service.test.ts +++ b/test/install-openshell-gateway-service.test.ts @@ -18,6 +18,7 @@ const SERVICE_TEMPLATE = path.join( "lib", "openshell-gateway.service.in", ); +const RUNNING_AS_ROOT = typeof process.getuid === "function" && process.getuid() === 0; const tempRoots: string[] = []; afterEach(() => { @@ -117,6 +118,51 @@ function writeSystemctlStub( return { bin, log }; } +function writeUpstreamSystemctlStub( + home: string, + options: { + diagnostic?: string; + execStart?: string; + fragmentPath?: string; + gatewayBin?: string; + status?: number; + }, +) { + const bin = path.join(home, "upstream-systemctl-bin"); + const log = path.join(home, "upstream-systemctl.log"); + const status = options.status ?? 0; + fs.mkdirSync(bin, { recursive: true }); + const response = + status === 0 + ? [ + `printf 'FragmentPath=%s\\n' ${JSON.stringify(options.fragmentPath ?? "")}`, + options.execStart === undefined + ? `printf 'ExecStart={ path=%s ; argv[]=%s ; ignore_errors=no ; }\\n' ${JSON.stringify(options.gatewayBin ?? "")} ${JSON.stringify(options.gatewayBin ?? "")}` + : `printf 'ExecStart=%s\\n' ${JSON.stringify(options.execStart)}`, + ] + : [ + ...(options.diagnostic ?? "systemctl failed") + .split(/\r?\n/) + .map((line) => `printf '%s\\n' ${JSON.stringify(line)} >&2`), + `exit ${status}`, + ]; + writeExecutable( + path.join(bin, "systemctl"), + [ + "#!/usr/bin/env bash", + `printf '%s\\n' "$*" >> ${JSON.stringify(log)}`, + 'case "$*" in', + ' "--user show openshell-gateway.service --property=FragmentPath --property=ExecStart")', + ...response.map((line) => ` ${line}`), + " ;;", + " *) exit 97 ;;", + "esac", + "", + ].join("\n"), + ); + return { bin, log }; +} + describe("install.sh OpenShell gateway service", () => { it.each([ "user-local", @@ -202,7 +248,7 @@ describe("install.sh OpenShell gateway service", () => { [ "upstream_openshell_gateway_user_service_installed() { return 0; }", `resolve_openshell_gateway_bin_for_service() { printf '%s\\n' ${JSON.stringify(nemoclawGatewayBin)}; }`, - `resolve_upstream_openshell_gateway_bin_for_service() { printf '%s\\n' ${JSON.stringify(upstreamGatewayBin)}; }`, + `inspect_upstream_openshell_gateway_user_service() { UPSTREAM_OPENSHELL_GATEWAY_SERVICE_BIN=${JSON.stringify(upstreamGatewayBin)}; return 0; }`, "trusted_openshell_gateway_bin_for_service() { return 0; }", "install_nemoclaw_openshell_gateway_user_service", ].join("\n"), @@ -232,7 +278,7 @@ describe("install.sh OpenShell gateway service", () => { [ "upstream_openshell_gateway_user_service_installed() { return 0; }", `resolve_openshell_gateway_bin_for_service() { printf '%s\\n' ${JSON.stringify(nemoclawGatewayBin)}; }`, - `resolve_upstream_openshell_gateway_bin_for_service() { printf '%s\\n' ${JSON.stringify(upstreamGatewayBin)}; }`, + `inspect_upstream_openshell_gateway_user_service() { UPSTREAM_OPENSHELL_GATEWAY_SERVICE_BIN=${JSON.stringify(upstreamGatewayBin)}; return 0; }`, "trusted_openshell_gateway_bin_for_service() { return 0; }", "install_nemoclaw_openshell_gateway_user_service", ].join("\n"), @@ -251,26 +297,263 @@ describe("install.sh OpenShell gateway service", () => { const home = makeTempRoot(); const conventionalGatewayBin = path.join(home, "usr", "bin", "openshell-gateway"); const overriddenGatewayBin = path.join(home, "opt", "openshell", "openshell-gateway"); - const systemctlBin = path.join(home, "systemctl-bin"); + const upstreamUnit = path.join( + home, + "usr", + "lib", + "systemd", + "user", + "openshell-gateway.service", + ); fs.mkdirSync(path.dirname(conventionalGatewayBin), { recursive: true }); fs.mkdirSync(path.dirname(overriddenGatewayBin), { recursive: true }); - fs.mkdirSync(systemctlBin); writeExecutable(conventionalGatewayBin, "#!/usr/bin/env bash\nexit 0\n"); writeExecutable(overriddenGatewayBin, "#!/usr/bin/env bash\nexit 0\n"); - writeExecutable( - path.join(systemctlBin, "systemctl"), - `#!/usr/bin/env bash\nprintf '{ path=${overriddenGatewayBin} ; argv[]=${overriddenGatewayBin} ; ignore_errors=no ; }\\n'\n`, - ); - - const result = runInstallHelper(home, "resolve_upstream_openshell_gateway_bin_for_service", { - PATH: `${systemctlBin}:${path.dirname(process.execPath)}:${TEST_SYSTEM_PATH}`, + const systemctl = writeUpstreamSystemctlStub(home, { + fragmentPath: upstreamUnit, + gatewayBin: overriddenGatewayBin, }); + const result = runInstallHelper( + home, + [ + `trusted_upstream_openshell_gateway_unit_for_service() { [[ "$1" == ${JSON.stringify(upstreamUnit)} ]]; }`, + `trusted_upstream_openshell_gateway_bin_for_service() { [[ "$1" == ${JSON.stringify(overriddenGatewayBin)} ]]; }`, + "resolve_upstream_openshell_gateway_bin_for_service", + ].join("\n"), + { PATH: `${systemctl.bin}:${path.dirname(process.execPath)}:${TEST_SYSTEM_PATH}` }, + ); + expect(result.status).toBe(0); expect(result.stdout.trim()).toBe(overriddenGatewayBin); expect(result.stdout).not.toContain(conventionalGatewayBin); }); + it("rejects duplicate effective ExecStart records even when their paths match (#8926)", () => { + const home = makeTempRoot(); + const gatewayBin = path.join(home, "usr", "bin", "openshell-gateway"); + const upstreamUnit = path.join( + home, + "usr", + "lib", + "systemd", + "user", + "openshell-gateway.service", + ); + fs.mkdirSync(path.dirname(gatewayBin), { recursive: true }); + writeExecutable(gatewayBin, "#!/usr/bin/env bash\nexit 0\n"); + const systemctl = writeUpstreamSystemctlStub(home, { + execStart: `{ path=${gatewayBin} ; }; { path=${gatewayBin} ; }`, + fragmentPath: upstreamUnit, + }); + + const result = runInstallHelper( + home, + [ + `trusted_upstream_openshell_gateway_unit_for_service() { [[ "$1" == ${JSON.stringify(upstreamUnit)} ]]; }`, + `trusted_upstream_openshell_gateway_bin_for_service() { [[ "$1" == ${JSON.stringify(gatewayBin)} ]]; }`, + 'inspect_upstream_openshell_gateway_user_service || { printf "%s\\n" "$UPSTREAM_OPENSHELL_GATEWAY_SERVICE_ERROR" >&2; exit 1; }', + ].join("\n"), + { PATH: `${systemctl.bin}:${path.dirname(process.execPath)}:${TEST_SYSTEM_PATH}` }, + ); + + expect(result.status).toBe(1); + expect(result.stderr).toContain("did not return one executable path"); + }); + + it("keeps the standalone gateway when the systemd user manager is unavailable (#8926)", () => { + const home = makeTempRoot(); + const systemctl = writeUpstreamSystemctlStub(home, { + diagnostic: "Failed to connect to bus: No medium found", + status: 1, + }); + + const result = runInstallHelper( + home, + [ + "upstream_openshell_gateway_user_service_installed() { return 0; }", + "install_nemoclaw_openshell_gateway_user_service", + ].join("\n"), + { PATH: `${systemctl.bin}:${path.dirname(process.execPath)}:${TEST_SYSTEM_PATH}` }, + ); + + expect(result.status, result.stdout + result.stderr).toBe(0); + expect(result.stdout).toContain("existing standalone gateway"); + expect(result.stdout).toContain("port 8080"); + expect(fs.existsSync(servicePath(home))).toBe(false); + expect(fs.readFileSync(systemctl.log, "utf-8").trim()).toBe( + "--user show openshell-gateway.service --property=FragmentPath --property=ExecStart", + ); + }); + + it.each([ + "openshell-gateway", + "nemoclaw-openshell-gateway", + ])("blocks standalone fallback when an enabled %s user service could claim port 8080 (#8926)", (serviceName) => { + const home = makeTempRoot(); + const activationPath = path.join( + home, + ".config", + "systemd", + "user", + "default.target.wants", + `${serviceName}.service`, + ); + fs.mkdirSync(path.dirname(activationPath), { recursive: true }); + fs.symlinkSync(path.join(home, "missing-package-unit.service"), activationPath); + const systemctl = writeUpstreamSystemctlStub(home, { + diagnostic: "Failed to connect to bus: No medium found", + status: 1, + }); + + const result = runInstallHelper( + home, + [ + "upstream_openshell_gateway_user_service_installed() { return 0; }", + "install_nemoclaw_openshell_gateway_user_service", + ].join("\n"), + { PATH: `${systemctl.bin}:${path.dirname(process.execPath)}:${TEST_SYSTEM_PATH}` }, + ); + + expect(result.status).toBe(1); + expect(result.stderr).toContain(activationPath); + expect(result.stderr).toContain("claim port 8080"); + expect(result.stderr).toContain("Restore the systemd user manager"); + expect(fs.lstatSync(activationPath).isSymbolicLink()).toBe(true); + expect(fs.existsSync(servicePath(home))).toBe(false); + }); + + it.each([ + ["user data", ".local/share/systemd/user/session.target.wants"], + ["user runtime", "runtime/systemd/user/default.target.wants"], + ["user control", ".config/systemd/user.control/default.target.wants"], + ["runtime control", "runtime/systemd/user.control/default.target.requires"], + ["early generator", "runtime/systemd/generator.early/default.target.wants"], + ["generator", "runtime/systemd/generator/default.target.requires"], + ["late generator", "runtime/systemd/generator.late/default.target.wants"], + ["transient", "runtime/systemd/transient/default.target.requires"], + ["upheld", "xdg-data/systemd/user/default.target.upholds"], + ["data directory", "xdg-data/systemd/user/default.target.requires"], + ])("blocks standalone fallback for an activation link in the %s root (#8926)", (_root, relativeDirectory) => { + const home = makeTempRoot(); + const activationDirectory = path.join(home, relativeDirectory); + const activationPath = path.join(activationDirectory, "openshell-gateway.service"); + fs.mkdirSync(activationDirectory, { recursive: true }); + fs.symlinkSync(path.join(home, "missing-package-unit.service"), activationPath); + const systemctl = writeUpstreamSystemctlStub(home, { + diagnostic: "Failed to connect to bus: No medium found", + status: 1, + }); + const env: NodeJS.ProcessEnv = { + PATH: `${systemctl.bin}:${path.dirname(process.execPath)}:${TEST_SYSTEM_PATH}`, + ...(relativeDirectory.startsWith("runtime/") + ? { XDG_RUNTIME_DIR: path.join(home, "runtime") } + : {}), + ...(relativeDirectory.startsWith("xdg-data/") + ? { XDG_DATA_DIRS: path.join(home, "xdg-data") } + : {}), + }; + + const result = runInstallHelper( + home, + [ + "upstream_openshell_gateway_user_service_installed() { return 0; }", + "install_nemoclaw_openshell_gateway_user_service", + ].join("\n"), + env, + ); + + expect(result.status).toBe(1); + expect(result.stderr).toContain(activationPath); + expect(fs.lstatSync(activationPath).isSymbolicLink()).toBe(true); + }); + + it("fails closed when the upstream service query returns an unknown error (#8926)", () => { + const home = makeTempRoot(); + const systemctl = writeUpstreamSystemctlStub(home, { + diagnostic: + "Failed to connect to bus: No medium found\nFailed to connect to bus: Permission denied", + status: 1, + }); + + const result = runInstallHelper( + home, + [ + "upstream_openshell_gateway_user_service_installed() { return 0; }", + "install_nemoclaw_openshell_gateway_user_service", + ].join("\n"), + { PATH: `${systemctl.bin}:${path.dirname(process.execPath)}:${TEST_SYSTEM_PATH}` }, + ); + + expect(result.status).toBe(1); + expect(result.stderr).toContain("Failed to connect to bus: Permission denied"); + expect(result.stdout).not.toContain("existing standalone gateway"); + expect(fs.existsSync(servicePath(home))).toBe(false); + }); + + it("fails closed when SYSTEMD_UNIT_PATH overrides the unit search path (#8926)", () => { + const home = makeTempRoot(); + const systemdUnitPath = `${path.join(home, "custom-systemd", "user")}\nspoofed-log-line`; + const systemctl = writeUpstreamSystemctlStub(home, { + diagnostic: "Failed to connect to bus: No medium found", + status: 1, + }); + + const result = runInstallHelper( + home, + [ + "upstream_openshell_gateway_user_service_installed() { return 0; }", + "install_nemoclaw_openshell_gateway_user_service", + ].join("\n"), + { + PATH: `${systemctl.bin}:${path.dirname(process.execPath)}:${TEST_SYSTEM_PATH}`, + SYSTEMD_UNIT_PATH: systemdUnitPath, + }, + ); + + expect(result.status).toBe(1); + expect(result.stderr).toContain("SYSTEMD_UNIT_PATH="); + expect(result.stderr).toContain("\\nspoofed-log-line"); + expect(result.stderr).not.toContain("\nspoofed-log-line"); + expect(result.stdout).not.toContain("existing standalone gateway"); + }); + + it.skipIf(RUNNING_AS_ROOT)( + "fails closed when an activation root cannot be inspected (#8926)", + () => { + const home = makeTempRoot(); + const dataHome = path.join(home, "xdg-data"); + const activationRoot = path.join(dataHome, "systemd", "user"); + fs.mkdirSync(activationRoot, { recursive: true }); + fs.chmodSync(activationRoot, 0o000); + const systemctl = writeUpstreamSystemctlStub(home, { + diagnostic: "Failed to connect to bus: No medium found", + status: 1, + }); + + let result: ReturnType; + try { + result = runInstallHelper( + home, + [ + "upstream_openshell_gateway_user_service_installed() { return 0; }", + "install_nemoclaw_openshell_gateway_user_service", + ].join("\n"), + { + PATH: `${systemctl.bin}:${path.dirname(process.execPath)}:${TEST_SYSTEM_PATH}`, + XDG_DATA_HOME: dataHome, + }, + ); + } finally { + fs.chmodSync(activationRoot, 0o700); + } + + expect(result.status).toBe(1); + expect(result.stderr).toContain("could not inspect"); + expect(result.stderr).toContain(activationRoot); + }, + ); + it("stops an active trusted NemoClaw gateway user service during upgrade retirement (#8800)", () => { const home = makeTempRoot(); const gatewayBin = userGatewayBin(home);