Skip to content

[Bug]: Gateway model switch crashes with cannot import name 'env_float' from 'utils' after a code update #51564

Description

@fyzanshaik-atlan

Bug Description

Switching the model of a live gateway session (anthropic/opus -> opencode/glm) crashed the session with:

Sorry, I encountered an error (ImportError).
cannot import name 'env_float' from 'utils' (~/.hermes/hermes-agent/utils.py)

The confusing part: the utils.py on disk does define env_float, so the file named in the error looks fine. Session id: 1518671026962174144.

Steps to Reproduce

  1. Start a gateway and open a session on a checkout that predates the env_float addition (commit 06ca1e99).
  2. Update the code on disk under the running gateway (a git pull, or the window before hermes update's restart fires). utils.py and ~22 consumers now reference env_float.
  3. In the live session, switch model to a different provider: /model <glm/opencode model>.
  4. The switch crashes with: cannot import name 'env_float' from 'utils'.

Expected Behavior

Switching models on a gateway whose code changed underneath it should not crash on a stale-module import. At minimum it should fail with a clear, actionable message (e.g. "restart the gateway") rather than a cryptic ImportError that points at a file which actually contains the symbol.

Actual Behavior

ImportError: cannot import name 'env_float' from 'utils' (~/.hermes/hermes-agent/utils.py)

The gateway is a single long-lived process, so its sys.modules is frozen at boot. A process that imported utils before env_float existed keeps the old module object in memory. After the on-disk code is updated, the first-time lazy import of a consumer module on the new provider's code path runs from utils import env_float against the stale cached utils and raises.

Affected Component

Gateway (Telegram/Discord/Slack/WhatsApp)

Messaging Platform

Discord — but the client is incidental; the trigger is the model switch on a stale process, not the platform. The same failure reproduces with any platform whose adapter (or any consumer) does a first-time from utils import env_float after the update.

Debug Report

Post-hoc report — the original session is gone, so hermes debug share links are not available. Full RCA below; fix is in PR #51561.

Operating System

macOS 15 (Darwin 24.6.0)

Python Version

3.11.10

Hermes Version

v0.17.0 (2026.6.19), upstream bb7ff7dc

Root Cause Analysis

Long-lived gateway process + hot code update + first-time lazy import.

  • env_float added to utils.py in 06ca1e99 (2026-06-20 14:00); ~22 consumers got from utils import env_float 54 min later in a7dd98c86. These are module-level imports.
  • A gateway booted before that holds a stale utils (no env_float) in sys.modules. The on-disk source is then updated.
  • The anthropic/opus path never imported the opencode/chat-completions consumer module. Switching providers triggers its first import, which executes from utils import env_float against the stale cached utils -> ImportError. Python formats the message from the consumer's view of utils.__file__ (the on-disk path), which is why the named file looks correct.
  • Not a circular import (utils imports only stdlib + yaml) and not file shadowing. Reproduced exactly in tests/test_stale_utils_module_import.py.
  • hermes update already gracefully restarts gateways after a pull (hermes_cli/main.py:9638), so this only bites when code changes outside that flow, or in the window before the restart completes.

Proposed Fix

Snapshot the git revision at gateway boot; refuse a /model switch with a clear "restart the gateway" message when the on-disk checkout drifted. Implemented in PR #51561 (gateway/code_skew.py + _model_switch_skew_guard in gateway/slash_commands.py), with regression tests.

Metadata

Metadata

Assignees

No one assigned

    Labels

    P2Medium — degraded but workaround existscomp/gatewayGateway runner, session dispatch, deliverysweeper:risk-compatibilitySweeper risk: may break existing users, config, migrations, defaults, or upgradestype/bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions