Skip to content

fix: prevent cron/scheduler_provider ModuleNotFoundError from sys.path shadowing - #49414

Closed
ochsec wants to merge 4 commits into
NousResearch:mainfrom
ochsec:fix/issue-49410-cron-syspath-shadow
Closed

fix: prevent cron/scheduler_provider ModuleNotFoundError from sys.path shadowing#49414
ochsec wants to merge 4 commits into
NousResearch:mainfrom
ochsec:fix/issue-49410-cron-syspath-shadow

Conversation

@ochsec

@ochsec ochsec commented Jun 20, 2026

Copy link
Copy Markdown

Fixes #49410

Problem

Gateway crashes on startup with ModuleNotFoundError: No module named 'cron.scheduler_provider' when any platform adapter that inserts plugins/ into sys.path is loaded (discord, raft). The crash creates an infinite restart loop (79+ crash cycles observed).

The root cause: plugins/platforms/discord/adapter.py and plugins/platforms/raft/adapter.py both execute sys.path.insert(0, str(_Path(__file__).resolve().parents[2])) at module level, which inserts the plugins/ directory at sys.path[0]. Since plugins/cron/__init__.py exists (it's the cron provider discovery module), Python resolves from cron.scheduler_provider import ... to plugins/cron/ instead of the real cron/ package. But plugins/cron/ is the provider discovery module — it does NOT contain scheduler_provider.py, so the import fails.

This happens because the cron.scheduler_provider imports were lazy (inside start_gateway() and _start_cron_ticker()), executed after the platform adapters had already mutated sys.path.

Root Cause

Import order: gateway code loads → platform adapters insert plugins/ at sys.path[0] → later from cron.scheduler_provider import ... finds plugins/cron/ first → ModuleNotFoundError.

Fix

Move the cron.scheduler_provider and cron.scheduler imports to module level in gateway/run.py, before the from gateway.platforms.base import ... line that can transitively trigger adapter loading. This ensures the correct cron package is cached in sys.modules before any sys.path mutation occurs.

Specific changes:

  • gateway/run.py: Add module-level imports of InProcessCronScheduler, resolve_cron_scheduler, and _resolve_home_env_var before gateway.platforms.base import
  • gateway/run.py: Remove the three now-redundant lazy from cron.* imports inside functions (_home_target_env_var, _start_cron_ticker, start_gateway)
  • tests/gateway/test_cron_import_shadowing.py: New test file with three tests verifying the shadowing scenario cannot recur

Changes

  • gateway/run.py — promoted 3 lazy cron imports to module level; added comment explaining why ordering matters
  • tests/gateway/test_cron_import_shadowing.py — new test: test_cron_scheduler_provider_importable_after_plugins_path_insert, test_plugins_cron_does_not_have_scheduler_provider, test_module_level_import_before_platform_base

Impact

This is a minimal, surgical fix. No behavioral change — the same symbols are imported from the same modules, just earlier and deterministically. The module-level imports have no side effects beyond populating sys.modules. No circular dependency risk: cron.scheduler_provider only depends on threading, abc, typing, and hermes_cli.config (lazily inside resolve_cron_scheduler).

The same sys.path.insert pattern exists in plugins/platforms/raft/adapter.py:39, so this fix also prevents the crash when raft is the active adapter. The gateway/platforms/base.py:491 insert is safe (it adds the repo root, not plugins/).

@alt-glitch alt-glitch added type/bug Something isn't working comp/gateway Gateway runner, session dispatch, delivery comp/cron Cron scheduler and job management P1 High — major feature broken, no workaround labels Jun 20, 2026
@legacycode

Copy link
Copy Markdown

Tested successfully on NixOS

I tested commit 9ed31fe (PR #49414) on a NixOS host running hermes-agent.service (default gateway) with the messaging extra group enabled.

Before the fix:

  • Gateway crashed immediately on startup with ModuleNotFoundError: No module named 'cron.scheduler_provider'
  • Restart counter exceeded 170 cycles

After the fix:

  • hermes-agent.service starts and stays stable
  • messaging group works without crashes
  • No more import errors

The module-level import order change resolves the issue reliably.

@teknium1

Copy link
Copy Markdown
Contributor

Closing in favor of #49913, which fixes the root cause (parents[2] -> parents[3] across all 5 migrated platform adapters so they insert the repo root, not plugins/). Your approach pre-imported cron.* at module level to cache sys.modules before the bad sys.path insert — that prevents this specific crash, but the broken parents[2] insert stays in place, so any other module imported after a platform adapter loads remains at risk of the same plugins/ shadowing. The kanban change in this PR was also unrelated. Thanks for digging into this.

@teknium1 teknium1 closed this Jun 21, 2026
@ochsec
ochsec deleted the fix/issue-49410-cron-syspath-shadow branch June 22, 2026 01:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/cron Cron scheduler and job management comp/gateway Gateway runner, session dispatch, delivery P1 High — major feature broken, no workaround type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: Gateway crashes on startup with ModuleNotFoundError: No module named 'cron.scheduler_provider'

4 participants