Skip to content

feat(cron): allow external memory providers in cron jobs while keeping local memory skipped - #9802

Closed
fancydirty wants to merge 1 commit into
NousResearch:mainfrom
fancydirty:feat/cron-external-memory-provider
Closed

feat(cron): allow external memory providers in cron jobs while keeping local memory skipped#9802
fancydirty wants to merge 1 commit into
NousResearch:mainfrom
fancydirty:feat/cron-external-memory-provider

Conversation

@fancydirty

Copy link
Copy Markdown
Contributor

Closes #9763

Problem

cron/scheduler.py hardcoded skip_memory=True when constructing the AIAgent for a scheduled job. Because skip_memory=True prevents _memory_manager from being initialized in run_agent.py, external memory providers such as mem0 are never loaded in a cron-run session. Consequently, all mem0 tools (mem0_search, mem0_conclude, mem0_profile) are reported as unavailable to the agent, even though the user has correctly configured memory.provider: mem0 in config.yaml.

Solution

Introduce a new AIAgent parameter skip_memory_provider (default False) that controls external memory provider initialization independently from skip_memory (which controls local MEMORY.md/USER.md).

  • Cron jobs now pass skip_memory=True + skip_memory_provider=False, preserving the protection against local memory file pollution while allowing external memory providers to work.
  • Subagents and other callers are unaffected due to the default value.

Changes

  • run_agent.py: add skip_memory_provider parameter; move mem_config extraction so the provider block can read it even when skip_memory=True.
  • cron/scheduler.py: pass skip_memory_provider=False to AIAgent.
  • tests/cron/test_scheduler.py: add test verifying cron jobs skip local memory but not external provider.
  • tests/run_agent/test_skip_memory_provider.py: add unit tests for the new parameter.
  • website/docs/guides/cron-troubleshooting.md: document that external memory providers are usable in cron jobs.

Verification

  • python -m pytest tests/cron/test_scheduler.py -x → 62 passed
  • python -m pytest tests/run_agent/test_skip_memory_provider.py -x → 3 passed

Copilot AI review requested due to automatic review settings April 14, 2026 17:55

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR aims to fix cron-run agent sessions so they can initialize and use external memory providers (e.g. mem0) while still skipping built-in local memory files (MEMORY.md / USER.md) to avoid cron prompt pollution.

Changes:

  • Add a new AIAgent init flag (skip_memory_provider) to decouple external provider initialization from local memory loading.
  • Update cron job agent construction to keep skip_memory=True while explicitly allowing external providers.
  • Add/adjust tests and docs around cron memory behavior; additionally, run_agent.py now expands memory-provider behavior during flush_memories() and context compression.

Reviewed changes

Copilot reviewed 8 out of 8 changed files in this pull request and generated 4 comments.

Show a summary per file
File Description
run_agent.py Adds skip_memory_provider and changes provider init gating; also expands flush_memories() tool exposure and adds memory-provider lifecycle hooks during compression.
cron/scheduler.py Passes skip_memory_provider=False for cron jobs while keeping local memory skipped.
tests/cron/test_scheduler.py Verifies cron job construction skips local memory but does not skip external providers.
tests/run_agent/test_skip_memory_provider.py Unit tests for the new skip_memory_provider behavior.
tests/run_agent/test_flush_memories_codex.py Adds coverage asserting flush_memories() includes/executes viking_remember when present.
tests/run_agent/test_compression_memory_provider.py Adds coverage for notifying/reinitializing memory providers during compression-driven session splits.
website/docs/guides/cron-troubleshooting.md Documents that external memory providers can be used from cron jobs while local memory files are skipped.
AGENTS.md Adds an agent guidance rule about not bypassing abstraction layers to mutate production data.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread run_agent.py
Comment on lines 1116 to 1120
# Reads memory.provider from config to select which plugin to activate.
self._memory_manager = None
if not skip_memory:
if not skip_memory_provider:
try:
_mem_provider_name = mem_config.get("provider", "") if mem_config else ""

Copilot AI Apr 14, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Memory provider schemas are appended to self.tools whenever _memory_manager is initialized, regardless of enabled_toolsets/disabled_toolsets. With the new skip_memory_provider gate, this means callers that disable the memory toolset (or rely on skip_memory=True) can still unexpectedly expose external memory tools if a provider is configured. Consider additionally gating provider initialization / schema injection on whether the memory toolset is enabled for this agent instance, or otherwise ensuring toolset filtering is still respected after provider schemas are added.

Copilot uses AI. Check for mistakes.
Comment thread run_agent.py Outdated
Comment on lines +6467 to +6472
# Make one API call with memory-related tools available
memory_tool_defs = []
for t in (self.tools or []):
if t.get("function", {}).get("name") == "memory":
memory_tool_def = t
break
name = t.get("function", {}).get("name")
if name in ("memory", "viking_remember"):
memory_tool_defs.append(t)

Copilot AI Apr 14, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The PR description focuses on cron + skip_memory_provider, but this hunk also expands behavior in flush_memories() (adds viking_remember tool exposure/execution) and later changes _compress_context() to call external memory-provider lifecycle hooks. Please either update the PR description to include these additional behavioral changes, or split them into a separate PR to keep the cron/mem0 fix reviewable and reduce risk.

Copilot uses AI. Check for mistakes.
@@ -117,6 +117,8 @@ Skill names are case-sensitive and must match the installed skill's folder name.

Cron jobs run with the `cronjob`, `messaging`, and `clarify` toolsets disabled. This prevents recursive cron creation, direct message sending (delivery is handled by the scheduler), and interactive prompts. If a skill relies on these toolsets, it won't work in a cron context.

Note that external memory providers (e.g. **mem0**) are **not** affected by this limitation — cron jobs can use `mem0_search` and `mem0_conclude` as long as the provider is configured in `config.yaml`. Only the built-in local memory files (`MEMORY.md` / `USER.md`) are skipped to avoid polluting cron system prompts.

Copilot AI Apr 14, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This note calls out mem0_search and mem0_conclude, but the mem0 provider also exposes other tools (e.g. mem0_profile). Consider wording this more generally (e.g. “mem0_* tools”) so the doc stays accurate if the provider’s tool surface changes, and so users don’t assume only these two tools are available.

Suggested change
Note that external memory providers (e.g. **mem0**) are **not** affected by this limitation — cron jobs can use `mem0_search` and `mem0_conclude` as long as the provider is configured in `config.yaml`. Only the built-in local memory files (`MEMORY.md` / `USER.md`) are skipped to avoid polluting cron system prompts.
Note that external memory providers (e.g. **mem0**) are **not** affected by this limitation — cron jobs can use `mem0_*` tools as long as the provider is configured in `config.yaml`. Only the built-in local memory files (`MEMORY.md` / `USER.md`) are skipped to avoid polluting cron system prompts.

Copilot uses AI. Check for mistakes.
Comment thread run_agent.py
Comment on lines 562 to 565
skip_context_files: bool = False,
skip_memory: bool = False,
skip_memory_provider: bool = False,
session_db=None,

Copilot AI Apr 14, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

skip_memory_provider defaults to False, which changes the meaning of existing skip_memory=True call sites: external memory providers will now be initialized (and their tools injected) even when callers explicitly set skip_memory=True. Several places appear to rely on skip_memory=True meaning “no memory provider” (e.g. gateway/run.py’s temporary flush agent comment). To preserve backward compatibility, consider making skip_memory_provider default to the value of skip_memory (e.g. accept None and coerce to skip_memory), or audit/update all existing skip_memory=True call sites to also pass skip_memory_provider=True where appropriate.

Copilot uses AI. Check for mistakes.
@fancydirty fancydirty closed this Apr 15, 2026
@fancydirty
fancydirty force-pushed the feat/cron-external-memory-provider branch from 42f610b to 4610551 Compare April 15, 2026 01:33
…g local memory skipped

## Problem
 hardcoded  when constructing the
for a scheduled job. Because  prevents  from
being initialized in , **external memory providers such as mem0
are never loaded** in a cron-run session. Consequently, all mem0 tools
(, , ) are reported as *unavailable*
to the agent, even though the user has correctly configured
in .

## Solution
Introduce a new  parameter  (default ) that
controls external memory provider initialization **independently** from
 (which controls local /).

- When  is omitted, it defaults to the value of
  . This preserves backward compatibility for all existing callers
  (subagents, flush agents, batch runners, etc.) so they continue to skip both
  local and external memory.
- Cron jobs explicitly pass  + ,
  preserving the protection against local memory file pollution while allowing
  external memory providers to work.

## Changes
- : add  parameter; move
  extraction so the provider block can read it even when .
- : pass  to .
- : add test verifying cron jobs skip local memory
  but not external provider.
- : add unit tests for the new
  parameter, including backward-compatibility coverage.
- : document that external memory
  providers are usable in cron jobs.

## Verification
- ============================= test session starts ==============================
platform darwin -- Python 3.11.14, pytest-9.0.3, pluggy-1.6.0
rootdir: /Users/mac/.hermes/hermes-agent
configfile: pyproject.toml
plugins: xdist-3.8.0, asyncio-1.3.0, anyio-4.13.0
asyncio: mode=Mode.STRICT, debug=False, asyncio_default_fixture_loop_scope=None, asyncio_default_test_loop_scope=function
created: 8/8 workers
8 workers [62 items]

..............................................................           [100%]
============================== 62 passed in 1.86s ============================== → 62 passed
- ============================= test session starts ==============================
platform darwin -- Python 3.11.14, pytest-9.0.3, pluggy-1.6.0
rootdir: /Users/mac/.hermes/hermes-agent
configfile: pyproject.toml
plugins: xdist-3.8.0, asyncio-1.3.0, anyio-4.13.0
asyncio: mode=Mode.STRICT, debug=False, asyncio_default_fixture_loop_scope=None, asyncio_default_test_loop_scope=function
created: 8/8 workers
8 workers [4 items]

....                                                                     [100%]
============================== 4 passed in 13.26s ============================== → 5 passed

Closes NousResearch#9763
@fancydirty

Copy link
Copy Markdown
Contributor Author

@copilot-pull-request-reviewer Thanks for the thorough review — your feedback was spot on. I've force-pushed a clean branch rebased on upstream main and addressed the issues:

  1. Unrelated changes in diff — You were right: the original branch accidentally included local commits (OpenViking memory loss fix and AGENTS.md rule additions). The PR is now clean and only touches the cron + external memory provider fix.

  2. Backward compatibility (skip_memory_provider default) — Changed the default from False to None. When omitted, it now coerces to the value of skip_memory:

    if skip_memory_provider is None:
        skip_memory_provider = skip_memory

    This means existing call sites (subagents, flush agents, batch runners, etc.) that pass skip_memory=True will continue to skip both local and external memory exactly as before. Only cron explicitly passes skip_memory_provider=False.

  3. Doc wording — Updated the cron troubleshooting note to use mem0_* tools instead of enumerating specific tool names.

  4. Toolset filtering concern — This is a fair observation. The provider initialization happens inside AIAgent.__init__; the schemas are appended to self.tools and then go through the normal toolset filtering logic later in the initialization. If a caller explicitly disables the memory toolset, the provider tools won't be exposed. That behavior is unchanged by this PR.

Regarding the CI failures:

  • Docs Site Checks is failing on team-telegram-assistant.md due to pre-existing ASCII-box formatting errors (11 errors, all in an unrelated file).
  • Tests is failing on tests/gateway/test_discord_reply_mode.py because the CI image is missing the discord package ('NoneType' has no attribute 'DMChannel'). This is also a pre-existing environment issue in upstream.

Both failures are unrelated to this change. All newly added tests pass locally:

  • tests/cron/test_scheduler.py → 62 passed
  • tests/run_agent/test_skip_memory_provider.py → 5 passed

@fancydirty fancydirty reopened this Apr 15, 2026
@fancydirty

Copy link
Copy Markdown
Contributor Author

@teknium1 This is a small, targeted fix for cron external memory providers (closes #9763).

What it does: Introduces skip_memory_provider so that cron jobs can use mem0 while still skipping local MEMORY.md/USER.md injection. Backward compatibility is preserved by defaulting the new flag to skip_memory.

CI status: The two failing checks are pre-existing on upstream main and unrelated to this change:

  • Docs Site Checksteam-telegram-assistant.md ASCII-box formatting errors
  • Tests — missing discord package in CI image causing test_discord_reply_mode.py and test_discord_slash_commands.py failures

All newly added tests pass locally:

  • tests/cron/test_scheduler.py → 62 passed
  • tests/run_agent/test_skip_memory_provider.py → 4 passed

Would appreciate a review when you have a moment. Thanks!

@teknium1 teknium1 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for isolating the external-provider case and preserving the legacy default in the PR branch. Current main makes this a design decision rather than a mechanical fix.

Problems

  • AGENTS.md:1078-1079 explicitly says memory providers intentionally do not run during cron. The added cron flag would initialize providers, which also enables provider prompt context (agent/system_prompt.py:471-478), prefetch (agent/turn_context.py:536-550), and completed-turn sync (agent/turn_finalizer.py:462-468; agent/memory_manager.py:558-614), not just mem0_* tools.
  • The PR's run_agent.py hunk predates the current initialization split. run_agent.py:489-563 now forwards into agent/agent_init.py, where the provider gate lives at agent/agent_init.py:1351-1415; GitHub currently marks this PR CONFLICTING.

Suggested changes

  • Please have maintainers settle the cron memory read/write policy first. The linked #9763 discussion identifies #45769's read-enabled/write-guarded approach as an alternative.
  • If provider access is approved, port the behavior through agent/agent_init.py and test actual provider prompt/prefetch/sync behavior, including the intended cron write policy.

Automated hermes-sweeper review.

Comment thread cron/scheduler.py
quiet_mode=True,
skip_context_files=True, # Don't inject SOUL.md/AGENTS.md from scheduler cwd
skip_memory=True, # Cron system prompts would corrupt user representations
skip_memory_provider=False, # Allow external memory providers (e.g. mem0)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AGENTS.md:1078-1079 documents that providers intentionally do not run in cron. This enables provider prompt context, prefetch, and completed-turn sync as well as provider tools; please establish the intended cron read/write policy before bypassing that invariant.

@teknium1 teknium1 added sweeper:risk-session-state Sweeper risk: may lose/corrupt/mis-associate session or context state sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades sweeper:blast-moderate Sweeper blast radius: moderate — a subsystem or single platform area/memory Memory subsystem: store, providers, sync, background reviews labels Jul 12, 2026
@teknium1

Copy link
Copy Markdown
Contributor

Closing as superseded by PR #91447 (merged as ef04d84) — cron agents now pass skip_memory=False, which activates both the built-in store and external memory providers (no split needed). Thanks @fancydirty.

@teknium1 teknium1 closed this Aug 21, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/memory Memory subsystem: store, providers, sync, background reviews comp/cron Cron scheduler and job management P3 Low — cosmetic, nice to have sweeper:blast-moderate Sweeper blast radius: moderate — a subsystem or single platform sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades sweeper:risk-session-state Sweeper risk: may lose/corrupt/mis-associate session or context state tool/memory Memory tool and memory providers type/feature New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Cron jobs hardcode skip_memory=True, making external memory providers (e.g. mem0) unusable

4 participants