Skip to content

feat(fox-overlay): agent plugin + monkey-patches + relocated mem0_oss (Phase 3b) - #221

Merged
roadhero merged 2 commits into
mainfrom
feat/v0.6.0-p3-agent-overlay
May 17, 2026
Merged

roadhero merged 2 commits into
mainfrom
feat/v0.6.0-p3-agent-overlay

Conversation

@roadhero

@roadhero roadhero commented May 17, 2026 •

Copy link
Copy Markdown
Contributor

Phase 3b (monorepo side) of the v0.6.0 upstream-separation migration (epic #155). Closes #171. Companion to 3 merged fork PRs in fox-in-the-box-ai/hermes-agent: #1 (restore upstream + pause notify), #2 (gateway bootstrap shim), and #3 (restore mem0_oss after architectural rollback).

What this does

Populates packages/fox-overlay/fox_overlay/agent_plugins/ with the 3 Fox monkey-patches relocated from the agent fork. After this PR merges, the agent-side overlay is live in runtime: gateway boot fires register(ctx) via the bootstrap shim, which applies all 3 monkey-patches (8 sentinels) before any patched code path runs.

mem0_oss stays as a bundled fork plugin per the Phase 3 architectural rollback below.

File changes

Path Change
forks/hermes-agent submodule pin 44a6808f → 0a278fc1 (fork main after PRs #1+#2+#3)
packages/integration/Dockerfile +22 lines — insert COPY packages/fox-overlay /app/fox-overlay + pip install -e /app/fox-overlay between upstream pip install (L141) and Phase 2 static ENV block (L143)
packages/fox-overlay/pyproject.toml +7 lines — entry-point fox_overlay = "fox_overlay.agent_plugins:register" (mem0_oss entry-point dropped per rollback)
packages/fox-overlay/MANIFEST.toml +5 lines — populate [python] section with 5 new entries
packages/fox-overlay/fox_overlay/agent_plugins/__init__.py +29 lines — register(ctx) invokes the 3 monkey-patches
packages/fox-overlay/fox_overlay/agent_plugins/fox_overlay_plugin/ NEW package — plugin.yaml (kind: backend) + monkey_patches/{_helpers,runtime_provider,auxiliary_client,cron_diagnostics}.py
packages/fox-overlay/tests/test_agent_overlay.py NEW — 9 pytest cases (8 sentinel checks + 1 idempotency check)

8 monkey-patches applied via substitute_function helper

Upstream target Substitutions Sentinel
hermes_cli.runtime_provider.resolve_runtime_provider 1 _fox_patched_target_model
agent.auxiliary_client._resolve_task_provider_model 1 _fox_patched_provider_auto_fallback
agent.auxiliary_client._get_auxiliary_task_config 2 _fox_patched_auxiliary_default_fallback
cron.jobs.create_job 1 _fox_patched_create_job_failure_history
cron.jobs.mark_job_run 1 _fox_patched_mark_job_run_failure_history
cron.scheduler.run_job 1 (+ traceback injected via extra_globals) _fox_patched_run_job_diagnostics
cron.scheduler.tick 1 _fox_patched_tick_structured_failure
tools.cronjob_tools._format_job 1 _fox_patched_format_job_failure_history

Each patch reads upstream source via inspect.getsource, applies textual substitution with EXACTLY-ONCE anchor check, recompiles in a copy of the upstream module's globals, rebinds the attribute. Anchor drift → AssertionError at register time (caught by test_agent_overlay.py in CI; Phase 9 nightly upstream-watch surfaces failures against new tags). Idempotent via per-target sentinel.

Engineer-discovered architectural rollback (mem0_oss)

The original Phase 3 plan relocated mem0_oss to the overlay as a second entry-point plugin. Two reviewers (#2 DevOps + #3 Adversarial) caught the gap during PR review: upstream's memory-provider discovery is a separate path (plugins/memory/__init__.py) with its own FakePluginContext exposing register_memory_provider — incompatible with the standard PluginContext that entry-point-discovered plugins receive. Relocated mem0_oss could not register a MemoryProvider at gateway runtime via either upstream's PluginManager OR the bootstrap shim.

Dennis approved Option A (revert relocation) over Option B (build a memory-provider context construction in overlay) and Option C (drop mem0_oss from v0.6.0). Net effect:

  • Fork PR install: allow setting hostname for better Tailscale URL #3 restores plugins/memory/mem0_oss/ byte-identical to pre-PR-ci: push :stable tag on main push (alongside :latest) #1 state (3 files, 1218 lines).
  • This PR removes the relocated packages/fox-overlay/fox_overlay/agent_plugins/mem0_oss/ + reverts pyproject.toml entry-point + reverts agent_plugins/__init__.py mem0_oss call + reverts MANIFEST.toml entry.
  • mem0_oss remains a bundled fork plugin, working as it did in v0.5.4 — zero user-visible regression.
  • Future work (out of scope for v0.6.0): dedicated phase to properly relocate memory providers, requiring either (a) upstream PR adding register_memory_provider to PluginContext, or (b) Fox-side fake-context construction.

Validation gates (all green)

  • ✅ Container rebuilds clean (FITB_VERSION=v0.6.0-p3-final, 1.45 GB)
  • ✅ Container starts, /health 200, supervisord all RUNNING
  • ✅ Gateway startup log (/data/data/hermes/logs/agent.log) shows all 8 [fox-overlay] patched ... lines
  • ✅ mem0_oss restored as bundled plugin at /app/hermes-agent/plugins/memory/mem0_oss/ (3 files)
  • ✅ Relocated overlay mem0_oss removed from /app/fox-overlay/fox_overlay/agent_plugins/
  • ✅ pytest tests/test_agent_overlay.py — 9/9 PASS:
    • 8 sentinel assertions (one per monkey-patch)
    • 1 idempotency assertion (register() called twice, sentinels still True)
  • ✅ Zero AssertionError / Traceback / ERROR in any log

Reviewer findings — all actionable items addressed

Finding Severity Resolution
mem0_oss doesn't load at gateway runtime (no register_memory_provider on PluginContext) HIGH × 2 Revert relocation; mem0_oss back in fork as bundled plugin (fork PR #3)
mem0ai + qdrant-client undeclared deps HIGH Moot — deps moved back to upstream's plugin.yaml install path
Bootstrap shim swallows AssertionError → silent drift HIGH Caught instead by test_agent_overlay.py pytest cases (Rule 3 compliance) — anchor drift fails CI test, not silent in production. The shim's existing except Exception log-warning behavior is appropriate for production (degrade gracefully, alert via warning log).
Rule 3 — no pytest cases for 8 monkey-patches FAIL→PASS Added test_agent_overlay.py with 9 cases
_helpers.py shallow-copy semantics docstring LOW Already documented in _helpers.py (line 47-53 about textwrap.dedent) — the related shallow-copy note can fit there as a Phase 4+ follow-up
Eager upstream imports in agent_plugins/__init__.py LOW (Adversarial) Defer — the bootstrap shim's except Exception catch covers the case if upstream renames a target module
_resolve_task_provider_model anchor uniqueness comment LOW Defer to a docstring follow-up

_helpers.py textual-substitution pattern

substitute_function(upstream_module, function_name, substitutions, sentinel, extra_globals=None) reads upstream source via inspect.getsource, applies substitutions sequentially. Anchor must appear EXACTLY ONCE or AssertionError. Idempotent. extra_globals injects names not imported in upstream (used for traceback in cron.scheduler.run_job).

Notably does NOT call textwrap.dedent — dedent normalizes whitespace-only blank lines to bare \n, breaking anchors that match verbatim file content (Engineer-discovered during integration smoke).

Out of scope (per architect brief)

  • bootstrap.py changes — Phase 4 owns those (webui-side shim)
  • Webui-side patches — Phase 4-7
  • Submodule re-point to virgin upstream — Phase 8
  • Memory-provider proper relocation — dedicated future phase (see "Engineer-discovered architectural rollback" above)

Cross-repo cohesion

Three fork PRs merged first per feedback_cross_repo_cohesion.md. This PR bumps the submodule pointer to the final fork main SHA (0a278fc1). Container behavior moves from "Fox patches active via fork edits" to "Fox patches active via overlay monkey-patches" with mem0_oss continuing as a bundled plugin — net zero functional change for users; massive reduction in upstream-tracking burden for the patched files.

Authorship

All commits authored by `roadhero` (Dennis Vorobyov) per Fox migration policy.

Closes #171.

roadhero added 2 commits May 17, 2026 03:10
Phase 3b (monorepo side) of the v0.6.0 upstream-separation migration
(epic #155, sub-issue #171). Companion to merged fork PRs
fox-in-the-box-ai/hermes-agent#1 (revert Fox edits + remove mem0_oss
+ pause notify-monorepo) and #2 (bootstrap shim).

## What this does

Populates packages/fox-overlay/fox_overlay/agent_plugins/ with the
relocated Fox customizations so the gateway boots with Fox bug fixes
restored on top of virgin upstream:

  1. Three monkey-patches against upstream hermes-agent functions,
     applied via `inspect.getsource` + textual substitution at
     register-time (anchor-based; raises AssertionError at startup if
     upstream refactors the target — caught by Phase 9 nightly watch).

     - hermes_cli.runtime_provider.resolve_runtime_provider:
         target_model used for Bedrock api_mode routing
     - agent.auxiliary_client._resolve_task_provider_model:
         provider=auto resolves to explicit provider when cfg_model set
     - agent.auxiliary_client._get_auxiliary_task_config:
         auxiliary.default falls back when no task-specific config
     - cron.jobs.create_job + .mark_job_run: rolling failure_history
     - cron.scheduler.run_job: enriched FAILED output (diagnostics +
       traceback + session-log path)
     - cron.scheduler.tick: structured multi-line failure delivery
     - tools.cronjob_tools._format_job: surface last_error +
       failure_history + consecutive_failures when present

  2. mem0_oss memory provider relocated wholesale from
     forks/hermes-agent/plugins/memory/mem0_oss/ (byte-identical
     restored via git show; SHA256 verified). plugin.yaml gains
     `kind: backend` for upstream v0.13.0's plugin manifest validation.

  3. pyproject.toml entry-points wire fox_overlay + mem0_oss into
     upstream's hermes_agent.plugins group (discovery mechanism
     documented in plan §3 Agent 2).

  4. Dockerfile inserts `COPY packages/fox-overlay /app/fox-overlay` +
     `pip install -e /app/fox-overlay` between the upstream pip
     install (line 141) and the Phase 2 static-overlay ENV block
     (line 143). Layer-cache friendly: static-asset edits don't
     invalidate the (slow) upstream install layer.

  5. Submodule pin bumped to e301105ec (fork main after PR#1 + PR#2
     merged).

## Validation gates

  - Container rebuilds clean (FITB_VERSION=v0.6.0-p3-shim-qa, 1.45 GB)
  - Container starts, /health 200, supervisord all RUNNING
  - Gateway startup log (/data/data/hermes/logs/agent.log) shows all
    8 `[fox-overlay] patched ...` lines confirming each substitution
  - All 8 runtime sentinel attributes True after gateway boot
  - Zero AssertionError / Traceback / ERROR in any log

## Engineer-discovered gap (handled by fork PR #2)

Upstream's PluginManager.discover_and_load() is invoked only from
hermes_cli commands and tui_gateway, never from gateway/run.py. The
plan assumed entry-points auto-fire at gateway boot; they don't.
Resolved via the ~23-line bootstrap shim in fork PR #2 (plan
deviation flagged + Dennis-approved). Net agent patch surface goes
from plan's optimistic 0 files to 1 file / ~23 lines — still within
Phase 8's patch-series model.

## _helpers.py textual-substitution pattern

`substitute_function` reads upstream source via inspect.getsource,
applies (old, new) substitutions sequentially with EXACTLY-ONCE
anchor check, recompiles in a copy of the upstream module's globals,
and rebinds the attribute. Anchor drift → AssertionError at register
time. Idempotent via per-target sentinel.

Notably does NOT call textwrap.dedent — dedent normalizes
whitespace-only blank lines to bare \\n, breaking anchors that match
verbatim file content. Engineer-discovered during integration smoke
(cron.jobs.mark_job_run anchor with 16-space blank line failed
until dedent removed); fix documented in _helpers.py docstring.

## Out of scope (deferred per architect brief)

  - Bootstrap.py changes — Phase 4 owns those (webui-side shim)
  - Webui-side patches — Phase 4-7
  - Submodule re-point to virgin upstream — Phase 8 (extracts the
    fork-PR-#2 shim to packages/fox-overlay/patches/agent/001-*.patch
    at that time)

Closes #171.
Reverts the mem0_oss-relocation portion of this PR per Reviewer #2 and
Reviewer #3's HIGH finding during Phase 3b review. Architectural
mismatch: upstream's memory-provider discovery is a separate
directory-scan path (plugins/memory/__init__.py) with its own
FakePluginContext exposing register_memory_provider — incompatible
with the standard PluginContext that entry-point-discovered plugins
receive. Relocated mem0_oss could not register a MemoryProvider at
gateway runtime via either the upstream PluginManager OR the gateway
bootstrap shim.

## File changes

  - removed packages/fox-overlay/fox_overlay/agent_plugins/mem0_oss/
    (3 files, ~1218 lines)
  - reverted pyproject.toml: drop mem0ai + qdrant-client dependencies
    (mem0_oss is back as a bundled fork plugin; upstream's own
    discovery path installs its plugin.yaml deps if needed)
  - reverted pyproject.toml entry-points: drop mem0_oss entry; keep
    fox_overlay entry for the 3 monkey-patches
  - reverted agent_plugins/__init__.py: register() no longer imports
    or invokes mem0_oss; docstring updated with deferral rationale
  - reverted MANIFEST.toml: drop mem0_oss entry; add deferral comment
  - bumped hermes-agent submodule pin from e301105e to 0a278fc1 (fork
    main after PR #3 merge which restores plugins/memory/mem0_oss/
    byte-identical to pre-PR-#1 baseline)
  - added packages/fox-overlay/tests/test_agent_overlay.py — 9 pytest
    cases verifying each of the 8 monkey-patch sentinels + idempotency
    of register()

## Net Phase 3 scope (after this revert)

The 3 Fox monkey-patches (Bedrock target_model, auxiliary.default
fallback, cron failure diagnostics) ship as planned via the
fox-overlay entry-point + gateway bootstrap shim. mem0_oss continues
as a bundled fork plugin, working as it did in v0.5.4 — zero user-
visible regression.

## Future work (out of scope for v0.6.0)

A dedicated future phase handles proper memory-provider relocation,
requiring either:
  (a) An upstream PR to NousResearch/hermes-agent adding
      register_memory_provider to the standard PluginContext, OR
  (b) A Fox-side fake-context construction in the overlay that mirrors
      plugins/memory/__init__.py's discovery path.

Refs Reviewer #2 + Reviewer #3 findings on PR #221; rollback decision
by Dennis 2026-05-17 (Option A of three presented).
@roadhero
roadhero merged commit 4e44537 into main May 17, 2026
5 checks passed
roadhero added a commit that referenced this pull request May 17, 2026
… refresh auxiliary_client anchor (#242) (#250)

Closes #242.

## Root cause discovered

Phase 3 (PR #221) added a gateway/run.py bootstrap shim that called
`fox_overlay.agent_plugins.register(None)` so the agent monkey-patches
fire at gateway start. The shim was a FORK COMMIT
(fox-in-the-box-ai/hermes-agent#2). Phase 8 ATOMIC re-pointed the
agent submodule at virgin upstream (NousResearch/hermes-agent v2026.5.16)
— the fork commit was lost, and with it the bootstrap shim. Agent
monkey-patches stopped firing entirely (even though entry-points were
discoverable, hermes-agent's PluginManager.discover_and_load() is only
invoked from `hermes` CLI commands, NOT from gateway startup).

Phase 3 commit body explicitly anticipated this:
> "Extracts to a managed patch in packages/fox-overlay/patches/agent/
>  at Phase 8."

This PR delivers that extraction.

## What ships

### Bootstrap shim as managed patch (NEW)

* `packages/fox-overlay/patches/agent/001-gateway-bootstrap-shim.patch`
  — re-applies the Phase 3 bootstrap block to virgin upstream's
  gateway/run.py at Docker build time. Same `FITB_DISABLE_AGENT_OVERLAY=1`
  bypass mechanism as before, gated at TWO points:
  1. Dockerfile patch loop (skips applying the patch entirely)
  2. The patched code itself (runtime env check)
* `packages/fox-overlay/patches/agent/series` — first non-empty entry

### Dockerfile agent patch series consumer (NEW)

New RUN block mirrors the webui patch series, between the webui patch
RUN and the .fox-removals RUN. Apply ordering preserves layer cache
predictability.

### Anchor refresh: auxiliary_client._resolve_task_provider_model

v2026.5.16 changed the cfg_provider!=auto branch return from
`None, None` to `cfg_base_url, cfg_api_key`. Fox's anchor expected
the old form. Updated to match new upstream signature; Fox's
fallback insertion still uses `None, None` (auto path has no explicit
URL/key).

### What stayed unchanged

* runtime_provider patch — anchor still matches v2026.5.16
* auxiliary_client._get_auxiliary_task_config — both anchors still match
* cron_diagnostics patches (create_job + mark_job_run) — anchors still match

## Verification

Container against v2026.5.16:

* Gateway log: NO warning about fox-overlay agent-plugin bootstrap
* `register(None)` runs cleanly via the bootstrap shim
* All 5 Phase 3 monkey-patches sentinels = True:
  - `resolve_runtime_provider._fox_patched_target_model`
  - `_resolve_task_provider_model._fox_patched_provider_auto_fallback`
  - `_get_auxiliary_task_config._fox_patched_auxiliary_default_fallback`
  - `create_job._fox_patched_create_job_failure_history`
  - `mark_job_run._fox_patched_mark_job_run_failure_history`
@roadhero
roadhero deleted the feat/v0.6.0-p3-agent-overlay branch June 1, 2026 05:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[v0.6.0 P3b] (monorepo) implement fox-overlay agent plugin + monkey-patches

2 participants