fix(sprites): make cache artifacts and skill helpers usable on Toolbox - #115
Merged
Merged
Conversation
… Toolbox paths Every host-side producer that hands the model a file (stored web pages, browser snapshots, worker artifacts, execute_code output) used to need its own bridge to the paired Toolbox, and several still published host paths that read_file cannot open there. Adopt upstream's single mechanism instead: the cache subdirectories in credential_files._CACHE_DIRS are synced to the Brand's private /tmp/.omnio-session/cache before every command and before any read under it, and to_agent_visible_cache_path() renders the Toolbox path for the sprites backend at every footer that publishes one (web_extract, browser snapshots, delegate summaries). from_agent_visible_cache_path() maps back so host-side media reads keep their fast path. browser_vision keeps publishing the host screenshot path: the omnio_paths plugin owns that field and translates it itself once it detects the projection. The delegation-only sync from #112 folds into the generic projection: same redaction of text artifacts, same refusal of paths outside the projected root, plus a 50 MiB per-file ceiling (logged once) so media never stalls the per-command sync. Text files above the 2 MiB JSON read cap are read and paged through the bounded raw stream (5 MiB) instead of failing with "File too large". execute_code stdout recovery (upstream NousResearch#97043/NousResearch#97048) rides on it: when stdout is truncated to its head/tail window, the sanitized full output (up to 5 MB, complete lines) is saved under cache/exec and the result names the Toolbox path, so the middle can be paged instead of re-running the script. A failed transfer surfaces as a read error, never as a host path. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…aming pages Review of the cache projection found three gaps: - The projected root was a dot directory. The Omnio proxy refuses hidden path segments in sandbox: links, so a screenshot or artifact published under it could never be handed over. The root is now /tmp/omnio-session. - Paths were published before the file reached the Toolbox, so consumers that read the Toolbox directly (the proxy's deliverable warm-up, the video wrapper) could miss a fresh artifact until the next Hermes command or read. publish_cache_path() now pushes the projection before it hands out a path; the Sprites environment registers its live instances for that flush, and a failed push still falls back to the read-time sync. - Above the 5 MiB whole-read ceiling the model was told to page with offset/limit, but paging loaded the whole file and failed the same way. Pages are now computed from the raw stream one chunk at a time, so any size pages and the recipe always works. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
ppazosp
changed the title
ppazosp
changed the title
Retain the direct remote stdout publication from #114 (930b3c0), adapted from upstream NousResearch#97043 and NousResearch#97048, while Sprites keeps #115's shared cache projection. Cover once-only remote execution, recovery of middle output, and failed publication without a host path. Co-authored-by: Teknium <127238744+teknium1@users.noreply.github.com>
ppazosp
marked this pull request as ready for review
September 21, 2026 13:06
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Hermes loads skills and creates cache files on Omnio while commands and file tools use the paired Toolbox. Host paths are unusable there; translated paths can also arrive before their files. Categorized skill names, duplicate copies and host HOME defaults cause additional failed loads and retries.
This PR makes cache artifacts and skill helpers usable in their execution environment. This also consolidates and supersedes #114: bounded, redacted stdout recovery is retained without repeating execution. Bulk execution completeness, execution deadlines, cancellation and queuing are separate work.
Design
The host remains authoritative. Allowed cache subdirectories project into the Brand-private
/tmp/omnio-session/cache; unrelated files and credentials do not cross. Producers use the shared lazy transport to attempt publication before returning a path, including on a cold gateway. Failed publication retains the translated path for a read-time retry. This is not a durable delivery queue: direct consumers can still fail during an outage.Model-facing skill directories, supporting-file hints and
${HERMES_SKILL_DIR}use the existing/skillsprojection, including its external-catalog layout. Management and inline shell preprocessing retain host paths. Commands and skill reads refresh the projection even inside the ordinary sync interval; only changed files upload. Failed helper transfer stops execution rather than running stale code. Directory mapping itself is read-only and cannot replace or delete a live sanitized Docker mount.Skill lookup accepts categorized management names, offers the actual inventory for directory/missing-file requests, and suggests installed names for stale category references. Only byte-identical, same-root copies with a uniquely preferred location are deduplicated. Different content, equal ranks and cross-root collisions stay explicit. Active-profile ownership and local-first management remain unchanged. Missing customer-authored documents still need content repair.
Cache transfer preserves binary bytes and redacts text with the existing transcript redactor, retaining the 50 MiB file ceiling. Large text reads use bounded raw-stream pagination; whole reads stop at 5 MiB. Truncated stdout has a sanitized recovery file capped at 5 MB on complete lines. Local execution and Sprites use the existing host cache publisher; other remote backends retain #114's direct transfer into their execution filesystem, including containers started before the cache directory existed. Transfer failure preserves the execution outcome and warns that recovery is unavailable.
flowchart LR H[Host skill catalog and cache] --> P[Existing Brand-scoped projection] P --> T[Toolbox /skills and visible cache] H --> M[Host-side skill management] P --> R[Runtime paths given to the agent] R --> C[Commands and file reads] C -->|Refresh before use| TUpstream approaches
Adapted to this fork without adopting a different skill-precedence policy:
Merged #97043 and #97048: bounded stdout recovery, consolidated from fix(execute_code): retain readable stdout recovery artifacts #114 with its contributor credit preserved.
Merged #98099: categorized management and directory-file recovery.
Merged #113126: identical same-root duplicate resolution.
Merged #109194: config HOME expansion against the execution environment.
Open proposal #80879: backend-visible skill paths, extended for the secret-free Sprites projection and non-mutating mount lookup. This is not represented as an upstream-merged fix.
Backward compatibility
Backward compatible. Stdout recovery fields are additive, and remote artifacts use existing backend file transport; results and persisted history need no migration. No Toolbox API, schema or persisted-data migration. Local skill paths and internal host loads stay unchanged. Containers use their existing layout; no live mount is replaced. Sprites keeps Brand-scoped copying without credential mounts.
Old Omnia retains legacy screenshot relocation because Hermes still supplies its host screenshot input. Updated Omnia uses the publisher. Either PR can land first; the full screenshot behavior requires both. New helpers work with the existing Toolbox endpoints. Old hidden delegation output remains scratch while new output uses the visible root.
Verification
Consolidation: 39 stdout/cache-delivery tests pass; four new regressions failed before restoring the other-remote fallback. Tests execute a real child in a separate filesystem fixture for SSH/Modal/Docker dispatch, recover the missing middle record and verify a once-only receipt. This fixture is not a live provider deployment. Ruff and
git diff --checkpass. Full GitHub CI passed on final consolidation heada509bbc2486892dd441991439419f8edb3d0cbac.Prior head: full GitHub CI passed on
b2a5ab6e1a.Related Hermes suites cover 464 tests across skills, management, profile isolation, mounting, Sprites and file sync. The initial expanded run had one existing timestamp-dependent config-cache test failure; Linux reproduced unchanged timestamps after 92/100 immediate equal-size writes. Its fixture now advances mtime explicitly; the 36-test file passes. Production cache semantics are unchanged.
Another 30 cache/delegation/video tests pass on the final source; changed-file Ruff and
git diff --checkpass.CI exposed an existing gateway wall-clock assertion (658 ms versus a 500 ms whole-turn bound). Its test now holds the hook blocked until the terminal event, so missing timeout behavior still fails without depending on machine scheduling. All 75 gateway contract tests pass with this test-only repair; production gateway behavior is unchanged.
New regressions failed before repair for runtime paths, categorized lookup, directory reads, duplicate resolution, HOME expansion, live-mount preservation, warm helper synchronization and failed-transfer handling.
The real Omnia plugin also passes through Hermes's actual registry, model-facing schema and argument coercion with READ + sort + null optional fields, producing the unchanged backend payload.
Earlier cache/attachment/media verification remains recorded on the preceding revision; it is not claimed as a fresh full-suite run here.
The paired
omnio.runtime.personalization.real-user-skill-deliveryscenario now requires successful helper creation, the exact returned runtime path, command exit code and unique output, and an exact durable conversation receipt. The cold-screenshot scenario remains registered. Final-pair browser validation is outstanding; keep draft. Server connectivity is restored, but no run against these paired revisions has completed. No customer Sprite was changed.Dependencies
Pairs with useomnia/omnia#4701 for screenshot publication, raw-file streaming and the Omnia session-search contract. No backend migration is required.
Conformance record: tracked as Verifying. The zero-retry product attempt stopped at preflight: missing development database configuration, no app listener on port 3000, and no public callback configured for the isolated checkout. No browser or Sprite action occurred.