Skip to content

feat(gateway): add staged document handling and safe archive support - #4819

Closed
kshitijk4poor wants to merge 1 commit into
NousResearch:mainfrom
kshitijk4poor:feat/gateway-archive-support
Closed

feat(gateway): add staged document handling and safe archive support#4819
kshitijk4poor wants to merge 1 commit into
NousResearch:mainfrom
kshitijk4poor:feat/gateway-archive-support

Conversation

@kshitijk4poor

@kshitijk4poor kshitijk4poor commented Apr 3, 2026 •
edited
Loading

Copy link
Copy Markdown
Collaborator

Summary

Add shared staged document handling for messaging platforms and a safe host-side archive extraction path for uploaded files.

Architecture

Gateway-uploaded files are cached on the Hermes host, but terminal may run in docker/modal/ssh backends that cannot directly see those host cache paths. This PR keeps uploaded-file handling on the gateway/document pipeline and adds a dedicated host-side extract_archive tool for archive workflows.

What Changed

Shared document plumbing

  • centralize document cache helpers in gateway/platforms/base.py
  • support markdown/plaintext/CSV/TSV text attachment injection and bounded CSV/TSV previews
  • extend supported document types to include .zip alongside the existing text/document formats

Adapter behavior

  • normalize Telegram, Slack, and Discord document handling around the shared helpers
  • keep markdown/plaintext/CSV/TSV attachments consistently available as cached documents with prompt enrichment where appropriate
  • update Telegram, Slack, and Discord zip handling to cache uploaded archives instead of rejecting/skipping them
  • keep archives as cached documents referenced by path instead of inlining archive content into the prompt

Archive extraction

  • add tools/archive_tool.py
  • register extract_archive in tool discovery
  • expose it via the file toolset and Hermes core tools
  • reject unsafe archive member paths and common symlink-based escapes
  • enforce basic archive size / extracted size / entry-count limits

Test Results

  • python -m pytest tests/gateway/test_document_cache.py tests/gateway/test_telegram_documents.py tests/gateway/test_slack.py tests/gateway/test_discord_document_handling.py tests/tools/test_archive_tool.py tests/tools/test_modal_sandbox_fixes.py tests/test_model_tools.py -q

Notes

If broader gateway failures appear during validation, they should be called out separately only when they reproduce on current upstream/main and are unrelated to this change.

@kshitijk4poor
kshitijk4poor force-pushed the feat/gateway-archive-support branch from 4b9f191 to 4e122b8 Compare April 3, 2026 18:44
teknium1 added a commit that referenced this pull request Apr 3, 2026
@teknium1
feat: add .zip document support and auto-mount cache dirs into remote…
f15a624 
… backends

- Add .zip to SUPPORTED_DOCUMENT_TYPES so gateway platforms (Telegram,
  Slack, Discord) cache uploaded zip files instead of rejecting them.
- Add get_cache_directory_mounts() and iter_cache_files() to
  credential_files.py for host-side cache directory passthrough
  (documents, images, audio, screenshots).
- Docker: bind-mount cache dirs read-only alongside credentials/skills.
  Changes are live (bind mount semantics).
- Modal: mount cache files at sandbox creation + resync before each
  command via _sync_files() with mtime+size change detection.
- Handles backward-compat with legacy dir names (document_cache,
  image_cache, audio_cache, browser_screenshots) via get_hermes_dir().
- Container paths always use the new cache/<subdir> layout regardless
  of host layout.

This replaces the need for a dedicated extract_archive tool (PR #4819)
— the agent can now use standard terminal commands (unzip, tar) on
uploaded files inside remote containers.

Closes: related to PR #4819 by kshitijk4poor
@teknium1 teknium1 mentioned this pull request Apr 3, 2026
Merged
teknium1 added a commit that referenced this pull request Apr 3, 2026
@teknium1
feat: add .zip document support and auto-mount cache dirs into remote…
b175608 
… backends (#4846)

- Add .zip to SUPPORTED_DOCUMENT_TYPES so gateway platforms (Telegram,
  Slack, Discord) cache uploaded zip files instead of rejecting them.
- Add get_cache_directory_mounts() and iter_cache_files() to
  credential_files.py for host-side cache directory passthrough
  (documents, images, audio, screenshots).
- Docker: bind-mount cache dirs read-only alongside credentials/skills.
  Changes are live (bind mount semantics).
- Modal: mount cache files at sandbox creation + resync before each
  command via _sync_files() with mtime+size change detection.
- Handles backward-compat with legacy dir names (document_cache,
  image_cache, audio_cache, browser_screenshots) via get_hermes_dir().
- Container paths always use the new cache/<subdir> layout regardless
  of host layout.

This replaces the need for a dedicated extract_archive tool (PR #4819)
— the agent can now use standard terminal commands (unzip, tar) on
uploaded files inside remote containers.

Closes: related to PR #4819 by kshitijk4poor
@teknium1

teknium1 commented Apr 3, 2026

Copy link
Copy Markdown
Contributor

Thanks for the contribution @kshitijk4poor! Your PR identified a real gap — zip uploads were rejected and remote backends couldn't access host-cached files.

We took a different architectural approach in PR #4846: instead of adding a dedicated extract_archive tool, we auto-mount the host cache directories (documents, images, audio, screenshots) into Docker/Modal backends so the agent can use standard terminal commands (unzip, tar, etc.) on any cached file. Your .zip support in SUPPORTED_DOCUMENT_TYPES and the gateway test updates were incorporated.

This avoids tool schema bloat while solving the underlying problem more broadly — not just archives, but all cached media is now accessible from remote terminals.

@teknium1 teknium1 closed this Apr 3, 2026
Tommyeds pushed a commit to Tommyeds/hermes-agent that referenced this pull request Apr 12, 2026
@teknium1 AURA
feat: add .zip document support and auto-mount cache dirs into remote…
2d4ae0a 
… backends (NousResearch#4846)

- Add .zip to SUPPORTED_DOCUMENT_TYPES so gateway platforms (Telegram,
  Slack, Discord) cache uploaded zip files instead of rejecting them.
- Add get_cache_directory_mounts() and iter_cache_files() to
  credential_files.py for host-side cache directory passthrough
  (documents, images, audio, screenshots).
- Docker: bind-mount cache dirs read-only alongside credentials/skills.
  Changes are live (bind mount semantics).
- Modal: mount cache files at sandbox creation + resync before each
  command via _sync_files() with mtime+size change detection.
- Handles backward-compat with legacy dir names (document_cache,
  image_cache, audio_cache, browser_screenshots) via get_hermes_dir().
- Container paths always use the new cache/<subdir> layout regardless
  of host layout.

This replaces the need for a dedicated extract_archive tool (PR NousResearch#4819)
— the agent can now use standard terminal commands (unzip, tar) on
uploaded files inside remote containers.

Closes: related to PR NousResearch#4819 by kshitijk4poor
angelburgosrosado pushed a commit to angelburgosrosado/hermes-agent that referenced this pull request Apr 27, 2026
@teknium1
feat: add .zip document support and auto-mount cache dirs into remote…
0129e53 
… backends (NousResearch#4846)

- Add .zip to SUPPORTED_DOCUMENT_TYPES so gateway platforms (Telegram,
  Slack, Discord) cache uploaded zip files instead of rejecting them.
- Add get_cache_directory_mounts() and iter_cache_files() to
  credential_files.py for host-side cache directory passthrough
  (documents, images, audio, screenshots).
- Docker: bind-mount cache dirs read-only alongside credentials/skills.
  Changes are live (bind mount semantics).
- Modal: mount cache files at sandbox creation + resync before each
  command via _sync_files() with mtime+size change detection.
- Handles backward-compat with legacy dir names (document_cache,
  image_cache, audio_cache, browser_screenshots) via get_hermes_dir().
- Container paths always use the new cache/<subdir> layout regardless
  of host layout.

This replaces the need for a dedicated extract_archive tool (PR NousResearch#4819)
— the agent can now use standard terminal commands (unzip, tar) on
uploaded files inside remote containers.

Closes: related to PR NousResearch#4819 by kshitijk4poor
02356abc pushed a commit to 02356abc/hermes-agent that referenced this pull request May 14, 2026
@teknium1
feat: add .zip document support and auto-mount cache dirs into remote…
6331c00 
… backends (NousResearch#4846)

- Add .zip to SUPPORTED_DOCUMENT_TYPES so gateway platforms (Telegram,
  Slack, Discord) cache uploaded zip files instead of rejecting them.
- Add get_cache_directory_mounts() and iter_cache_files() to
  credential_files.py for host-side cache directory passthrough
  (documents, images, audio, screenshots).
- Docker: bind-mount cache dirs read-only alongside credentials/skills.
  Changes are live (bind mount semantics).
- Modal: mount cache files at sandbox creation + resync before each
  command via _sync_files() with mtime+size change detection.
- Handles backward-compat with legacy dir names (document_cache,
  image_cache, audio_cache, browser_screenshots) via get_hermes_dir().
- Container paths always use the new cache/<subdir> layout regardless
  of host layout.

This replaces the need for a dedicated extract_archive tool (PR NousResearch#4819)
— the agent can now use standard terminal commands (unzip, tar) on
uploaded files inside remote containers.

Closes: related to PR NousResearch#4819 by kshitijk4poor
gweeteve pushed a commit to gweeteve/hermes-agent that referenced this pull request Jun 2, 2026
@teknium1
feat: add .zip document support and auto-mount cache dirs into remote…
abaf5ce 
… backends (NousResearch#4846)

- Add .zip to SUPPORTED_DOCUMENT_TYPES so gateway platforms (Telegram,
  Slack, Discord) cache uploaded zip files instead of rejecting them.
- Add get_cache_directory_mounts() and iter_cache_files() to
  credential_files.py for host-side cache directory passthrough
  (documents, images, audio, screenshots).
- Docker: bind-mount cache dirs read-only alongside credentials/skills.
  Changes are live (bind mount semantics).
- Modal: mount cache files at sandbox creation + resync before each
  command via _sync_files() with mtime+size change detection.
- Handles backward-compat with legacy dir names (document_cache,
  image_cache, audio_cache, browser_screenshots) via get_hermes_dir().
- Container paths always use the new cache/<subdir> layout regardless
  of host layout.

This replaces the need for a dedicated extract_archive tool (PR NousResearch#4819)
— the agent can now use standard terminal commands (unzip, tar) on
uploaded files inside remote containers.

Closes: related to PR NousResearch#4819 by kshitijk4poor
waefrebeorn pushed a commit to waefrebeorn/slermes that referenced this pull request Jul 2, 2026
@teknium1
feat: add .zip document support and auto-mount cache dirs into remote…
483d530 
… backends (NousResearch#4846)

- Add .zip to SUPPORTED_DOCUMENT_TYPES so gateway platforms (Telegram,
  Slack, Discord) cache uploaded zip files instead of rejecting them.
- Add get_cache_directory_mounts() and iter_cache_files() to
  credential_files.py for host-side cache directory passthrough
  (documents, images, audio, screenshots).
- Docker: bind-mount cache dirs read-only alongside credentials/skills.
  Changes are live (bind mount semantics).
- Modal: mount cache files at sandbox creation + resync before each
  command via _sync_files() with mtime+size change detection.
- Handles backward-compat with legacy dir names (document_cache,
  image_cache, audio_cache, browser_screenshots) via get_hermes_dir().
- Container paths always use the new cache/<subdir> layout regardless
  of host layout.

This replaces the need for a dedicated extract_archive tool (PR NousResearch#4819)
— the agent can now use standard terminal commands (unzip, tar) on
uploaded files inside remote containers.

Closes: related to PR NousResearch#4819 by kshitijk4poor
@kshitijk4poor
kshitijk4poor deleted the feat/gateway-archive-support branch August 5, 2026 07:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@kshitijk4poor @teknium1